AI and language models

Hello Fredrik Carlsson, who are you?

- I am an algorithm researcher at the Centre for Applied AI, since 2018, researching on large neural networks. Examples of large neural networks are ChatGPT and similar language models.

Can you briefly describe your research?

- Currently, I am researching about how to train large neural networks more effectively. These networks can be used for everything from reading and writing text, to generating or controlling images.

What research question do you want to answer?

- My current research question is how we can train large neural networks with less data, which we call data efficiency. 

What have you found? 

- It is clear that different learning algorithms affect not only the amount of data required, but also the type of "knowledge" the network learns. We are now methodically trying to figure out what leads to what, and then see what conclusions we can se.

How does research work in practice, what do you do on a typical day?

- A large part of the time is spent in front of a computer screen where I implement and run experiments. The rest of the time I spend thinking at a whiteboard or in discussion with colleagues. Recently, there have also been a lot of presentations about this kind of technology.

Can you name some AI applications in society in your research area?

- Yes, for example, GPT technology can be used as a "study buddy" for students. In the longer term, it is also conceivable that teachers could use the technology as an examination tool. However, it is clear that schools and higher education institutions will have a major challenge in terms of examinations in the form of essays and homework. Automatic translation is also an application, where technology can provide a good first draft of a translation that can then be corrected by a human translator. Finally, the technology can be used to find information on the internet, which can affect the market for search engines.  

What do Chat GPT and big language models mean for research and society? 

- There is a big change happening now. In AI and language technology, it is clear that this will have a major impact on research. Research actors will have to consider whether they have the resources to conduct this type of research. For society, this will mean a restructuring of the labour market, where certain job roles can be automated and become more efficient. 

Facts about neural networks

• The idea of Neural Networks was originally based on the idea of imitate the human brain's nerve cells, neurons.
• The neuron handles a simple function, but connected to billions of other neurons, complex tasks can be solved.
• Pattern/image recognition is a strength of neural networks.
• An algorithm is a "recipe" (instructions) on how the computer should solve problems.