Learning Machines seminars – arkiv 2022

The advent of new powerful deep neural networks (DNNs) has fostered their application in a wide range of research areas, including more recently in fluid mechanics. In this presentation, we will cover some of the fundamentals of deep learning applied to computational fluid dynamics (CFD). Furthermore, we explore the capabilities of DNNs to perform various predictions in turbulent flows: we will use convolutional neural networks (CNNs) for non-intrusive sensing, i.e. to predict the flow in a turbulent open channel based on quantities measured at the wall. We show that it is possible to obtain very good flow predictions, outperforming traditional linear models, and we showcase the potential of transfer learning between friction Reynolds numbers of 180 and 550. We also discuss other modelling methods based on autoencoders (AEs) and generative adversarial networks (GANs), and we present results of deep-reinforcement-learning-based flow control. 

Dr. Ricardo Vinuesa is an Associate Professor at the Department of Engineering Mechanics, at KTH Royal Institute of Technology in Stockholm. He is also a Researcher at the KTH Climate Action Centre and Vice Director of the KTH Digitalization Platform. He studied Mechanical Engineering at the Polytechnic University of Valencia (Spain), and he received his PhD in Mechanical and Aerospace Engineering from the Illinois Institute of Technology in Chicago. His research combines numerical simulations and data-driven methods to understand and model complex wall-bounded turbulent flows, such as the boundary layers developing around wings and urban environments. Dr. Vinuesa has received, among others, an ERC Consolidator Grant, the Göran Gustafsson Award for Young Researchers and he is coordinating a number of EU-funded projects.

Despite their fantastic achievements in fields such as computer vision and natural language processing, state-of-the-art deep learning approaches differ from human cognition in fundamental ways. While humans can learn new concepts from just a single or few examples, and effortlessly extrapolate new knowledge from concepts learned in other contexts, deep learning methods generally rely on large amounts of data for their learning. Moreover, while humans can make use of contextual knowledge of e.g. laws of nature and insights into how others reason, such information is generally hard to exploit in deep learning methods.

Current deep learning approaches are indeed purposeful for a wide range of applications where there are large volumes of training data and/or well defined problem settings. However, models that learn in a more human-like manner have the potential to be more adaptable to new situations, be more data efficient and also more interpretable to humans - a desirable property e.g. for intelligence augmentation applications with a human in the loop, e.g. medical decision support systems or social robots.

In this talk I will describe a number of projects in my group where we explore disentanglement, temporality, multimodality, and cause-effect representations to accomplish compositional, structured, and interpretable models of the world.

Hedvig Kjellström is a Professor in the Division of Robotics, Perception and Learning at KTH Royal Institute of Technology, Sweden. She is also a Principal AI Scientist at Silo AI, Sweden and an affiliated researcher in the Max Planck Institute for Intelligent Systems, Germany. She received an MSc in Engineering Physics and a PhD in Computer Science from KTH in 1997 and 2001, respectively, and thereafter worked at the Swedish Defence Research Agency, before returning to a faculty position at KTH.

Her present research focuses on methods for enabling artificial agents to interpret human and animal behavior. These ideas are applied in the study of human aesthetic bodily expressions such as in music and dance, modeling and interpreting human communicative behavior, the understanding of animal behavior and cognition, and intelligence amplification - AI systems that collaborate with and help humans.

Hedvig has received several prizes for her research, including the 2010 Koenderink Prize for fundamental contributions in computer vision. She has written around 130 papers in the fields of computer vision, machine learning, robotics, information fusion, cognitive science, speech, and human-computer interaction. She is mostly active within computer vision, where she is an Associate Editor for IEEE TPAMI and regularly serves as Area Chair for the major conferences.

Today, both science and industry rely heavily on machine learning models, predominantly artificial neural networks, that become increasingly complex and demand a significant amount of computational resources.

The problem of model computational complexity is well known to the computer science community, yet existing methods typically attempt to solve it by shrinking the models, e.g. by quantizing them, or limiting their access to resources.

In this talk, I will look holistically at the efficiency of machine learning models and draw the inspirations to address their main challenges from the green sustainable economy principles.

Instead of constraining some computations or memory used by the models, I will focus on reusing what is available to them: computations done in the previous processing steps, partial information accessible at run-time, or knowledge gained by the model during previous training sessions in continually learned models. This new research path of zero-waste machine learning opens a plethora of research opportunities, both for academia and industry.

Tomasz Trzciński (DSc, WUT'20; PhD, EPFL'14; MSc, UPC/PoliTo'10) is an Associate Professor at Warsaw University of Technology, where he leads a Computer Vision Lab, and at Jagiellonian University of Cracow (GMUM).

He is also a Computer Vision Group Leader at IDEAS NCBR, a publicly-funded Polish Center for AI. He was a Visiting Scholar at Stanford University in 2017 and at Nanyang Technological University in 2019.

Previously, he worked at Google in 2013, Qualcomm in 2012 and Telefónica in 2010. He is an Associate Editor of IEEE Access and MDPI Electronics and frequently serves as a reviewer in major computer science conferences (CVPR, ICCV, ECCV, NeurIPS, ICML) and journals (TPAMI, IJCV, CVIU).

He is a Senior Member of IEEE, member of ELLIS Society, member of the ALICE Collaboration at CERN and an expert of National Science Centre and Foundation for Polish Science.

He is a Chief Scientist at Tooploox and a co-founder of Comixify, a technology startup focused on using machine learning algorithms for video editing.

Natural language processing has never been more popular than now and the success of neural NLP paved the way for ever-increasing expectations in general AI research and intelligent system development. The way that large language models can generalize to highly valuable representations of meaning is astonishing. Neural models support and combine sub-tasks of natural language understanding and generation and, in Helsinki, we are especially interested in translation models that naturally combine both sides of NLP research.

FoTran is a project that studies the use of massively multilingual machine translation as a backbone for representation learning. The idea is that translations provide a rich signal for semantic abstractions, more than generative or masked language models. Our hypothesis is that increased linguistic diversity leads to better abstractions and we look at the internals of a neural model and their applications in various downstream tasks. In this talk I will present some of our studies that investigate the impact of multilingual training objectives on embedding spaces and their application in downstream tasks, and I will discuss ideas and directions for future work.

Jörg Tiedemann is professor of language technology at the Department of Digital Humanities at the University of Helsinki. He received his PhD in computational linguistics for work on bitext alignment and machine translation from Uppsala University before moving to the University of Groningen for 5 years of post-doctoral research on question answering and information extraction. His main research interests are connected with massively multilingual data sets and data-driven natural language processing and he currently runs an ERC-funded project on representation learning and natural language understanding.

Audio codecs (mp3, Opus), are compression algorithms used whenever one needs to transmit audio, whether when streaming a song or during a conference call. In this talk, I will present SoundStream, a novel neural audio codec that can efficiently compress speech, music and general audio at bitrates normally targeted by speech-tailored codecs.

SoundStream relies on a model architecture composed by a fully convolutional encoder/decoder network and a residual vector quantizer, which are trained jointly end-to-end. Training leverages recent advances in text-to-speech and speech enhancement, which combine adversarial and reconstruction losses to allow the generation of high-quality audio content from quantized embeddings. By training with structured dropout applied to quantizer layers, a single model can operate across variable bitrates from 3kbps to 18kbps, with a negligible quality loss when compared with models trained at fixed bitrates.

In addition, the model is amenable to a low latency implementation, which supports streamable inference and runs in real time on a smartphone CPU. In subjective evaluations using audio at 24kHz sampling rate, SoundStream at 3kbps outperforms Opus at 12kbps and approaches EVS at 9.6kbps. Moreover, we are able to perform joint compression and enhancement either at the encoder or at the decoder side with no additional latency, which we demonstrate through background noise suppression for speech.

Neil Zeghidour is a Senior Research Scientist at Google Brain in Paris, and teaches automatic speech processing at Ecole Normale Supérieure. He previously graduated with a PhD in Machine Learning from Ecole Normale Superieure in Paris, jointly with Facebook AI Research. His main research interest is to integrate signal processing and deep learning into fully learnable architectures for audio understanding and generation.

Recognition of pain in animals is important because pain compromises animal welfare and can be a manifestation of disease. This is a difficult task for veterinarians and caretakers, partly because horses, being prey animals, display subtle pain behavior, and because they cannot verbalize their pain. An automated video-based system has a large potential to improve the consistency and efficiency of pain predictions.

Video recording is desirable for ethological studies because it interferes minimally with the animal, in contrast to more invasive measurement techniques, such as accelerometers. Moreover, to be able to say something meaningful about animal behavior, the subject needs to be studied for longer than the exposure of single images. In deep learning, we have not come as far for video as we have for single images, and even more questions remain regarding what types of architectures should be used and what these models are actually learning. Collecting video data with controlled moderate pain labels is both laborious and involves real animals, and the amount of such data should therefore be limited. The low-data scenario, in particular, is under-explored in action recognition, in favor of the ongoing exploration of how well large models can learn large datasets.

The first theme of the talk is automated recognition of pain in horses. Here, we propose a method for end-to-end horse pain recognition from video, finding, in particular, that the temporal modeling ability of the artificial neural network is important to improve the classification. We surpass veterinarian experts on a dataset with horses undergoing well-defined moderate experimental pain induction. Next, we investigate domain transfer to another type of pain in horses: less defined, longer-acting and lower-grade orthopedic pain. We find that a smaller, recurrent video model is more robust to domain shift on a target dataset than a large, pre-trained, 3D CNN, having equal performance on a source dataset. We also discuss challenges with learning video features on real-world datasets.

Motivated by questions arisen within the application area, the second theme of the talk is empirical properties of deep video models. Here, we study the spatiotemporal features that are learned by deep video models in end-to-end video classification and propose an explainability method as a tool for such investigations. Further, the question of whether different approaches to frame dependency treatment in video models affect their cross-domain generalization ability is explored through empirical study. We also propose new datasets for light-weight temporal modeling and to investigate texture bias within action recognition.

Sofia Broomé a PhD student in machine learning at KTH Royal Institute of Technology in Stockholm, at the Robotics, Perception & Learning (RPL) division. Her advisor is Prof. Hedvig Kjellström. Sofia will defend her PhD thesis on September 2nd 2022 entitled "Learning Spatiotemporal Features in Low-Data and Fine-Grained Action Recognition with an Application to Equine Pain Behavior".

Observing and measuring clouds and rain is essential for climate science, meteorology, and an increasing range of societal and economic activities. This is due to the importance of clouds and precipitation for the hydrological cycle and the weather and climate of the Earth. Since patterns of cloudiness and precipitation vary across continental scales, their study and monitoring require observations with global coverage, which currently can only be provided by satellite observations.

My research focuses on development of machine-learning-based methods to improve satellite measurements of hydrometeors, i. e., the particles that make up clouds and precipitation.

I have developed neural-network-based methods for the processing of satellite measurements that can handle the uncertainties caused by their inverse-problem character. These methods only require only minor changes to the training and architecture of the neural network and can be combined with any currently available architectures

The methods have been successfully applied to the processing of observations from the Global Precipitation Measurement, an international satellite mission led by NASA and the Japanese Space Agency, as well as near real-time measurements of rain over Brazil. In both applications, the novel methods lead to considerable improvements in the accuracy of the satellite measurements.
The key advantage of the methods is that they allow combining the power of modern, deep neural networks with the mathematically sound handling of uncertainties offered by conventional methods for solving inverse problems. The methods unlocked significant improvements in global measurements of precipitation and will therefore be used operationally in the upcoming processing scheme for the Global Precipitation Measurement mission.

Simon Pfreundschuh obtained a BSc in Computer Science from the University of Kiel and two MScs in Engineering Mathematics and Physics and Astronomy from Chalmers University of Technology. From 2017 until 2022 he was a graduate student at the Department of Space, Earth and Environment at Chalmers University of Technology. He was a visiting scholar at the Department of Atmospheric Science at Colorado State University for his work on the Global Precipitation Measurement satellite mission. He completed his graduate studies in May 2022 and will continue his work as a post-doctoral fellow at Chalmers University of Technology and associate scientist at the Swedish Meteorological and Hydrological Institute (SMHI).

In this presentation we will first give an overview of the recent achievements of the machine learning group at LTU and then dive deep into the young field of Technical Language (TL) Understanding and Supervision.

Recent advances of NLP and NLU generated a vast amount of practically useful applications. When it comes to technical language with very domain-specific terms, however, the general deep learning models are not directly applicable.

The terms, abbreviations and TL in general, create difficulties, as those do not appear in standard communication. We present algorithms to address these challenges and even benefit from typical structures in technical environments. The practical use is show-cased on an actual technical monitoring systems in process industry.

Marcus Liwicki is professor, Chair of Machine Learning and Vice-Rector for Applied AI at Luleå University of Technology and a senior assistant in the University of Fribourg. He received his PhD degree from the University of Bern, Switzerland, in 2007. His research interests include machine learning, pattern recognition, artificial intelligence, human computer interaction, digital humanities, knowledge management, ubiquitous intuitive input devices, document analysis, and graph matching.

Fredrik Sandin is a professor in Machine Learning at the Lulea University of Technology (LTU) in Sweden. He is interested in brain-inspired machine learning, neuromorphic computing, and applications requiring new efficient approaches to sensing, computing, and artificial intelligence. He has a Ph.D. in Physics (2007) from the Swedish Graduate School of Space Technology.

Karl Ekström is a PhD candidate at Luleå University of Technology. He is working in technical language processing and applies novel fundamental research results to real applications in the processing industry.

The rapid development and deployment of machine learning systems is having a profound impact on our society. In recent years these systems have excelled in a wide range of data-driven prediction problems.

However, in many cases, such as safety-critical applications, high predictive accuracy is not enough. To make the systems robust we also need to reliably reason about the uncertainties in their predictions.

In this talk I will first discuss the notion of calibration as a fundamental requirement for probabilistic predictive models to report reliable uncertainty estimates.

I will introduce a kernel-based calibration error that can be used to test for calibration for general probabilistic models. Second, I will discuss how label noise might influence both accuracy and calibration of a probabilistic classifier, and the role of so called robust loss functions in this setting.

Fredrik Lindsten is an associate professor and head of division at the division of statistics and machine learning at Linköping University, and a senior lecturer at the department of information technology, Uppsala University.

Fredrik received his PhD in automatic control from Linköping University in 2013. During 2014 and 2015 he was a postdoc at the department of engineering at the University of Cambridge. Fredrik has been a visiting researcher at the SAIL lab at University of California, Berkeley and at the department of statistics at the University of Oxford. His research interests are in probabilistic models, uncertainty quantification for machine learning, computational statistics and signal processing.

The introduction of immensely large Causal Language Models (CLMs) has rejuvenated the interest in open-ended text generation. However, controlling the generative process for these Transformer-based models is at large an unsolved problem.

Earlier work has explored either plug-and-play decoding strategies, or more powerful but blunt approaches such as prompting. There hence currently exists a trade-off between fine-grained control, and the capability for more expressive high-level instructions.

To alleviate this trade-off, we propose an encoder-decoder architecture that enables intermediate text prompts at arbitrary time steps. We propose a resource-efficient method for converting a pre-trained CLM into this architecture, and demonstrate its potential on various experiments, including the novel task of contextualized word inclusion. Our method provides strong results on multiple experimental settings, proving itself to be both expressive and versatile.

Fredrik Carlsson is a researcher at RISE working on deep learning for language and vision and combinations of the two. He has published papers in venues such as ICLR and ACL on contrastive learning and representation learning for natural language processing.

One of the key lessons to take away from contemporary machine learning is that flexible models offer the best predictive performance. This has implications in many situations.

In this lecture I will try to make this concrete by looking at a few constructions that we are working with. I will start with a classification task from ECG interpretation and then continue to the more under-researched area of how to formulate and solve regression problems using deep learning.

There are currently several different approaches used for deep regression and there is still room for innovation. I will illustrate this landscape in general and introduce our rather general deep regression method which has a clear probabilistic interpretation. We show good performance on several computer vision regression tasks, system identification problems and 3D object detection using laser data.

Thomas B. Schön is the Beijer Professor Artificial Intelligence in the Department of Information Technology at Uppsala University. In 2018, he was elected to The Royal Swedish Academy of Engineering Sciences (IVA) and The Royal Society of Sciences at Uppsala. He received the Tage Erlander prize for natural sciences and technology in 2017 and the Arnberg prize in 2016, both awarded by the Royal Swedish Academy of Sciences (KVA). He was awarded the best PhD thesis award by The European Association for Signal Processing in 2013. He received the best teacher award at the Institute of Technology, Linköping University in 2009.

Multiple and competing water demands can require complex water allocation decisions that must be made with imperfect information about the future. Near-term forecasts of environmental outcomes can inform real-time decision making by providing predicted future conditions with associated uncertainty. In this talk, we describe our approach of combining process-guided deep learning models with a data assimilation algorithm to generate 7-day forecasts of maximum stream water temperature in the Delaware River Basin to support New York City reservoir management decisions. We evaluate the forecast accuracy and reliability during the summer of 2021 and discuss future workflow and model improvements to better deliver valuable information in support of environmental management decisions.

Jake Zwart (he/him) is a data scientist at the U.S. Geological Survey (USGS) in the Data Science Branch of the Water Mission Area. Jake's work currently focuses on producing short-term forecasts of stream temperature at regional scales to aid in water resources decision making. The data science group at USGS has developed techniques to inject scientific knowledge into machine learning models (process-guided deep learning) to make accurate predictions of environmental variables, and developed methods for assimilating real-time observations into these models to improve near-term forecasts of water quality.

Neural autoregressive sequence models smear the probability among many possible sequences including degenerate ones, such as empty or repetitive sequences.

In this work, we tackle one specific case where the model assigns a high probability to unreasonably short sequences. We define the oversmoothing rate to quantify this issue. After confirming the high degree of oversmoothing in neural machine translation, we propose to explicitly minimize the oversmoothing rate during training.

We conduct a set of experiments to study the effect of the proposed regularization on both model distribution and decoding performance. We use a neural machine translation task as the testbed and consider three different datasets of varying size.

Our experiments reveal three major findings.

First, we can control the oversmoothing rate of the model by tuning the strength of the regularization.

Second, by enhancing the oversmoothing loss contribution, the probability and the rank of <eos> token decrease heavily at positions where it is not supposed to be.

Third, the proposed regularization impacts the outcome of beam search especially when a large beam is used. The degradation of translation quality (measured in BLEU) with a large beam significantly lessens with lower oversmoothing rate, but the degradation compared to smaller beam sizes remains to exist.

From these observations, we conclude that the high degree of oversmoothing is the main reason behind the degenerate case of overly probable short sequences in a neural autoregressive model.
 

Kyunghyun Cho is an associate professor of computer science and data science at New York University and CIFAR Fellow of Learning in Machines & Brains. He is also a senior director of frontier research at the Prescient Design team within Genentech Research & Early Development (gRED). He was a research scientist at Facebook AI Research from June 2017 to May 2020 and a postdoctoral fellow at University of Montreal until Summer 2015 under the supervision of Prof. Yoshua Bengio, after receiving PhD and MSc degrees from Aalto University April 2011 and April 2014, respectively, under the supervision of Prof. Juha Karhunen, Dr. Tapani Raiko and Dr. Alexander Ilin. He tries his best to find a balance among machine learning, natural language processing, and life, but almost always fails to do so.

Many of the most widely used neural network architectures make use of rectified linear activations (ReLU, i.e. f(x) = max(x, 0)) and are therefore piecewise linear functions. The maximal subsets of the input space on which the network function is linear are called linear regions.

If we want to better understand ReLU networks, it may be beneficial to understand these regions. There is the common intuition that the number of linear regions of neural networks measures their expressivity. Therefore a lot of focus has been placed on trying to obtain estimates of this number.

However, this number is staggeringly high: Even very small networks have many more linear regions than there are atoms in the universe (1080). This number is also much larger than the number of points in the dataset. This raises the question of how representative the number of linear regions is for network performance and how information extracted from training samples passes on to the many linear regions free of data for successful generalisation to test data.

Our approach differs from previous ones focused on counting in that it investigates the linear coefficients associated to the linear linear regions. We propose TropEx, a tropical algebra-based algorithm extracting linear terms of the network function. This allows us to compare the network function with an extracted tropical function that agrees with the original network around all training data points, but which has much fewer linear regions.

We also use our algorithm to compare different network types from the perspective of linear regions and their coefficients.

Martin Trimmel has been a doctoral student at Lund University since October 2017 under the supervision of Professor Cristian Sminchisescu. Martin obtained a Master of Advanced Studies degree from the University of Cambridge and a BSc degree from the University of Edinburgh. He did both his Master’s and Bachelor’s degrees in mathematics, focusing on algebra and algebraic geometry during his time at Cambridge. During his undergraduate studies Martin also spent a semester at Université Toulouse-Jean-Jaurès in France. In his research, Martin is interested in using mathematics to obtain a better theoretical understanding of deep learning. In particular, his previous work has focused on the effect the activations have on the network function and its performance.

We have recently seen the development of complex language models capable of everything between writing news articles and generating stories about unicorns. These models outclass several natural language understanding tasks, and despite this they cannot answer basic questions relating to multiplication, how to get a table through a doorway or what the color of a dove is. Arguments that recently have arisen claim that this is expected, since a model that only ever has seen text cannot be expected to understand the meaning behind the words it sees, since the meaning lies somewhere outside of text. Thus, these powerful language models have recently been referred to as “stochastic parrots”.

A way that has been proposed to mitigate this issue is to connect the model with more information than text, such as images, sounds and effects of actions, that may provide the meaning behind the text. This is also referred to as “grounding the model"". The proposed setup is not entirely dissimilar to how humans can read and write, but also interact with the outside world, by e.g. being able to see and hear. In my presentation, I will give an overview of this problem and the proposed solutions. I will also talk about my contributions to this research area.

Lovisa Hagström is a PhD student at the department of Computer Science and Engineering at Chalmers University of Technology. She does research within Natural Language Processing (NLP) with a focus on making language models learn from more than text, where she is currently working with vision-language models. She has a master’s degree in Engineering Physics from Chalmers and spent one year studying abroad at the National University of Singapore. Her personal interests include cross-country skiing and running.

A widely applied technique in early drug discovery to identify novel active molecules against a protein target is modelling quantitative structure-activity relationships (QSAR). However, small datasets are ubiquitous in drug discovery as data generation is expensive and can be restricted for ethical reasons (e.g. in vivo experiments).

Therefore, QSAR prediction is known to be extremely challenging, as available measurements of compound activities range in the low dozens or hundreds. However, there are often large databases of related datasets, each with a small number of datapoints, opening up the opportunity for few-shot learning after pretraining on a substantially larger corpus of data.

While many few-shot learning methods were developed and evaluated in the computer-vision domain, we demonstrate that application in the explicitly graph-structured molecular domain provides an increase in QSAR modelling performance. In particular, through use of a purpose-made few-shot learning dataset of molecules and evaluation procedure, we are able to compare the performance of a number of few-shot and meta-learning methods against standard single-task learners.

We show that prototypical networks in particular are valuable in this domain, and that model-agnostic meta-learning is capable of AUPRC enhancement in the very low-data regime. We also implement a range of pre-trained, single-task and multitask baselines, and seek to encourage further development of methods to address tasks with very limited data availability.

Magnus Röding is a scientist at RISE in the department of Agriculture and Food and the Product Design unit in Gothenburg. He is also Adjunct Associate Professor in Mathematical Statistics at the Department of Mathematical Sciences, Chalmers University of Technology. His research interests concern modelling, statistics, and machine learning in connection to materials science and data analysis in imaging, microscopy, and other experimental techniques.

A widely applied technique in early drug discovery to identify novel active molecules against a protein target is modelling quantitative structure-activity relationships (QSAR). However, small datasets are ubiquitous in drug discovery as data generation is expensive and can be restricted for ethical reasons (e.g. in vivo experiments).

Therefore, QSAR prediction is known to be extremely challenging, as available measurements of compound activities range in the low dozens or hundreds. However, there are often large databases of related datasets, each with a small number of datapoints, opening up the opportunity for few-shot learning after pretraining on a substantially larger corpus of data.

While many few-shot learning methods were developed and evaluated in the computer-vision domain, we demonstrate that application in the explicitly graph-structured molecular domain provides an increase in QSAR modelling performance. In particular, through use of a purpose-made few-shot learning dataset of molecules and evaluation procedure, we are able to compare the performance of a number of few-shot and meta-learning methods against standard single-task learners.

We show that prototypical networks in particular are valuable in this domain, and that model-agnostic meta-learning is capable of AUPRC enhancement in the very low-data regime. We also implement a range of pre-trained, single-task and multitask baselines, and seek to encourage further development of methods to address tasks with very limited data availability.

Megan Stanley is a researcher at Microsoft Research, Cambridge, UK, currently working on applications of machine learning in the sciences and with particular interest in few-shot learning and generalization. She graduated from her PhD in Physics at the University of Cambridge in 2017.

Natural soundscapes contain valuable information, readily available, about biodiversity and animal behaviour. Indeed a revolution in acoustic monitoring is already underway, driven by deep learning (DL) and big data, as in other domains. But beyond basic recognition tools, what can we gain from high-resolution DL acoustic tasks? Can we use DL to explore animal perception, and would that help with biodiversity monitoring?

I will present recent work on modelling animal acoustic cognitive tasks: song similarity in birds and echolocation flower recognition in bats, using data collected non-invasively from the animals themselves. I will focus on DL training scenarios designed to reflect the cognitive tasks.

Dan Stowell is an Associate Professor of AI & Biodiversity working in the Netherlands (Tilburg University, Naturalis Biodiversity Centre, JADS). His research is about machine listening and computational bioacoustics - which means using computation (especially machine learning) to understand animal sounds and other sound signals.

- I develop automatic processes to analyse large amounts of sound
recordings - for example detecting the bird sounds in there and how they vary, how they relate to each other, how the birds' behaviour relates to the sounds they make. The research work is focussed on the machine learning and signal processing methods that can help with these questions. I also work with others to apply these methods to biodiversity monitoring.

Dan is a Fellow at Alan Turing Institute, working with OpenClimateFix and OpenStreetMap on addressing climate change through solar panel mapping.

This presentation will focus on the application of machine learning methods in the interpretation of data gathered in experiments aimed at describing the shape, structure or molecular composition of materials, via small angle x-ray scattering (SAXS) and nuclear magnetic resonance spectroscopy (NMR).

The challenges in both of these experimental methods lies in the interpretation of complex data sets, which can be time consuming and cumbersome even for trained experts – which also run the risk of introducing bias to the interpretations. Cases where ML-applications could both accelerate the interpretation and enhance the degree of understanding of a system have been identified.

More specifically, the systems studied and methods applied are as follows: For SAXS, both diluted and solid porous systems were studied. In the diluted systems, classification of the shapes of particles in liquid was investigated, using several classification algorithms. For solid porous systems classification of the type of material was investigated, but also regression analysis parameters describing the shape of the materials.

For the NMR spectroscopic investigations, the investigations were focused on determining the individual concentrations of a group of compounds in a mixture, through image analysis of two-dimensional spectra. There have also been investigations on predicting the spectral appearance from molecular structures.

The results from these studies and their significance for experimental scientists in their everyday work will be discussed.

Jerk holds a PhD in Organic Chemistry from Stockholm University. He has been employed by RISE since 2014 and have primarily been working with development of methods for analysis of biobased materials with NMR spectroscopy, and other techniques. He is currently working on development of sustainable multifunctional materials.

This presentation will have two parts describing recent research I have been involved with. Initially, I will focus on zero-shot learning - learning how to recognise novel visual classes from only text descriptions. This work introduces the Wiki-ImageNet database and shows that incorporating quality text descriptions within existing simple and SOTA zero-shot learning algorithms improves performance. The second part will deal with a more theoretical issue. The function parameterized by most ConvNets is piecewise affine. There is a body of work debating whether the density of regions in the piecewise function of a well-trained over-parametrized network is correlated with its expressiveness. In our recent work, we show empirically for the first time on realistic network architectures that region density may fall short of capturing meaningful nonlinearity of over-parameterized deep networks.

Josephine Sullivan is a lecturer in computer vision within the School of Computer Science and Communication (CSC) at the Royal Institute of Technology (KTH). Her research efforts are devoted to the field of computer vision, including visual tracking, human/object pose recognition and multi-target tracking. Josephine obtained a D.Phil from University of Oxford University, Dept. Of Engineering Sciences in 2001 under the supervision of Prof. Andrew Blake.

This talk is divided into two parts. In the first part I provide a brief overview of how artificial intelligence (AI) is being or can be used to mitigate and adapt to the effects of climate change. The content here will to some extent be based on the report "Tackling Climate Change with Machine Learning" by Rolnick et al. In the second part I outline some potential considerations and discussion topics regarding AI before and/or during a potential (climate change-induced) societal collapse, which hundreds of scholars now say we should discuss and prepare for. The purpose of this second part is to initiate dialogue on how AI may be used to mitigate and adapt to (parts of) collapse, as well as to highlight risks with and limits of AI in this context.

Aleksis is a senior machine learning researcher at RISE Research Institutes of Sweden since September 2021. His current main research interest is to develop AI methodologies for climate adaptation. Prior to joining RISE, he received his PhD degree in computer vision from Lund University, where his research focused on using reinforcement learning for various computer vision tasks such as object detection, semantic segmentation and human pose estimation. Concrete research interests at RISE include investigating how reinforcement learning can be used to control mobile systems (e.g. drones) for climate adaptation-related tasks such as crop monitoring.

Many physical events and processes create sound, often in the hearable spectrum. This makes sound an interesting source of information about such processes. By combining sound sensors with machine learning, these can be tracked and analyzed. Here we will discuss how we have applied this combination for several different usecases, including: Noise Monitoring of shooting ranges and construction sites, Process Monitoring of coffeebean cracking during roasting, and Condition Monitoring of equipment such as pumps and ventilation. We will discuss some of the techniques in use, such as Convolutional and Recurrent Neural Networks, spectrogram pre-processing, and edge computing on embedded devices and microcontrollers. We will also discuss some of the challenges that come when deploying continious monitoring, such as data collection, heavily imbalanced data and preserving privacy.

Jon is a Machine Learning Engineer that specializes in audio and IoT applications. He has a Master in Data Science and a Bachelor in Electronics Engineering, and has worked as a software engineer in electronics and web projects for 10 years. Since 2019 he is the Head of Machine Learning and Data Science at Soundsensing, a provider of IoT sensors for sound with built in Machine Learning capabilities.

How will a patient's health be affected by taking a given medication? How will a user's question be answered by a google search result? How will cloud reflectively and precipitation be influenced by a local emissions policy? Making effective decisions depends on being able to answer such questions.

Answering such questions requires knowledge about the causal effect that a decision (medication, search result, emissions policy) has in a specific context. And knowing the effect of the decision requires knowledge about the context.

– In this talk, I will review recent collaborative works that aim to:
1) quantify our uncertainties about decision outcomes at the individual/unit, group, or population level
2) understand decision making under such uncertainty
3) how we can learn more about the world in order to reduce our uncertainty.

I will also discuss recent applied work looking at using causal models to understand Aerosol-Cloud Interactions

Andrew is a PhD student in the Department of Computer Science, University of Oxford

He works in the Applied and Theoretical Machine Learning Group (OATML) under the supervision of Yarin Gal. 
Prior to joining the group, he was a program manager and researcher at Imagia in Montreal. 

He obtained his undergraduate and master’s degrees from the Department of Electrical and Computer Engineering at McGill University, working in the Probabilistic Vision Group (PVG) under the supervision of Tal Arbel. 

His research focuses on personalized decision making under uncertainty in causal-effect estimates. 
Specifically, he is interested in quantifying, integrating, and reducing uncertainty arising from the relaxation of causal assumptions.