- title: 'FastStitch: Speech editing by hitch-hiking a pre-trained FastSpeech2 model' abstract: 'We present an innovative approach to speech editing, mitigating the time-consuming process of training acoustic models from scratch. Our methodology involves fine-tuning the upper layers of a pre-trained FastSpeech2 model and fusing it with information from a reference mel-spectrogram during inference via a convolution-based, or an attention-based, blending network. Comparative evaluations against baseline methods and against state-of-the-art techniques on single-speaker (LJSpeech) as well as multi-speaker (VCTK) datasets, employing both subjective and objective measures, demonstrate the superior quality of our approach, yielding significantly more natural-sounding speech edits.' volume: 233 URL: https://proceedings.mlr.press/v233/alexos24a.html PDF: https://proceedings.mlr.press/v233/alexos24a/alexos24a.pdf edit: https://github.com/mlresearch//v233/edit/gh-pages/_posts/2024-01-23-alexos24a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the 5th Northern Lights Deep Learning Conference ({NLDL})' publisher: 'PMLR' author: - given: Antonios family: Alexos - given: Pierre family: Baldi editor: - given: Tetiana family: Lutchyn - given: Adín family: Ramírez Rivera - given: Benjamin family: Ricaud page: 1-6 id: alexos24a issued: date-parts: - 2024 - 1 - 23 firstpage: 1 lastpage: 6 published: 2024-01-23 00:00:00 +0000 - title: 'Deep Learning Over-Parameterization: the Shallow Fallacy' abstract: 'A major tenet of conventional wisdom dictates that models should not be over-parameterized: the number of free parameters should not exceed the number of training data points. This tenet originates from centuries of shallow learning, primarily in the form of linear or logistic regression. It is routinely applied to all kinds of data analyses and modeling and even to infer properties of the brain. However, through a variety of precise mathematical examples, we show that this conventional wisdom is completely wrong as soon as one moves from shallow to deep learning. In particular, we construct sequences of both linear and non-linear deep learning models whose number of parameters can grow to infinity, while the training set can remain very small (e.g. a single example). In deep models, the parameter space is partitioned into large equivalence classes. Learning can be viewed as a communication process where information is communicated from the data to the synaptic weights. The information in the training data only needs to specify an equivalence class of the parameters, and not the exact parameter values. As such, the number of training examples can be significantly smaller than the number of free parameters.' volume: 233 URL: https://proceedings.mlr.press/v233/baldi24a.html PDF: https://proceedings.mlr.press/v233/baldi24a/baldi24a.pdf edit: https://github.com/mlresearch//v233/edit/gh-pages/_posts/2024-01-23-baldi24a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the 5th Northern Lights Deep Learning Conference ({NLDL})' publisher: 'PMLR' author: - given: Pierre family: Baldi editor: - given: Tetiana family: Lutchyn - given: Adín family: Ramírez Rivera - given: Benjamin family: Ricaud page: 7-12 id: baldi24a issued: date-parts: - 2024 - 1 - 23 firstpage: 7 lastpage: 12 published: 2024-01-23 00:00:00 +0000 - title: 'Deep Reinforcement Learning for Goal-Based Investing Under Regime-Switching' abstract: 'Goal-based investing focuses on helping investors achieve specific financial goals, shifting away from the volatility-based risk paradigm. While numerous methods exist for this type of problem, the majority of them struggle to properly capture the non-stationary dynamics of real-world financial markets. This paper introduces a novel deep reinforcement learning framework for goal-based investing that addresses market non-stationarity through prompt reactions to regime switches. It relies on the integration of regime probability estimates directly into the state space. The experimental results indicate that the proposed method significantly outperforms several benchmarks commonly used in goal-based investing.' volume: 233 URL: https://proceedings.mlr.press/v233/bauman24a.html PDF: https://proceedings.mlr.press/v233/bauman24a/bauman24a.pdf edit: https://github.com/mlresearch//v233/edit/gh-pages/_posts/2024-01-23-bauman24a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the 5th Northern Lights Deep Learning Conference ({NLDL})' publisher: 'PMLR' author: - given: Tessa family: Bauman - given: Sven family: Goluža - given: Bruno family: Gasperov - given: Zvonko family: Kostanjcar editor: - given: Tetiana family: Lutchyn - given: Adín family: Ramírez Rivera - given: Benjamin family: Ricaud page: 13-19 id: bauman24a issued: date-parts: - 2024 - 1 - 23 firstpage: 13 lastpage: 19 published: 2024-01-23 00:00:00 +0000 - title: 'A Dual Convolutional Neural Network Pipeline for Melanoma Diagnostics and Prognostics' abstract: 'Melanoma is a type of cancer that begins in the cells controlling the pigment of the skin, and it is often referred to as the most dangerous skin cancer. Diagnosing melanoma can be time-consuming, and a recent increase in melanoma incidents indicates a growing demand for a more efficient diagnostic process. This paper presents a pipeline for melanoma diagnostics, leveraging two convolutional neural networks, a diagnosis, and a prognosis model. The diagnostic model is responsible for localizing malignant patches across whole slide images and delivering a patient-level diagnosis as malignant or benign. Further, the prognosis model utilizes the diagnostic model’s output to provide a patient-level prognosis as good or bad. The full pipeline has an F1 score of 0.79 when tested on data from the same distribution as it was trained on.' volume: 233 URL: https://proceedings.mlr.press/v233/bo-sande24a.html PDF: https://proceedings.mlr.press/v233/bo-sande24a/bo-sande24a.pdf edit: https://github.com/mlresearch//v233/edit/gh-pages/_posts/2024-01-23-bo-sande24a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the 5th Northern Lights Deep Learning Conference ({NLDL})' publisher: 'PMLR' author: - given: Marie family: Bø-Sande - given: Edvin family: Benjaminsen - given: Neel family: Kanwal - given: Saul family: Fuster - given: Helga family: Hardardottir - given: Ingrid family: Lundal - given: Emilius A.M. family: Janssen - given: Kjersti family: Engan editor: - given: Tetiana family: Lutchyn - given: Adín family: Ramírez Rivera - given: Benjamin family: Ricaud page: 20-26 id: bo-sande24a issued: date-parts: - 2024 - 1 - 23 firstpage: 20 lastpage: 26 published: 2024-01-23 00:00:00 +0000 - title: 'Towards AI for approximating hydrodynamic simulations as a 2D segmentation task' abstract: 'Traditional predictive simulations and remote sensing techniques for forecasting floods are based on fixed and spatially restricted physics-based models. These models are computationally expensive and can take many hours to run, resulting in predictions made based on outdated data. They are also spatially fixed, and unable to scale to unknown areas. By modelling the task as an image segmentation problem, an alternative approach using artificial intelligence to approximate the parameters of a physics-based model in 2D is demonstrated, enabling rapid predictions to be made in real-time.' volume: 233 URL: https://proceedings.mlr.press/v233/bryan-smith24a.html PDF: https://proceedings.mlr.press/v233/bryan-smith24a/bryan-smith24a.pdf edit: https://github.com/mlresearch//v233/edit/gh-pages/_posts/2024-01-23-bryan-smith24a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the 5th Northern Lights Deep Learning Conference ({NLDL})' publisher: 'PMLR' author: - given: Lydia family: Bryan-Smith - given: Nina family: Dethlefs editor: - given: Tetiana family: Lutchyn - given: Adín family: Ramírez Rivera - given: Benjamin family: Ricaud page: 27-35 id: bryan-smith24a issued: date-parts: - 2024 - 1 - 23 firstpage: 27 lastpage: 35 published: 2024-01-23 00:00:00 +0000 - title: 'TraCE: Trajectory Counterfactual Explanation Scores' abstract: 'Counterfactual explanations, and their associated algorithmic recourse, are typically leveraged to understand and explain predictions of individual instances coming from a black-box classifier. In this paper, we propose to extend the use of counterfactuals to evaluate progress in sequential decision making tasks. To this end, we introduce a model-agnostic modular framework, TraCE (Trajectory Counterfactual Explanation) scores, to distill and condense progress in highly complex scenarios into a single value. We demonstrate TraCE’s utility by showcasing its main properties in two case studies spanning healthcare and climate change.' volume: 233 URL: https://proceedings.mlr.press/v233/clark24a.html PDF: https://proceedings.mlr.press/v233/clark24a/clark24a.pdf edit: https://github.com/mlresearch//v233/edit/gh-pages/_posts/2024-01-23-clark24a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the 5th Northern Lights Deep Learning Conference ({NLDL})' publisher: 'PMLR' author: - given: Jeffrey Nicholas family: Clark - given: Edward Alexander family: Small - given: Nawid family: Keshtmand - given: Michelle Wing Lam family: Wan - given: Elena Fillola family: Mayoral - given: Enrico family: Werner - given: Christopher family: Bourdeaux - given: Raul family: Santos-Rodriguez editor: - given: Tetiana family: Lutchyn - given: Adín family: Ramírez Rivera - given: Benjamin family: Ricaud page: 36-45 id: clark24a issued: date-parts: - 2024 - 1 - 23 firstpage: 36 lastpage: 45 published: 2024-01-23 00:00:00 +0000 - title: 'Scheduling conditional task graphs with deep reinforcement learning' abstract: 'Industrial applications often depend on costly computation infrastructures. Well optimised schedulers provide cost efficient utilization of these computational resources, but they can take significant effort to implement. It can also be beneficial to split the application into a hierarchy of tasks represented as a conditional task graph. In such case, the tasks in the hierarchy are conditionally executed, depending on the output of the earlier tasks. While such conditional task graphs can save computational resources, they also add complexity to scheduling. Recently, there has been research on Deep Reinforcement Learning (DRL) based schedulers, but they mostly do not address conditional task graphs. We design a DRL based scheduler for conditional task graphs in a heterogeneous execution environment. We measure how the probabilities of a conditional task graph affects the scheduler and how these adverse effects can be mitigated. We show that our solution learns to beat traditional baseline schedulers in a fraction of an hour.' volume: 233 URL: https://proceedings.mlr.press/v233/debner24a.html PDF: https://proceedings.mlr.press/v233/debner24a/debner24a.pdf edit: https://github.com/mlresearch//v233/edit/gh-pages/_posts/2024-01-23-debner24a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the 5th Northern Lights Deep Learning Conference ({NLDL})' publisher: 'PMLR' author: - given: Anton family: Debner - given: Maximilian family: Krahn - given: Vesa family: Hirvisalo editor: - given: Tetiana family: Lutchyn - given: Adín family: Ramírez Rivera - given: Benjamin family: Ricaud page: 46-52 id: debner24a issued: date-parts: - 2024 - 1 - 23 firstpage: 46 lastpage: 52 published: 2024-01-23 00:00:00 +0000 - title: 'Understanding Neural ODE prediction decision using SHAP' abstract: 'Neural ordinary differential equations (NODEs) have emerged as a powerful approach for modelling complex dynamic systems using continuous-time transformations. Although NODEs offer superior modelling capabilities, little research has been conducted on understanding the factors that contribute to their predictions on image datasets. In this paper, we propose the leveraging of SHapley Additive exPlanations (SHAP), which is an influential explainable artificial intelligence method, to gain insights into the NODEs prediction process. We enable the interpretable analysis of important pixels that contribute to the prediction decisions of NODEs by adapting SHAP to the continuous-time nature thereof. Experiments on synthetic datasets demonstrate the efficacy of our proposed approach in revealing the dynamics and important features that drive NODEs predictions. Our empirical findings provide insights into how NODEs determine important features and the distributions of the Shapley values of each class. The proposed integration of SHAP with NODEs contributes to the broader goal of enhancing transparency and trustworthiness in the application of continuous-time models to complex real-world systems.' volume: 233 URL: https://proceedings.mlr.press/v233/dinh24a.html PDF: https://proceedings.mlr.press/v233/dinh24a/dinh24a.pdf edit: https://github.com/mlresearch//v233/edit/gh-pages/_posts/2024-01-23-dinh24a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the 5th Northern Lights Deep Learning Conference ({NLDL})' publisher: 'PMLR' author: - given: Phuong family: Dinh - given: Deddy family: Jobson - given: Takashi family: Sano - given: Hirotada family: Honda - given: Shugo family: Nakamura editor: - given: Tetiana family: Lutchyn - given: Adín family: Ramírez Rivera - given: Benjamin family: Ricaud page: 53-58 id: dinh24a issued: date-parts: - 2024 - 1 - 23 firstpage: 53 lastpage: 58 published: 2024-01-23 00:00:00 +0000 - title: 'Comparative Analysis of Binary and Multiclass Activity Recognition in High-Quality Newborn Resuscitation Videos' abstract: 'Globally, 3-10% of newborns do not breathe spontaneously at birth and need resuscitation. Prompt initiation of resuscitative interventions such as tactile stimulation and positive pressure ventilation can reduce neonatal mortality and morbidity associated with birth asphyxia. Automated video analysis of resuscitation episodes may be beneficial for evaluation and debriefing purposes. In this work, a dataset of 220 newborn resuscitation videos collected at the Stavanger University Hospital (Norway) is used to develop NBT-I3D, a deep neural network pipeline to automatically recognize resuscitation activities. To assess the task, both binary and multiclass networks have undergone training, allowing for a comparison of the two approaches. Results obtained for binary classification show a mean precision and recall of 84.76% and 80.92%, respectively. For multiclass, a mean precision and recall of 72.26% and 74.80% are reported.' volume: 233 URL: https://proceedings.mlr.press/v233/garcia-torres24a.html PDF: https://proceedings.mlr.press/v233/garcia-torres24a/garcia-torres24a.pdf edit: https://github.com/mlresearch//v233/edit/gh-pages/_posts/2024-01-23-garcia-torres24a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the 5th Northern Lights Deep Learning Conference ({NLDL})' publisher: 'PMLR' author: - given: Jorge family: García-Torres - given: Øyvind family: Meinich-Bache - given: Siren Irene family: Rettedal - given: Amalie family: Kibsgaard - given: Sara family: Brunner - given: Kjersti family: Engan editor: - given: Tetiana family: Lutchyn - given: Adín family: Ramírez Rivera - given: Benjamin family: Ricaud page: 59-66 id: garcia-torres24a issued: date-parts: - 2024 - 1 - 23 firstpage: 59 lastpage: 66 published: 2024-01-23 00:00:00 +0000 - title: 'Beyond Accuracy: Fairness, Scalability, and Uncertainty Considerations in Facial Emotion Recognition' abstract: 'Facial emotion recognition (FER) from images or videos is an emerging subfield of emotion recognition that in recent years has achieved increased traction resulting in a wide range of models, datasets, and applications. Benchmarking computer vision methods often provide accuracy rates above 90% in controlled settings. However, little focus has been given to aspects of fairness, uncertainty, and scalability within facial emotion recognition systems. The increasing applicability of FER models within assisted psychiatry and similar domains underlines the importance of fair and computational resource compliant decision-making. The primary objective of this paper is to propose methods for assessment of existing open source FER models to establish a thorough understanding of their current fairness, scalability, and robustness.' volume: 233 URL: https://proceedings.mlr.press/v233/fromberg24a.html PDF: https://proceedings.mlr.press/v233/fromberg24a/fromberg24a.pdf edit: https://github.com/mlresearch//v233/edit/gh-pages/_posts/2024-01-23-fromberg24a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the 5th Northern Lights Deep Learning Conference ({NLDL})' publisher: 'PMLR' author: - given: Laurits family: Fromberg - given: Troels family: Nielsen - given: Flavia Dalia family: Frumosu - given: Line H family: Clemmensen editor: - given: Tetiana family: Lutchyn - given: Adín family: Ramírez Rivera - given: Benjamin family: Ricaud page: 67-74 id: fromberg24a issued: date-parts: - 2024 - 1 - 23 firstpage: 67 lastpage: 74 published: 2024-01-23 00:00:00 +0000 - title: 'Is diverse and inclusive AI trapped in the gap between reality and algorithmizability?' abstract: 'We investigate the preconditions of an operationalization of ethics on the example algorithmization, i.e. the mathematical implementation, of the concepts of fairness and diversity in AI. From a non-technical point of view in ethics, this implementation entails two major drawbacks, (1) as it narrows down big concepts to a single model that is deemed manageable, and (2) as it hides unsolved problems of humanity in a system that could be mistaken as the ‘solution’ to these problems. We encourage extra caution when dealing with such issues and vote for human oversight.' volume: 233 URL: https://proceedings.mlr.press/v233/geldhauser24a.html PDF: https://proceedings.mlr.press/v233/geldhauser24a/geldhauser24a.pdf edit: https://github.com/mlresearch//v233/edit/gh-pages/_posts/2024-01-23-geldhauser24a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the 5th Northern Lights Deep Learning Conference ({NLDL})' publisher: 'PMLR' author: - given: Carina family: Geldhauser - given: Hermann family: Diebel-Fischer editor: - given: Tetiana family: Lutchyn - given: Adín family: Ramírez Rivera - given: Benjamin family: Ricaud page: 75-80 id: geldhauser24a issued: date-parts: - 2024 - 1 - 23 firstpage: 75 lastpage: 80 published: 2024-01-23 00:00:00 +0000 - title: 'Effects of Foreground Augmentations in Synthetic Training Data on the Use of UAVs for Weed Detection' abstract: 'This study addresses the issue of black-grass, a herbicide-resistant weed that threatens wheat yields in Western Europe, through the use of high- resolution Unmanned Aerial Vehicles (UAVs) and synthetic data augmentation in precision agriculture. We mitigate challenges such as the need for large labeled datasets and environmental variability by employing synthetic data augmentations in training a Mask R-CNN model. Using a minimal dataset of 43 black-grass and 12 wheat field images, we achieved a 37% increase in Area Under the Curve (AUC) over the non-augmented baseline, with scaling as the most effective augmentation. The best model attained a recall of 53% at a precision of 64%, offering a promising approach for future precision agriculture applications.' volume: 233 URL: https://proceedings.mlr.press/v233/hallosta24a.html PDF: https://proceedings.mlr.press/v233/hallosta24a/hallosta24a.pdf edit: https://github.com/mlresearch//v233/edit/gh-pages/_posts/2024-01-23-hallosta24a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the 5th Northern Lights Deep Learning Conference ({NLDL})' publisher: 'PMLR' author: - given: Simon family: Hallösta - given: Mats Ingemar family: Pettersson - given: Mattias family: Dahl editor: - given: Tetiana family: Lutchyn - given: Adín family: Ramírez Rivera - given: Benjamin family: Ricaud page: 81-88 id: hallosta24a issued: date-parts: - 2024 - 1 - 23 firstpage: 81 lastpage: 88 published: 2024-01-23 00:00:00 +0000 - title: 'Guiding drones by information gain' abstract: 'The accurate estimation of locations and emission rates of gas sources is crucial across various domains, including environmental monitoring and greenhouse gas emission analysis. This study investigates two drone sampling strategies for inferring source term parameters of gas plumes from atmospheric measurements. Both strategies are guided by the goal of maximizing information gain attained from observations at sequential locations. Our research compares the myopic approach of infotaxis to a far-sighted navigation strategy trained through deep reinforcement learning. We demonstrate the superior performance of deep reinforcement learning over infotaxis in environments with non-isotropic gas plumes.' volume: 233 URL: https://proceedings.mlr.press/v233/hove24a.html PDF: https://proceedings.mlr.press/v233/hove24a/hove24a.pdf edit: https://github.com/mlresearch//v233/edit/gh-pages/_posts/2024-01-23-hove24a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the 5th Northern Lights Deep Learning Conference ({NLDL})' publisher: 'PMLR' author: - given: Alouette van family: Hove - given: Kristoffer family: Aalstad - given: Norbert family: Pirk editor: - given: Tetiana family: Lutchyn - given: Adín family: Ramírez Rivera - given: Benjamin family: Ricaud page: 89-96 id: hove24a issued: date-parts: - 2024 - 1 - 23 firstpage: 89 lastpage: 96 published: 2024-01-23 00:00:00 +0000 - title: 'Beyond output-mask comparison: A self-supervised inspired object scoring system for building change detection' abstract: 'Updating urban-area maps is crucial for urban planning and development. Traditional methods of updating urban-area maps based on aerial photography are labor-intensive and struggle to keep pace with rapid urban development. Automated algorithms for detecting new and removed buildings based on bi-temporal images typically either rely on comparing mono-temporal building detection outputs or requiring examples of new and removed buildings for training. This study presents a novel method using self-supervised learning principles to train a distinct object-change scoring network. It repurposes segments of the (potentially imperfect) delineations used in single-temporal detector training, harnesses bi-temporal data attributes, and leverages the assumption that most buildings remain unchanged over time. This eliminates the need for explicit examples of new or removed buildings, while still overcome usual constraints of post-detection output-mask comparison methods. We provide precision-recall curves and examples demonstrating the improved performance of the suggested approach. Furthermore, we discuss several immediate algorithmic variations that hold the potential for even further enhancements in performance.' volume: 233 URL: https://proceedings.mlr.press/v233/jensen24a.html PDF: https://proceedings.mlr.press/v233/jensen24a/jensen24a.pdf edit: https://github.com/mlresearch//v233/edit/gh-pages/_posts/2024-01-23-jensen24a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the 5th Northern Lights Deep Learning Conference ({NLDL})' publisher: 'PMLR' author: - given: Are C family: Jensen editor: - given: Tetiana family: Lutchyn - given: Adín family: Ramírez Rivera - given: Benjamin family: Ricaud page: 97-103 id: jensen24a issued: date-parts: - 2024 - 1 - 23 firstpage: 97 lastpage: 103 published: 2024-01-23 00:00:00 +0000 - title: 'Neural machine translation for automated feedback on children’s early-stage writing' abstract: 'In this work, we address the problem of assessing and constructing feedback for early-stage writing automatically using machine learning. Early-stage writing is typically vastly different from conventional writing due to phonetic spelling and lack of proper grammar, punctuation, spacing etc. Consequently, early-stage writing is highly non-trivial to analyze using common linguistic metrics. We propose to use sequence-to-sequence models for translating early-stage writing by students into conventional writing, which allows the translated text to be analyzed using linguistic metrics. Furthermore, we propose a novel robust likelihood to mitigate the effect of label noise in the dataset. We investigate the proposed methods using a set of numerical experiments and demonstrate that the conventional text can be predicted with high accuracy.' volume: 233 URL: https://proceedings.mlr.press/v233/jensen24b.html PDF: https://proceedings.mlr.press/v233/jensen24b/jensen24b.pdf edit: https://github.com/mlresearch//v233/edit/gh-pages/_posts/2024-01-23-jensen24b.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the 5th Northern Lights Deep Learning Conference ({NLDL})' publisher: 'PMLR' author: - given: Jonas Vestergaard family: Jensen - given: Mikkel family: Jordahn - given: Michael Riis family: Andersen editor: - given: Tetiana family: Lutchyn - given: Adín family: Ramírez Rivera - given: Benjamin family: Ricaud page: 104-112 id: jensen24b issued: date-parts: - 2024 - 1 - 23 firstpage: 104 lastpage: 112 published: 2024-01-23 00:00:00 +0000 - title: 'SafetyCage: A misclassification detector for feed-forward neural networks' abstract: 'Deep learning classifiers have reached state-of-the-art performance in many fields, particularly so image classification. Wrong class assignment by the classifiers can often be inconsequential when distinguishing pictures of cats and dogs, but in more critical operations like autonomous driving vehicles or process control in industry, wrong classifications can lead to disastrous events. While reducing the error rate of the classifier is of primary importance, it is impossible to completely remove it. Having a system that is able to flag wrong or suspicious classifications is therefore a necessary component for safety and robustness in operations. In this work, we present a general statistical inference framework for detection of misclassifications. We test our approach on two well-known benchmark datasets: MNIST and CIFAR-10. We show that, given the underlying classifier is well trained, SafetyCage is effective at flagging wrong classifications. We also include a detailed discussion of the drawbacks, and what can be done to improve the approach.' volume: 233 URL: https://proceedings.mlr.press/v233/johnsen24a.html PDF: https://proceedings.mlr.press/v233/johnsen24a/johnsen24a.pdf edit: https://github.com/mlresearch//v233/edit/gh-pages/_posts/2024-01-23-johnsen24a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the 5th Northern Lights Deep Learning Conference ({NLDL})' publisher: 'PMLR' author: - given: Pål Vegard family: Johnsen - given: Filippo family: Remonato editor: - given: Tetiana family: Lutchyn - given: Adín family: Ramírez Rivera - given: Benjamin family: Ricaud page: 113-119 id: johnsen24a issued: date-parts: - 2024 - 1 - 23 firstpage: 113 lastpage: 119 published: 2024-01-23 00:00:00 +0000 - title: 'Typicality-based point OOD detection with contrastive learning' abstract: 'Typicality-based inference methods for OOD detection find a typical value (often the mean value) of a model statistic from the training data and then flag test points as anomalous if the model statistic of the test data point deviates significantly from the typical value. These methods are effective for detecting a group of OOD data points when OOD data points are labeled into groups, but ineffective for the detection of individual OOD data points. In this paper, we extend typicality-based inference to be effective for point OOD detection by utilizing latent features learned from contrastive learning and then obtaining the nearest neighbors of a test data point to provide additional context used for point OOD detection. The typicality-based inference approach is shown to improve point OOD detection relative to several benchmarks.' volume: 233 URL: https://proceedings.mlr.press/v233/keshtmand24a.html PDF: https://proceedings.mlr.press/v233/keshtmand24a/keshtmand24a.pdf edit: https://github.com/mlresearch//v233/edit/gh-pages/_posts/2024-01-23-keshtmand24a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the 5th Northern Lights Deep Learning Conference ({NLDL})' publisher: 'PMLR' author: - given: Nawid family: Keshtmand - given: Raul family: Santos-Rodriguez - given: Jonathan family: Lawry editor: - given: Tetiana family: Lutchyn - given: Adín family: Ramírez Rivera - given: Benjamin family: Ricaud page: 120-129 id: keshtmand24a issued: date-parts: - 2024 - 1 - 23 firstpage: 120 lastpage: 129 published: 2024-01-23 00:00:00 +0000 - title: 'Neural Langevin Dynamics: Towards Interpretable Neural Stochastic Differential Equations' abstract: 'Neural Stochastic Differential Equations (NSDE) have been trained as both Variational Autoencoders, and as GANs. However, the resulting Stochastic Differential Equations can be hard to interpret or analyse due to the generic nature of the drift and diffusion fields. By restricting our NSDE to be of the form of Langevin dynamics and training it as a VAE, we obtain NSDEs that lend themselves to more elaborate analysis and to a wider range of visualisation techniques than a generic NSDE. More specifically, we obtain an energy landscape, the minima of which are in one-to-one correspondence with latent states underlying the used data. This not only allows us to detect states underlying the data dynamics in an unsupervised manner but also to infer the distribution of time spent in each state according to the learned SDE. In general, restricting an NSDE to Langevin dynamics enables the use of a large set of tools from computational molecular dynamics for the analysis of the obtained results.' volume: 233 URL: https://proceedings.mlr.press/v233/koop24a.html PDF: https://proceedings.mlr.press/v233/koop24a/koop24a.pdf edit: https://github.com/mlresearch//v233/edit/gh-pages/_posts/2024-01-23-koop24a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the 5th Northern Lights Deep Learning Conference ({NLDL})' publisher: 'PMLR' author: - given: Simon Martinus family: Koop - given: Mark A family: Peletier - given: Jacobus Willem family: Portegies - given: Vlado family: Menkovski editor: - given: Tetiana family: Lutchyn - given: Adín family: Ramírez Rivera - given: Benjamin family: Ricaud page: 130-137 id: koop24a issued: date-parts: - 2024 - 1 - 23 firstpage: 130 lastpage: 137 published: 2024-01-23 00:00:00 +0000 - title: 'Heterogeneous Learning for Brain Lesion Segmentation, Detection, and Classification' abstract: 'Brain lesions detected in magnetic resonance images often vary in type and rarity across different cohorts, posing a challenge for deep learning techniques that are typically specialized in recognizing single lesion types from homogenous data. This limitation restricts their practicality in diverse clinical settings. In this study, we explore different deep-learning approaches to develop robust models handling both subject and imaging variability, while recognizing multiple lesion types. Our research focuses on segmentation and detection tasks across four distinct datasets, encompassing six cohorts of subjects with white matter hyperintensities, multiple sclerosis lesions, or stroke abnormalities. Our findings reveal that a cascade approach, comprising a fully convolutional network and a fully connected classifier, offers optimal accuracy for robust multiclass lesion segmentation and detection. Notably, our proposed model remains competitive with models trained solely on one dataset and applied to the same dataset while showing robustness against domain shifts. Additionally, in related tasks, our model consistently produces results comparable with the state-of-the-art methods. This study contributes to advancing clinically applicable deep learning techniques for brain lesion recognition, offering a promising solution for handling lesion diversity in uncontrolled clinical environments.' volume: 233 URL: https://proceedings.mlr.press/v233/llambias24a.html PDF: https://proceedings.mlr.press/v233/llambias24a/llambias24a.pdf edit: https://github.com/mlresearch//v233/edit/gh-pages/_posts/2024-01-23-llambias24a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the 5th Northern Lights Deep Learning Conference ({NLDL})' publisher: 'PMLR' author: - given: Sebastian Nørgaard family: Llambias - given: Mads family: Nielsen - given: Mostafa Mehdipour family: Ghazi editor: - given: Tetiana family: Lutchyn - given: Adín family: Ramírez Rivera - given: Benjamin family: Ricaud page: 138-144 id: llambias24a issued: date-parts: - 2024 - 1 - 23 firstpage: 138 lastpage: 144 published: 2024-01-23 00:00:00 +0000 - title: 'Instruction-guided deidentification with synthetic test cases for Norwegian clinical text' abstract: 'Deidentification methods, which remove directly identifying information, can be useful tools to mitigate the privacy risks associated with sharing healthcare data. However, benchmarks to evaluate deidentification methods are themselves often derived from real clinical data, making them sensitive themselves and therefore harder to share and apply. Given the rapid advances in generative language modelling, we would like to leverage large language models to construct freely available deidentification benchmarks, and to assist in the deidentification process. We apply the GPT-4 language model to, for the first time, construct a synthetic and publicly available dataset of synthetic Norwegian discharge summaries with annotated identifying details, consisting of 1200 summaries averaging 100 words each. In our sample of documents, we find that the generated annotations highly agree with human annotations, with an $F_1$ score of $0.983$. We then examine whether large language models can be applied directly to perform deidentification themselves, proposing methods where an instruction-tuned language model is prompted to either annotate or redact identifying details. Comparing the methods on our synthetic dataset and the NorSynthClinical-PHI dataset, we find that GPT-4 underperforms the baseline method proposed by Bråthen et al. (2021), suggesting that named entity recognition problems are still challenging for instruction-tuned language models.' volume: 233 URL: https://proceedings.mlr.press/v233/lund24a.html PDF: https://proceedings.mlr.press/v233/lund24a/lund24a.pdf edit: https://github.com/mlresearch//v233/edit/gh-pages/_posts/2024-01-23-lund24a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the 5th Northern Lights Deep Learning Conference ({NLDL})' publisher: 'PMLR' author: - given: Jørgen Aarmo family: Lund - given: Karl Øyvind family: Mikalsen - given: Joel family: Burman - given: Ashenafi Zebene family: Woldaregay - given: Robert family: Jenssen editor: - given: Tetiana family: Lutchyn - given: Adín family: Ramírez Rivera - given: Benjamin family: Ricaud page: 145-152 id: lund24a issued: date-parts: - 2024 - 1 - 23 firstpage: 145 lastpage: 152 published: 2024-01-23 00:00:00 +0000 - title: 'Loop closure with a low power millimeter wave radar sensor using an autoencoder' abstract: 'In this paper, we will consider place recognition, more commonly know as loop closure, with a low resolution single-chip millimeter wave (mmWave) radar in indoor environments. It is an essential part in simultaneous localization and mapping (SLAM) systems to avoid drift. By using a novel method to create descriptors or latent codes with an autoencoder in combination with exploiting the temporal similarity between our latent codes, we are able to successfully extract loop closures with a radar-only system without requiring ground truth. Our proposed method is validated in an industrial IoT lab on an Unmanned Aerial Vehicle (UAV) and on a cargo bike in a parking building.' volume: 233 URL: https://proceedings.mlr.press/v233/meiresone24a.html PDF: https://proceedings.mlr.press/v233/meiresone24a/meiresone24a.pdf edit: https://github.com/mlresearch//v233/edit/gh-pages/_posts/2024-01-23-meiresone24a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the 5th Northern Lights Deep Learning Conference ({NLDL})' publisher: 'PMLR' author: - given: Pieter family: Meiresone - given: David Van family: Hamme - given: Wilfried family: Philips editor: - given: Tetiana family: Lutchyn - given: Adín family: Ramírez Rivera - given: Benjamin family: Ricaud page: 153-157 id: meiresone24a issued: date-parts: - 2024 - 1 - 23 firstpage: 153 lastpage: 157 published: 2024-01-23 00:00:00 +0000 - title: 'Local Gamma Augmentation for Ischemic Stroke Lesion Segmentation on MRI' abstract: 'The identification and localisation of pathological tissues in medical images continues to command much attention among deep learning practitioners. When trained on abundant datasets, deep neural networks can match or exceed human performance. However, the scarcity of annotated data complicates the training of these models. Data augmentation techniques can compensate for a lack of training samples. However, many commonly used augmentation methods can fail to provide meaningful samples during model fitting. We present local gamma augmentation, a technique for introducing new instances of intensities in pathological tissues. We leverage local gamma augmentation to compensate for a bias in intensities corresponding to ischemic stroke lesions in human brain MRIs. On three datasets, we show how local gamma augmentation can improve the image-level sensitivity of a deep neural network tasked with ischemic lesion segmentation on magnetic resonance images.' volume: 233 URL: https://proceedings.mlr.press/v233/middleton24a.html PDF: https://proceedings.mlr.press/v233/middleton24a/middleton24a.pdf edit: https://github.com/mlresearch//v233/edit/gh-pages/_posts/2024-01-23-middleton24a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the 5th Northern Lights Deep Learning Conference ({NLDL})' publisher: 'PMLR' author: - given: Jon family: Middleton - given: Marko family: Bauer - given: Kaining family: Sheng - given: Jacob family: Johansen - given: Mathias family: Perslev - given: Silvia family: Ingala - given: Mads family: Nielsen - given: Akshay family: Pai editor: - given: Tetiana family: Lutchyn - given: Adín family: Ramírez Rivera - given: Benjamin family: Ricaud page: 158-164 id: middleton24a issued: date-parts: - 2024 - 1 - 23 firstpage: 158 lastpage: 164 published: 2024-01-23 00:00:00 +0000 - title: 'Large Neural Networks at a Fraction' abstract: 'Large-scale deep learning models are known for their large amount of parameters, weighing down on the computational resources. The core of the Lottery Ticket Hypothesis showed us the potential of pruning to reduce such parameters without a significant drop in accuracy. Quaternion neural networks achieve comparable accuracy to equivalent real-valued networks on multi-dimensional prediction tasks. In our work, we implement pruning on real and quaternion-valued implementations of large-scale networks in the task of image recognition. For instance, our implementation of the ResNet-101 architecture on the CIFAR-100 and ImageNet64x64 datasets resulted in pruned quaternion models outperforming their real-valued counterparts by 4% and 7% in accuracy at sparsities of about 6% and 0.4%, respectively. We also got quaternion implementations of ResNet-101 and ResNet-152 on CIFAR-100 with steady Lottery tickets, whereas the Real counterpart failed to train at the same sparsity. Our experiments show that the pruned quaternion implementations perform better at higher sparsity than the corresponding real-valued counterpart, even in some larger neural networks.' volume: 233 URL: https://proceedings.mlr.press/v233/mukhopadhyay24a.html PDF: https://proceedings.mlr.press/v233/mukhopadhyay24a/mukhopadhyay24a.pdf edit: https://github.com/mlresearch//v233/edit/gh-pages/_posts/2024-01-23-mukhopadhyay24a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the 5th Northern Lights Deep Learning Conference ({NLDL})' publisher: 'PMLR' author: - given: Aritra family: Mukhopadhyay - given: Adhilsha family: A - given: Subhankar family: Mishra editor: - given: Tetiana family: Lutchyn - given: Adín family: Ramírez Rivera - given: Benjamin family: Ricaud page: 165-173 id: mukhopadhyay24a issued: date-parts: - 2024 - 1 - 23 firstpage: 165 lastpage: 173 published: 2024-01-23 00:00:00 +0000 - title: 'MDD-UNet: Domain Adaptation for Medical Image Segmentation with Theoretical Guarantees, a Proof of Concept' abstract: 'The current state-of-the art techniques for image segmentation are often based on U-Net architectures, a U-shaped encoder-decoder networks with skip connections. Despite the powerful performance, the architecture often does not perform well when used on data which has different characteristics than the data it was trained on. Many techniques for improving performance in the presence of domain shift have been developed, however typically only have loose connections to the theory of domain adaption. In this work, we propose an unsupervised domain adaptation framework for U-Nets with theoretical guarantees based on the Margin Disparity Discrepancy called the MDD-UNet. We evaluate the proposed technique on the task of hippocampus segmentation, and find that the MDD-UNet is able to learn features which are domain-invariant with no knowledge about the labels in the target domain. The MDD-UNet improves performance over the standard U-Net on 11 out of 12 combinations of datasets. This work serves as a proof of concept by demonstrating an improvement on the U-Net in it’s standard form without modern enhancements, which opens up a new avenue of studying domain adaptation for models with very large hypothesis spaces from both methodological and practical perspectives.' volume: 233 URL: https://proceedings.mlr.press/v233/munk24a.html PDF: https://proceedings.mlr.press/v233/munk24a/munk24a.pdf edit: https://github.com/mlresearch//v233/edit/gh-pages/_posts/2024-01-23-munk24a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the 5th Northern Lights Deep Learning Conference ({NLDL})' publisher: 'PMLR' author: - given: Asbjørn family: Munk - given: Mads family: Nielsen editor: - given: Tetiana family: Lutchyn - given: Adín family: Ramírez Rivera - given: Benjamin family: Ricaud page: 174-180 id: munk24a issued: date-parts: - 2024 - 1 - 23 firstpage: 174 lastpage: 180 published: 2024-01-23 00:00:00 +0000 - title: 'Hubness Reduction Improves Sentence-BERT Semantic Spaces' abstract: 'Semantic representations of text, i.e. representations of natural language which capture meaning by geometry, are essential for areas such as information retrieval and document grouping. High-dimensional trained dense vectors have received much attention in recent years as such representations. We investigate the structure of semantic spaces that arise from embeddings made with Sentence-BERT and find that the representations suffer from a well-known problem in high dimensions called hubness. Hubness results in asymmetric neighborhood relations, such that some texts (the hubs) are neighbours of many other texts while most texts (so-called anti-hubs), are neighbours of few or no other texts. We quantify the semantic quality of the embeddings using hubness scores and error rate of a neighbourhood based classifier. We find that when hubness is high, we can reduce error rate and hubness using hubness reduction methods. We identify a combination of two methods as resulting in the best reduction. For example, on one of the tested pretrained models, this combined method can reduce hubness by about 75% and error rate by about 9%. Thus, we argue that mitigating hubness in the embedding space provides better semantic representations of text.' volume: 233 URL: https://proceedings.mlr.press/v233/nielsen24a.html PDF: https://proceedings.mlr.press/v233/nielsen24a/nielsen24a.pdf edit: https://github.com/mlresearch//v233/edit/gh-pages/_posts/2024-01-23-nielsen24a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the 5th Northern Lights Deep Learning Conference ({NLDL})' publisher: 'PMLR' author: - given: Beatrix Miranda Ginn family: Nielsen - given: Lars Kai family: Hansen editor: - given: Tetiana family: Lutchyn - given: Adín family: Ramírez Rivera - given: Benjamin family: Ricaud page: 181-204 id: nielsen24a issued: date-parts: - 2024 - 1 - 23 firstpage: 181 lastpage: 204 published: 2024-01-23 00:00:00 +0000 - title: 'Clifford Group Equivariant Neural Network Layers for Protein Structure Prediction' abstract: 'We employ Clifford Group Equivariant Neural Network (CGENN) layers to predict protein coordinates in a Protein Structure Prediction (PSP) pipeline. PSP is the estimation of the 3D structure of a protein, generally through deep learning architectures. Information about the geometry of the protein chain has been proven to be crucial for accurate predictions of 3D structures. However, this information is usually flattened as machine learning features that are not representative of the geometric nature of the problem. Leveraging recent advances in geometric deep learning, we redesign a PSP architecture with the addition of CGENN layers. CGENNs can achieve better generalization and robustness when dealing with data that show rotational or translational invariance such as protein coordinates, which are independent of the chosen reference frame. CGENNs inputs, outputs, weights and biases are objects in the Geometric Algebra of 3D Euclidean space, i.e. $\mathcal{G}_{3,0,0}$, and hence are interpretable from a geometrical perspective. We test 6 approaches to PSP and show that CGENN layers increase the prediction accuracy by up to 2.1%, with fewer trainable parameters compared to linear layers and give a clear geometric interpretation of their outputs.' volume: 233 URL: https://proceedings.mlr.press/v233/pepe24a.html PDF: https://proceedings.mlr.press/v233/pepe24a/pepe24a.pdf edit: https://github.com/mlresearch//v233/edit/gh-pages/_posts/2024-01-23-pepe24a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the 5th Northern Lights Deep Learning Conference ({NLDL})' publisher: 'PMLR' author: - given: Alberto family: Pepe - given: Sven family: Buchholz - given: Joan family: Lasenby editor: - given: Tetiana family: Lutchyn - given: Adín family: Ramírez Rivera - given: Benjamin family: Ricaud page: 205-211 id: pepe24a issued: date-parts: - 2024 - 1 - 23 firstpage: 205 lastpage: 211 published: 2024-01-23 00:00:00 +0000 - title: 'Deep Perceptual Similarity is Adaptable to Ambiguous Contexts' abstract: 'This work examines the adaptability of Deep Perceptual Similarity (DPS) metrics to context beyond those that align with average human perception and contexts in which the standard metrics have been shown to perform well. Prior works have shown that DPS metrics are good at estimating human perception of similarity, so-called perceptual similarity. However, it remains unknown whether such metrics can be adapted to other contexts. In this work, DPS metrics are evaluated for their adaptability to different contradictory similarity contexts. Such contexts are created by randomly ranking six image distortions. Metrics are adapted to consider distortions more or less disruptive to similarity depending on their place in the random rankings. This is done by training pretrained CNNs to measure similarity according to given contexts. The adapted metrics are also evaluated on a perceptual similarity dataset to evaluate whether adapting to a ranking affects their prior performance. The findings show that DPS metrics can be adapted with high performance. While the adapted metrics have difficulties with the same contexts as baselines, performance is improved in 99% of cases. Finally, it is shown that the adaption is not significantly detrimental to prior performance on perceptual similarity. The implementation of this work is available online.' volume: 233 URL: https://proceedings.mlr.press/v233/pihlgren24a.html PDF: https://proceedings.mlr.press/v233/pihlgren24a/pihlgren24a.pdf edit: https://github.com/mlresearch//v233/edit/gh-pages/_posts/2024-01-23-pihlgren24a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the 5th Northern Lights Deep Learning Conference ({NLDL})' publisher: 'PMLR' author: - given: Gustav Grund family: Pihlgren - given: Fredrik family: Sandin - given: Marcus family: Liwicki editor: - given: Tetiana family: Lutchyn - given: Adín family: Ramírez Rivera - given: Benjamin family: Ricaud page: 212-219 id: pihlgren24a issued: date-parts: - 2024 - 1 - 23 firstpage: 212 lastpage: 219 published: 2024-01-23 00:00:00 +0000 - title: 'A Hybrid Spiking-Convolutional Neural Network Approach for Advancing Machine Learning Models' abstract: 'In this article, we propose a novel standalone hybrid Spiking-Convolutional Neural Network (SC-NN) model and test on using image inpainting tasks. Our approach uses the unique capabilities of SNNs, such as event-based computation and temporal processing, along with the strong representation learning abilities of CNNs, to generate high-quality inpainted images. The model is trained on a custom dataset specifically designed for image inpainting, where missing regions are created using masks. The hybrid model consists of SNNConv2d layers and traditional CNN layers. The SNNConv2d layers implement the leaky integrate-and-fire (LIF) neuron model, capturing spiking behavior, while the CNN layers capture spatial features. In this study, a mean squared error (MSE) loss function demonstrates the training process, where a training loss value of 0.015, indicates accurate performance on the training set and the model achieved a validation loss value as low as 0.0017 on the testing set. Furthermore, extensive experimental results demonstrate state-of-the-art performance, showcasing the potential of integrating temporal dynamics and feature extraction in a single network for image inpainting.' volume: 233 URL: https://proceedings.mlr.press/v233/sanaullah24a.html PDF: https://proceedings.mlr.press/v233/sanaullah24a/sanaullah24a.pdf edit: https://github.com/mlresearch//v233/edit/gh-pages/_posts/2024-01-23-sanaullah24a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the 5th Northern Lights Deep Learning Conference ({NLDL})' publisher: 'PMLR' author: - given: Sanaullah family: Sanaullah editor: - given: Tetiana family: Lutchyn - given: Adín family: Ramírez Rivera - given: Benjamin family: Ricaud page: 220-227 id: sanaullah24a issued: date-parts: - 2024 - 1 - 23 firstpage: 220 lastpage: 227 published: 2024-01-23 00:00:00 +0000 - title: 'Lidar-based Norwegian tree species detection using deep learning' abstract: 'Background: The mapping of tree species within Norwegian forests is a time-consuming process, involving forest associations relying on manual labeling by experts. The process can involve aerial imagery, personal familiarity, on-scene references, and remote sensing data. The state-of-the-art methods usually use high-resolution aerial imagery with semantic segmentation methods. Methods: We present a deep learning based tree species classification model utilizing only lidar (Light Detection And Ranging) data. The lidar images are segmented into four classes (Norway Spruce, Scots Pine, Birch, background) with a U-Net based network. The model is trained with focal loss over partial weak labels. A major benefit of the approach is that both the lidar imagery and the base map for the labels have free and open access. Results: Our tree species classification model achieves a macro-averaged $\mathrm{F}_1$ score of 0.70 on an independent validation with National Forest Inventory (NFI) in-situ sample plots. That is close to, but below the performance of aerial, or aerial and lidar combined models.' volume: 233 URL: https://proceedings.mlr.press/v233/vermeer24a.html PDF: https://proceedings.mlr.press/v233/vermeer24a/vermeer24a.pdf edit: https://github.com/mlresearch//v233/edit/gh-pages/_posts/2024-01-23-vermeer24a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the 5th Northern Lights Deep Learning Conference ({NLDL})' publisher: 'PMLR' author: - given: Martijn family: Vermeer - given: Jacob Alexander family: Hay - given: David family: Völgyes - given: Zsofia family: Koma - given: Johannes family: Breidenbach - given: Daniele family: Fantin editor: - given: Tetiana family: Lutchyn - given: Adín family: Ramírez Rivera - given: Benjamin family: Ricaud page: 228-234 id: vermeer24a issued: date-parts: - 2024 - 1 - 23 firstpage: 228 lastpage: 234 published: 2024-01-23 00:00:00 +0000 - title: 'An Inductive Bias for Emergent Communication in a Continuous Setting' abstract: 'We study emergent communication in a multi-agent reinforcement learning setting, where the agents solve cooperative tasks and have access to a communication channel. The communication channel may consist of either discrete symbols or continuous variables. We introduce an inductive bias to aid with the emergence of good communication protocols for continuous messages, and we look at the effect this type of inductive bias has for continuous and discrete messages in itself or when used in combination with reinforcement learning. We demonstrate that this type of inductive bias has a beneficial effect on the communication protocols learnt in two toy environments, Negotiation and Sequence Guess.' volume: 233 URL: https://proceedings.mlr.press/v233/villanger24a.html PDF: https://proceedings.mlr.press/v233/villanger24a/villanger24a.pdf edit: https://github.com/mlresearch//v233/edit/gh-pages/_posts/2024-01-23-villanger24a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the 5th Northern Lights Deep Learning Conference ({NLDL})' publisher: 'PMLR' author: - given: John Isak Fjellvang family: Villanger - given: Troels Arnfred family: Bojesen editor: - given: Tetiana family: Lutchyn - given: Adín family: Ramírez Rivera - given: Benjamin family: Ricaud page: 235-243 id: villanger24a issued: date-parts: - 2024 - 1 - 23 firstpage: 235 lastpage: 243 published: 2024-01-23 00:00:00 +0000 - title: 'Efficient Node Selection in Private Personalized Decentralized Learning' abstract: 'Personalized decentralized learning is a promising paradigm for distributed learning, enabling each node to train a local model on its own data and collaborate with other nodes to improve without sharing any data. However, this approach poses significant privacy risks, as nodes may inadvertently disclose sensitive information about their data or preferences through their collaboration choices. In this paper, we propose Private Personalized Decentralized Learning (PPDL), a novel approach that combines secure aggregation and correlated adversarial multi-armed bandit optimization to protect node privacy while facilitating efficient node selection. By leveraging dependencies between different arms, represented by potential collaborators, we demonstrate that PPDL can effectively identify suitable collaborators solely based on aggregated models. Additionally, we show that PPDL surpasses previous non-private methods in model performance on standard benchmarks under label and covariate shift scenarios.' volume: 233 URL: https://proceedings.mlr.press/v233/zec24a.html PDF: https://proceedings.mlr.press/v233/zec24a/zec24a.pdf edit: https://github.com/mlresearch//v233/edit/gh-pages/_posts/2024-01-23-zec24a.md series: 'Proceedings of Machine Learning Research' container-title: 'Proceedings of the 5th Northern Lights Deep Learning Conference ({NLDL})' publisher: 'PMLR' author: - given: Edvin Listo family: Zec - given: Johan family: Östman - given: Olof family: Mogren - given: Daniel family: Gillblad editor: - given: Tetiana family: Lutchyn - given: Adín family: Ramírez Rivera - given: Benjamin family: Ricaud page: 244-250 id: zec24a issued: date-parts: - 2024 - 1 - 23 firstpage: 244 lastpage: 250 published: 2024-01-23 00:00:00 +0000