Proceedings of UniReps: the Third Edition of the Workshop on Unifying Representations in Neural Models Held in San Diego Convention Center, San Diego, USA on 06 December 2025 Published as Volume 322 by the Proceedings of Machine Learning Research on 27 February 2026. Volume Edited by: Marco Fumero Clementine Domine Zorah L"ahner Irene Cannistraci Bo Zhao Alex Williams Series Editors: Neil D. Lawrence https://proceedings.mlr.press/v322/ Sun, 05 Jul 2026 15:06:36 +0000 Sun, 05 Jul 2026 15:06:36 +0000 Jekyll v3.10.0 Task arithmetic has emerged as a simple yet powerful technique for model merging, enabling the combination of multiple finetuned models into a single model. Despite its empirical success, a clear theoretical understanding of why and when it works has been lacking. This paper provides a rigorous theoretical foundation for task arithmetic by establishing a direct connection between task vectors and gradients of the task losses. We show that under standard gradient descent, a task vector generated from one epoch of finetuning is exactly equivalent to the negative gradient of the loss, scaled by the learning rate. For the practical multi-epoch setting, we prove that this equivalence holds approximately, with a second-order error term that we explicitly bound for feed-forward networks. Our empirical analysis across seven vision benchmarks corroborates our theory, demonstrating that the first-epoch gradient dominates the finetuning trajectory in both norm and direction. A key implication is that merging models finetuned for only a single epoch often yields performance comparable to merging fully converged models. These findings reframe task arithmetic as a form of approximate multitask learning, providing a clear rationale for its effectiveness and highlighting the critical role of early training dynamics in model merging. Fri, 27 Feb 2026 00:00:00 +0000 https://proceedings.mlr.press/v322/zhou26a.html https://proceedings.mlr.press/v322/zhou26a.html Multimodal Retrieval-Augmented Generation (MRAG) has emerged as a promising method to generate factual and up-to-date responses of Multimodal Large Language Models (MLLMs) by incorporating non-parametric knowledge from external knowledge bases. However, existing MRAG approaches suffer from static retrieval strategies, inflexible modality selection, and suboptimal utilization of retrieved information, leading to three critical challenges: determining when to retrieve, what modalities to incorporate, and how to utilize retrieved information effectively. To address these challenges, we introduce Windsock, a query-dependent module making decisions on retrieval necessity and modality selection, effectively reducing computational overhead and improving response quality. Additionally, we propose Dynamic Noise-Resistance (DANCE) Instruction Tuning, an adaptive training strategy that enhances MLLMs’ ability to utilize retrieved information while maintaining robustness against noise. Moreover, we adopt a self-assessment approach leveraging knowledge within MLLMs to convert question-answering datasets to MRAG training datasets. Extensive experiments demonstrate that our proposed method significantly improves both efficiency and generation quality by 17.70% while reducing 8.95% retrieval times. Fri, 27 Feb 2026 00:00:00 +0000 https://proceedings.mlr.press/v322/zhao26a.html https://proceedings.mlr.press/v322/zhao26a.html Independently trained vision and language models inhabit disjoint representational spaces, shaped by their respective modalities, learning objectives, and architectures. The Platonic Representation Hypothesis (PRH) suggests these models may nonetheless converge toward a shared statistical model of reality. This raises a fundamental question: can we move beyond post-hoc detection of such alignment and explicitly optimize for it? We argue this challenge is particularly important for tasks such as fine-grained contextual distinctions—where multiple descriptions share global semantics but differ in subtle compositional details. We tackle this setting with the Joint Autoencoder Modulator (JAM), which aligns frozen unimodal models by jointly training modality-specific autoencoders with coordinated reconstruction and cross-modal alignment objectives. We systematically evaluate JAM across three design axes: (i) alignment objectives, introducing our multimodal Spread Loss that outperforms classic contrastive methods; (ii) the layer depth at which alignment is most effective; and (iii) the role of foundation model scale in representational convergence. Our findings show that JAM reliably induces alignment (outperforming innately multimodal models and post-hoc alignment baselines with absolute error reduction of up to 10%, and relative error reduction of up to 80%), offering both fundamental insight into the structure of shared semantics and practical guidance for transforming generalist unimodal foundations into specialist multimodal models. Fri, 27 Feb 2026 00:00:00 +0000 https://proceedings.mlr.press/v322/yoon26a.html https://proceedings.mlr.press/v322/yoon26a.html Wide-field amacrine cells (ACs) play a unique role in retinal processing by integrating visual information across a large spatial area. Their inhibitory influence has been implicated in multiple retinal functions such as differential motion detection and the suppression of retinal activity during eye movements. However, a coherent understanding of their general function is lacking due to difficulties in directly recording from these cells and identifying effective visual stimuli to activate them. In this study, we used convolutional neural networks (CNNs) to investigate wide-field inhibition mediated by wide-field ACs in the marmoset retina. We trained CNNs to mimic the function of the retina by predicting retinal ganglion cell (RGC) population responses to naturalistic movie stimuli and optimising the most exciting inputs (MEIs) to visualise RGCs’ receptive field (RF) structures. We then optimized suppressive surrounds beyond classical RGC RF boundaries, intended to capture the inhibitory effect of wide-field ACs on RGC activity. These optimized surrounds reduced MEI-elicited activity by 10% to 30%, demonstrating that CNNs not only mimic retinal responses but can also reveal hidden computational aspects of wide-field inhibition. However, suppression strength and generalization varied across architectures and datasets, indicating potential model-specific effects, highlighting the importance of cautious interpretation. Overall, our approach illustrates how interpretability methods applied to artificial neural networks can offer new hypotheses regarding biological retinal computation, paving the way for targeted experimental validation. The code is available at https://anonymous.4open.science/r/wac_supsur-6FF7. Fri, 27 Feb 2026 00:00:00 +0000 https://proceedings.mlr.press/v322/vystrcilova26a.html https://proceedings.mlr.press/v322/vystrcilova26a.html Prior works have investigated the integration of large language models (LLMs) with rectified flow for image synthesis, but systematic studies of remain scarce. In this study, we examine how controlling the interaction between stochastic and semantic inputs during encoding, while integrating them during decoding, influences the alignment between noised latents and LLM hidden states. Our investigation shows that architectural refinements, such as dual-stream encoding and single-stream decoding, can accelerate training and improve image quality relative to LLM-adapted rectified flow baselines by enhancing representational similarity between text and visual domains. We evaluate our approach on standard image benchmarks and observe gains in both training speed and output detail preservation, indicating that structural choices in the integration of LLM features matter for cross-modal representational alignment in generative modeling. Beyond empirical improvements, our findings contribute to understanding how foundation models trained on text can develop representations that align with visual domains, revealing insights into the emergence of similar representational structures across distinct modalities. These results highlight a promising direction at the intersection of LLMs, rectified flow, and cross-modal representational analysis and motivate further explorations into unified representation learning. Fri, 27 Feb 2026 00:00:00 +0000 https://proceedings.mlr.press/v322/vysotskyi26a.html https://proceedings.mlr.press/v322/vysotskyi26a.html Recent advances in text-to-image diffusion models have shown remarkable performance in generating realistic images from text descriptions. However, high-quality visual text generation in generated images remains a major challenge. Gibberish text generation is particularly problematic when the model has to generate proper nouns and text that is not commonly present in training data. Unlike existing methods to improve visual text generation which are based on data-intensive and time-consuming fine-tuning approaches, we propose an inference-time representation alignment algorithm, TextIT, that does not need additional data or training. First, we propose an inference-time self-attention manipulation loss that exposes and aligns latent intermediate self-attention (SA) representations governing visual text generation with those of correctly-rendered text. Next, we impose fine-grained control over the generated text by aligning character-wise control points, obtained through self-attention map vectorization, with ground truth character control points. We provide evidence that inference-time representational manipulation enables controllable and interpretable improvements in text-to-image generation, validating our method with character and word-level visual text generation results that retain the overall generative diversity of diffusion models. Fri, 27 Feb 2026 00:00:00 +0000 https://proceedings.mlr.press/v322/tripathy26a.html https://proceedings.mlr.press/v322/tripathy26a.html Model stitching is a technique for assembling new neural networks from the parts of existing networks, without having to re-train or fine-tune the existing weights. It has shown promise for new forms of neural architecture search, decentralized training, and transfer learning. But what are the upper bounds on this technique? Little investigation has gone into determining exactly what types of blocks can (or cannot) be stitched together, and how. In this work, we investigate the feasibility of adapting very low layers to very high layers, and stitching across different architectures, in the context of image classification models. We develop some modifications to the original stitching methods to make it possible to achieve good performance while stitching such disparate layers: (1) We interpolate the spatial dimensions of the input; (2) we propose adapters with more complex, nonlinear transformations; and (3) we propose the use of bottleneck adapters for computational efficiency. With these modifications, we are able to stitch, for example, the lower layers of a ResNet-50 to the upper layers of a Swin-Tiny, achieving ImageNet test accuracy near to the original models. Fri, 27 Feb 2026 00:00:00 +0000 https://proceedings.mlr.press/v322/traft26a.html https://proceedings.mlr.press/v322/traft26a.html Vision-language models (VLM) bring image and textual representations close together in a joint embedding space, which is useful for tagging and retrieval from content stores. However such associations are not very stable in that a synonymous textual query does not retrieve the same set of images or with a high degree of overlap. This is due to the absence of linkages between semantically related concepts in vision-language models. In contrast, the episodic memory store in the brain has linkages to the semantic conceptual memory subsystem which helps in both the formation and recall of memories. In this paper, we exploit this paradigm to link a VLM to a semantic memory thereby producing a new semantic vision-language model called SemCLIP. Specifically, we develop a semantic memory model for the language of object-naming nouns reflecting their semantic similarity. We then link a vision language model to the semantic memory model through a semantic alignment transform. This leads to a richer and more stable understanding of the concepts by bringing synonymous visual concepts and their associated images closer. Both the semantic memory model and the alignment transform can be learned from word knowledge sources thus avoiding large-scale retraining of VLMs from real-world image-text pairs. The resulting model is shown to outperform existing embedding models for semantic similarity and downstream tasks of retrieval on multiple datasets. Fri, 27 Feb 2026 00:00:00 +0000 https://proceedings.mlr.press/v322/syeda-mahmood26a.html https://proceedings.mlr.press/v322/syeda-mahmood26a.html Recommending matches in a text-rich, dynamic two-sided marketplace presents unique challenges due to evolving content and interaction graphs. We introduce GraphMatch, a new large-scale recommendation framework that fuses pre-trained language models with graph neural networks to overcome these challenges. Unlike prior approaches centered on standalone models, GraphMatch is a comprehensive recipe built on powerful text encoders and GNNs working in tandem. It employs adversarial negative sampling alongside point-in-time subgraph training to learn representations that capture both the fine-grained semantics of evolving text and the time-sensitive structure of the graph. We evaluated extensively on interaction data from Upwork, a leading labor marketplace, at large scale, and discuss our approach towards low-latency inference suitable for real-time use. In our experiments, GraphMatch outperforms language-only and graph-only baselines on matching tasks while being efficient at runtime. These results demonstrate that unifying language and graph representations yields a highly effective solution to text-rich, dynamic two-sided recommendations, bridging the gap between powerful pretrained LMs and large-scale graphs in practice. Fri, 27 Feb 2026 00:00:00 +0000 https://proceedings.mlr.press/v322/sacha26a.html https://proceedings.mlr.press/v322/sacha26a.html Diffusion models have achieved impressive performance in generating high-quality and diverse synthetic data. However, their success typically assumes a class-balanced training distribution. In real-world settings, multi-class data often follow a long-tailed distribution, where standard diffusion models struggle—producing low-diversity and lower-quality samples for underrepresented (tail) classes. While this degradation is well-documented, its underlying cause remains poorly understood. In this work, we investigate the behavior of diffusion models trained on long-tailed datasets and identify a key issue: the latent representations (from the bottleneck layer of the U-Net) for tail class subspaces exhibit significant overlap with those of head classes, leading to feature borrowing and poor generation quality. Importantly, we show that this is not merely due to limited data per class, but that the relative class imbalance significantly contributes to this phenomenon. To address this, we propose **CO**ntrastive **R**egularization for **A**ligning **L**atents (CORAL), a contrastive latent alignment framework that leverages supervised contrastive losses to encourage well-separated latent class representations. Experiments demonstrate that CORAL significantly improves both the diversity and visual quality of samples generated for tail classes relative to state-of-the-art methods. Fri, 27 Feb 2026 00:00:00 +0000 https://proceedings.mlr.press/v322/rodriguez26a.html https://proceedings.mlr.press/v322/rodriguez26a.html We propose a probabilistic interpretation of transformers as unrolled inference steps assuming a probabilistic Laplacian Eigenmaps model from the ProbDR framework. Our derivation shows that at initialisation, transformers perform “linear” dimensionality reduction. We also show that within the transformer block, a graph Laplacian term arises from our arguments, rather than an attention matrix (which we interpret as an adjacency matrix). We demonstrate that simply subtracting the identity from the attention matrix (and thereby taking a graph diffusion step) improves validation performance on a language model and a simple vision transformer. Fri, 27 Feb 2026 00:00:00 +0000 https://proceedings.mlr.press/v322/ravuri26a.html https://proceedings.mlr.press/v322/ravuri26a.html Spatial omics technologies provide rich insights into biological processes by jointly capturing molecular profiles and the spatial organization of cells. The resulting high-dimensional data can be naturally represented as graphs, where Graph Neural Networks (GNNs) offer an effective framework to model interactions in the tissue. Self-supervised pretraining methods such as Bootstrapped Graph Latents (BGRL) and GRACE leverage graph augmentations to build invariances without costly labels. Yet, the design of augmentation strategies remains underexplored, particularly in the context of spatial omics. In this work, we systematically investigate how different graph augmentations affect embedding quality and downstream performance in spatial omics. We evaluate a suite of existing and novel augmentations, including transformations tailored to biological variation, across two representative tasks: unsupervised domain identification in healthy tissue and supervised phenotype prediction in cancer tissue. Our results show that carefully chosen augmentations substantially improve performance, whereas poorly aligned or overly complex augmentations may fail to help or even degrade performance. These findings highlight the central role of augmentation design in enforcing meaningful invariances for graph contrastive pretraining in spatial omics. Fri, 27 Feb 2026 00:00:00 +0000 https://proceedings.mlr.press/v322/rabuzin26a.html https://proceedings.mlr.press/v322/rabuzin26a.html The mouse is one of the most studied animal models in the field of systems neuroscience. Understanding the generalized patterns and decoding the neural representations that are evoked by the diverse range of natural scene stimuli in the mouse visual cortex is one of the key quests in computational vision. In recent years, significant parallels have been drawn between the primate visual cortex and hierarchical deep neural networks. However, their generalized efficacy in understanding mouse vision has been limited. In this study, we investigate the functional alignment between the mouse visual cortex and deep learning models for object classification tasks. We first introduce a generalized representational learning strategy that uncovers a striking resemblance between the functional mapping of the mouse visual cortex and high-performing deep learning models on both top-down (population-level) and bottom-up (single cell-level) scenarios. Next, this representational similarity across the two systems is further enhanced by the addition of Neural Response Normalization (NeuRN) layer, inspired by the activation profile of excitatory and inhibitory neurons in the visual cortex. To test the performance effect of NeuRN on real-world tasks, we integrate it into deep learning models and observe significant improvements in their robustness against data shifts in domain generalization tasks. Our work proposes a novel framework for comparing the functional architecture of the mouse visual cortex with deep learning models. Our findings carry broad implications for the development of advanced AI models that draw inspiration from the mouse visual cortex, suggesting that these models serve as valuable tools for studying the neural representations of the mouse visual cortex and, as a result, enhancing their performance on real-world tasks. Fri, 27 Feb 2026 00:00:00 +0000 https://proceedings.mlr.press/v322/qazi26a.html https://proceedings.mlr.press/v322/qazi26a.html Interpreting graph neural networks (GNNs) is difficult because message passing mixes signals and internal channels rarely align with human concepts. We study superposition, the sharing of directions by multiple features, directly in the latent space of GNNs. Using controlled experiments with unambiguous graph concepts, we extract features as (i) class-conditional centroids at the graph level and (ii) linear-probe directions at the node level, and then analyze their geometry with simple basis-invariant diagnostics. Across GCN/GIN/GAT we find: increasing width produces a phase pattern in overlap; topology imprints overlap onto node-level features that pooling partially remixes into task-aligned graph axes; sharper pooling increases axis alignment and reduces channel sharing; and shallow models can settle into metastable low-rank embeddings. These results connect representational geometry with concrete design choices (width, pooling, and final-layer activations) and suggest practical approaches for more interpretable GNNs. Fri, 27 Feb 2026 00:00:00 +0000 https://proceedings.mlr.press/v322/pertl26a.html https://proceedings.mlr.press/v322/pertl26a.html Previous work has shown correlations between the hidden states of large language models and fMRI brain responses, on language tasks. These correlations have been taken as evidence of the representational similarity of these models and brain states. This study tests whether these previous results are robust to several possible concerns. Specifically this study shows: (i) that the previous results are still found after dimensionality reduction, and thus are not attributable to the curse of dimensionality; (ii) that previous results are confirmed when using new measures of similarity; (iii) that correlations between brain representations and those from models are specific to models trained on human language; and (iv) that the results are dependent on the presence of positional encoding in the models. These results confirm and strengthen the results of previous research and contribute to the debate on the biological plausibility and interpretability of state-of-the-art large language models. Fri, 27 Feb 2026 00:00:00 +0000 https://proceedings.mlr.press/v322/parra26a.html https://proceedings.mlr.press/v322/parra26a.html Pre-trained neural networks are commonly used and reused in the machine learning community. Alice trains a model for a particular task, and a part of her neural network is reused by Bob for a different task, often to great effect. To what can we ascribe Bob’s success? This paper introduces an experimental setup through which factors contributing to Bob’s empirical success could be studied in silico. As a result, we demonstrate that Bob might just be lucky: his task accuracy increases monotonically with the correlation between his task and Alice’s. Even when Bob has provably uncorrelated tasks and input features from Alice’s pre-trained network, he can achieve significantly better than random performance due to Alice’s choice of network and optimizer. When there is little correlation between tasks, only reusing lower pre-trained layers is preferable, and we hypothesize the converse: that the optimal number of retrained layers is indicative of task and feature correlation. Finally, we show in controlled real-world scenarios that Bob can effectively reuse Alice’s pre-trained network if there are semantic correlations between his and Alice’s task. Fri, 27 Feb 2026 00:00:00 +0000 https://proceedings.mlr.press/v322/mohamud26a.html https://proceedings.mlr.press/v322/mohamud26a.html Do brains and language models converge toward the same internal representations of the world? Recent years have seen a rise in studies of neural activations and model alignment. In this work, we review 25 fMRI-based studies published between 2023 and 2025 and explicitly confront their findings with two key hypotheses: (i) the Platonic Representation Hypothesis—that as models scale and improve, they converge to a representation of the real world, and (ii) the Intermediate-Layer Advantage—that intermediate (mid-depth) layers often encode richer, more generalizable features. Our findings provide converging evidence that models and brains may share abstract representational structures, supporting both hypotheses and motivating further research on brain–model alignment. Fri, 27 Feb 2026 00:00:00 +0000 https://proceedings.mlr.press/v322/lopez-cardona26a.html https://proceedings.mlr.press/v322/lopez-cardona26a.html The superposition hypothesis states that single neurons may participate in representing multiple features in order for the neural network to represent more features than it has neurons. In neuroscience and AI, representational alignment metrics measure the extent to which different deep neural networks (DNNs) or brains represent similar information. In this work, we explore a critical question: does superposition interact with alignment metrics in any undesirable way? We hypothesize that models which represent the same features in different superposition arrangements, i.e., their neurons have different linear combinations of the features, will interfere with predictive mapping metrics (semi-matching, soft-matching, linear regression), producing lower alignment than expected. We develop a theory for how permutation metrics are dependent on superposition arrangements. This is tested by training sparse autoencoders (SAEs) to disentangle superposition in toy models, where alignment scores are shown to typically increase when a model’s base neurons are replaced with its sparse overcomplete latent codes. We find similar increases for DNN-DNN and DNN-brain linear regression alignment in the visual domain. Our results suggest that superposition disentanglement is necessary for mapping metrics to uncover the true representational alignment between neural networks. Fri, 27 Feb 2026 00:00:00 +0000 https://proceedings.mlr.press/v322/longon26a.html https://proceedings.mlr.press/v322/longon26a.html Knowledge distillation (KD) is a widely adopted technique for compressing large models into smaller, more efficient student models that can be deployed on devices with limited computational resources. Among various KD methods, Relational Knowledge Distillation (RKD) improves student performance by aligning relational structures in the feature space, such as pairwise distances and angles. In this work, we propose Quantum Relational Knowledge Distillation (QRKD), which extends RKD by incorporating quantum relational information. Specifically, we map classical features into a Hilbert space, interpret them as quantum states, and compute quantum kernel values to capture richer inter-sample relationships. These quantum-informed relations are then used to guide the distillation process. We evaluate QRKD on both vision and language tasks, including CNNs on MNIST and CIFAR-10, and GPT-2 on WikiText-2, Penn Treebank, and IMDB. Across all benchmarks, QRKD consistently improves student model performance compared to classical RKD. Importantly, both teacher and student models remain classical and deployable on standard hardware, with quantum computation required only during training. This work presents the first demonstration of quantum-enhanced knowledge distillation in a fully classical deployment setting. Fri, 27 Feb 2026 00:00:00 +0000 https://proceedings.mlr.press/v322/liu26b.html https://proceedings.mlr.press/v322/liu26b.html Unified visual representations from language supervision and self-supervision offer the potential to advance general-purpose vision models. In this work, we present an empirical study on unifying joint-embedding predictive architecture (I-JEPA) with language supervision from CLIP for visual representation learning. I-JEPA is unique among self-supervised learning methods in that it is predictive rather than contrastive or generative, enabling faster convergence with less compute while still producing strong representations. Existing works have shown that joint training with language supervision and other visual self-supervision methods yield improved model performance, but combining language supervision with I-JEPA remains unexplored. We introduce CLIPred, a framework that jointly optimizes the two objectives, and systematically evaluate it across zero-shot classification, retrieval, and probing tasks. CLIPred outperforms CLIP-only, I-JEPA-only, and sequentially applying the two, and offers better zero-shot transfer than DINOv2+CLIP with lower training cost, though with trade-offs in probing performance. Our experiments further examine the effects of loss weighting, amount of data used by each objective, and batch size on our framework, We conduct further analysis on design choices of the architecture and the semantics of the patch embeddings generated by CLIPred. This work provides the first comprehensive assessment of combining I-JEPA and CLIP, highlighting both the benefits and limitations of the framework as well as recommendations on when and how to apply the framework. Fri, 27 Feb 2026 00:00:00 +0000 https://proceedings.mlr.press/v322/liu26a.html https://proceedings.mlr.press/v322/liu26a.html Benchmarking 3D spatial understanding of foundation models is essential for real-world applications such as robotics and autonomous driving. Existing evaluations often rely on downstream fine-tuning with linear heads or task-specific decoders, making it difficult to isolate the intrinsic 3D reasoning ability of pre-trained encoders. In this work, we introduce a novel benchmark for in-context 3D scene understanding that requires no fine-tuning and directly probes the quality of dense visual features. Building on the Hummingbird framework, which evaluates in-context 2D scene understanding, we extend the setup to the 3D Multi-View ImageNet (MVImgNet) dataset. Given a set of images depicting objects at specific camera angles (keys), we benchmark the performance of segmenting novel views (queries) and report the scores in 4 categories of easy, medium, hard, and extreme based on the key-query view contrast. We benchmark 7 state-of-the-art foundation models and show that DINO-based encoders remain competitive across large viewpoint shifts. Our code is publicly available at https://github.com/ToyeshC/open-hummingbird-3d-eval. Fri, 27 Feb 2026 00:00:00 +0000 https://proceedings.mlr.press/v322/lilova26a.html https://proceedings.mlr.press/v322/lilova26a.html What does it even mean to average neural networks? We investigate the problem of synthesizing a single neural network from a collection of pretrained models, each trained on disjoint data shards, using only their final weights and no access to training data. In forming a definition of neural averaging, we take insight from model soup, which appears to aggregate multiple models into a singular model while enhancing generalization performance. In this work, we reinterpret model souping as a special case of a broader framework: Amortized Model Ensembling (AME) for neural averaging, a data-free meta-optimization approach that treats model differences as pseudogradients to guide neural weight updates. We show that this perspective not only recovers model soup but enables more expressive and adaptive ensembling strategies. Empirically, AME produces averaged neural solutions that outperform both individual experts and model soup baselines, especially in out-of-distribution settings. Our results suggest a principled and generalizable notion of data-free model weight aggregation and defines, in one sense, how to perform neural averaging. Fri, 27 Feb 2026 00:00:00 +0000 https://proceedings.mlr.press/v322/lee26a.html https://proceedings.mlr.press/v322/lee26a.html Multimodal product retrieval systems in e-commerce platforms rely on effectively combining visual and textual signals to improve search relevance and user experience. However, vision-language models such as CLIP are vulnerable to typographic attacks, where misleading or irrelevant text embedded in images skews model predictions. In this work, we propose a novel method that reverses the logic of typographic attacks by overlaying relevant textual content (e.g., titles, descriptions) directly onto product images to perform additional vision-text compression, thereby strengthening image-text alignment and boosting multimodal product retrieval performance. We evaluate our method on three vertical-specific e-commerce datasets (sneakers, handbags, and trading cards) using five state-of-the-art vision foundation models. Our experiments demonstrate consistent improvements in unimodal and multimodal retrieval accuracy across categories and model families. Our findings suggest that visually rendering product metadata is a simple yet effective enhancement for zero-shot multimodal retrieval in e-commerce applications. Fri, 27 Feb 2026 00:00:00 +0000 https://proceedings.mlr.press/v322/jenq26a.html https://proceedings.mlr.press/v322/jenq26a.html Document image retrieval is a fundamental task for improving document understanding, where the goal is to retrieve relevant images in the document and to answer the question from the user. Unlike other text-to-image tasks, which mainly focus on the alignment between image caption and natural image, document image retrieval requires the model to understand the question from user and return related table image or scientific image. The significant domain difference between image caption and user question, as well as natural image and scientific images, prevents the off-the-shelf retrieval model from becoming applicable. To systematically study the degradation, we curate a novel multi-lingual Document Question-Image Retrieval benchmark, DocQIR, that covers questions in 5 different languages. Our preliminary study shows that off-the-shelf retrieval models fail to retrieve documents images when questions in various languages are presented. To address this issue, we proposed a novel architecture, DocQIR-Emb, that leverages a multi-lingual text embedder and a VLM to encode a question and an image into a shared feature space. Since the multi-lingual embedder is trained to align text in different languages, the text embedder is frozen and only the VLM is optimized. Experiments show that DocQIR-Emb outperforms the baseline by at least 40% on the proposed DocQIR dataset and the gain is consistent across table image and scientific image. Different architecture designs are also ablated to demonstrate the effectiveness of DocQIR-Emb. Fri, 27 Feb 2026 00:00:00 +0000 https://proceedings.mlr.press/v322/ho26a.html https://proceedings.mlr.press/v322/ho26a.html The inclusion of symmetries as an inductive bias, known as *equivariance*, often improves generalization on geometric data (e.g. grids, sets, and graphs). However, equivariant architectures are usually highly constrained, designed for symmetries chosen *a priori*, and not applicable to datasets with other symmetries. This precludes the development of flexible, multi-modal foundation models capable of processing diverse data equivariantly. In this work, we build a single model — the Any-Subgroup Equivariant Network (ASEN) — that can be simultaneously equivariant to several groups, simply by modulating a certain auxiliary input feature. In particular, we start with a fully permutation-equivariant base model, and then obtain subgroup equivariance by using a symmetry-breaking input whose automorphism group is that subgroup. However, finding an input with the desired automorphism group is computationally hard. We overcome this by relaxing from exact to approximate symmetry breaking, leveraging the notion of 2-closure to derive fast algorithms. Theoretically, we show that our subgroup-equivariant networks can simulate equivariant MLPs, and their universality can be guaranteed if the base model is universal. Empirically, we validate our method on symmetry selection for graph tasks, as well as multitask and transfer learning for sequence tasks, showing that a single network equivariant to multiple permutation subgroups outperforms both separate equivariant models and a single non-equivariant model. Fri, 27 Feb 2026 00:00:00 +0000 https://proceedings.mlr.press/v322/goel26a.html https://proceedings.mlr.press/v322/goel26a.html Discover why, when and how distinct learning processes yield similar representations, and the degree to which these can be unified. Fri, 27 Feb 2026 00:00:00 +0000 https://proceedings.mlr.press/v322/fumero26a.html https://proceedings.mlr.press/v322/fumero26a.html Sparse autoencoders (SAEs) are machine learning models that can be used to express the inner workings of certain other models as human-interpretable features. While sparse autoencoders work well when applied to language models, there has been little research that investigates the extent to which they generalize to other applications of machine learning. This work investigates the application of SAEs to a deep Q network trained to complete a simple task. We find that, although SAEs tend to perform well and find a number of human-interpretable features, they contain a large number of "dead features" that never activate, which suggests that more research is necessary to adapt SAEs to the unique tasks reinforcement learning models solve. In particular, we note that the most effective deep Q networks trained to complete a task tend to result in sparse autoencoders with a consistent quantity of dead features. This suggests that these sparse autoencoders may in some sense be capturing the "optimal" or "true" number of features needed to solve the toy problem we study, and the high number of dead features may simply imply that additional live features past a certain quantity are unhelpful. Fri, 27 Feb 2026 00:00:00 +0000 https://proceedings.mlr.press/v322/duplessie26a.html https://proceedings.mlr.press/v322/duplessie26a.html Model merging provides cost-effective, data-efficient combination of specialized deep neural networks through parameter integration. This technique exploits specialist model strengths across downstream tasks without retraining. Most model merging approaches critically depend on scaling hyper-parameters $\lambda$, weighting each model’s contribution globally or individually. Principled approaches for setting scaling factors without accessing any data (data-free) are scarce, often leading researchers to tune $\lambda$ using privileged data from the evaluation set, which is obviously unfeasible in practice. To address this limitation, we introduce Weight Weaving, a plug-and-play technique that pools model weights across $\lambda$ values search space using user-defined pooling functions, such as averaging, random selection, or even existing model merging methods. Our method demonstrates high modularity, imposing minimal search space constraints. It operates orthogonally to existing model merging methods and eliminates evaluation data requirements. We validate Weight Weaving across three ViT variants in three experimental setups: vision multi-task learning, vision continual learning, and domain generalization. Our method consistently improves the performance of several model merging methods, achieving average accuracy gains of up to 15.9 percentage points in a data-free setting. Fri, 27 Feb 2026 00:00:00 +0000 https://proceedings.mlr.press/v322/chaves26a.html https://proceedings.mlr.press/v322/chaves26a.html We propose MultiPersona-Align, a novel approach for personalized image generation that enhances multi-subject diffusion models through self-supervised feature alignment. While existing methods rely primarily on spatial masking for subject control, they often produce semantically inconsistent features that fail to preserve the subject-specific visual characteristics. Our method introduces Dual Alignment Framework: (1) Spatially-Aligned Subject-Specific Cross-Attention Mechanism that aligns subject-specific diffusion features with corresponding DINOv2 CLS tokens within spatial regions, and (2) Patch-Aligned Self-Attention that ensures global semantic consistency by aligning full-image diffusion features with DINOv2 patch representations. This approach leverages DINOv2’s robust semantic understanding without requiring additional training data or annotations. Experiments on multi-subject generation tasks demonstrate that our alignment losses significantly improve subject fidelity and semantic consistency while maintaining spatial control. The method integrates seamlessly into existing architectures, adding minimal computational overhead during training while providing substantial quality improvements in personalized image generation. Fri, 27 Feb 2026 00:00:00 +0000 https://proceedings.mlr.press/v322/chakravarthy26a.html https://proceedings.mlr.press/v322/chakravarthy26a.html Vision-language models (VLMs) achieve remarkable performance through large-scale image–text pretraining. However, their reliance on labeled image datasets limits scalability and leaves vast amounts of unlabeled image data underutilized. To address this, we propose Unified Vision-Language Alignment for Zero-Label Enhancement (ViZer), an enhancement training framework that enables zero-label learning in image captioning, providing a practical starting point for broader zero-label adaptation in vision-language tasks. Unlike prior approaches that rely on human or synthetically annotated datasets, ViZer actively aligns vision and language representation features during training, enabling existing VLMs to generate improved captions without requiring text labels or full retraining. We demonstrate ViZer’s advantage in qualitative evaluation, as automated caption metrics such as CIDEr and BERTScore often penalize details that are absent in reference captions. Applying ViZer on SmolVLM-Base and Qwen2-VL, we observe consistent qualitative improvements, producing captions that are more grounded and descriptive than their baseline. Fri, 27 Feb 2026 00:00:00 +0000 https://proceedings.mlr.press/v322/byun26a.html https://proceedings.mlr.press/v322/byun26a.html The strong performance of simple neural networks is often attributed to their nonlinear activations. However, a linear view of neural networks makes understanding and controlling networks much more approachable. We draw from a dynamical systems view of neural networks, offering a fresh perspective by using Koopman operator theory and its connections with dynamic mode decomposition (DMD). Together, they offer a framework for linearizing dynamical systems by embedding the system into an appropriate observable space. By reframing a neural network as a dynamical system, we demonstrate that we can replace the nonlinear layer in a pretrained multi-layer perceptron (MLP) with a finite-dimensional linear operator. In addition, we analyze the eigenvalues of DMD and the right singular vectors of SVD, to present evidence that time-delayed coordinates provide a straightforward and highly effective observable space for Koopman theory to linearize a network layer. Consequently, we replace layers of an MLP trained on the Yin-Yang dataset with predictions from a DMD model, achieving a mdoel accuracy of up to 97.3%, compared to the original 98.4%. In addition, we replace layers in an MLP trained on the MNIST dataset, achieving up to 95.8%, compared to the original 97.2% on the test set. Fri, 27 Feb 2026 00:00:00 +0000 https://proceedings.mlr.press/v322/aswani26a.html https://proceedings.mlr.press/v322/aswani26a.html