Do Coresets, Pruning, and Quantization Preserve Neural Network Representations?

Tushar Shinde, Avinash Kumar Sharma
Proceedings of GRaM: the Second Edition of the Workshop on Geometry-grounded Representation Learning and Generative Modeling, PMLR 326:489-496, 2026.

Abstract

Neural network compression techniques, such as coreset selection, pruning, and quantization, enable efficient deployment but often induce representational changes that traditional accuracy metrics fail to capture. We propose Representation Similarity (REPS), a multi-faceted diagnostic metric that unifies effective rank, neuron aliveness, class separation, and eigenvalue decay similarity into a single interpretable score, providing comprehensive evaluation of compression- induced representational degradation. Experiments on CIFAR-10 with ResNet-18 demonstrate that REPS correlates strongly with accuracy drops (Pearson r =0.988), substantially outperforming conventional baselines such as weight similarity (r = 0.141) and prediction agreement. We further provide a sensitivity analysis of REPS component weights and layer-wise analysis revealing dimensional collapse, neuron death, and class separation degradation, offering interpretable insights into representational integrity under compression. These results position REPS as a robust, lightweight diagnostic tool for guiding compression-aware model design and adaptive deployment in resource-constrained environments.

Cite this Paper

BibTeX
@InProceedings{pmlr-v326-shinde26a, title = {Do Coresets, Pruning, and Quantization Preserve Neural Network Representations?}, author = {Shinde, Tushar and Sharma, Avinash Kumar}, booktitle = {Proceedings of GRaM: the Second Edition of the Workshop on Geometry-grounded Representation Learning and Generative Modeling}, pages = {489--496}, year = {2026}, editor = {Pouplin, Alison and Vadgama, Sharvaree and Bekkers, Erik and Kaba, Sékou-Oumar and Lawrence, Hannah and Lecha, Manuel and Baker, Elizabeth and Suk, Julian and Walters, Robin and Tomczak, Jakub and Jegelka, Stefanie}, volume = {326}, series = {Proceedings of Machine Learning Research}, month = {26 Apr}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v326/main/assets/shinde26a/shinde26a.pdf}, url = {https://proceedings.mlr.press/v326/shinde26a.html}, abstract = {Neural network compression techniques, such as coreset selection, pruning, and quantization, enable efficient deployment but often induce representational changes that traditional accuracy metrics fail to capture. We propose Representation Similarity (REPS), a multi-faceted diagnostic metric that unifies effective rank, neuron aliveness, class separation, and eigenvalue decay similarity into a single interpretable score, providing comprehensive evaluation of compression- induced representational degradation. Experiments on CIFAR-10 with ResNet-18 demonstrate that REPS correlates strongly with accuracy drops (Pearson r =0.988), substantially outperforming conventional baselines such as weight similarity (r = 0.141) and prediction agreement. We further provide a sensitivity analysis of REPS component weights and layer-wise analysis revealing dimensional collapse, neuron death, and class separation degradation, offering interpretable insights into representational integrity under compression. These results position REPS as a robust, lightweight diagnostic tool for guiding compression-aware model design and adaptive deployment in resource-constrained environments.} }
Endnote
%0 Conference Paper %T Do Coresets, Pruning, and Quantization Preserve Neural Network Representations? %A Tushar Shinde %A Avinash Kumar Sharma %B Proceedings of GRaM: the Second Edition of the Workshop on Geometry-grounded Representation Learning and Generative Modeling %C Proceedings of Machine Learning Research %D 2026 %E Alison Pouplin %E Sharvaree Vadgama %E Erik Bekkers %E Sékou-Oumar Kaba %E Hannah Lawrence %E Manuel Lecha %E Elizabeth Baker %E Julian Suk %E Robin Walters %E Jakub Tomczak %E Stefanie Jegelka %F pmlr-v326-shinde26a %I PMLR %P 489--496 %U https://proceedings.mlr.press/v326/shinde26a.html %V 326 %X Neural network compression techniques, such as coreset selection, pruning, and quantization, enable efficient deployment but often induce representational changes that traditional accuracy metrics fail to capture. We propose Representation Similarity (REPS), a multi-faceted diagnostic metric that unifies effective rank, neuron aliveness, class separation, and eigenvalue decay similarity into a single interpretable score, providing comprehensive evaluation of compression- induced representational degradation. Experiments on CIFAR-10 with ResNet-18 demonstrate that REPS correlates strongly with accuracy drops (Pearson r =0.988), substantially outperforming conventional baselines such as weight similarity (r = 0.141) and prediction agreement. We further provide a sensitivity analysis of REPS component weights and layer-wise analysis revealing dimensional collapse, neuron death, and class separation degradation, offering interpretable insights into representational integrity under compression. These results position REPS as a robust, lightweight diagnostic tool for guiding compression-aware model design and adaptive deployment in resource-constrained environments.
APA
Shinde, T. & Sharma, A.K.. (2026). Do Coresets, Pruning, and Quantization Preserve Neural Network Representations?. Proceedings of GRaM: the Second Edition of the Workshop on Geometry-grounded Representation Learning and Generative Modeling, in Proceedings of Machine Learning Research 326:489-496 Available from https://proceedings.mlr.press/v326/shinde26a.html.

Related Material