Sequence Alignment-based Similarity Metric in Evolutionary Neural Architecture Search

Mateo Avila Pava, René Groh, Andreas M Kist
Proceedings of the Third International Conference on Automated Machine Learning, PMLR 256:8/1-21, 2024.

Abstract

Neural Architecture Search (NAS) has emerged as a powerful method for automating the design of deep neural networks across diverse applications, with evolutionary optimization showing particular promise in addressing its intricate demands. However, the effectiveness of this approach highly depends on balancing exploration and exploitation, ensuring that the search does not prematurely converge to suboptimal solutions while still achieving near-optimal outcomes. This paper addresses this challenge by proposing a novel similarity metric inspired by global sequence alignment from biology. Unlike most of the existing methods that require pre-trained models for comparison, our metric operates directly on neural network architectures within the defined search space, eliminating the need for model training. We outline the computation of the normalized similarity metric and demonstrate its application in quantifying diversity within populations in evolutionary NAS. Experimental results conducted on popular datasets for image classification, such as CIFAR-10, CIFAR-100, and ImageNet16-120, show the effectiveness of our approach in guiding diversity based on our suggested control function. Additionally, we highlight the usefulness of our similarity metric in comparing individuals to identify advantageous or disadvantageous architectural design choices. The code is available at \url{https://anon-github.automl.cc/r/evonas_similarity_metric-1BD3/}.

Cite this Paper

BibTeX
@InProceedings{pmlr-v256-pava24a, title = {Sequence Alignment-based Similarity Metric in Evolutionary Neural Architecture Search}, author = {Pava, Mateo Avila and Groh, Ren\'e and Kist, Andreas M}, booktitle = {Proceedings of the Third International Conference on Automated Machine Learning}, pages = {8/1--21}, year = {2024}, editor = {Eggensperger, Katharina and Garnett, Roman and Vanschoren, Joaquin and Lindauer, Marius and Gardner, Jacob R.}, volume = {256}, series = {Proceedings of Machine Learning Research}, month = {09--12 Sep}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v256/main/assets/pava24a/pava24a.pdf}, url = {https://proceedings.mlr.press/v256/pava24a.html}, abstract = {Neural Architecture Search (NAS) has emerged as a powerful method for automating the design of deep neural networks across diverse applications, with evolutionary optimization showing particular promise in addressing its intricate demands. However, the effectiveness of this approach highly depends on balancing exploration and exploitation, ensuring that the search does not prematurely converge to suboptimal solutions while still achieving near-optimal outcomes. This paper addresses this challenge by proposing a novel similarity metric inspired by global sequence alignment from biology. Unlike most of the existing methods that require pre-trained models for comparison, our metric operates directly on neural network architectures within the defined search space, eliminating the need for model training. We outline the computation of the normalized similarity metric and demonstrate its application in quantifying diversity within populations in evolutionary NAS. Experimental results conducted on popular datasets for image classification, such as CIFAR-10, CIFAR-100, and ImageNet16-120, show the effectiveness of our approach in guiding diversity based on our suggested control function. Additionally, we highlight the usefulness of our similarity metric in comparing individuals to identify advantageous or disadvantageous architectural design choices. The code is available at \url{https://anon-github.automl.cc/r/evonas_similarity_metric-1BD3/}.} }
Endnote
%0 Conference Paper %T Sequence Alignment-based Similarity Metric in Evolutionary Neural Architecture Search %A Mateo Avila Pava %A René Groh %A Andreas M Kist %B Proceedings of the Third International Conference on Automated Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Katharina Eggensperger %E Roman Garnett %E Joaquin Vanschoren %E Marius Lindauer %E Jacob R. Gardner %F pmlr-v256-pava24a %I PMLR %P 8/1--21 %U https://proceedings.mlr.press/v256/pava24a.html %V 256 %X Neural Architecture Search (NAS) has emerged as a powerful method for automating the design of deep neural networks across diverse applications, with evolutionary optimization showing particular promise in addressing its intricate demands. However, the effectiveness of this approach highly depends on balancing exploration and exploitation, ensuring that the search does not prematurely converge to suboptimal solutions while still achieving near-optimal outcomes. This paper addresses this challenge by proposing a novel similarity metric inspired by global sequence alignment from biology. Unlike most of the existing methods that require pre-trained models for comparison, our metric operates directly on neural network architectures within the defined search space, eliminating the need for model training. We outline the computation of the normalized similarity metric and demonstrate its application in quantifying diversity within populations in evolutionary NAS. Experimental results conducted on popular datasets for image classification, such as CIFAR-10, CIFAR-100, and ImageNet16-120, show the effectiveness of our approach in guiding diversity based on our suggested control function. Additionally, we highlight the usefulness of our similarity metric in comparing individuals to identify advantageous or disadvantageous architectural design choices. The code is available at \url{https://anon-github.automl.cc/r/evonas_similarity_metric-1BD3/}.
APA
Pava, M.A., Groh, R. & Kist, A.M.. (2024). Sequence Alignment-based Similarity Metric in Evolutionary Neural Architecture Search. Proceedings of the Third International Conference on Automated Machine Learning, in Proceedings of Machine Learning Research 256:8/1-21 Available from https://proceedings.mlr.press/v256/pava24a.html.

Related Material