Uncovering the Connections Between Adversarial Transferability and Knowledge Transferability

Kaizhao Liang, Jacky Y Zhang, Boxin Wang, Zhuolin Yang, Sanmi Koyejo, Bo Li
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:6577-6587, 2021.

Abstract

Knowledge transferability, or transfer learning, has been widely adopted to allow a pre-trained model in the source domain to be effectively adapted to downstream tasks in the target domain. It is thus important to explore and understand the factors affecting knowledge transferability. In this paper, as the first work, we analyze and demonstrate the connections between knowledge transferability and another important phenomenon–adversarial transferability, \emph{i.e.}, adversarial examples generated against one model can be transferred to attack other models. Our theoretical studies show that adversarial transferability indicates knowledge transferability, and vice versa. Moreover, based on the theoretical insights, we propose two practical adversarial transferability metrics to characterize this process, serving as bidirectional indicators between adversarial and knowledge transferability. We conduct extensive experiments for different scenarios on diverse datasets, showing a positive correlation between adversarial transferability and knowledge transferability. Our findings will shed light on future research about effective knowledge transfer learning and adversarial transferability analyses.

Cite this Paper

BibTeX
@InProceedings{pmlr-v139-liang21b, title = {Uncovering the Connections Between Adversarial Transferability and Knowledge Transferability}, author = {Liang, Kaizhao and Zhang, Jacky Y and Wang, Boxin and Yang, Zhuolin and Koyejo, Sanmi and Li, Bo}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {6577--6587}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/liang21b/liang21b.pdf}, url = {https://proceedings.mlr.press/v139/liang21b.html}, abstract = {Knowledge transferability, or transfer learning, has been widely adopted to allow a pre-trained model in the source domain to be effectively adapted to downstream tasks in the target domain. It is thus important to explore and understand the factors affecting knowledge transferability. In this paper, as the first work, we analyze and demonstrate the connections between knowledge transferability and another important phenomenon–adversarial transferability, \emph{i.e.}, adversarial examples generated against one model can be transferred to attack other models. Our theoretical studies show that adversarial transferability indicates knowledge transferability, and vice versa. Moreover, based on the theoretical insights, we propose two practical adversarial transferability metrics to characterize this process, serving as bidirectional indicators between adversarial and knowledge transferability. We conduct extensive experiments for different scenarios on diverse datasets, showing a positive correlation between adversarial transferability and knowledge transferability. Our findings will shed light on future research about effective knowledge transfer learning and adversarial transferability analyses.} }
Endnote
%0 Conference Paper %T Uncovering the Connections Between Adversarial Transferability and Knowledge Transferability %A Kaizhao Liang %A Jacky Y Zhang %A Boxin Wang %A Zhuolin Yang %A Sanmi Koyejo %A Bo Li %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-liang21b %I PMLR %P 6577--6587 %U https://proceedings.mlr.press/v139/liang21b.html %V 139 %X Knowledge transferability, or transfer learning, has been widely adopted to allow a pre-trained model in the source domain to be effectively adapted to downstream tasks in the target domain. It is thus important to explore and understand the factors affecting knowledge transferability. In this paper, as the first work, we analyze and demonstrate the connections between knowledge transferability and another important phenomenon–adversarial transferability, \emph{i.e.}, adversarial examples generated against one model can be transferred to attack other models. Our theoretical studies show that adversarial transferability indicates knowledge transferability, and vice versa. Moreover, based on the theoretical insights, we propose two practical adversarial transferability metrics to characterize this process, serving as bidirectional indicators between adversarial and knowledge transferability. We conduct extensive experiments for different scenarios on diverse datasets, showing a positive correlation between adversarial transferability and knowledge transferability. Our findings will shed light on future research about effective knowledge transfer learning and adversarial transferability analyses.
APA
Liang, K., Zhang, J.Y., Wang, B., Yang, Z., Koyejo, S. & Li, B.. (2021). Uncovering the Connections Between Adversarial Transferability and Knowledge Transferability. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:6577-6587 Available from https://proceedings.mlr.press/v139/liang21b.html.

Related Material