Coded-InvNet for Resilient Prediction Serving Systems

Tuan Dinh, Kangwook Lee
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:2749-2759, 2021.

Abstract

Inspired by a new coded computation algorithm for invertible functions, we propose Coded-InvNet a new approach to design resilient prediction serving systems that can gracefully handle stragglers or node failures. Coded-InvNet leverages recent findings in the deep learning literature such as invertible neural networks, Manifold Mixup, and domain translation algorithms, identifying interesting research directions that span across machine learning and systems. Our experimental results show that Coded-InvNet can outperform existing approaches, especially when the compute resource overhead is as low as 10%. For instance, without knowing which of the ten workers is going to fail, our algorithm can design a backup task so that it can correctly recover the missing prediction result with an accuracy of 85.9%, significantly outperforming the previous SOTA by 32.5%.

Cite this Paper

BibTeX
@InProceedings{pmlr-v139-dinh21a, title = {Coded-InvNet for Resilient Prediction Serving Systems}, author = {Dinh, Tuan and Lee, Kangwook}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {2749--2759}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/dinh21a/dinh21a.pdf}, url = {https://proceedings.mlr.press/v139/dinh21a.html}, abstract = {Inspired by a new coded computation algorithm for invertible functions, we propose Coded-InvNet a new approach to design resilient prediction serving systems that can gracefully handle stragglers or node failures. Coded-InvNet leverages recent findings in the deep learning literature such as invertible neural networks, Manifold Mixup, and domain translation algorithms, identifying interesting research directions that span across machine learning and systems. Our experimental results show that Coded-InvNet can outperform existing approaches, especially when the compute resource overhead is as low as 10%. For instance, without knowing which of the ten workers is going to fail, our algorithm can design a backup task so that it can correctly recover the missing prediction result with an accuracy of 85.9%, significantly outperforming the previous SOTA by 32.5%.} }
Endnote
%0 Conference Paper %T Coded-InvNet for Resilient Prediction Serving Systems %A Tuan Dinh %A Kangwook Lee %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-dinh21a %I PMLR %P 2749--2759 %U https://proceedings.mlr.press/v139/dinh21a.html %V 139 %X Inspired by a new coded computation algorithm for invertible functions, we propose Coded-InvNet a new approach to design resilient prediction serving systems that can gracefully handle stragglers or node failures. Coded-InvNet leverages recent findings in the deep learning literature such as invertible neural networks, Manifold Mixup, and domain translation algorithms, identifying interesting research directions that span across machine learning and systems. Our experimental results show that Coded-InvNet can outperform existing approaches, especially when the compute resource overhead is as low as 10%. For instance, without knowing which of the ten workers is going to fail, our algorithm can design a backup task so that it can correctly recover the missing prediction result with an accuracy of 85.9%, significantly outperforming the previous SOTA by 32.5%.
APA
Dinh, T. & Lee, K.. (2021). Coded-InvNet for Resilient Prediction Serving Systems. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:2749-2759 Available from https://proceedings.mlr.press/v139/dinh21a.html.

Related Material