The PetShop Dataset — Finding Causes of Performance Issues across Microservices

Michaela Hardt, William Roy Orchard, Patrick Blöbaum, Elke Kirschbaum, Shiva Kasiviswanathan
Proceedings of the Third Conference on Causal Learning and Reasoning, PMLR 236:957-978, 2024.

Abstract

Identifying root causes for unexpected or undesirable behavior in complex systems is a prevalent challenge. This issue becomes especially crucial in modern cloud applications that employ numerous microservices. Although the machine learning and systems research communities have proposed various techniques to tackle this problem, there is currently a lack of standardized datasets for quantitative benchmarking. Consequently, research groups are compelled to create their own datasets for experimentation. This paper introduces a dataset specifically designed for evaluating root cause analyses in microservice-based applications. The dataset encompasses latency, requests, and availability metrics emitted in 5-minute intervals from a distributed application. In addition to normal operation metrics, the dataset includes 68 injected performance issues, which increase latency and reduce availability throughout the system. We showcase how this dataset can be used to evaluate the accuracy of a variety of methods spanning different causal and non-causal characterisations of the root cause analysis problem. We hope the new dataset, available at https://github.com/amazon-science/petshop-root-cause-analysis, enables further development of techniques in this important area.

Cite this Paper

BibTeX
@InProceedings{pmlr-v236-hardt24a, title = {The PetShop Dataset — Finding Causes of Performance Issues across Microservices}, author = {Hardt, Michaela and Orchard, William Roy and Bl\"obaum, Patrick and Kirschbaum, Elke and Kasiviswanathan, Shiva}, booktitle = {Proceedings of the Third Conference on Causal Learning and Reasoning}, pages = {957--978}, year = {2024}, editor = {Locatello, Francesco and Didelez, Vanessa}, volume = {236}, series = {Proceedings of Machine Learning Research}, month = {01--03 Apr}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v236/hardt24a/hardt24a.pdf}, url = {https://proceedings.mlr.press/v236/hardt24a.html}, abstract = {Identifying root causes for unexpected or undesirable behavior in complex systems is a prevalent challenge. This issue becomes especially crucial in modern cloud applications that employ numerous microservices. Although the machine learning and systems research communities have proposed various techniques to tackle this problem, there is currently a lack of standardized datasets for quantitative benchmarking. Consequently, research groups are compelled to create their own datasets for experimentation. This paper introduces a dataset specifically designed for evaluating root cause analyses in microservice-based applications. The dataset encompasses latency, requests, and availability metrics emitted in 5-minute intervals from a distributed application. In addition to normal operation metrics, the dataset includes 68 injected performance issues, which increase latency and reduce availability throughout the system. We showcase how this dataset can be used to evaluate the accuracy of a variety of methods spanning different causal and non-causal characterisations of the root cause analysis problem. We hope the new dataset, available at https://github.com/amazon-science/petshop-root-cause-analysis, enables further development of techniques in this important area.} }
Endnote
%0 Conference Paper %T The PetShop Dataset — Finding Causes of Performance Issues across Microservices %A Michaela Hardt %A William Roy Orchard %A Patrick Blöbaum %A Elke Kirschbaum %A Shiva Kasiviswanathan %B Proceedings of the Third Conference on Causal Learning and Reasoning %C Proceedings of Machine Learning Research %D 2024 %E Francesco Locatello %E Vanessa Didelez %F pmlr-v236-hardt24a %I PMLR %P 957--978 %U https://proceedings.mlr.press/v236/hardt24a.html %V 236 %X Identifying root causes for unexpected or undesirable behavior in complex systems is a prevalent challenge. This issue becomes especially crucial in modern cloud applications that employ numerous microservices. Although the machine learning and systems research communities have proposed various techniques to tackle this problem, there is currently a lack of standardized datasets for quantitative benchmarking. Consequently, research groups are compelled to create their own datasets for experimentation. This paper introduces a dataset specifically designed for evaluating root cause analyses in microservice-based applications. The dataset encompasses latency, requests, and availability metrics emitted in 5-minute intervals from a distributed application. In addition to normal operation metrics, the dataset includes 68 injected performance issues, which increase latency and reduce availability throughout the system. We showcase how this dataset can be used to evaluate the accuracy of a variety of methods spanning different causal and non-causal characterisations of the root cause analysis problem. We hope the new dataset, available at https://github.com/amazon-science/petshop-root-cause-analysis, enables further development of techniques in this important area.
APA
Hardt, M., Orchard, W.R., Blöbaum, P., Kirschbaum, E. & Kasiviswanathan, S.. (2024). The PetShop Dataset — Finding Causes of Performance Issues across Microservices. Proceedings of the Third Conference on Causal Learning and Reasoning, in Proceedings of Machine Learning Research 236:957-978 Available from https://proceedings.mlr.press/v236/hardt24a.html.

Related Material