Learning Generalized Intersection Over Union for Dense Pixelwise Prediction

Jiaqian Yu, Jingtao Xu, Yiwei Chen, Weiming Li, Qiang Wang, Byungin Yoo, Jae-Joon Han
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:12198-12207, 2021.

Abstract

Intersection over union (IoU) score, also named Jaccard Index, is one of the most fundamental evaluation methods in machine learning. The original IoU computation cannot provide non-zero gradients and thus cannot be directly optimized by nowadays deep learning methods. Several recent works generalized IoU for bounding box regression, but they are not straightforward to adapt for pixelwise prediction. In particular, the original IoU fails to provide effective gradients for the non-overlapping and location-deviation cases, which results in performance plateau. In this paper, we propose PixIoU, a generalized IoU for pixelwise prediction that is sensitive to the distance for non-overlapping cases and the locations in prediction. We provide proofs that PixIoU holds many nice properties as the original IoU. To optimize the PixIoU, we also propose a loss function that is proved to be submodular, hence we can apply the Lovász functions, the efficient surrogates for submodular functions for learning this loss. Experimental results show consistent performance improvements by learning PixIoU over the original IoU for several different pixelwise prediction tasks on Pascal VOC, VOT-2020 and Cityscapes.

Cite this Paper

BibTeX
@InProceedings{pmlr-v139-yu21e, title = {Learning Generalized Intersection Over Union for Dense Pixelwise Prediction}, author = {Yu, Jiaqian and Xu, Jingtao and Chen, Yiwei and Li, Weiming and Wang, Qiang and Yoo, Byungin and Han, Jae-Joon}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {12198--12207}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/yu21e/yu21e.pdf}, url = {https://proceedings.mlr.press/v139/yu21e.html}, abstract = {Intersection over union (IoU) score, also named Jaccard Index, is one of the most fundamental evaluation methods in machine learning. The original IoU computation cannot provide non-zero gradients and thus cannot be directly optimized by nowadays deep learning methods. Several recent works generalized IoU for bounding box regression, but they are not straightforward to adapt for pixelwise prediction. In particular, the original IoU fails to provide effective gradients for the non-overlapping and location-deviation cases, which results in performance plateau. In this paper, we propose PixIoU, a generalized IoU for pixelwise prediction that is sensitive to the distance for non-overlapping cases and the locations in prediction. We provide proofs that PixIoU holds many nice properties as the original IoU. To optimize the PixIoU, we also propose a loss function that is proved to be submodular, hence we can apply the Lovász functions, the efficient surrogates for submodular functions for learning this loss. Experimental results show consistent performance improvements by learning PixIoU over the original IoU for several different pixelwise prediction tasks on Pascal VOC, VOT-2020 and Cityscapes.} }
Endnote
%0 Conference Paper %T Learning Generalized Intersection Over Union for Dense Pixelwise Prediction %A Jiaqian Yu %A Jingtao Xu %A Yiwei Chen %A Weiming Li %A Qiang Wang %A Byungin Yoo %A Jae-Joon Han %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-yu21e %I PMLR %P 12198--12207 %U https://proceedings.mlr.press/v139/yu21e.html %V 139 %X Intersection over union (IoU) score, also named Jaccard Index, is one of the most fundamental evaluation methods in machine learning. The original IoU computation cannot provide non-zero gradients and thus cannot be directly optimized by nowadays deep learning methods. Several recent works generalized IoU for bounding box regression, but they are not straightforward to adapt for pixelwise prediction. In particular, the original IoU fails to provide effective gradients for the non-overlapping and location-deviation cases, which results in performance plateau. In this paper, we propose PixIoU, a generalized IoU for pixelwise prediction that is sensitive to the distance for non-overlapping cases and the locations in prediction. We provide proofs that PixIoU holds many nice properties as the original IoU. To optimize the PixIoU, we also propose a loss function that is proved to be submodular, hence we can apply the Lovász functions, the efficient surrogates for submodular functions for learning this loss. Experimental results show consistent performance improvements by learning PixIoU over the original IoU for several different pixelwise prediction tasks on Pascal VOC, VOT-2020 and Cityscapes.
APA
Yu, J., Xu, J., Chen, Y., Li, W., Wang, Q., Yoo, B. & Han, J.. (2021). Learning Generalized Intersection Over Union for Dense Pixelwise Prediction. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:12198-12207 Available from https://proceedings.mlr.press/v139/yu21e.html.

Related Material