Real-time On-line Learning of Transformed Hidden Markov Models from Video

Nemanja Petrovic, Nebojsa Jojic, Brendan J. Frey, Thomas S. Huang
Proceedings of the Ninth International Workshop on Artificial Intelligence and Statistics, PMLR R4:225-232, 2003.

Abstract

The transformed hidden Markov model is a temporal model that captures three typical causes of variability in video - scene/object class, appearance variability within the class, and image motion. In our previous work, we showed that an exact EM algorithm can jointly learn the appearances of multiple objects and/or poses of an object, and track the objects or camera motion in video, starting simply from random initialization. As such, this model can serve as a basis for both video clustering and object tracking applications. However, the original algorithm requires a significant amount of computation that renders it impractical for video clustering and its off-line nature makes it unsuitable for real-time tracking applications. In this paper, we propose a new, significantly faster, on-line learning algorithm that enables real-time clustering and tracking. We demonstrate that the algorithm can extract objects using the constraints on their motion and also perform tracking while the appearance models are learned. We also demonstrate the clustering results on an example of typical unrestricted personal media - the vacation video.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR4-petrovic03a, title = {Real-time On-line Learning of Transformed Hidden Markov Models from Video}, author = {Petrovic, Nemanja and Jojic, Nebojsa and Frey, Brendan J. and Huang, Thomas S.}, booktitle = {Proceedings of the Ninth International Workshop on Artificial Intelligence and Statistics}, pages = {225--232}, year = {2003}, editor = {Bishop, Christopher M. and Frey, Brendan J.}, volume = {R4}, series = {Proceedings of Machine Learning Research}, month = {03--06 Jan}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/r4/petrovic03a/petrovic03a.pdf}, url = {https://proceedings.mlr.press/r4/petrovic03a.html}, abstract = {The transformed hidden Markov model is a temporal model that captures three typical causes of variability in video - scene/object class, appearance variability within the class, and image motion. In our previous work, we showed that an exact EM algorithm can jointly learn the appearances of multiple objects and/or poses of an object, and track the objects or camera motion in video, starting simply from random initialization. As such, this model can serve as a basis for both video clustering and object tracking applications. However, the original algorithm requires a significant amount of computation that renders it impractical for video clustering and its off-line nature makes it unsuitable for real-time tracking applications. In this paper, we propose a new, significantly faster, on-line learning algorithm that enables real-time clustering and tracking. We demonstrate that the algorithm can extract objects using the constraints on their motion and also perform tracking while the appearance models are learned. We also demonstrate the clustering results on an example of typical unrestricted personal media - the vacation video.}, note = {Reissued by PMLR on 01 April 2021.} }
Endnote
%0 Conference Paper %T Real-time On-line Learning of Transformed Hidden Markov Models from Video %A Nemanja Petrovic %A Nebojsa Jojic %A Brendan J. Frey %A Thomas S. Huang %B Proceedings of the Ninth International Workshop on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2003 %E Christopher M. Bishop %E Brendan J. Frey %F pmlr-vR4-petrovic03a %I PMLR %P 225--232 %U https://proceedings.mlr.press/r4/petrovic03a.html %V R4 %X The transformed hidden Markov model is a temporal model that captures three typical causes of variability in video - scene/object class, appearance variability within the class, and image motion. In our previous work, we showed that an exact EM algorithm can jointly learn the appearances of multiple objects and/or poses of an object, and track the objects or camera motion in video, starting simply from random initialization. As such, this model can serve as a basis for both video clustering and object tracking applications. However, the original algorithm requires a significant amount of computation that renders it impractical for video clustering and its off-line nature makes it unsuitable for real-time tracking applications. In this paper, we propose a new, significantly faster, on-line learning algorithm that enables real-time clustering and tracking. We demonstrate that the algorithm can extract objects using the constraints on their motion and also perform tracking while the appearance models are learned. We also demonstrate the clustering results on an example of typical unrestricted personal media - the vacation video. %Z Reissued by PMLR on 01 April 2021.
APA
Petrovic, N., Jojic, N., Frey, B.J. & Huang, T.S.. (2003). Real-time On-line Learning of Transformed Hidden Markov Models from Video. Proceedings of the Ninth International Workshop on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research R4:225-232 Available from https://proceedings.mlr.press/r4/petrovic03a.html. Reissued by PMLR on 01 April 2021.

Related Material