Pinview: Implicit Feedback in Content-Based Image Retrieval

Peter Auer, Zakria Hussain, Samuel Kaski, Arto Klami, Jussi Kujala, Jorma Laaksonen, Alex P. Leung, Kitsuchart Pasupa, John Shawe-Taylor
Proceedings of the First Workshop on Applications of Pattern Analysis, PMLR 11:51-57, 2010.

Abstract

This paper describes Pinview, a content-based image retrieval system that exploits implicit relevance feedback during a search session. Pinview contains several novel methods that infer the intent of the user. From relevance feedback, such as eye movements or clicks, and visual features of images Pinview learns a similarity metric between images which depends on the current interests of the user. It then retrieves images with a specialized reinforcement learning algorithm that balances the tradeoff between exploring new images and exploiting the already inferred interests of the user. In practise, we have integrated Pinview to the content-based image retrieval system PicSOM, in order to apply it to real-world image databases. Preliminary experiments show that eye movements provide a rich input modality from which it is possible to learn the interests of the user.

Cite this Paper

BibTeX
@InProceedings{pmlr-v11-auer10a, title = {Pinview: Implicit Feedback in Content-Based Image Retrieval}, author = {Auer, Peter and Hussain, Zakria and Kaski, Samuel and Klami, Arto and Kujala, Jussi and Laaksonen, Jorma and Leung, Alex P. and Pasupa, Kitsuchart and Shawe-Taylor, John}, booktitle = {Proceedings of the First Workshop on Applications of Pattern Analysis}, pages = {51--57}, year = {2010}, editor = {Diethe, Tom and Cristianini, Nello and Shawe-Taylor, John}, volume = {11}, series = {Proceedings of Machine Learning Research}, address = {Cumberland Lodge, Windsor, UK}, month = {01--03 Sep}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v11/auer10a/auer10a.pdf}, url = {https://proceedings.mlr.press/v11/auer10a.html}, abstract = {This paper describes Pinview, a content-based image retrieval system that exploits implicit relevance feedback during a search session. Pinview contains several novel methods that infer the intent of the user. From relevance feedback, such as eye movements or clicks, and visual features of images Pinview learns a similarity metric between images which depends on the current interests of the user. It then retrieves images with a specialized reinforcement learning algorithm that balances the tradeoff between exploring new images and exploiting the already inferred interests of the user. In practise, we have integrated Pinview to the content-based image retrieval system PicSOM, in order to apply it to real-world image databases. Preliminary experiments show that eye movements provide a rich input modality from which it is possible to learn the interests of the user.} }
Endnote
%0 Conference Paper %T Pinview: Implicit Feedback in Content-Based Image Retrieval %A Peter Auer %A Zakria Hussain %A Samuel Kaski %A Arto Klami %A Jussi Kujala %A Jorma Laaksonen %A Alex P. Leung %A Kitsuchart Pasupa %A John Shawe-Taylor %B Proceedings of the First Workshop on Applications of Pattern Analysis %C Proceedings of Machine Learning Research %D 2010 %E Tom Diethe %E Nello Cristianini %E John Shawe-Taylor %F pmlr-v11-auer10a %I PMLR %P 51--57 %U https://proceedings.mlr.press/v11/auer10a.html %V 11 %X This paper describes Pinview, a content-based image retrieval system that exploits implicit relevance feedback during a search session. Pinview contains several novel methods that infer the intent of the user. From relevance feedback, such as eye movements or clicks, and visual features of images Pinview learns a similarity metric between images which depends on the current interests of the user. It then retrieves images with a specialized reinforcement learning algorithm that balances the tradeoff between exploring new images and exploiting the already inferred interests of the user. In practise, we have integrated Pinview to the content-based image retrieval system PicSOM, in order to apply it to real-world image databases. Preliminary experiments show that eye movements provide a rich input modality from which it is possible to learn the interests of the user.
RIS
TY - CPAPER TI - Pinview: Implicit Feedback in Content-Based Image Retrieval AU - Peter Auer AU - Zakria Hussain AU - Samuel Kaski AU - Arto Klami AU - Jussi Kujala AU - Jorma Laaksonen AU - Alex P. Leung AU - Kitsuchart Pasupa AU - John Shawe-Taylor BT - Proceedings of the First Workshop on Applications of Pattern Analysis DA - 2010/09/30 ED - Tom Diethe ED - Nello Cristianini ED - John Shawe-Taylor ID - pmlr-v11-auer10a PB - PMLR DP - Proceedings of Machine Learning Research VL - 11 SP - 51 EP - 57 L1 - http://proceedings.mlr.press/v11/auer10a/auer10a.pdf UR - https://proceedings.mlr.press/v11/auer10a.html AB - This paper describes Pinview, a content-based image retrieval system that exploits implicit relevance feedback during a search session. Pinview contains several novel methods that infer the intent of the user. From relevance feedback, such as eye movements or clicks, and visual features of images Pinview learns a similarity metric between images which depends on the current interests of the user. It then retrieves images with a specialized reinforcement learning algorithm that balances the tradeoff between exploring new images and exploiting the already inferred interests of the user. In practise, we have integrated Pinview to the content-based image retrieval system PicSOM, in order to apply it to real-world image databases. Preliminary experiments show that eye movements provide a rich input modality from which it is possible to learn the interests of the user. ER -
APA
Auer, P., Hussain, Z., Kaski, S., Klami, A., Kujala, J., Laaksonen, J., Leung, A.P., Pasupa, K. & Shawe-Taylor, J.. (2010). Pinview: Implicit Feedback in Content-Based Image Retrieval. Proceedings of the First Workshop on Applications of Pattern Analysis, in Proceedings of Machine Learning Research 11:51-57 Available from https://proceedings.mlr.press/v11/auer10a.html.

Related Material