Collaborators: Marco Bertini, Giuseppe Serra, Alberto Del Bimbo

Nowadays, almost any photo-sharing service or social network provide tagging functionalities to let users annotate their visual media. The tags are then used to retrieve the uploaded content, and to ease browsing and exploration of these collections. However, not all media are equally tagged by users: using the current browsers is easy to tag a single photo, and even tagging a part of a photo, like a face, has become quite common; on the other hand tagging a video is more complicated and time consuming, so that users just tend to tag the overall content of a video.

We present a method for automatic video annotation that increases the number of tags originally provided by users, and localizes them temporally, associating tags to keyframes. Our approach exploits collective knowledge embedded in user-generated tags and web sources, and visual similarity of keyframes and images uploaded to social sites like YouTube and Flickr, as well as web sources like Google and Bing.

Related publications:

[bibtex file=ballan.bib key=cviu15][bibtex file=ballan.bib key=mm11][bibtex file=ballan.bib key=wsm10]

Datasets:

• MICC-YouTube60: this dataset is composed by 60 web-videos downloaded from YouTube in May 2010; 4 videos for each of the 15 YouTube categories are provided. For getting access to the youtube-dataset please send an e-mail requesting the download link to: lamberto.ballan@unifi.it.

If you use this dataset, please cite the paper: L. Ballan, M. Bertini, A. Del Bimbo, M. Meoni, G. Serra. “Tag suggestion and localization in user-generated videos based on social knowledge,” in Proc. of ACM Multimedia Int’l Workshop on Social Media (WSM),  Firenze, Italy, 2010.

Comments are closed