We gave a tutorial on “Image Tag Assignment, Refinement and Retrieval” at ACM MM 2015, based on our recent survey. Our tutorial focuses on challenges and solutions for content-based image retrieval in the context of online image sharing and tagging. We present a unified review on three closely linked problems: tag assignment, tag refinement, and tag-based image retrieval. We introduce a taxonomy to structure the growing literature, understand the ingredients of the main works, and recognize their merits and limitations.
We provided also an hands-on session with the main methods, software and datasets. All data, code and slides are online at: http://www.micc.unifi.it/tagsurvey
Andy Bagdanov and I are organizing a paper reading group on Multimedia and Vision at the MICC of University of Florence.
We plan a meeting once every three weeks (approximately), usually from 12.00 to 13.30. The schedule of our meetings and the material are available on this page.
Our paper “Commercials and Trademarks Recognition” has been accepted as book chapter in TV Content Analysis: Techniques and Applications that will be published by CRC Press, Taylor & Francis group, on March 2012.
Book summary: TV content is currently available through various communication channels and devices, including digital TV, mobile TV, and Internet TV. However, with the increase in TV content volume, both its management and consumption become more and more challenging. Thoroughly describing TV program analysis techniques, this book explores the systems, architectures, algorithms, applications, research results, new approaches, and open issues. Leading experts address a wide variety of related subject areas and present a scientifically sound treatment of state-of-the-art techniques for readers interested or involved in TV program analysis.
Our paper entitled “Enriching and Localizing Semantic Tags in Internet Videos” has been accepted by ACM Multimedia 2011.
Tagging of multimedia content is becoming more and more widespread as web 2.0 sites, like Flickr and Facebook for images, YouTube and Vimeo for videos, have popularized tagging functionalities among their users. These user-generated tags are used to retrieve multimedia content, and to ease browsing and exploration of media collections, e.g. using tag clouds. However, not all media are equally tagged by users: using the current browsers is easy to tag a single photo, and even tagging a part of a photo, like a face, has become common in sites like Flickr and Facebook; on the other hand tagging a video sequence is more complicated and time consuming, so that users just tend to tag the overall content of a video.
In this paper we present a system for automatic video annotation that increases the number of tags originally provided by users, and localizes them temporally, associating tags to shots. This approach exploits collective knowledge embedded in tags and Wikipedia, and visual similarity of keyframes and images uploaded to social sites like YouTube and Flickr. Our paper is now available online.
I have submitted my PhD thesis: “Object and event recognition in multimedia archives using local visual features” (supervisors: Prof. Alberto Del Bimbo and Dr. Marco Bertini). The dissertation will be defended on April 21, 2011. The thesis committee is comprised of three members: Prof. Enrico Vicario (Univ. of Florence, ING-INF/05), Prof. Giuliano Benelli (Univ. of Siena, ING-INF/04), Prof. Marco Scarpa (Univ. of Messina, INF/01).
“The digital revolution has converted old, analog technologies into a digital format. In this context, due to the widespread availability of personal and professional imaging devices, the low cost of multimedia storage and ease of content transmission and sharing, the need to automatically analyze and organize large amounts of visual data becomes more and more prominent. But although data processing capabilities of machines are truly impressive if compared to a human, data interpretation skills are very poor. It is mainly due to the fact that machines can only compute low level properties of data that have no clear relation with high level conceptual semantics. We present in this thesis a step-by-step methodology to reduce this semantic gap and to achieve automatic annotation and retrieval of visual content. This task may consist of determining whether the visual data contains some specific property, object or activity. […]”
I report in this page also the latex template used for my thesis at MICC (now it is the standard in our lab) and a zip file with the cover template (useful to produce a cool book in 17×24 format).
Our paper “Event detection and recognition for semantic annotation of video” was accepted for publication by Springer International Journal of Multimedia Tools and Applications (MTAP) in the special issue “Survey papers in Multimedia by World Experts”. The paper is available online now (see publications) and it is also available on SpringerLink (DOI).
In this paper we survey the field of event recognition, from interest point detectors and descriptors, to event modelling techniques and knowledge management technologies. We provide an overview of the methods, categorising them according to video production methods and video domains, and according to types of events and actions that are typical of these domains.
Our paper on “Tag suggestion and localization in user-generated videos based on social knowledge” won the best paper award at ACM SIGMM Workshop on Social Media (WSM’10) in conjunction with ACM Multimedia 2010.
Nowadays, almost any web site that provides means for sharing user-generated multimedia content, like Flickr, Facebook, YouTube and Vimeo, has tagging functionalities to let users annotate the material that they want to share. The tags are then used to retrieve the uploaded content, and to ease browsing and exploration of these collections, e.g. using tag clouds. However, while tagging a single image is straightforward, and sites like Flickr and Facebook allow also to tag easily portions of the uploaded photos, tagging a video sequence is more cumbersome, so that users just tend to tag the overall content of a video. While research on image tagging has received a considerable attention in the latest years, there are still very few works that address the problem of automatically assigning tags to videos, locating them temporally within the video sequence.
In this paper we present a system for video tag suggestion and temporal localization based on collective knowledge and visual similarity of frames. The algorithm suggests new tags that can be associated to a given keyframe exploiting the tags associated to videos and images uploaded to social sites like YouTube and Flickr and visual features. Our paper is now available online.
From April 6 to June 30, 2010, I will be a visiting PhD student at Telecom Paristech in Paris (France). Telecom Paristech (also known as ENST) is one of the most prestigious and selective grandes écoles in France and one of the finest institutions in the field of Telecommunications.
Together with Giuseppe Serra, we will work in the Image Processing and Interpretation (TII) group in the department of Signal and Image Processing (TSI), collaborating with Dr. Hichem Sahbi.
Our paper on “Video Annotation and Retrieval Using Ontologies and Rule Learning” was accepted for publication by the International IEEE MultiMedia Magazine.
In this paper we present an approach for automatic annotation and retrieval of video content, based on ontologies and semantic concept classifiers. A novel rule-based method is used to describe and recognize composite concepts and events. Our algorithm learns automatically rules expressed in Semantic Web Rules Language (SWRL), exploiting the knowledge embedded into the ontology. Concepts’ relationship of co-occurrence and the temporal consistency of video data are used to improve the performance of individual concept detectors. Finally, we present a web video search engine, based on ontologies, that permits queries using a composition of boolean and temporal relations between concepts.
MICC laboratories, Florence, 21st October 2009 (10.30-13.30). Course on Multimedia Databases (DBMM) – laboratory lecture.
- Goal: logo recognition in web images.
- Dataset/testset: find 2 different logos vs 100 images.
- Evaluation metrics: recognition performances will be evaluated in terms of Precision and Recall.
Tutors: Lamberto Ballan, Lorenzo Seidenari.
Download slides 
(with references) | Download Software & Dataset 
Comments Off on ACM MM’15 Tutorial on Image Tag Assignment, Refinement and Retrieval 
We gave a tutorial on “Image Tag Assignment, Refinement and Retrieval” at ACM MM 2015, based on our recent survey. Our tutorial focuses on challenges and solutions for content-based image retrieval in the context of online image sharing and tagging. We present a unified review on three closely linked problems: tag assignment, tag refinement, and tag-based image retrieval. We introduce a taxonomy to structure the growing literature, understand the ingredients of the main works, and recognize their merits and limitations.
Themocracy