Our paper entitled “Enriching and Localizing Semantic Tags in Internet Videos” has been accepted by ACM Multimedia 2011.
Tagging of multimedia content is becoming more and more widespread as web 2.0 sites, like Flickr and Facebook for images, YouTube and Vimeo for videos, have popularized tagging functionalities among their users. These user-generated tags are used to retrieve multimedia content, and to ease browsing and exploration of media collections, e.g. using tag clouds. However, not all media are equally tagged by users: using the current browsers is easy to tag a single photo, and even tagging a part of a photo, like a face, has become common in sites like Flickr and Facebook; on the other hand tagging a video sequence is more complicated and time consuming, so that users just tend to tag the overall content of a video.
In this paper we present a system for automatic video annotation that increases the number of tags originally provided by users, and localizes them temporally, associating tags to shots. This approach exploits collective knowledge embedded in tags and Wikipedia, and visual similarity of keyframes and images uploaded to social sites like YouTube and Flickr. Our paper is now available online.
I am involved in the technical program committee of the 1st International Workshop on Computer Vision Methods in Blind Image Forensics (CVBIF), in conjunction with ICCV 2011.
The verification of original images, as well as the detection of manipulations in digital images and multimedia content has become an increasingly important topic. The purpose of this workshop is to bring together leading experts from image forensics and the computer vision community. Its goal is to foster new vision-based approaches to image forensics problems and thus promote the advancement of vision-based solutions in forensics applications. Download a PDF version of the call for papers here!
The paper “A SIFT-based forensic method for copy-move attack detection and transformation recovery” by I. Amerini, L. Ballan, R. Caldelli, A. Del Bimbo, and G. Serra is now officially accepted for publication by the IEEE Transactions on Information Forensics and Security.
One of the principal problems in image forensics is determining if a particular image is authentic or not. This can be a crucial task when images are used as basic evidence to influence judgment like, for example, in a court of law. To carry out such forensic analysis, various technological instruments have been developed in the literature.
In this paper the problem of detecting if an image has been forged is investigated; in particular, attention has been paid to the case in which an area of an image is copied and then pasted onto another zone to create a duplication or to cancel something that was awkward. Generally, to adapt the image patch to the new context a geometric transformation is needed. To detect such modifications, a novel methodology based on Scale Invariant Features Transform (SIFT) is proposed. Such a method allows both to understand if a copy-move attack has occurred and, furthermore, to recover the geometric transformation used to perform cloning. Extensive experimental results are presented to confirm that the technique is able to precisely individuate the altered area and, in addition, to estimate the geometric transformation parameters with high reliability. The method also deals with multiple cloning.
More information about this project (there are also links to datasets used in the experiments) are available on this page.
I have submitted my PhD thesis: “Object and event recognition in multimedia archives using local visual features” (supervisors: Prof. Alberto Del Bimbo and Dr. Marco Bertini). The dissertation will be defended on April 21, 2011. The thesis committee is comprised of three members: Prof. Enrico Vicario (Univ. of Florence, ING-INF/05), Prof. Giuliano Benelli (Univ. of Siena, ING-INF/04), Prof. Marco Scarpa (Univ. of Messina, INF/01).
“The digital revolution has converted old, analog technologies into a digital format. In this context, due to the widespread availability of personal and professional imaging devices, the low cost of multimedia storage and ease of content transmission and sharing, the need to automatically analyze and organize large amounts of visual data becomes more and more prominent. But although data processing capabilities of machines are truly impressive if compared to a human, data interpretation skills are very poor. It is mainly due to the fact that machines can only compute low level properties of data that have no clear relation with high level conceptual semantics. We present in this thesis a step-by-step methodology to reduce this semantic gap and to achieve automatic annotation and retrieval of visual content. This task may consist of determining whether the visual data contains some specific property, object or activity. […]”
I report in this page also the latex template used for my thesis at MICC (now it is the standard in our lab) and a zip file with the cover template (useful to produce a cool book in 17×24 format).
Our paper “Event detection and recognition for semantic annotation of video” was accepted for publication by Springer International Journal of Multimedia Tools and Applications (MTAP) in the special issue “Survey papers in Multimedia by World Experts”. The paper is available online now (see publications) and it is also available on SpringerLink (DOI).
In this paper we survey the field of event recognition, from interest point detectors and descriptors, to event modelling techniques and knowledge management technologies. We provide an overview of the methods, categorising them according to video production methods and video domains, and according to types of events and actions that are typical of these domains.
Our paper on “Tag suggestion and localization in user-generated videos based on social knowledge” won the best paper award at ACM SIGMM Workshop on Social Media (WSM’10) in conjunction with ACM Multimedia 2010.
Nowadays, almost any web site that provides means for sharing user-generated multimedia content, like Flickr, Facebook, YouTube and Vimeo, has tagging functionalities to let users annotate the material that they want to share. The tags are then used to retrieve the uploaded content, and to ease browsing and exploration of these collections, e.g. using tag clouds. However, while tagging a single image is straightforward, and sites like Flickr and Facebook allow also to tag easily portions of the uploaded photos, tagging a video sequence is more cumbersome, so that users just tend to tag the overall content of a video. While research on image tagging has received a considerable attention in the latest years, there are still very few works that address the problem of automatically assigning tags to videos, locating them temporally within the video sequence.
In this paper we present a system for video tag suggestion and temporal localization based on collective knowledge and visual similarity of frames. The algorithm suggests new tags that can be associated to a given keyframe exploiting the tags associated to videos and images uploaded to social sites like YouTube and Flickr and visual features. Our paper is now available online.
From April 6 to June 30, 2010, I will be a visiting PhD student at Telecom Paristech in Paris (France). Telecom Paristech (also known as ENST) is one of the most prestigious and selective grandes écoles in France and one of the finest institutions in the field of Telecommunications.
Together with Giuseppe Serra, we will work in the Image Processing and Interpretation (TII) group in the department of Signal and Image Processing (TSI), collaborating with Dr. Hichem Sahbi.
Our paper on “Video Annotation and Retrieval Using Ontologies and Rule Learning” was accepted for publication by the International IEEE MultiMedia Magazine.
In this paper we present an approach for automatic annotation and retrieval of video content, based on ontologies and semantic concept classifiers. A novel rule-based method is used to describe and recognize composite concepts and events. Our algorithm learns automatically rules expressed in Semantic Web Rules Language (SWRL), exploiting the knowledge embedded into the ontology. Concepts’ relationship of co-occurrence and the temporal consistency of video data are used to improve the performance of individual concept detectors. Finally, we present a web video search engine, based on ontologies, that permits queries using a composition of boolean and temporal relations between concepts.
MICC laboratories, Florence, 21st October 2009 (10.30-13.30). Course on Multimedia Databases (DBMM) – laboratory lecture.
- Goal: logo recognition in web images.
- Dataset/testset: find 2 different logos vs 100 images.
- Evaluation metrics: recognition performances will be evaluated in terms of Precision and Recall.
Tutors: Lamberto Ballan, Lorenzo Seidenari.
Download slides 
(with references) | Download Software & Dataset 
Our paper on “Video Event Classification using String Kernels“ was accepted for publication by Springer International Journal on Multimedia Tools and Applications (MTAP) in the special issue on Content-Based Multimedia Indexing.
In this paper we present a method to introduce temporal information for video event recognition within the bag-of-words (BoW) approach. Events are modeled as a sequence composed of histograms of visual features, computed from each frame using the traditional BoW. The sequences are treated as strings phrases where each histogram is considered as a character. Event classification of these sequences of variable length, depending on the duration of the video clips, are performed using SVM classifiers with a string kernel that uses the Needlemann-Wunsch edit distance.
Comments Off on Enriching and Localizing Semantic Tags in Internet Videos 
Themocracy