Network of Excellence Peer-to-Peer Tagged Media

Video Lectures

  • Beyond Relevance (S3MR 2011)

    Finding similar and relevant media content given a user query or sample image has been at the core of the multimedia retrieval community for a long time. In this talk, I will identify and address multimedia challenges that play a role at Yahoo!, and which go beyond relevancy of images and video to a given multimedia retrieval task.The true challenge for multimedia is to find a balance between relevancy, freshness, quality, interestingness and diversity in order to provide an engaging rich media experience to the user.

  • ShareIt: Mining SocialMedia Activities for Detecting Events (S3MR 2011)

    The list of social networking websites is diverse across the globe but the popularity of social media is indisputable. The 640M+ Facebook users, the 480M+ QZone users or the 200M+ Twitter users are used to share observations, opinions and media acting as citizen sensors of the society. This has given an unprecedented access to a vast amount of data on which research communities can perform social media analytics to support socially intelligent applications such as targeted online content delivery, crisis management, organizing revolutions or promoting social development in underdeveloped countries.This lecture will address challenges for building applications using social media. It will focus on the notion of event for structuring the user online activities. We will briefly show how semantic web technologies can be used to break the silos in which each social network platform tends to lock the users. We will then present methods combining semantic inferencing and visual analysis for detecting events from social media activities and for finding automatically media (photos and videos) illustrating events. We will review the numerous works that try to make sense of microposts (e.g. challenges in extracting named entities and role of semantic/background knowledge enhanced techniques) in order to detect events. We will conclude this talk with research questions spanning multidisciplinary fields such as information retrieval, data mining, semantic web and web science communities.

  • Human Sensing, Implicit Tagging, and Implicit Interaction (S3MR 2011)

    Annotation of multimedia data is usually done manually by human annotators or automatically, using algorithms that directly analyze the multimedia data. An alternative to these methods is the use of implicit tagging, where the aim is to observe the user as he/she engages with the multimedia data and infer knowledge about the data by analyzing the user's reactions. Implicit tagging can be performed through various modalities, such as facial expressions, body gestures, EEG analysis, gaze tracking, verbal and non-verbal communication etc.In this talk we we focus on the continuum of human sensing in a sensor-equipped environment for 'off-line tagging' for future retrieval purposes and 'on-line tagging' for real-time support of a user. What can we learn from a user by observing his or her interaction patterns with an application or with another human? And how can we make off-line and on-line use of this knowledge? Sensor-equipped humans will become 'things' (or nodes) in the 'Internet of Things'. They will voluntarily and involuntarily feed the Internet of Things with their behavioral information, allowing the 'Internet of Things' to do behavioral retrieval, but also allowing the 'Internet of Things' to learn more about their life than they may find acceptable.In our talk we will also look at possibilities to deceive the environment, that is, feeding the environment with signals that hide our true feelings and intentions. This is not unusual in daily life, and being able to do so in 'everywhere and always' sensing environments will make it possible to maintain 'naturalness' in our behavior in such environments.

  • Image and Video Tagging in the Internet Era (S3MR 2011)

    Automatically converting visual content into textual description has long been a dream of a number of multimedia, computer vision and machine learning researchers. Image and video tagging, which has been studied heavily in recently years, can be regarded as a more realistic step to that ambitious goal. Especially, the explosion of media data and media users on the Internet, as well as the connections among users, among data and between users and data, bring us both challenges and opportunities. In this lecture, firstly we will review the evolution of multimedia tagging in the past decade, and then introduce state-of-the-art learning based tagging approaches, followed by summarizing manual tagging schemes on the Internet environment and presenting Internet-scale data-driven methods for scalable image and video tagging. Finally we will discuss the roles of models, data and users in multimedia tagging systems and study an sustainable ecosystem for multimedia tagging on the Internet environment. We will also discuss promising research and development directions in this area. 

  • Powering the Next Generation Media Search on the Web (S3MR 2010)

    This talk will address some of the concrete problems that we're facing, as well as some of the solutions proposed to deploy user click feedback with textual, social and visual features in image search.

  • Social Media and Personal Media (S3MR 2010)

    Social media are receiving a great share of interest. Large sites have attracted enormous numbers of users that upload and share digital media such as videos and images but also text as can be found in blogs or wikis. Even though these sites are often well used they may not be the central location where users actually store their content. With regard to photos we rather observe that the personal image collection mainly resides on the own hard disk. However, the sharing of one's own photos reveals much about our photos as well as the social media might well enrich to our personal photos. Consider just the simple fact that you have uploaded a few out of a large set of photos - this would mean that these are very important to me and maybe to others. On the other hand our personal media collections are connected with the experiences shared in the digital social networks - others have been traveling to the same place, taking the same pictures, making similar experiences that can be shared and augment the personal experience. In this presentation, we will share the ideas and approaches of personal photo collections meeting the wisdom of social media.This lecture was given at the S3MR Spring School in 2010.

  • Social Media: Creation, Distribution and Access (S3MR 2010)

    In the last few years, many of us have become active participants in social media. Social media brings in new technology challenges in creation, distribution, and access. Social media is clearly different from personal media and from broadcast media in its goal and usage and in fact is a bridge between the two. In this presentation, first we will discuss emergence of social media and discuss its characteristics in comparison to personal and broadcast media to put it in right user centered perspective. Next we will discuss some emerging approaches for creation, distribution, and access in social media. Finally, we will present some approaches being developed in our group in these areas for popular social media platforms such as Facebook and Twitter.