Learning Visual Semantics: Models, Massive Computation, and Innovative Applications
Tutorial at CVPR 2014
June 23rd, 1:00pm-5:00pm, Columbus, OH
Instructors:
 |
 |
 |
|
| Shih-Fu Chang | John Smith | Rogerio Feris | Liangliang Cao |
Overview:
The explosion of digital multimedia data - including visual content from surveillance cameras, mobile phones, personal photo collections, news footage, or medical images ▒ is creating significant opportunities for automated visual analysis. However, the most interesting content in multimedia files is often unconstrained and complex in nature, reflecting a diversity of human behaviors, scenes, activities, and events, which poses serious challenges for computer vision approaches. In this tutorial, we will present the state-of-the-art on large-scale visual semantic modeling, covering methods for obtaining intuitive mid-level semantic feature representations, while presenting innovative applications. The organizers will share their experience in achieving top performance on several recent competitions, including TRECVID, ImageNet, and ImageCLEF, and developing large-scale data and tool resources.
Outline:
01:00pm: Introduction (Rogerio Feris, pdf slides)
01:20pm: Feature Extraction, Coding, and Pooling (Liangliang Cao, pdf slides)
Brief Introduction to Local Feature Descriptors and Quantization
Focus on Modern Representations based on Sparse Coding and Fisher Vector Coding
Connections to Feature Learning Approaches
02:10pm: Large-Scale Semantic Modeling (John Smith, pdf slides)
Dealing with Class Imbalance
Ensemble Learning / Model-Shared Subspace Boosting
Hierarchical Multi-Faceted Classification
Scaling to Millions of Semantic Unit Models
03:00pm: Coffee Break
03:30pm: Shifting from Naming to Describing: Semantic Attribute Models (Rogerio Feris, pdf slides)
Zero-shot Learning for Recognition and Search
04:15pm: High-level Semantic Modeling (Shih-Fu Chang, pdf slides)
Beyond Semantics: Visual Aesthetics, Interestingness, and Memorability
Emotion Attributes: Visual Sentiment Analysis
05:00pm: Concluding Remarks
Innovative Applications and Datasets:
We will provide pointers to datasets and cover various applications for large-scale visual semantic analysis, including:
Visual Sentiment Analysis - Large-Scale Ontology for Analysis of Emotion and Sentiment in Social Media
Snapshot Serengeti - Large-Scale Classification of Animals in the Wild
Galaxy Zoo - Fine-Grained Classification of Galaxy Morphological Attributes
Medical Imaging - Classification of Medical Image Modality and Anatomy
Smart Surveillance - Search based on Fine-Grained Semantic Attributes of Objects and Scenes.
And many more....
Resources: