Semantic Information Pursuit for Multimodal Data Analysis

MURI: Semantic Information Pursuit for Multimodal Data Analysis

About

In 1948 Shanon laid the foundations of information theory which revolutionized statistics, physics, engineering, and computer science. A strong limitation, however, is that the semantic content of data is not taken into account. This research will produce a novel framework for characterizing semantic information content in multimodal data. It will combine non-convex optimization with advanced statistical methods, leading to new representation learning algorithms, measures of uncertainty, and sampling methods. Given data from a scene, the algorithms will be able find the most informative representations of the data for a specific task. These methods will be applied to complex datasets from real situations, including text, images, videos, sensor signals, and cameras, resulting in intelligent decision based algorithms. 

Current work 

Our group is working on characertizing uncertainty in multimodal representations. We will develop a statistical framework for characterizing the uncertainty of the information representations using both frequentists and Bayesian approaches. We will also develop efficient statistical sampling methods, which will be useful for both characterizing uncertainty and performing inference in the information pursuit framework.

Grant websites

Co-Principal Investigators