Scientific research and development established at our laboratory focuses on various aspects of video coding, image and video analysis, and multimedia security. The research is funded largely by the U.S. Government grants, and in part by the internal FAU funds. The following is a summary of our research and development efforts at MLAB.

3D and multi-view video coding. Efficient coding of sequences corresponding to the same scene that is captured with multiple cameras represents an emerging area of research. We are currently studying how to effectively code 3D and multi-view sequences by exploiting the additional spatiotemporal redundancies that are naturally present in such sequences. In addition, we are also exploiting properties of human visual system to code different views with different qualities and still keep a high-quality visual perception.

Video coding for mobile applications. Limited computing environment and power awareness represent important aspects that need to be addressed in a typical mobile application. Our research in this area is focused on a design and evaluation of innovative techniques and algorithms used to ease the computational requirements on the mobile client side. Methodologies and tools to estimate the power consumption due to video playback on a given architecture are also being investigated.

Transcoding. The wide use of the MPEG-2 video today and the expected adoption of H.264 creates a need for tools and techniques for MPEG-2 to H.264 video transcoding. We are developing a transcoder that uses the MPEG-2 macroblock modes, DCT coefficients, and motion vectors, for direct estimation of H.264 coding modes and prediction modes for inter and intra macroblocks.

Object segmentation and tracking. The problem of video object segmentation and tracking in sequences with complex, moving background is a challenging one. Several approaches for foreground object detection based on Bayesian decision framework and neural networks are being developed for standard and high resolution (HDTV and quad-HDTV) sequences. We study how to combine a particular foreground object detection method with a graph-based image color segmentation technique to improve the segmentation tracking accuracy. Also we investigate the potential of enhancing the segmentation and tracking results using the additional depth information contained in stereo or multi-view sequences.

Stereo matching and 3D reconstruction. In the area of stereo matching, we are studying how the computed disparity map of a pair of stereo images can be used for higher level tasks such as scene understanding, classification and tracking. Furthermore, our research is directed towards developing effective methods and tools for reconstructing a 3D model of the scene from stereo and multi-view images and video frames in order to assist the higher level tasks using the gathered 3D information.

Perceptually-motivated multimedia analysis. Modeling the behavior of a part of the human brain responsible for visual perception is an incredibly complex task. Such biologically-inspired models can be used effectively in many multimedia applications. At MLAB, we are studying how biologically-inspired models based on visual attention can help determining the regions of interests (ROIs) within an images or video frames in an unsupervised environment, such as automated content-based image retrieval (CBIR) system.

3D video playback in autostereoscopic displays. The current generation of autostereoscopic displays cannot support the quad-HDTV resolutions and video has to be down sampled. A 3D video player for such displays robust to camera vibrations and capable of a real-time down sampling and interpolation is being developed at MLAB.

Retrieval. We designed and implemented a content-based image retrieval (CBIR) system with relevance feedback capabilities. It supports browsing, query-by-example, and two different relevance feedback modes that allow users to refine their queries. This work is being extended to a distributed, Web-based, scenario, with added semantic image annotation, search and retrieval capabilities. More recently, we study how biologically-motivated CBIR methods can enhance the retrieval results.

Secure multimedia communications. In the area of secure multimedia communications we are studying the impact of network intrusion detection techniques in the design and operation of secure and robust multimedia communication systems. Furthermore, we investigate possible improvements on the performance of Intrusion Detection Systems (IDSs), thanks to additional knowledge of multimedia traffic and data.

Video encryption and steganography. Many application-related requirements of a typical video system are ignored by conventional cryptography. We focus on studying encryption and steganography for digital videos that support these requirements. A tool (SimViKi) for evaluation of video encryption methods is also being developed at MLAB.

Biometrics-based authentication. An emerging research area of biometrics-based authentication is studied at MLAB. We investigate various issues and challenges in authentication systems that are based on user’s biometric data, in context of content protection and digital rights management (DRM) for multimedia.