We focus on computer vision and machine learning. We seek to enable computers to see, think, and learn like humans. Our current interests include:
Action Understanding. Current systems can recognize "what is there" (objects and their locations), but not much beyond that. We want computers to also understand "what is going on", including interactions, intents, causality, and dynamics.
3D Perception. Recovering 3D representations from 2D imagery is a foundation for high-level vision, but remains unsolved in unconstrained settings. We investigate algorithms to perceive 3D (in particular, from a single view) in the wild, making no assumptions about the types of objects or scenes.
Deep Learning and Representation Learning. We develop fundamental representations and architectures for vision and cognition. We study methods to make deep learning more scalable, flexible, and resource-efficient. We also study how to learn like humans, in particular, using strong prior knowledge and few training examples.
Vision, Language, and Cognition. We are interested in the intersection of vision, language, and cognition. In particular, we are interested in equipping computers with common sense, the ability to acquire and represent basic facts about the world and the ability to learn and reason using such knowledge.


Faculty / Principal Investigator: Jia Deng
PhD Students: Yu-Wei Chao . Weifeng Chen . Ankit Goyal . Lanlan Liu . Alejandro Newell . Jonathan Stroud . Dawei Yang . Kaiyu Yang
MS Students: Hei Law . Mingzhe Wang


Computer Science and Engineering
Bob Betty Beyster Building
2260 Hayward St, Ann Arbor, MI 48109-2121


- 8/2016: One paper is accepted to NIPS 2016.
- 7/2016: Two papers are accepted to ECCV 2016.
- 3/2016: We have released a state-of-the-art human pose estimator. Check it out here.
- 3/2016: Congratulations to Jonathan Stroud for winning the NSF Graduate Fellowship Honorable Mention!
- 3/2016: Congratulations to Yu-Wei Chao for winning the Google PhD Fellowship!


Dynamic Deep Neural Networks: Optimizing Accuracy-Efficiency Trade-offs by Selective Execution
Lanlan Liu, Jia Deng
[ paper ]

Associative Embedding: End-to-End Learning for Joint Detection and Grouping
Alejandro Newell, Jia Deng
[ paper ]

Single-Image Depth Perception in the Wild
Weifeng Chen, Zhao Fu, Dawei Yang, Jia Deng
Neural Information Processing Systems (NIPS), 2016
[ paper ] [ data ] [ code ]

Stacked Hourglass Networks for Human Pose Estimation
Alejandro Newell, Kaiyu Yang, Jia Deng
European Conference on Computer Vision (ECCV), 2016
[ paper ] [ code ]

Structured Matching for Phrase Localization
Mingzhe Wang, Mahmoud Azab, Noriyuki Kojima, Rada Mihalcea, Jia Deng
European Conference on Computer Vision (ECCV), 2016
[ paper ] [ code ]

Leveraging the Wisdom of the Crowd for Fine-Grained Recognition
Jia Deng, Jonathan Krause, Michael Stark, Li Fei-Fei.
IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI). 2016.
[ paper ]

Learning to Name Objects
Vicente Ordonez, Wei Liu, Jia Deng, Yejin Choi, Alexander Berg, Tamara Berg
Communications of the ACM. March 2016 (Vol. 59, No. 3).
[ paper ]

HICO: A Benchmark for Recognizing Human-Object Interactions in Images
Yu-Wei Chao, Zhan Wang, Yugeng He, Jiaxuan Wang, Jia Deng
International Conference on Computer Vision (ICCV) 2015
[ paper ] [ data ] [ code ]

Probabilistic Label Relation Graphs with Ising Models
Nan Ding, Jia Deng, Kevin Murphy, Hartmut Neven
International Conference on Computer Vision (ICCV) 2015
[ paper ]

Learning Semantic Relationships for Better Action Retrieval in Images
Vignesh Ramanathan, Congcong Li, Jia Deng, Wei Han, Zhen Li, Kunlong Gu, Yang Song, Samy Bengio, Charles Rosenberg, Fei-Fei Li
In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2015
[ paper] [ project ]

Mining Semantic Affordances of Visual Object Categories
Yu-Wei Chao, Zhan Wang, Rada Mihalcea, Jia Deng
In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2015.
[ paper ] [project ] [ data ] [ code ] [ poster ]

ImageNet Large Scale Visual Recognition Challenge
Olga Russakovsky*, Jia Deng*, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg and Li Fei-Fei. (* = equal contribution)
International Journal of Computer Vision (IJCV), 2015.
[ paper ] [ bibtex ] [ MIT Technology Review ]