Xinlei Chen, Abhinav Shrivastava and Abhinav Gupta at Carnegie Mellon University present NEIL (Never-Ending Image Learner) at ICCV 2013
In December 2013, Xinlei Chen, Abhinav Shrivastava and Abhinav Gupta at Carnegie Mellon University presented NEIL (Never-Ending Image Learner) at ICCV 2013, a continuously running system that autonomously mined semantic relationships between visual concepts from unlabelled web images without human supervision.
Background
Teaching a computer to recognise objects in photos was already hard work by the early 2010s. The standard approach was supervised learning: researchers collected thousands of images, paid people to label each one, and used that labelled data to train a model. Systems trained this way could identify a dog or a chair with reasonable accuracy, but they needed a constant supply of human effort to grow. Every new category meant more labels, more time, more money.
What these systems could not do was learn relationships. Knowing that something is a car is one thing. Knowing that cars have wheels, that wheels are round, that tyres are black, and that this cluster of facts hangs together as common sense about the physical world is something else entirely. That kind of structured knowledge was mostly written by hand into databases, rule by rule. It did not come from looking at images.
The web was full of images. Billions of them, free, covering almost every object and scene a person might encounter. But they carried no labels, and building a system that could learn reliably from that raw, noisy material was an open problem.
What happened
In December 2013, Xinlei Chen, Abhinav Shrivastava and Abhinav Gupta at Carnegie Mellon University’s Robotics Institute presented NEIL (Never-Ending Image Learner) at the IEEE International Conference on Computer Vision (ICCV 2013) in Sydney. NEIL was designed to run continuously, pulling images from the web and extracting structured knowledge from them without human supervision.
The system worked in two interleaved loops. In one, it used image classifiers to identify objects and scenes in web images. In the other, it used what it had already learned to propose relationships between concepts: that a concept seen frequently alongside another probably shares some attribute or part. Those relationships were fed back in to improve the classifiers, and the updated classifiers generated better relationship candidates in the next pass. Each loop made the other more reliable. After running for several months, NEIL had processed millions of images and assembled a large graph of visual relationships, things like “zebras have stripes” and “buses are larger than cars”, drawn not from text or hand-coded rules but from the images themselves.
The approach sat between fully supervised learning, where humans label everything, and fully unsupervised learning, where the system gets no guidance at all. NEIL used a small amount of seed knowledge to get started, then grew its own knowledge base from there. Chen, Shrivastava and Gupta showed that this kind of semi-supervised, never-ending learner could accumulate grounded, common-sense visual knowledge at a scale that manual methods could not match.
Why it mattered
NEIL demonstrated that a computer vision system could bootstrap common-sense visual knowledge, such as 'grass is green' or 'cars have wheels', directly from raw web images at scale, without hand-labelled training data or manually encoded rules. This connected computer vision to the emerging agenda of never-ending learning, in which systems accumulate structured knowledge incrementally rather than being trained on a fixed dataset. The work showed that semi-supervised, continuously running learners could serve as a practical route toward machine common sense grounded in perception.
People
Xinlei Chen, Abhinav Shrivastava, Abhinav Gupta
Organisations
Carnegie Mellon University, Robotics Institute Cmu
Sources
- NEIL: Extracting Visual Knowledge from Web Data.Carnegie Mellon University Robotics Institute.Primary source
- NEIL: Extracting Visual Knowledge from Web Data.IEEE Xplore / IEEE International Conference on Computer Vision (ICCV 2013).Primary source
- NEIL: Extracting Visual Knowledge from Web Data (author copy).Carnegie Mellon University (author page).Secondary
Cite this page
AI Achievements. (2013). Xinlei Chen, Abhinav Shrivastava and Abhinav Gupta at Carnegie Mellon University present NEIL (Never-Ending Image Learner) at ICCV 2013. Retrieved 2026-08-22, from https://achievements.ai/milestone/never-ending-image-learner-neil
@misc{achievements_never_ending_image_learner_neil,
title = {Xinlei Chen, Abhinav Shrivastava and Abhinav Gupta at Carnegie Mellon University present NEIL (Never-Ending Image Learner) at ICCV 2013},
author = {{AI Achievements}},
year = {2013},
url = {https://achievements.ai/milestone/never-ending-image-learner-neil}
}