Summary: Researchers have developed MovieNet, an AI design inspired by the human mind, to comprehend and analyze moving pictures with unparalleled accuracy. MovieNet uses considerably less data and energy than standard AI to mimic how neurons interpret visual sequences and detect subtle changes in powerful scenes.
In terms of recognizing behavioural designs, such as caterpillar swimming under various circumstances, MovieNet outperformed existing AI models and even individual observers in testing. The breakthrough’s transformative potential and its potential to revolutionize areas like healthcare and drug testing are highlighted by its eco-friendly design.
Important Information:
- Brain-Like Processing: MovieNet resembles neurons to process video patterns with great precision, distinguishing active scenes better than traditional AI versions.
- High Performance: MovieNet achieves outstanding reliability while using less power and data, making it more responsible and flexible for different applications.
- Medical Potential: The AI may help in early detection of disorders like Parkinson’s by identifying simple changes in activity, as well as enhancing drug testing methods.
Origin: Scripps Research Institute
Imagine a model of artificial intelligence ( AI ) that can watch and interpret moving images with human-like subtlety.
Scientists at Scripps Research have created MovieNet, an impressive Artificial that interprets videos in a manner that mirrors how our brains interpret real-world events as they pass through time, to fulfill this dream.
This brain-inspired AI design, detailed in a research published in the , Proceedings of the National Academy of Sciences , on November 19, 2024, you consider moving moments by simulating how neurons—or head cells—make real-time perception of the world.
MovieNet introduces a strategy for machine-learning models to identify sophisticated, changing scenes, a breakthrough that could change fields from clinical diagnostics to intelligent driving, where it is crucial to recognize subtle changes over time. Conventional AI excels at recognizing still images.
MovieNet is also more accurate and sustainable than conventional AI, in addition.
” The brain doesn’t just see still frames, it creates an ongoing visual narrative”, says senior author , Hollis Cline, PhD, the director of the Dorris Neuroscience Center and the Hahn Professor of Neuroscience at Scripps Research.
” Static image recognition has come a long way, but the brain’s capacity to process flowing scenes—like watching a movie—requires a much more sophisticated form of pattern recognition. We have applied the same principles to AI by studying how neurons can capture these sequences.
To create MovieNet, Cline and first author Masaki Hiramoto, a staff scientist at Scripps Research, examined how the brain processes real-world scenes as short sequences, similar to movie clips. Specifically, the researchers studied how tadpole neurons responded to visual stimuli.
” Tadpoles have a very good visual system, plus we know that they can detect and respond to moving stimuli efficiently”, explains Hiramoto.
He and Cline found neurons that can recognize objects as they move and change and respond to movie-like characteristics like brightness shifts and image rotation shifts. These neurons are housed in the optic tectum, the area where the brain’s visual processing unit gathers moving image components into a coherent sequence.
Think of this process as a lenticular puzzle: each piece by itself may not make sense, but combined they create a complete image in motion.
Different neurons process various “puzzle pieces” of a real-life moving image, which the brain then integrates into a continuous scene.
The researchers also discovered that the tadpoles ‘ optic tectum neurons, which captured information in dynamic clips lasting between 100 and 600 milliseconds, could detect subtle changes in visual stimuli over time.
These neurons have a high level of sensitivity to patterns of light and shadow, and each neuron’s response to a particular area of the visual field aids in the creation of a detailed map of a scene into the shape of a “movie clip.”
Cline and Hiramoto taught MovieNet to emulate this brain-like processing and encrypt video clips as a series of small, recognizable visual cues. This made it possible for the AI model to distinguish subtle variations between dynamic scenes.
To test MovieNet, the researchers showed it video clips of tadpoles swimming under different conditions.
MovieNet not only exceeded the abilities of trained human observers by about 18 %, but it also achieved 82.3 % accuracy in distinguishing normal from abnormal swimming behaviors. Even with its extensive training and processing resources, it outperformed current AI models like Google’s GoogLeNet, which had only 72 percent accuracy.
” This is where we saw real potential”, points out Cline.
The team came to the conclusion that MovieNet saved less time and data than current AI models by understanding changing scenes even better.
MovieNet stands out from conventional AI thanks to its ability to simplify data without sacrificing accuracy. MovieNet effectively compresses data like a zipped file that retains crucial details by breaking down visual information into necessary sequences.
Beyond its high accuracy, MovieNet is an eco-friendly AI model. The processing of conventional AI uses a lot of energy and leaves a lot of waste. A greener alternative that performs at a high standard while maintaining its reduced data requirements is offered by MovieNet.
” By mimicking the brain, we’ve managed to make our AI far less demanding, paving the way for models that aren’t just powerful but sustainable”, says Cline. This efficiency also allows for the expansion of AI in fields where conventional methods are expensive.
In addition, MovieNet has potential to reshape medicine. As the technology develops, it might be able to identify subtle changes in early stages of disease, such as detecting irregular heart rhythms or identifying the first warning signs of neurodegenerative conditions like Parkinson’s.
For instance, the AI might detect subtle motor changes that are frequently invisible to human eyes early, giving doctors time to take the necessary steps.
Additionally, MovieNet’s ability to identify changes in tadpole swimming patterns as a result of better drug-switching techniques, as researchers can study dynamic cellular responses rather than static snapshots.
” Current methods miss crucial changes because they can only analyze images captured at regular intervals,” says Hiramoto.
MovieNet can track the subtlest changes in a drug test by monitoring cells over time.
Looking ahead, Cline and Hiramoto plan to continue refining MovieNet’s ability to adapt to different environments, enhancing its versatility and potential applications.
” Taking inspiration from biology will continue to be a fertile area for advancing AI,” says Cline. We can achieve levels of efficiency that aren’t possible with conventional approaches by creating models that think like living organisms.
Funding: This work for the study” Movie recognition AI is enabled by the identification of movie encoding neurons.”, was supported by funding from the National Institutes of Health ( RO1EY011261, RO1EY027437 and RO1EY031597 ),  , the Hahn Family Foundation and the Harold L. Dorris Neurosciences Center Endowment Fund.
About this news about AI research
Author: Press Office
Source: Scripps Research Institute
Contact: Press Office – Scripps Research Institute
Image: The image is credited to Neuroscience News
Original Research: Open access.
Hollis Cline et al.,” Identifying movie encoding neurons enables movie recognition AI.” PNAS
Abstract
Movie recognition AI is enabled by the identification of movie encoding neurons.
Although spatiotemporal image dynamics dominates natural visual scenes, how the visual system integrates “movie” information over time is not well understood.
We used sparse noise stimuli and reverse correlation analysis to characterize optic tectal neuronal receptive fields.
Neurons recognized movies of ~200-600 ms durations with defined start and stop stimuli. Through a hierarchical algorithm, the duration of the movies from start to stop responses were tuned by sensory experience.
Following trigonometric functions, neurons encoded families of image sequences. Spike sequence and information flow suggest that repetitive circuit motifs underlie movie identification.
In machine learning networks for static image recognition, the principles of movie encoding in the brain, such as how image sequences and duration are encoded, may be of use to movie recognition technology.
We created and trained a machine learning network that mimicked neural rules of movie encoders in the visual system.
In classifying natural movie scenes, the network, called MovieNet, outperformed current machine learning image recognition networks while reducing the number of steps and data required to complete the task.
This study demonstrates that brain-based movie processing principles enable effective machine learning by revealing how movie sequences and time are encoded in the brain.
