Neuron reluctantly: the neural network created an image that directly affects the brain
See this image above? With this strange image, MIT neuroscientists were able to activate individual neurons in the brain. Using the best available model of the visual neural network of the brain, scientists have developed a new way to precisely control individual neurons and their populations in the middle of this network. During animal testing, the team showed that the information obtained from the computational model allowed them to create images that strongly activated certain brain neurons.
Scientists, in fact, got a way to access the brain through the image "directly", bypassing the long way of understanding images. But before you think about the grim future, in which we will really be zombied from the TV screens, let's do everything in order.
This is definitely a breakthrough.
The main conclusions of the work are the existing calculated versions of the models of the visual neural systems are sufficiently similar to the present ones so that they can be used to monitor the state of the brain in animals. How accurately these models mimic the work of the visual cortex - this issue is very heated debate, says James DiCarlo, head of the department of brain and cognitive sciences MIT, the senior author of the study, which appeared on May 2 in the journal Science.
“People have long wondered whether these models provide an understanding of the visual system,” he says. “Instead of discussing it in academic circles, we have shown that these models are already powerful enough to be used in new and important ways. Regardless of whether you understand how this model works or not, in a sense, it already benefits. ”
That is, it does not matter how the computational model of the visual system of the brain works - it is important that we can already use it, that it is sufficiently accurate and that new experiments can be developed on its basis. This is the first consequence of the work that needs to be taken into account.
Control of neurons through images - this is possible
Over the past few years, DiCarlo and others have developed models of the visual system based on artificial neural networks. Each network starts with an arbitrary architecture consisting of model neurons, or nodes that can be connected to each other by different strength indicators, or “weight”.
Then scientists teach these models in a library of over 1 million images. Looking through each image and label of the most important object in the image - an airplane or chair, for example - the model learns to recognize objects by changing the strength of the connections. It is difficult to determine exactly how the model achieves this kind of recognition, but DiCarlo and his colleagues previously showed that the “neurons” in these models create activity patterns that are very similar to those that are observed in the visual cortex of animals when reacting to the same images. That is, the neural network seems to be trying to learn to think or see for real.
In the new study, scientists wanted to check whether their models could perform some tasks that had not previously been demonstrated. In particular, they wondered whether it was possible to use these models to control neural activity in the visual cortex of animals.
“So far, we have tried to predict using these models what the neural responses to other stimuli they haven’t seen before,” the scientist says. "The main difference here is that we go one step further and use models to bring neurons into desired states."
To achieve this, scientists first created an accurate "one to one" map of brain neurons in the visual region of the V4 brain from nodes in the computational model. They did this by showing images to animals and models and comparing their responses to the same pictures. In the V4 region there are millions of neurons, but for this study, maps of subpopulations with 5-40 neurons were created simultaneously.
“As soon as each neuron receives an assignment, the model allows you to make predictions about this neuron,” says DiCarlo.The scientists then decided to find out if they could use these predictions to control the activity of individual neurons in the visual cortex. The first type of control, which they called “stretching,” includes displaying an image that will bring the activity of a particular neuron far beyond the limits of activity usually caused by “natural” images, such as those used to train neural networks.
The researchers found that when animals demonstrated such “synthetic” images that were created by models and did not resemble natural objects, the target neurons reacted as expected. On average, neurons exhibited about 40 percent more activity in response to these images than when they were shown natural images. No one has ever achieved this kind of control before.
“The fact that they managed to do this is amazing. From the point of view of a neuron, its ideal image appears to be in its focus. The neuron is suddenly given the stimulus it has always been looking for, ”says Aaron Batista, an associate professor of bioengineering at the University of Pittsburgh, who did not participate in the study. “This is a great idea, and implementing it is a real feat. Perhaps this is the most powerful proof of the need to use artificial neural networks to understand real neural networks. ”
Just think: scientists have created a simple (so far) image generator, causing a certain effect in the brain of an animal (so far). In theory, so far only in theory, it would be possible to create a “perfect” image for controlling hormone emissions, creating specific memories, programming human actions, because all of this is the result of neurons. The picture created by the neural network, which no one has ever seen, and which only a neural network that understands the internal workings of the brain can only cure and kill.
In a similar series of experiments, scientists tried to create images that would “maximize” the neuron from itself, while maintaining activity in neighboring neurons at a very low level, which is more difficult. With the majority of neurons tested, scientists were able to increase the activity of the target neuron with a slight increase in the surrounding neurons.
“The general trend in neurobiology is that experimental data collection and computer modeling are performed slightly separately, which makes it impossible to significantly confirm the model, therefore there is no measurable progress. Our efforts are bringing back a “closed loop” approach, scientists say. This is important for the success of building and testing models that are most like the brain.
Measurement accuracy
Scientists have also shown that they can use their model to predict how neurons from the V4 region will react to the synthesized images - like the one above. Most of the previous model tests used the same type of naturalistic images on which the model was trained. Scientists from MIT have found that models with an accuracy of 54% predict how the brain should respond to the synthesized images, and with an accuracy of 90% predict how the brain will react to natural images.
“In a sense, we quantify how accurate these models are when forecasting outside the area in which they were trained,” says one of the researchers. “Ideally, the model should be able to accurately predict the response, regardless of the input signal.”
Now scientists are hoping to improve the accuracy of models, allowing them to include new information that they comprehend by looking at the synthesized images. This study did not apply. Simply put, models will learn from their own generated images.
This kind of control will be useful for neuroscientists who want to study how different neurons communicate and interact with each other. In the future, this approach will potentially be helpful in alleviating mood problems, such as depression.Now scientists are working on expanding their model to the lower temporal (inferotemporal) cortex, which feeds on the amygdala, which is involved in the processing of emotions.
“If we had a good model of neurons that trigger a surge of emotion or cause different types of disorders, we could use this model to control neurons in a way that helps alleviate these disorders.