Neural networks translation breakthrough made at MIT
Mon 11 Dec 2017
MIT researchers have found that changing the way neural networks look at data makes them better at understanding languages.
Researchers discovered that by applying a recently developed interpretative technique, they could analyse neural networks and assess the way they work.
What they found is that the systems typically concentrated on lower-level tasks, such as sound recognition, before moving onto higher-level tasks, like understanding the nuances of sentences and different ways they could be interpreted.
Importantly, the researchers also found that while carrying out this process, the systems were choosing not to assess a certain type of data. Once told to include this, the systems improved.
Though the performance improvement was relatively small, it is significant because it shows that analysing the way neural networks carry out tasks can help improve artificial intelligence systems.
Artificial intelligence: UK government makes AI key part of industrial strategy
Jim Glass, a senior research scientist at CSAIL, the MIT department that carried out the research, said: “In machine translation, historically, there was sort of a pyramid with different layers.
“At the lowest level, there was the word, the surface forms, and the top of the pyramid was some kind of interlingual representation, and you’d have different layers where you were doing syntax, semantics.
“This was a very abstract notion, but the idea was the higher up you went in the pyramid, the easier it would be to translate to a new language, and then you’d go down again. So, part of what we’re doing is trying to figure out what aspects of this notion are being encoded in the network.”
Neural networks analyse very large sets of data in order to learn how to perform tasks and have been a key part of the development of artificial intelligence, particularly in areas like speech-recognition and translation.
But fully understanding the way they work has been a challenge for some time. When a network trains itself to perform a task, it adjusts its internal settings in an unpredictable way. This study has had some success in this, alongside its developments in AI.