Artificial intelligence from IBM, MIT and Harvard uses grammar rules to capture the linguistic nuances of American English

Join today’s top leaders online at the Data Summit on March 9. Register here.


What is the difference between independent and dependent clauses? Is it “me” or is it “I”? And how does “affect” really differ from “effect”? Plenty of evidence suggests a strong correlation between grammatical knowledge and writing ability, and new research suggests the same may be true for AI. In one pair of preprinted papersscientists from IBM, Harvard and MIT are extensively testing a natural language processing system trained in grammar rules – rules they say helped it learn faster and perform better.

The work is expected to be presented at the North American chapter of the Association for Computational Linguistics conference in June.

“The grammar helps the model behave more humanely,” said Miguel Ballesteros, a researcher at the MIT-IBM Watson AI Lab and co-author of the two papers, in a statement. “Sequential models don’t seem to care if you end a sentence with an ungrammatical expression. Why? Because they don’t see this hierarchy.

The IBM team, along with scientists from MIT, Harvard, the University of California, Carnegie Mellon University and Kyoto University, have designed a set of tools to determine the linguistic prowess of models. grammar-aware AIs. As the co-authors explain, a model in question was trained on a sentence structure called Recurrent Neural Network Grammars, or RNNGs, which imbued it with basic knowledge of grammar.

The RNNG model and similar models with little or no grammatical training were fed sentences with good, bad, or ambiguous syntax. AI systems assigned probabilities to each word, so that in grammatically “off” sentences, low-probability words appeared instead of high-probability words. These were used to measure surprise.

The co-authors found that the RNNG system performed consistently better than systems trained on little or no grammar using a fraction of the data, and that it could understand “fairly sophisticated” rules. In one instance, he identified that “that” in the sentence “I know the lion devoured at sunrise” erroneously appeared instead of “what” to introduce the embedded clause, a construct linguists call dependency. between a filler (a word like “who or what”) and a space (the absence of a sentence where one is generally required).

Padding and padding dependencies are more complicated than you might think. In the sentence “The policeman who the criminal shot the politician with his shocked weapon during the trial”, for example, the gap corresponding to the filling “who” is a little abnormal. Technically, it should come after the verb “shoot”, not “shocked”. Here is the rewritten sentence: “The police officer at whom the criminal fired his weapon shocked the jury during the trial.”

“Without being trained on tens of millions of words, state-of-the-art sequential models don’t care where the gaps are and aren’t in sentences like these,” the science department professor said. Brain and Cognitive Studies from MIT and study co-author Roger Levy. “A human would find that really weird, and apparently the grammar-enriched patterns too.”

The team says their work is a promising step towards more accurate language models, but they admit it requires validation on larger datasets. They leave that for future work.

VentureBeat’s mission is to be a digital public square for technical decision makers to learn about transformative enterprise technology and conduct transactions. Learn more

Comments are closed.