News

The results of the first round of tests of the Up Great PRO competition were summed up//READING

21.12.2020

Within the framework of the Up Great Pro//READING technology competition, the first cycle of tests was completed and the winners of the “Grammar” and “Grammar.Eng” nominations were announced.

The Up Great Pro//READING contest is aimed at creating an AI assistant capable of helping teachers identify logical, factual, stylistic and other semantic errors in the writings of schoolchildren in the Russian language, literature, history, social science and English. The aim of the competition is to stimulate the development of strong artificial intelligence technologies and support the development community in the field of machine learning.

More than 180 development teams took part in the first round of the competition. According to the results of the qualification, 11 teams were admitted to the tests and showed effective solutions. As part of the tests, AI assistants checked 500 essays and essays of the unified state exam in Russian and the same number in English. The automated platform compared the texts verified by AI assistants and real teachers and identified the best quality solutions.

The winners of the special categories of “Grammar” and “Grammar. Eng”, whose AI systems correctly identified the largest number of grammatical and speech errors, divided the prize Fund of 20 million rubles.

Results of the nomination “Grammar” for texts in Russian:

  1. Rocket (developers from MIPT and Lomonosov Moscow State University) — 1st place, 7 million roubles
  2. “Antiplagiat” (developers “Antiplagiat”) — 2nd place*
  3. Chemist (developers from MIPT) — 3rd place, 2 million roubles
  4. NugumanovTeam (developers from Ufa) — 4th place, 1 million roubles

* In accordance with the rules of the competition, one team can claim the prize Fund only in one of the categories. The Antiplagiat team took the 2nd place in both the “Grammar” and “Grammar.ENG” categories. According to the decision of the team, the cash prize is awarded in the category “Grammar.ENG”.

Results of the nomination “Grammar.ENG” for English texts:

  1. “Neurosthenia” (developers DeepPavlov) — 1st place, 7 million rubles.
  2. “Antiplagiat” (developers “Antiplagiat”) — 2nd place, 2 million rubles.
  3. Nanosemantika (developers of the Ashmanov neural Network and the ORFO service) — 3rd place, 1 million roubles

As part of the main task of the competition, no team has yet achieved a result corresponding to the level of teachers. Participants will be able to compete again for the prize Fund of the competition in the amount of 200 million rubles at the second cycle trials planned for the spring of 2021. In the course of the analysis of texts in Russian, the best intermediate result was shown by the team “Antiplagiat” (accuracy level — 57.8%). For the English language, the best quality solution was demonstrated by the NeuroChtenie team (accuracy level — 84.5%). Among the leaders was the team from St. Petersburg Seedh AI3, as well as the teams “Rocket” and “Nanosemantika”. At this stage, the participants ' AI assistants most successfully coped with identifying lexical, grammatical and speech errors. The greatest difficulties arise when determining logical errors.

Participants ' solutions are planned for pilot implementation and operation in 2021-22 in a number of schools in different regions of Russia.

Yuriy Molodykh, head of the organizing Committee of the Up Great PRO technology competition / / READING
The test results showed that the task of creating an AI assistant capable of detecting all types of semantic errors in texts is quite solvable on the basis of existing technologies. The teams have shown effective solutions that can already be used as an effective teacher assistant in school. Such an assistant will be able to facilitate the daily routine work of the teacher, allowing teachers to concentrate their efforts on the most important and complex pedagogical tasks, the formation of logical and critical thinking.
Mikhail Burtsev, head of the Laboratory of neural systems and deep learning of MIPT, team “Neurochemistry”
We are very pleased that we have managed to create the best system for checking essays in English and take first place in the nomination for English grammar. Our DeepPavlov library helped us in this, and an Orchestrator of error annotators and skills for their correction was built on it. The high processing speed of the composition-only 5–6 seconds-confirmed that the library is ready to work with large amounts of data in the conditions of real industrial load. And, of course, the team work of our laboratory and the support of the physicist were the most powerful factors on the way to victory. I hope that we will be able to participate in the competition in the future, although it may take more than one year to overcome the technological barrier.
Yuri Chekhovich, Executive Director of “Antiplagiat”
For the first time we participated in a competition that did not concern the area of our main competence — the detection of borrowings in texts. And, of course, we are pleased that the results of our solutions for the Russian language have confirmed the competence of the team “Antiplagiat”. I am sure that we can overcome the technological barrier at the upcoming stages, that is, create a solution that will assess written works as well as a live teacher.

The Up Great Pro//READING technology competition was organized by the SKOLKOVO Foundation and ASI WITHIN the framework of the National technology initiative, with the support of the ROSNANO School League and FEFU. The technical partner of the competition is the NTI competence Center in the area of “Artificial intelligence” on the basis of MIPT.

Detailed information is available on the official website of the competition: https://ai.upgreat.one/participants/leaderboard/
https://ai.upgreat.one/



Information

About Up Great technology competitions

Up Great technology competitions were initiated by RVC, SKOLKOVO Foundation and ASI for Russian innovative companies as part of the National technology initiative. Participants are invited to find solutions to the most complex technological problems that the world has no solutions yet. The winning team, which has overcome the global technological barrier, receives a large cash prize for demonstrating a clear and repeatable solution to the competition task before experts, the jury and the General public. The mechanics of the competitions are developed in accordance with the best practices of the world technological competitions: XPrize, Darpa Grand Challenge, etc. The operator of up Great Technology competitions is RVC.
http://upgreat.one/

Search by name:

Search by date:

Select date in calendar
Select date in calendar
Venue: