New trends in student composition grading: cooperation model between teachers and AI

As technology advances, a long-standing question is how it will change or replace traditional human jobs. From self-service checkouts in supermarkets to AI’s ability to detect serious illnesses in medical scans, workers in all fields find themselves working with tools that do parts of their jobs. As the epidemic has accelerated the popularity of AI tools in the classroom, and this trend has not slowed down, teaching has become another field that shares professional work with tools such as AI.

We have developed a strong interest in a specific application of artificial intelligence in teaching, which is to assess student learning outcomes. Grading and giving feedback to students often consumes a lot of teachers' time, which prevents many teachers from assigning more important writing tasks. At the same time, students often have to wait for a long time to get grades and feedback. In this case, if AI can help evaluate students' homework, it will undoubtedly save time and improve learning efficiency. However, we are also thinking about a question, that is, can AI scoring and feedback systems really help students as effectively as real teachers?

The teacher will ask: "What do you want to express? I don't quite understand. The main task of AI is to solve the problems that have arisen in the writing process and format, rather than trying to understand the meaning of the student's true intention."

We recently conducted an evaluation of an AI platform that allows middle school students to write, submit, and revise essays in response to set essay questions. Whenever students submit an essay, they receive immediate access to the AI Scores and suggestions based on their mastery (1-4 points) in four writing areas (argument and focus, support and evidence, organization, language and style) to help them improve their articles.

To compare AI ratings and feedback with those of actual teachers, we invited 16 middle school writing teachers who had used the platform during the 2021-2022 school year for a face-to-face meeting. After ensuring they had an accurate understanding and application of the grading criteria, We asked each teacher to evaluate 10 random articles that were not their students' and give feedback. This gave us 160 articles evaluated by teachers for us to compare with the AI's ratings and feedback.

Are teachers' scores similar to or different from those given by AI?

On average, we find that teachers give lower scores to articles than AI. Regardless of the aspect, there are significant differences between teachers and AI, In addition to claims and focus. Overall, in terms of the total score of the four dimensions (minimum 4 points, maximum 16 points), the average score of teachers for these 160 articles is 7.6, while the average score of AI for the same set of articles is 8.8. Specifically, teachers and AI tended to agree on high-scoring (score 4) and low-scoring (score 1) articles in terms of claims and focus and support and evidence, but differed on intermediate scores. Teachers are more likely to give the article a score of 2, while the AI ​​is more likely to give the article a score of 3. On the other hand, in terms of organization and language style, teachers are more likely to give the article a score of 1 or 2, while the AI's scores are distributed between Between 1 and 4, more articles received 3 or even 4 points.

Are the teachers’ written comments similar or different to the comments given by the AI?

In our meeting with 16 teachers, we gave them the opportunity to discuss the scores and feedback they had given on 10 essays. Before talking specifically about the essays, we heard a common observation: last year when they were When using this grading program in class, most students need help understanding and interpreting the comments given by the AI. For example, many times, students read a comment and don’t know how to improve their writing. So, according to the teacher According to them, one noticeable change is that they are now able to express comments in language more appropriate to students' understanding ability levels.

"In our discussion, we reflected on how friendly AI can be when it comes to comments and feedback. . Today’s kids are used to direct and honest feedback. They don't always need to placate their egos, but rather want to solve problems. So, it’s not always about being rhetorical, it’s about being direct”.

Another difference we found was that teachers focused more on the quality of the essay as a whole—flow, tone, whether it simply summarized or established an argument, whether the evidence fit the argument, and whether the whole was coherent. Teachers explained that they were more likely to give an article a 2 when focusing on claims and focus and support and evidence because they were able to see the entire article—something many AIs can’t really do. , because many AIs are trained at the sentence level rather than providing training on the entire article.

Teachers evaluate the organizational structure more rigorously because they, unlike AI, can understand the sequence and flow of the entire article. For example, teachers shared that the AI ​​might find transition words or suggest students use more transition words as evidence of a well-structured argument, but teachers can see whether the transitions truly flow or are just inserted into a set of in unrelated sentences. In terms of language and style, teachers again pointed to the problem of AI being more easily disrupted, for example by using seemingly complex vocabulary - which may impress the AI, but teachers will see is just a string of incomprehensible words. Words that form sentences or express ideas.

Can AI help teachers grade?

Assessing student work is a very important and time-consuming part of teaching, especially when students are learning to write. Students need frequent practice and prompt feedback to become confident and proficient writers. However, most teachers lack the time for planning and grading, and they have too many students to teach to schedule regular or long-form writing assignments, as well as maintain work-life balance in their careers.

AI is very important in reducing the burden on teachers. Although our preliminary research found some differences in assessment between teachers and AI, we believe that if the AI ​​system can look at students' articles as comprehensively as teachers can, and give feedback in a way that adapts to students' growth and specific situations, students can be more independent. In response to these opinions, then AI can indeed help teachers in grading. We believe that improving AI is valuable in these areas, not only to reduce the grading burden on teachers, but also to ensure that students have more opportunities to write and receive timely and helpful feedback to enhance their development as writers.

The above is the detailed content of New trends in student composition grading: cooperation model between teachers and AI. For more information, please follow other related articles on the PHP Chinese website!