backtop


Print 10 comment(s) - last by BRB29.. on Apr 8 at 7:49 AM

The software learns how to do so from human professors

Short answer and essay questions on exams could one day be graded by computer software, but many educators aren't happy about it. 

EdX, a nonprofit enterprise started by Harvard University and the Massachusetts Institute of Technology (MIT), has created a software system that is capable of learning how to grade the way a human teacher would and then grades both short answer and essay questions based on this method.

The major benefit is that students will receive a grade instantly rather than waiting days or weeks. Not only that, but students could use this software to keep rewriting essays until they reach a desired grade (since the grading process is instant), helping them to learn rather than just accept a grade and move on.


“There is a huge value in learning with instant feedback,” said Dr. Anant Agarwal, an electrical engineer who is president of EdX. “Students are telling us they learn much better with instant feedback. 
 
"This is machine learning and there is a long way to go, but it’s good enough and the upside is huge. "We found that the quality of the grading is similar to the variation you find from instructor to instructor.” 

The EdX software allows a human teacher to grade about 100 essays first while it observes the grading technique. From there, the tool learns how to grade and does so on its own. 

While this software could help students learn faster and free up some of the professor's time to tend to other things, it has been criticized in the education sector. Les Perelman, a retired director of writing and a current researcher at MIT, said that software can never compare to a human professor. Perelman has written fake essays with bogus "facts" in it and tricked computer software into giving him a decent grade.

“My first and greatest objection to the research is that they did not have any valid statistical test comparing the software directly to human graders,” said Perelman.

While software for grading basic tests like multiple choice are widely adopted, it is yet to be seen whether the same will happen for essays and more detailed tests, which require the knowledge of a professor to score accurately. 

Source: The New York Times



Comments     Threshold


This article is over a month old, voting and posting comments is disabled

What's the fun in that?
By freedom4556 on 4/6/2013 3:10:05 AM , Rating: 2
But then the professor can't give you a bad grade because they didn't like the stance you chose when answering the prompt. Or the way your face looks. Or the font you chose. Or what size your margins were. Teachers will never use this; it takes all the subjectivity out of grading papers.




RE: What's the fun in that?
By TourGuide on 4/6/13, Rating: 0
RE: What's the fun in that?
By Solandri on 4/6/2013 3:33:37 PM , Rating: 2
quote:
You are an ignorant person who clearly has had a bad experience in school. Grading papers isn't about being 'subjective'. It is about achieving the desired response.

I'm of two minds about this. First, what you're describing is the ideal. OP is completely correct that in reality bias exists among professors and teachers. I've tested for and confirmed it (by turning in sequential assignments with vastly different levels of effort put into them, but with viewpoints agreeing with or contradictory to the teacher's own viewpoints). If you view school as a means of education, then this sort of thing is unacceptable, and teachers/professors should be disciplined for it.

But if you view school as preparation for real life, then you want this sort of mentor bias to rear its ugly head from time to time. You're not going encounter perfectly objective people at your job either. And much of life is about learning to prioritize, compromise, and to bite one's tongue against a particularly annoying person so you can complete a greater goal.

So in that respect, turning schools into an ideally objective learning experience means they'd fail to teach you one of the skills crucial to coping with real life. i.e. This sort of imperfection among teachers/professors is actually desirable to a small degree. Just make sure the students have a means to deal with it when it becomes insufferable (in my case, after talking about it with administration I was allowed to switch courses and/or instructors).


RE: What's the fun in that?
By arthur449 on 4/6/2013 9:33:06 AM , Rating: 2
I detect sarcasm.

That being said, the only thing teachers and professors hate more than coming up with lesson plans and trying to teach to standardized tests is grading papers.


This will not help students learn!
By Schrag4 on 4/6/2013 10:45:00 AM , Rating: 2
The grade isn't the feedback that helps students learn. Instead, it's the notes that the teacher provides explaining what's wrong or praising what's done correctly that helps a student learn. I imagine if students had instant feedback about what grade they would receive for an essay, they would simply start typing random crap until the grade becomes good enough for their own satisfaction. What they would learn is how to more efficiently feed the software what it's looking for rather than what they're really supposed to be learning.

Let me give an analogy. I'm a somewhat young guy, but I understand that many decades ago, getting your software tested was quite an ordeal, involving scheduling time on a mainframe, dealing with punch cards, etc. When it came your turn to test something, you probably learned early on that your life would be much easier if you spent some time to debug the code before it actually ran anywhere. Contrast that with today's development environments, where in many instances, you write 10, 20, 100 lines of code and just let it fly because building takes mere seconds, maybe minutes, so there's no significant penalty for mistakes. IMO that at the very least allows for "learning" by trying stuff until it works rather than learning how it works and then trying it.

Am I way off?




By geddarkstorm on 4/6/2013 12:28:09 PM , Rating: 2
Exactly. And so much for innovative thought or creativity -- two things machines can't handle, since they deviate from standardized parameters.


Circle, complete
By Aloonatic on 4/6/2013 3:22:57 PM , Rating: 3
So software (a Google search, then copy & paste) writes the essay with little human involvement. It's only right that software should mark it, with little human involvement too.




nice idea
By Bubbacub on 4/6/2013 4:14:07 PM , Rating: 2
it all depends on execution.

on the plus side, all the pretty girls won't automatically score highly on coursework.

the flipside is that if the software is not particulary smart it will be figured out by students fairly quickly and they will realise what type of formulaic garbage tends to score highly.

also i imagine that a really good candidate displaying higher level analytical thinking may write in a way that requires more effort to comprehend than simple plain unambiguous statements. there would be instances where it could be difficult for an algorithm to differentiate a real high flying candidate from one that just can't write properly.

i short unless this software has the comprehension and understanding of a teacher in the particular subject being marked then this will introduce a whole new layer of inaccuracy and error.




Enter python
By toyotabedzrock on 4/6/2013 8:31:37 PM , Rating: 2
Last I checked programmers do not make ideal writers.

And with instant feedback grades become pointless. The first proficient python programmer is gonna find it easy to pump out jiberish and ace the test.

Also teachers would not hate this. It doesn't do anything to replace actual teaching.




This will never work
By BRB29 on 4/8/2013 7:49:25 AM , Rating: 2
Unless computers can think like humans, there is no chance this will ever be effective. Whether it's a bad essay or good is purely subjective because it's a combination of persuasion and artwork of language. No computers, not even supercomputers, can simulate the human mind.

The fact that our consciousness controls our mass is still a mystery to science. Our creativity is an illogical process while computers can only think logically or black and white or 0s and 1s. Ironically, the biggest anomaly in science that we still cannot explain is our existence. How can you simulate something you know nothing about?




"I f***ing cannot play Halo 2 multiplayer. I cannot do it." -- Bungie Technical Lead Chris Butcher














botimage
Copyright 2014 DailyTech LLC. - RSS Feed | Advertise | About Us | Ethics | FAQ | Terms, Conditions & Privacy Information | Kristopher Kubicki