{"id":3163,"date":"2021-02-18T17:04:39","date_gmt":"2021-02-18T17:04:39","guid":{"rendered":"https:\/\/thenextweb.com\/?p=1339582"},"modified":"2021-02-18T17:04:39","modified_gmt":"2021-02-18T17:04:39","slug":"who-writes-better-essays-college-students-or-gpt-3","status":"publish","type":"post","link":"https:\/\/www.londonchiropracter.com\/?p=3163","title":{"rendered":"Who writes better essays: College students or GPT-3?"},"content":{"rendered":"\n

The GPT-3 text generator has proven adept at producing code<\/a>, blogs<\/a>, and, err, bigotry<\/a>. But is the AI a good student?<\/p>\n

Education resource site EduRef<\/a> has tried to find out \u2014 by testing the system\u2019s essay-writing skills.<\/p>\n

The company hired a panel of professors to create writing prompts for essays on US history, research methods, creative writing, and law.<\/p>\n

They fed the prompts to GPT-3, and also gave them to a group of recent college graduates and undergrad students.<\/span><\/p>\n

The anonymized papers were then marked by the panel, to test whether AI can get better grades than human pupils.<\/p>\n

Some of the results could unnerve professors \u2014 and excite unscrupulous students. But others showed GPT-3 still has a lot to learn.<\/p>\n

[Read: How do you build a pet-friendly gadget? We asked experts and animal owners<\/a>]<\/em><\/p>\n

GPT-3\u2019s highest grades were B-minuses for a history essay<\/a> on American exceptionalism and a policy memo<\/a> for a law class.<\/span><\/p>\n

Its human rivals earned similar marks for their history papers: a B and a C+. But only one of three students got a higher grade than the AI for the law assignment.<\/span><\/p>\n

GPT-3 also received a solid C for its research methods paper<\/a> on COVID-19 vaccine efficacy, while the students got a B and a D. <\/span><\/p>\n

However, the AI\u2019s creative writing abilities couldn\u2019t match its technical skills. Its story<\/a> received the model\u2019s solitary fail, while the student writers\u2019 grades ranged from A to D+.<\/p>\n

Credit: EduRef<\/figcaption>
<\/i><\/a>EduRef asked the graders what they thought of GPT-3\u2019s papers before they knew it was AI.<\/figcaption><\/figure>\n

Overall, GPT-3 showed an impressive grasp of grammar, syntax, and word frequency. <\/span>But it failed to craft a strong narrative for the creative writing assignment.<\/p>\n

Project manager Sam Larson told TNW that this could be due to how GPT-3 recalls information:<\/p>\n

\n

The creative task asked for memories and stories using the five senses, which GPT-3 has no direct experience with, so it (probably) would have to iterate through a different type of information search, which the prompt was not designed to activate. AI is drawing from a repository of events, history, and law information (because it\u2019s drawing its language prediction from readily available databases). So for law and history topics, this is useful and relevant to cite this data for those topics. But creative writing rests more on imagination and synthesis of thoughts into ideas, versus a regurgitation of pre-existing data.<\/span><\/p>\n<\/blockquote>\n

Still, what GPT-3 lacked in craft it made up for in speed. The model spent between three and 20 minutes generating content for each assignment, while the humans took three days on average.<\/p>\n

Assessing the assessment<\/h2>\n

EduRef stressed that the experiment was only an exploratory study. GPT-3\u2019s outputs were lightly edited for length and repetition, although its content, factual information, and grammar were left untouched.<\/p>\n

In addition, the AI produced two papers for the history, research, and law assignments. Lorson then picked which ones to use:<\/p>\n

\n

Whichever one was more essay-like was the one I picked to expand on. One of the two usually ended up simply restating the prompt or outputting a longer version of the prompt, so picking the more essay-like \u2018answer\u2019 was pretty straightforward.<\/span><\/p>\n<\/blockquote>\n

Lorson said the creative writing task required additional human interference:<\/p>\n

\n

Eventually I gave in and encouraged it with a \u2018Once upon a time\u2026\u2019 leader and it got the hint I wanted an actual story. After a couple of generations, I had two variations that looked like creative essays, but one got stuck in a loop describing the same scene on repeat so I picked the other!<\/span><\/p>\n<\/blockquote>\n

Lorson \u2014 who is himself an academic \u2014 was nonetheless impressed by GPT\u2019s performance. He hopes that this type of AI-generated content gives instructors and policy-makers pause for thought about how they quantify what makes a successful student.<\/span><\/p>\n

But students may be more interested in AI\u2019s ability to lend them a devious helping-hand.<\/p>\n

Published February 18, 2021 \u2014 17:04 UTC <\/p>\n

Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

The GPT-3 text generator has proven adept at producing code, blogs, and, err, bigotry. But is the AI a good student? Education resource site EduRef has tried to find out \u2014 by testing…<\/p>\n","protected":false},"author":1,"featured_media":3164,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[1],"tags":[],"_links":{"self":[{"href":"https:\/\/www.londonchiropracter.com\/index.php?rest_route=\/wp\/v2\/posts\/3163"}],"collection":[{"href":"https:\/\/www.londonchiropracter.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.londonchiropracter.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.londonchiropracter.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.londonchiropracter.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=3163"}],"version-history":[{"count":0,"href":"https:\/\/www.londonchiropracter.com\/index.php?rest_route=\/wp\/v2\/posts\/3163\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.londonchiropracter.com\/index.php?rest_route=\/wp\/v2\/media\/3164"}],"wp:attachment":[{"href":"https:\/\/www.londonchiropracter.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=3163"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.londonchiropracter.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=3163"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.londonchiropracter.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=3163"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}