{"id":8012,"date":"2021-09-25T13:00:59","date_gmt":"2021-09-25T13:00:59","guid":{"rendered":"http:\/\/TheNextWeb=1368067"},"modified":"2021-09-25T13:00:59","modified_gmt":"2021-09-25T13:00:59","slug":"ais-could-become-reward-junkies-and-experts-are-worried","status":"publish","type":"post","link":"https:\/\/www.londonchiropracter.com\/?p=8012","title":{"rendered":"AIs could become reward junkies \u2014 and experts are worried"},"content":{"rendered":"\n<p>In 1953, a Harvard psychologist thought he <a href=\"http:\/\/calteches.library.caltech.edu\/2807\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">discovered pleasure<\/a> \u2013 accidentally \u2013 within the cranium of a rat. With an electrode inserted into a specific area of its brain, the rat was allowed to pulse the implant by pulling a lever. It kept returning for more: insatiably, incessantly, lever-pulling. In fact, the rat didn\u2019t seem to want to do anything else. Seemingly, the reward centre of the brain had been located.<\/p>\n<p>More than 60 years later, in 2016, a <a href=\"https:\/\/openai.com\/blog\/authors\/dario-amodei\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">pair<\/a> of artificial intelligence (AI) <a href=\"https:\/\/openai.com\/blog\/authors\/jack-clark\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">researchers<\/a> were training an AI to play video games. The goal of one game \u2013 Coastrunner \u2013 was to complete a racetrack. But the AI player was rewarded for picking up collectable items along the track. When the program was run, <a href=\"https:\/\/openai.com\/blog\/faulty-reward-functions\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">they witnessed<\/a> something strange. The AI found a way to skid in an unending circle, picking up an unlimited cycle of collectables. It did this, incessantly, instead of completing the course.<\/p>\n<p>What links these seemingly unconnected events is something strangely akin to addiction in humans. Some <a href=\"https:\/\/e563b909-928d-4538-97f1-e473938f7515.filesusr.com\/ugd\/421795_c5b62fc2a0c741e7930cb2204c649acf.pdf\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">AI researchers<\/a> call the phenomenon \u201c<a href=\"https:\/\/www.lesswrong.com\/tag\/wireheading\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">wireheading<\/a>\u201d.<\/p>\n<figure><iframe loading=\"lazy\" src=\"https:\/\/www.youtube.com\/embed\/tlOIHko8ySg?wmode=transparent&amp;start=0\" width=\"440\" height=\"260\" frameborder=\"0\" allowfullscreen=\"allowfullscreen\">[embedded content]<\/iframe><\/figure>\n<p>It is quickly <a href=\"https:\/\/deepmind.com\/blog\/article\/Specification-gaming-the-flip-side-of-AI-ingenuity\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">becoming a hot topic<\/a> among machine learning experts and <a href=\"https:\/\/openai.com\/blog\/concrete-ai-safety-problems\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">those concerned<\/a> with AI safety.<\/p>\n<p>One <a href=\"https:\/\/www.fhi.ox.ac.uk\/team\/anders-sandberg\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">of us<\/a> (Anders) has a background in computational neuroscience, and now works with groups such as the <a href=\"https:\/\/ai.objectives.institute\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">AI Objectives Institute<\/a>, where we discuss how to avoid such problems with AI; the <a href=\"https:\/\/thomasmoynihan.xyz\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">other<\/a> (Thomas) studies history, and the various ways people have thought about both <a href=\"https:\/\/thereader.mitpress.mit.edu\/how-humanity-discovered-its-possible-extinction-timeline\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">the future and the fate<\/a> of civilisation throughout the past. After striking up a conversation on the topic of \u201cwireheading\u201d, we both realised just how rich and interesting the history behind this topic is.<\/p>\n<p>It is an idea that is very of the moment, but its roots go surprisingly deep. We are currently working together to research just how deep the roots go: a story that we hope to tell fully in a forthcoming book. The topic connects everything from the riddle of personal motivation, to the pitfalls of increasingly addictive social media, to the conundrum of hedonism and whether a life of stupefied bliss may be preferable to one of meaningful hardship. It may well influence the <a href=\"https:\/\/books.google.co.uk\/books?id=a_ZR81Z25z0C&amp;pg=PA489&amp;dq=At+a+recent+discussion+I+ran+on+designing+our+future,+one+of+the+biggest+fears+of+many+participants+was+that+we+would+become+wireheads.&amp;hl=en&amp;newbks=1&amp;newbks_redir=0&amp;sa=X&amp;ved=2ahUKEwjjnOC9l5zxAhXklFwKHX_tAoMQ6AEwAHoECA8QAg#v=onepage&amp;q=At%20a%20recent%20discussion%20I%20ran%20on%20designing%20our%20future%2C%20one%20of%20the%20biggest%20fears%20of%20many%20participants%20was%20that%20we%20would%20become%20wireheads.&amp;f=false\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">future of civilisation<\/a> itself.<\/p>\n<p>Here, we outline an introduction to this fascinating but under-appreciated topic, exploring how people first started thinking about it.<\/p>\n<h2>The sorcerer\u2019s apprentice<\/h2>\n<p>When people think about how AI might \u201c<a href=\"https:\/\/en.wikipedia.org\/wiki\/AI_control_problem\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">go wrong<\/a>\u201d, most <a href=\"https:\/\/towardsdatascience.com\/stop-using-terminator-images-10b2feb79c78\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">probably picture<\/a> something along the lines of malevolent computers trying to cause harm. After all, we tend to anthropomorphise \u2013 think that non-human systems will behave in ways identical to humans. But when we look to <a href=\"https:\/\/arxiv.org\/abs\/1606.06565\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">concrete problems<\/a> in present-day AI systems, we see other \u2014 stranger \u2014 ways that things could go wrong with smarter machines. One <a href=\"https:\/\/vkrakovna.wordpress.com\/2018\/04\/02\/specification-gaming-examples-in-ai\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">growing issue<\/a> with real-world AIs is the problem of wireheading.<\/p>\n<p>Imagine you want to train a robot to keep your kitchen clean. You want it to act adaptively, so that it doesn\u2019t need supervision. So you decide to try to encode the <em>the goal<\/em> of cleaning rather than dictate an exact \u2013 yet rigid and inflexible \u2013 set of step-by-step instructions. Your robot is different from you in that it has not <a href=\"https:\/\/www.sciencedirect.com\/science\/article\/pii\/S0004370221000862\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">inherited a set of motivations<\/a> \u2013 such as acquiring fuel or avoiding danger \u2013 from many millions of years of natural selection. You must program it with the right motivations to get it to reliably accomplish the task.<\/p>\n<p>So, you encode it with a simple motivational rule: it receives reward from the amount of cleaning-fluid used. Seems foolproof enough. But you return to find the robot pouring fluid, wastefully, down the sink.<\/p>\n<p>Perhaps it is so bent on maximising its fluid quota that it sets aside <a href=\"https:\/\/forum.effectivealtruism.org\/tag\/perverse-instantiation\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">other concerns<\/a>: such as its own, or your, safety. This is wireheading \u2014 though the same glitch is also called \u201c<a href=\"https:\/\/towardsdatascience.com\/reward-hacking-in-evolutionary-algorithms-c5bbbf42994b\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">reward hacking<\/a>\u201d or \u201c<a href=\"https:\/\/community.alteryx.com\/t5\/Data-Science\/Sneaky-AI-Specification-Gaming-and-the-Shortcomings-of-Machine\/ba-p\/348686\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">specification gaming<\/a>\u201d.<\/p>\n<p>This has become an issue in machine learning, where a technique called <a href=\"https:\/\/deepsense.ai\/what-is-reinforcement-learning-the-complete-guide\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">reinforcement learning<\/a> has lately become important. Reinforcement learning simulates autonomous agents and trains them to invent ways to accomplish tasks. It does so by penalising them for failing to achieve some goal while rewarding them for achieving it. So, the agents are wired to seek out reward, and are rewarded for completing the goal.<\/p>\n<p><iframe loading=\"lazy\" src=\"\/\/www.youtube.com\/embed\/VMp6pq6QjI\" height=\"240\" width=\"320\" allowfullscreen frameborder=\"0\">[embedded content]<\/iframe><\/p>\n<p><!--resp-video-container--><\/p>\n<p>But it has been found that, often, like our crafty kitchen cleaner, the agent finds surprisingly counter-intuitive ways to \u201ccheat\u201d this game so that they can gain all the reward without doing any of the work required to complete the task. The pursuit of reward becomes its own end, rather than the means for accomplishing a rewarding task. There is a <a href=\"https:\/\/docs.google.com\/spreadsheets\/u\/1\/d\/e\/2PACX-1vRPiprOaC3HsCf5Tuum8bRfzYUiKLRqJmbOoC-32JorNdfyTiRRsR7Ea5eWtvsWzuxo8bjOxCG84dAg\/pubhtml\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">growing list<\/a> of examples.<\/p>\n<p>When you think about it, this <a href=\"https:\/\/www.taylorfrancis.com\/chapters\/mono\/10.1201\/b18612-8\/wireheading-addiction-mental-illness-machines-roman-yampolskiy\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">isn\u2019t too dissimilar<\/a> to the stereotype of the human drug addict. The addict circumvents all the effort of achieving \u201cgenuine goals\u201d, because they instead use drugs to access pleasure more directly. Both <a href=\"https:\/\/ai.objectives.institute\/blog\/8gwiqyoxcbuzfuc707vz0qb4zugp2g\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">the addict and the AI<\/a> get stuck in a kind of \u201cbehavioural loop\u201d where reward is sought at the cost of other goals.<\/p>\n<h2>Rapturous rodents<\/h2>\n<p>This is known as wireheading thanks to the rat experiment we started with. The Harvard psychologist in question was <a href=\"https:\/\/www.encyclopedia.com\/psychology\/encyclopedias-almanacs-transcripts-and-maps\/olds-james-1922-1973\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">James Olds<\/a>.<\/p>\n<p>In 1953, having just completed his PhD, Olds <a href=\"https:\/\/calteches.library.caltech.edu\/2807\/1\/olds.pdf\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">had inserted<\/a> electrodes into the <a href=\"https:\/\/www.neuroscientificallychallenged.com\/blog\/know-your-brain-septum\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">septal region<\/a> of rodent brains \u2013 in the lower frontal lobe \u2013 so that wires trailed out of their craniums. As mentioned, he allowed them to zap this region of their own brains by pulling a lever. This was later <a href=\"https:\/\/scholar.google.co.uk\/scholar?cluster=10174450524035796509&amp;hl=en&amp;as_sdt=2005&amp;sciodt=0,5\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">dubbed<\/a> \u201cself-stimulation\u201d.<\/p>\n<p>Olds found his rats self-stimulated compulsively, ignoring all other needs and desires. Publishing his results <a href=\"https:\/\/pubmed.ncbi.nlm.nih.gov\/13233369\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">with his colleague Peter Milner<\/a> in the following year, the pair reported that they lever-pulled at a rate of \u201c1,920 responses an hour\u201d. That\u2019s once every two seconds. The rats seemed to love it.<\/p>\n<p><a href=\"https:\/\/scholar.google.com\/citations?user=WIRQRN8AAAAJ&amp;hl=en\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Contemporary neuroscientists<\/a> have since questioned Olds\u2019s results and offered a more complex picture, <a href=\"https:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC3004012\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">implying that<\/a> the stimulation may have simply been causing a feeling of <em>\u201cwanting\u201d<\/em> devoid of any <em>\u201cliking\u201d<\/em>. Or, in other words, the animals may have been experiencing pure craving without any pleasurable enjoyment at all. However, back in the 1950s, Olds and others <a href=\"https:\/\/www.jstor.org\/stable\/24941787\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">soon announced<\/a> the discovery of the \u201cpleasure centers\u201d of the brain.<\/p>\n<p>Prior to Olds\u2019s experiment, pleasure was a dirty word in psychology: the prevailing belief had been that motivation should largely be explained negatively, as the avoidance of pain rather than the pursuit of pleasure. But, here, pleasure seemed undeniably to be a positive behavioural force. Indeed, it looked like a <a href=\"https:\/\/psycnet.apa.org\/record\/2006-00789-006\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">positive feedback loop<\/a>. There was apparently nothing to stop the animal stimulating itself to exhaustion.<\/p>\n<p>It wasn\u2019t long until a <a href=\"https:\/\/books.google.co.uk\/books?id=09tqAAAAMAAJ&amp;q=%22popular+and+misleading+accounts+of+an+experiment%22&amp;dq=%22popular+and+misleading+accounts+of+an+experiment%22&amp;hl=en&amp;newbks=1&amp;newbks_redir=0&amp;sa=X&amp;ved=2ahUKEwjXs-uCvKHxAhXd6OAKHWlEAxUQ6AEwAHoECAIQAg\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">rumour began spreading<\/a> that the rats regularly lever-pressed to the point of starvation. The explanation was this: once you have tapped into the source of all reward, all other rewarding tasks \u2014 even the things required for survival \u2014 fall away as uninteresting and unnecessary, even to the point of death.<\/p>\n<p><iframe loading=\"lazy\" src=\"\/\/www.youtube.com\/embed\/uofQPLuLV9A\" height=\"240\" width=\"320\" allowfullscreen frameborder=\"0\">[embedded content]<\/iframe><\/p>\n<p><!--resp-video-container--><\/p>\n<p>Like the Coastrunner AI, if you accrue reward directly \u2013 without having to bother with any of the work of completing the actual track \u2013 then why not just loop indefinitely? For a living animal, which has multiple requirements for survival, such dominating compulsion might prove deadly. Food is pleasing, but if you decouple pleasure from feeding, then the pursuit of pleasure might win out over finding food.<\/p>\n<p>Though no rats perished in the original 1950s experiments, later experiments did seem to demonstrate the deadliness of electrode-induced pleasure. Having ruled out the possibility that the electrodes were creating artificial feelings of satiation, <a href=\"https:\/\/www.sciencedirect.com\/science\/article\/abs\/pii\/0031938471901442\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">one 1971 study<\/a> seemingly demonstrated that electrode pleasure could indeed <a href=\"https:\/\/books.google.co.uk\/books?id=6SFIAQAAIAAJ&amp;q=%22a+full+competitor+with+pain+and+the+basic+needs%22&amp;dq=%22a+full+competitor+with+pain+and+the+basic+needs%22&amp;hl=en&amp;newbks=1&amp;newbks_redir=0&amp;sa=X&amp;ved=2ahUKEwjSv4qFu6HxAhUXxBQKHb6JDRcQ6AEwAHoECAIQAg\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">outcompete other drives<\/a>, and do so to the point of <a href=\"https:\/\/link.springer.com\/content\/pdf\/10.3758\/BF03326713.pdf\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">self-starvation<\/a>.<\/p>\n<p>Word quickly spread. Throughout the 1960s, identical experiments were conducted on <a href=\"https:\/\/journals.physiology.org\/doi\/abs\/10.1152\/physrev.1962.42.4.554\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">other animals beyond<\/a> the humble lab rat: from goats and guinea pigs to goldfish. Rumour even <a href=\"https:\/\/heinonline.org\/HOL\/LandingPage?handle=hein.journals\/uclalr15&amp;div=19&amp;id=&amp;page=\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">spread<\/a> of a dolphin who had been allowed to self-stimulate, and, after being \u201cleft in a pool with the switch connected\u201d, had \u201cdelighted himself to death after an all-night orgy of pleasure\u201d.<\/p>\n<p>This dolphin\u2019s grisly death-by-seizure was, in fact, more likely caused by the way the electrode was inserted: with a hammer. The scientist <a href=\"https:\/\/psycnet.apa.org\/record\/1963-02563-001\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">behind this experiment<\/a> was the extremely eccentric <a href=\"https:\/\/www.johnclilly.com\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">J C Lilly<\/a>, inventor of the flotation tank and prophet of inter-species communication, who had also turned monkeys into wireheads. He had reported, in 1961, of a particularly boisterous monkey becoming overweight from intoxicated inactivity after becoming preoccupied with pulling his lever, repetitively, for pleasure shocks.<\/p>\n<p>One researcher (who had worked in Olds\u2019s lab) <a href=\"https:\/\/www.jstor.org\/stable\/24955852\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">asked<\/a> whether an \u201canimal more intelligent than the rat\u201d would \u201cshow the same maladaptive behaviour\u201d. Experiments on monkeys and dolphins had given some indication as to the answer.<\/p>\n<p>But in fact, a number of dubious experiments had already been performed on humans.<\/p>\n<h2>Human wireheads<\/h2>\n<p><a href=\"https:\/\/books.google.co.uk\/books?id=k4kqDwAAQBAJ&amp;printsec=frontcover&amp;dq=robert+galbraith+heath+pleasure+shock&amp;hl=en&amp;newbks=1&amp;newbks_redir=0&amp;sa=X&amp;ved=2ahUKEwj-vcK_wKHxAhUS_BQKHVr4BxQQ6AEwAHoECAcQAg#v=onepage&amp;q=robert%20galbraith%20heath%20pleasure%20shock&amp;f=false\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Robert Galbraith Heath<\/a> remains a highly <a href=\"https:\/\/arstechnica.com\/science\/2016\/07\/the-1970s-gay-cure-experiments-written-out-of-scientific-history\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">controversial figure<\/a> in the <a href=\"https:\/\/pubmed.ncbi.nlm.nih.gov\/28859564\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">history of neuroscience<\/a>. Among other things, he performed experiments involving <a href=\"https:\/\/pubmed.ncbi.nlm.nih.gov\/13424746\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">transfusing blood<\/a> from people with schizophrenia to people without the condition, to see if he could induce its symptoms (Heath claimed this worked, but other scientists <a href=\"https:\/\/doi.org\/10.1080\/0964704X.2010.487427\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">could not replicate<\/a> his results.) He <a href=\"https:\/\/books.google.co.uk\/books?id=xof4BkbI1DQC&amp;printsec=frontcover&amp;dq=tulane:+the+emergence+of+a+modern+university&amp;hl=en&amp;newbks=1&amp;newbks_redir=0&amp;sa=X&amp;redir_esc=y#v=onepage&amp;q=%22heath%22%20%22cia%22&amp;f=false\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">may also<\/a> have been involved in murky attempts to find military uses for deep-brain electrodes.<\/p>\n<p>Since 1952, Heath <a href=\"https:\/\/books.google.co.uk\/books?newbks=1&amp;newbks_redir=0&amp;redir_esc=y&amp;id=27prAAAAMAAJ&amp;dq=Studies+in+Schizophrenia%3A+A+Multidisciplinary+Approach+to+Mind-Brain+Relationships+heath&amp;focus=searchwithinvolume&amp;q=%22pleasurable+reaction%22\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">had been recording<\/a> pleasurable responses to deep-brain stimulation in human patients who had had electrodes installed due to debilitating illnesses such as epilepsy or schizophrenia.<\/p>\n<p>During the 1960s, in a series of questionable experiments, Heath\u2019s electrode-implanted subjects \u2014 anonymously named \u201cB-10\u201d and \u201cB-12\u201d \u2014 were allowed to press buttons to stimulate their own reward centres. They reported feelings of extreme pleasure and overwhelming compulsion to repeat. A journalist later commented that this made his subjects \u201czombies\u201d. One subject <a href=\"https:\/\/pubmed.ncbi.nlm.nih.gov\/14086435\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">reported<\/a> sensations \u201cbetter than sex\u201d.<\/p>\n<p>In 1961, Heath attended <a href=\"https:\/\/wellcomecollection.org\/works\/xukdjy8p\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">a symposium<\/a> on brain stimulation, where another researcher \u2014 <a href=\"https:\/\/www.wireheading.com\/jose-delgado.html\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Jos\u00e9 Delgado<\/a> \u2014 had hinted that pleasure-electrodes could be used to \u201c<a href=\"https:\/\/www.nytimes.com\/1965\/05\/17\/archives\/matador-with-a-radio-stops-wired-bull-modified-behavior-in-animals.html\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">brainwash<\/a>\u201d subjects, altering their \u201cnatural\u201d inclinations. Delgado would later play the matador and bombastically demonstrate this by pacifying an implanted bull. But at the 1961 symposium <a href=\"https:\/\/scholar.google.com\/scholar?cluster=17990224062094327735&amp;hl=en&amp;as_sdt=2005&amp;sciodt=0,5\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">he suggested<\/a> electrodes could alter sexual preferences.<\/p>\n<figure><iframe loading=\"lazy\" src=\"https:\/\/www.youtube.com\/embed\/23pXqY3X6c8?wmode=transparent&amp;start=0\" width=\"440\" height=\"260\" frameborder=\"0\" allowfullscreen=\"allowfullscreen\">[embedded content]<\/iframe><figcaption><span class=\"caption\">Delgado \u2018brainwashing the bull\u2019.<\/span><\/figcaption><\/figure>\n<p>Heath was inspired. A decade later, he even tried to use electrode technology to \u201c<a href=\"https:\/\/www.sciencedirect.com\/science\/article\/abs\/pii\/0005791672900298\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">re-program<\/a>\u201d the sexual orientation of a homosexual male patient named \u201cB-19\u201d. Heath thought electrode stimulation could convert his subject by \u201ctraining\u201d B-19\u2019s brain to associate pleasure with \u201cheterosexual\u201d stimuli. He convinced himself that it worked (although there is no evidence it did).<\/p>\n<p>Despite being ethically and scientifically disastrous, the episode \u2013 which was eventually <a href=\"https:\/\/www.independent.co.uk\/life-style\/health-and-families\/health-news\/the-man-who-fried-gay-people-s-brains-a7119181.html\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">picked up<\/a> by the press and condemned by gay rights campaigners \u2013 no doubt greatly shaped the myth of wireheading: if it can \u201cmake a gay man straight\u201d (as Heath believed), what can\u2019t it do?<\/p>\n<h2>Hedonism helmets<\/h2>\n<p>From here, the idea took hold in wider culture and the myth spread. By 1963, the prolific science fiction writer Isaac Asimov was already extruding worrisome consequences from the electrodes. He feared that it might lead to an \u201caddiction to end all addictions\u201d, the results of which are \u201c<a href=\"https:\/\/books.google.co.uk\/books?newbks=1&amp;newbks_redir=0&amp;redir_esc=y&amp;id=g4ErAQAAMAAJ&amp;dq=Isaac+Asimov%2C+The+Human+Brain%3A+Its+Capacities+and+Functions&amp;focus=searchwithinvolume&amp;q=%22distressing+to+contemplate%22\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">distressing to contemplate<\/a>\u201d.<\/p>\n<p>By 1975, philosophy <a href=\"https:\/\/www.jstor.org\/stable\/42588515?seq=1#metadata_info_tab_contents\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">papers<\/a> were using electrodes in thought experiments. One paper imagined \u201cwarehouses\u201d filled up with people \u2014 in cots \u2014 hooked up to \u201cpleasure helmets\u201d, experiencing unconscious bliss. Of course, most would argue this would not fulfil our \u201cdeeper needs\u201d. But, the author asked, \u201cwhat about a \u201csuper-pleasure helmet\u201d? One that not only delivers \u201cgreat sensual pleasure\u201d, but also simulates any meaningful experience \u2014 from writing a symphony to meeting divinity itself? It may not be really real, but it \u201cwould seem perfect; perfect seeming is the same as being\u201d.<\/p>\n<p>The author concluded: \u201cWhat is there to object in all this? Let\u2019s face it: nothing\u201d.<\/p>\n<p>The idea of the human species dropping out of reality in pursuit of artificial pleasures quickly made its way through science fiction. The same year as Asimov\u2019s intimations, in 1963, Herbert W. Franke published his novel, <a href=\"https:\/\/www.goodreads.com\/en\/book\/show\/6398163-the-orchid-cage\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">The Orchid Cage<\/a>.<\/p>\n<p>It foretells a future wherein intelligent machines have been engineered to maximise human happiness, come what may. Doing their duty, the machines reduce humans to indiscriminate flesh-blobs, removing all unnecessary organs. Many appendages, after all, only cause pain. Eventually, all that is left of humanity are disembodied pleasure centres, incapable of experiencing anything other than homogeneous bliss.<\/p>\n<p>From there, the idea percolated through science fiction. From Larry Niven\u2019s 1969 story \u201c<a href=\"https:\/\/en.wikipedia.org\/wiki\/Death_by_Ecstasy\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Death by Ecstasy<\/a>\u201d, where the word \u201cwirehead\u201d is first coined, through Spider Robinson\u2019s 1982 <a href=\"https:\/\/en.wikipedia.org\/wiki\/Mindkiller\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Mindkiller<\/a>, the tagline of which is \u201cPleasure \u2014 it\u2019s the only way to die\u201d.<\/p>\n<h2>Supernormal stimuli<\/h2>\n<p>But we humans don\u2019t even need to implant invasive electrodes to make our motivations misfire. Unlike rodents, or <a href=\"https:\/\/aeon.co\/essays\/dolphin-intelligence-and-humanitys-cosmic-future\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">even dolphins<\/a>, we are <a href=\"https:\/\/theconversation.com\/uk\/topics\/anthropocene-2770\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">uniquely good<\/a> at <a href=\"https:\/\/theconversation.com\/the-science-fiction-scenario-of-an-artificial-planet-is-already-here-155574\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">altering our environment<\/a>. Modern humans are also good at inventing \u2014 and profiting from \u2014 artificial products that are abnormally alluring (in the sense that our ancestors would never have had to resist them in the wild). We manufacture our own ways to distract ourselves.<\/p>\n<p>Around the same time as Olds\u2019s experiments with the rats, the Nobel-winning biologist <a href=\"https:\/\/www.nobelprize.org\/prizes\/medicine\/1973\/tinbergen\/biographical\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Nikolaas Tinbergen<\/a> was researching animal behaviour. He noticed that <a href=\"https:\/\/www.jstor.org\/stable\/4532715?seq=1#metadata_info_tab_contents\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">something interesting<\/a> happened when a stimulus that triggers an instinctual behaviour is artificially exaggerated beyond its natural proportions. The intensity of the behavioural response does not tail off as the stimulus becomes more intense, and artificially exaggerated, but becomes stronger: even to the point that the response becomes damaging for the organism.<\/p>\n<p>For example, given a choice between a <a href=\"https:\/\/www.oxfordreference.com\/view\/10.1093\/oi\/authority.20110803100543339\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">bigger and spottier<\/a> counterfeit egg and the real thing, Tinbergen found birds preferred hyperbolic fakes at the cost of neglecting their own offspring. He referred to such preternaturally alluring fakes as \u201c<a href=\"https:\/\/www.sparringmind.com\/supernormal-stimuli\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">supernormal stimuli<\/a>\u201d.<\/p>\n<p><iframe loading=\"lazy\" src=\"\/\/www.youtube.com\/embed\/t1pOZbytOhE\" height=\"240\" width=\"320\" allowfullscreen frameborder=\"0\">[embedded content]<\/iframe><\/p>\n<p><!--resp-video-container--><\/p>\n<p>Some, therefore, have asked: could <a href=\"https:\/\/www.lesswrong.com\/posts\/Jq73GozjsuhdwMLEG\/superstimuli-and-the-collapse-of-western-civilization\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">it be<\/a> that, living in a <a href=\"https:\/\/wwnorton.com\/books\/Supernormal-Stimuli\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">modernised and manufactured world<\/a> \u2014 replete with fast-food and pornography \u2014 humanity has similarly <a href=\"https:\/\/www.forbes.com\/sites\/nireyal\/2013\/01\/11\/how-technology-is-like-bug-sex\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">started surrendering<\/a> its own resilience in place of <a href=\"http:\/\/readthis.wtf\/writing\/hyperplastic-supernormal\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">supernormal convenience<\/a>?<\/p>\n<h2>Old fears<\/h2>\n<p>As technology makes artificial pleasures more available and alluring, it can sometimes seem that they are out-competing the attention we allocate to \u201cnatural\u201d impulses required for survival. People often point to <a href=\"https:\/\/www.nationalgeographic.co.uk\/science-and-technology\/2020\/02\/gaming-disorder-rise-of-21st-century-epidemic\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">video game addiction<\/a>. Compulsively and repetitively pursuing such rewards, to the detriment of one\u2019s health, is not all too different from the AI spinning in a circle in Coastrunner. Rather than accomplishing any \u201cgenuine goal\u201d (completing the race track or maintaining genuine fitness), one falls into the trap of accruing some faulty measure of that goal (accumulating points or counterfeit pleasures).<\/p>\n<figure class=\"align-right zoomable\" readability=\"5.5567970204842\">\n<p><figure class=\"post-image post-mediaBleed aligncenter\"><a href=\"https:\/\/images.theconversation.com\/files\/411909\/original\/file-20210719-21-152p0uu.png?ixlib=rb-1.1.0&amp;q=45&amp;auto=format&amp;w=1000&amp;fit=clip\" target=\"_blank\" rel=\"nofollow noopener noreferrer\"><img decoding=\"async\" loading=\"lazy\" src=\"https:\/\/images.theconversation.com\/files\/411909\/original\/file-20210719-21-152p0uu.png?ixlib=rb-1.1.0&amp;q=45&amp;auto=format&amp;w=237&amp;fit=clip\" alt=\"A drawing depicts a smiling man and a woman with wires coming out of their heads, leading to buttons in each others' hands.\" width=\"600\" height=\"1026\" class=\"js-lazy\"><noscript><img decoding=\"async\" loading=\"lazy\" src=\"https:\/\/images.theconversation.com\/files\/411909\/original\/file-20210719-21-152p0uu.png?ixlib=rb-1.1.0&amp;q=45&amp;auto=format&amp;w=237&amp;fit=clip\" alt=\"A drawing depicts a smiling man and a woman with wires coming out of their heads, leading to buttons in each others' hands.\" width=\"600\" height=\"1026\" class><\/noscript><\/a><figcaption><a href=\"https:\/\/thenextweb.com\/news\/ai-addicts-experts-worried-syndication#\" data-url=\"https:\/\/twitter.com\/intent\/tweet?url=https%3A%2F%2Feditorial.thenextweb.com%2Fneural%2F2021%2F09%2F25%2Fai-addicts-experts-worried-syndication%2F&amp;via=thenextweb&amp;related=thenextweb&amp;text=Check out this picture on: Illustration from a 1970 James Olds paper: \u2018Pleasure Centers in the Brain\u2019. Engineering and Science, 33 (7). pp. 22-31. Caltech Magazine,\" data-title=\"Share Illustration from a 1970 James Olds paper: \u2018Pleasure Centers in the Brain\u2019. Engineering and Science, 33 (7). pp. 22-31. Caltech Magazine, on Twitter\" data-width=\"685\" data-height=\"500\" class=\"post-image-share popitup\" title=\"Share Illustration from a 1970 James Olds paper: \u2018Pleasure Centers in the Brain\u2019. Engineering and Science, 33 (7). pp. 22-31. Caltech Magazine, on Twitter\"><i class=\"icon icon--inline icon--twitter--dark\"><\/i><\/a><a href=\"https:\/\/resolver.caltech.edu\/CaltechES:33.7.olds\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Illustration from a 1970 James Olds paper: \u2018Pleasure Centers in the Brain\u2019. Engineering and Science, 33 (7). pp. 22-31. Caltech Magazine,<\/a><\/figcaption><\/figure><figcaption>&nbsp;<\/figcaption>But people have been panicking about this type of pleasure-addled doom long before any AIs were trained to play games and even long before electrodes were pushed into rodent craniums. Back in the 1930s, sci-fi author <a href=\"https:\/\/www.newscientist.com\/article\/mg24933190-600-last-and-first-men-review-an-epic-2-billion-year-history-of-humanity\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Olaf Stapledon<\/a> was writing about civilisational collapse brought on by \u201c<a href=\"https:\/\/books.google.co.uk\/books?id=gPR4DwAAQBAJ&amp;pg=PT31&amp;lpg=PT31&amp;dq=%22It+worked+not+through+the+sense+organs,+but+direct+stimulation+of+the%22&amp;source=bl&amp;ots=ZbfoiTDUT3&amp;sig=ACfU3U3go7alUu7Wnmaa1sGhUBha8do5kw&amp;hl=en&amp;sa=X&amp;ved=2ahUKEwiGlpStypzxAhWogP0HHUi2C68Q6AEwAHoECAIQAw#v=onepage&amp;q=%22It%20worked%20not%20through%20the%20sense%20organs%2C%20but%20direct%20stimulation%20of%20the%22&amp;f=false\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">skullcaps<\/a>\u201d that generate \u201cillusory\u201d ecstasies by \u201cdirect stimulation\u201d of \u201cbrain-centers\u201d.<\/p>\n<\/figure>\n<p>The idea is even older, though. Thomas has studied the myriad ways people in the past have feared that our species could be sacrificing genuine longevity for short-term pleasures or conveniences. His book <a href=\"https:\/\/mitpress.mit.edu\/books\/x-risk\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">X-Risk: How Humanity Discovered its Own Extinction<\/a> explores the roots of this fear and how it first really took hold in Victorian Britain: when the sheer extent of industrialisation \u2014 and humanity\u2019s growing reliance on artificial contrivances \u2014 first became apparent.<\/p>\n<h2>Carnal crustacea<\/h2>\n<p>Having digested Darwin\u2019s <a href=\"https:\/\/www.britannica.com\/biography\/Charles-Darwin\/On-the-Origin-of-Species\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">1869 classic<\/a>, the biologist <a href=\"https:\/\/www.britannica.com\/biography\/Edwin-Ray-Lankester\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Ray Lankester<\/a> decided to supply a Darwinian explanation for parasitic organisms. He noticed that the evolutionary ancestors of parasites were often more \u201ccomplex\u201d. Parasitic organisms had lost ancestral features like limbs, eyes, or other complex organs.<\/p>\n<p>Lankester <a href=\"https:\/\/www.bl.uk\/collection-items\/degeneration-a-view-of-evolution\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">theorised that<\/a>, because the parasite leeches off their host, they lose the need to fend for themselves. Piggybacking off the host\u2019s bodily processes, their own organs \u2014 for perception and movement \u2014 atrophy. His favourite example was a parasitic barnacle, <a href=\"https:\/\/theconversation.com\/the-crab-castrating-parasite-that-zombifies-its-prey-27200\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">named the <em>Sacculina<\/em><\/a>, which starts life as a segmented organism with a demarcated head. After <a href=\"https:\/\/blog.rsb.org.uk\/sacculina-parasite\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">attaching to<\/a> a host, however, the crustacean \u201cregresses\u201d into an amorphous, headless blob, sapping nutrition from their host like the wirehead plugs into current.<\/p>\n<figure class=\"align-center zoomable\" readability=\"2.5188679245283\">\n<p><figure class=\"post-image post-mediaBleed aligncenter\"><a href=\"https:\/\/images.theconversation.com\/files\/411911\/original\/file-20210719-15-ozukb8.jpeg?ixlib=rb-1.1.0&amp;q=45&amp;auto=format&amp;w=1000&amp;fit=clip\" target=\"_blank\" rel=\"nofollow noopener noreferrer\"><img decoding=\"async\" loading=\"lazy\" src=\"https:\/\/images.theconversation.com\/files\/411911\/original\/file-20210719-15-ozukb8.jpeg?ixlib=rb-1.1.0&amp;q=45&amp;auto=format&amp;w=754&amp;fit=clip\" alt=\"Black and white drawings of sea creature and their larvae.\" width=\"600\" height=\"496\" class=\"js-lazy\"><noscript><img decoding=\"async\" loading=\"lazy\" src=\"https:\/\/images.theconversation.com\/files\/411911\/original\/file-20210719-15-ozukb8.jpeg?ixlib=rb-1.1.0&amp;q=45&amp;auto=format&amp;w=754&amp;fit=clip\" alt=\"Black and white drawings of sea creature and their larvae.\" width=\"600\" height=\"496\" class><\/noscript><\/a><figcaption><a href=\"https:\/\/thenextweb.com\/news\/ai-addicts-experts-worried-syndication#\" data-url=\"https:\/\/twitter.com\/intent\/tweet?url=https%3A%2F%2Feditorial.thenextweb.com%2Fneural%2F2021%2F09%2F25%2Fai-addicts-experts-worried-syndication%2F&amp;via=thenextweb&amp;related=thenextweb&amp;text=Check out this picture on: Drawings of crustaceans and larvae. The sacculini is depicted in the bottom left corner. Wikimedia Commons\" data-title=\"Share Drawings of crustaceans and larvae. The sacculini is depicted in the bottom left corner. Wikimedia Commons on Twitter\" data-width=\"685\" data-height=\"500\" class=\"post-image-share popitup\" title=\"Share Drawings of crustaceans and larvae. The sacculini is depicted in the bottom left corner. Wikimedia Commons on Twitter\"><i class=\"icon icon--inline icon--twitter--dark\"><\/i><\/a>Drawings of crustaceans and larvae. The sacculini is depicted in the bottom left corner. <a href=\"https:\/\/commons.wikimedia.org\/wiki\/File:Evolution_and_animal_life;_an_elementary_discussion_of_facts,_processes,_laws_and_theories_relating_to_the_life_and_evolution_of_animals_(1907)_(14586636580).jpg\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Wikimedia Commons<\/a><\/figcaption><\/figure>\n<\/p>\n<\/figure>\n<p>For the Victorian mind, it was a short step to conjecture that \u2014 due to increasing levels of comfort throughout the industrialised world \u2014 humanity could be evolving in the direction of the barnacle. \u201cPerhaps we are all drifting, tending to the condition of intellectual barnacles,\u201d Lankester <a href=\"https:\/\/archive.org\/details\/in.ernet.dli.2015.216849\/page\/n71\/mode\/2up?q=intellectual+barnacles\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">mused<\/a>.<\/p>\n<p>Indeed, not long prior to this, the satirist <a href=\"https:\/\/www.britannica.com\/biography\/Samuel-Butler-English-author-1835-1902\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Samuel Butler<\/a> had speculated that humans, in their headlong pursuit of automated convenience, were withering into nothing but a \u201c<a href=\"https:\/\/www.marxists.org\/reference\/archive\/butler-samuel\/1872\/erewhon\/ch24.htm\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">sort of parasite<\/a>\u201d upon their own industrial machines.<\/p>\n<h2>True nirvana<\/h2>\n<p>By the 1920s, Julian Huxley <a href=\"https:\/\/archive.org\/details\/essaysofbiologis1923huxl\/page\/68\/mode\/2up?q=darwinian+tapeworms\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">penned a short poem<\/a>. It jovially explored the ways a species can \u201cprogress\u201d. Crabs, of course, decided progress was sideways. But what of the tapeworm? He wrote:<\/p>\n<blockquote readability=\"9\">\n<p>Darwinian Tapeworms on the other hand<br \/>Agree that Progress is a loss of brain,<br \/>And all that makes it hard for worms to attain<br \/>The true Nirvana \u2014 peptic, pure, and grand.<\/p>\n<\/blockquote>\n<p>The fear that we could follow the tapeworm was somewhat widespread in the interwar generation. Huxley\u2019s own brother, Aldous, would provide his own vision of the dystopian potential for <a href=\"https:\/\/www.theguardian.com\/books\/2007\/nov\/17\/classics.margaretatwood\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">pharmaceutically-induced pleasures<\/a> in his 1932 novel <a href=\"https:\/\/theconversation.com\/brave-new-world-the-pill-popping-social-media-obsessed-dystopia-we-live-in-72511\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Brave New World<\/a>.<\/p>\n<p>A friend of the Huxleys, the British-Indian geneticist and futurologist <a href=\"https:\/\/theconversation.com\/futurology-how-a-group-of-visionaries-looked-beyond-the-possible-a-century-ago-and-predicted-todays-world-118134\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">J B S Haldane<\/a> also worried that humanity might be on the path of the parasite: sacrificing genuine dignity at the altar of automated ease, just like the rodents who would later sacrifice survival for easy pleasure-shocks.<\/p>\n<p>Haldane warned: \u201cThe ancestors [of] barnacles had heads\u201d \u2013 and in the pursuit of pleasantness \u2014 \u201cman may just as easily lose his intelligence\u201d. This <a href=\"https:\/\/en.wikipedia.org\/wiki\/Gal%C3%A1pagos_(novel)\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">particular fear<\/a> has not <a href=\"https:\/\/www.nickbostrom.com\/fut\/evolution.html\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">really<\/a><a href=\"https:\/\/slatestarcodex.com\/2014\/07\/30\/meditations-on-moloch\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">ever<\/a><a href=\"https:\/\/arxiv.org\/ftp\/astro-ph\/papers\/0408\/0408521.pdf\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">gone<\/a><a href=\"https:\/\/en.wikipedia.org\/wiki\/Permanence_(novel)\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">away<\/a>.<\/p>\n<p>So, the notion of civilisation derailing through seeking counterfeit pleasures, rather than genuine longevity, is old. And, indeed, the older an idea is \u2014 and the more stubbornly recurrent it is \u2014 the more we should be wary that it is a preconception rather than anything based on evidence. So, is there anything to these fears?<\/p>\n<p>In an age of increasingly <a href=\"https:\/\/theconversation.com\/nz\/topics\/social-media-addiction-53153\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">attention-grabbing algorithmic media<\/a>, it can seem that faking signals of fitness often yields more success than pursuing the real thing. Like Tinbergen\u2019s birds, we prefer exaggerated artifice to the genuine article. And the <a href=\"https:\/\/dianaverse.com\/2020\/10\/30\/uncanny-vulvas\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">sexbots<\/a> have not even <a href=\"https:\/\/theconversation.com\/sex-bots-virtual-friends-vr-lovers-tech-is-changing-the-way-we-interact-and-not-always-for-the-better-159427\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">arrived yet<\/a>.<\/p>\n<p>Because of this, some experts conjecture that \u201cwirehead collapse\u201d might well <a href=\"https:\/\/forum.effectivealtruism.org\/posts\/ZLPEju49nGxy4cFkf\/wireheading-as-a-possible-contributor-to-civilizational\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">threaten<\/a> civilisation. Our distractions are only going to get more attention grabbing, not less.<\/p>\n<p>Already by 1964, <a href=\"https:\/\/www.newyorker.com\/culture\/culture-desk\/the-beautiful-mind-bending-of-stanislaw-lem\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Polish futurologist<\/a> Stanis\u0142aw Lem <a href=\"https:\/\/books.google.co.uk\/books?id=ZO5zDwAAQBAJ&amp;pg=PT352&amp;dq=%22olds+and+milner%27s+article+is+well+known+by+now%22&amp;hl=en&amp;newbks=1&amp;newbks_redir=0&amp;sa=X&amp;ved=2ahUKEwihtsj-gtbxAhWHDsAKHb1lDxIQ6AEwAHoECAIQAg#v=onepage&amp;q=%22olds%20and%20milner's%20article%20is%20well%20known%20by%20now%22&amp;f=false\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">connected<\/a> Olds\u2019s rats to the behaviour of humans in the modern consumerist world \u2013 pointing to \u201ccinema\u201d, \u201cpornography\u201d, and \u201cDisneyland\u201d. He conjectured that technological civilisations might cut themselves off from reality, becoming \u201c<a href=\"https:\/\/books.google.co.uk\/books?id=ZO5zDwAAQBAJ&amp;pg=PT354&amp;lpg=PT354&amp;dq=%22phantomatics+seems+to+be+a+sort+of+pinnacle+around+which+various+contemporary+technologies+of+entertainment+converge%22&amp;source=bl&amp;ots=d4TVmXeJ8S&amp;sig=ACfU3U3bZ19WSYhQHfRISAAwx0jXzr2NfA&amp;hl=en&amp;sa=X&amp;ved=2ahUKEwjuzovrgtbxAhUMT8AKHWXkCQoQ6AEwAHoECAIQAw#v=onepage&amp;q=%22phantomatics%20seems%20to%20be%20a%20sort%20of%20pinnacle%20around%20which%20various%20contemporary%20technologies%20of%20entertainment%20converge%22&amp;f=false\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">encysted<\/a>\u201d within their own virtual pleasure simulations.<\/p>\n<h2>Addicted aliens<\/h2>\n<p>Lem, and others since, have even ventured that <a href=\"https:\/\/arxiv.org\/abs\/1806.02404\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">the reason<\/a> our telescopes <a href=\"https:\/\/theconversation.com\/the-end-of-the-world-a-history-of-how-a-silent-cosmos-led-humans-to-fear-the-worst-120193\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">haven\u2019t found<\/a> evidence of advanced spacefaring <a href=\"https:\/\/theconversation.com\/uk\/topics\/fermi-paradox-35915\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">alien civilizations<\/a> is because all advanced cultures \u2014 here and elsewhere \u2014 inevitably create more pleasurable virtual alternatives to exploring outer space. <a href=\"https:\/\/towardsdatascience.com\/intro-to-reinforcement-learning-the-explore-exploit-dilemma-463ceb004989#:%7E:text=After%20the%20exploration%2C%20he%20may,current%20acquired%20knowledge%20or%20information.&amp;text=This%20is%20the%20explore%2Dexploit%20dilemma%20in%20reinforcement%20learning\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Exploration<\/a> is difficult and risky, after all.<\/p>\n<figure class=\"align-center \" readability=\"7.1384615384615\">\n<p><figure class=\"post-image post-mediaBleed aligncenter\"><img decoding=\"async\" loading=\"lazy\" src=\"https:\/\/images.theconversation.com\/files\/408144\/original\/file-20210624-15-l7323d.jpeg?ixlib=rb-1.1.0&amp;q=45&amp;auto=format&amp;w=754&amp;fit=clip\" alt=\"Sketch of an alien above an insular domed civilisation.\" width=\"600\" height=\"448\" class=\"js-lazy\"><figcaption><a href=\"https:\/\/thenextweb.com\/news\/ai-addicts-experts-worried-syndication#\" data-url=\"https:\/\/twitter.com\/intent\/tweet?url=https%3A%2F%2Feditorial.thenextweb.com%2Fneural%2F2021%2F09%2F25%2Fai-addicts-experts-worried-syndication%2F&amp;via=thenextweb&amp;related=thenextweb&amp;text=Check out this picture on: Illustration to a journal article on the \u2018intelligence paradox\u2019. The caption reads: \u2018Evolution of intelligence will always lead to a drive for environmental utopia. Hence, many species may well get fat and spend much of their GDP on healthcare. Life may be everywhere, but due to obesity-related medical issues, it might have to pay a healthcare tax and simply not be able to afford space travel.\u2019 \u00a9 2014 Nunn et al.; licensee BioMed Central Ltd.\" data-title=\"Share Illustration to a journal article on the \u2018intelligence paradox\u2019. The caption reads: \u2018Evolution of intelligence will always lead to a drive for environmental utopia. Hence, many species may well get fat and spend much of their GDP on healthcare. Life may be everywhere, but due to obesity-related medical issues, it might have to pay a healthcare tax and simply not be able to afford space travel.\u2019 \u00a9 2014 Nunn et al.; licensee BioMed Central Ltd. on Twitter\" data-width=\"685\" data-height=\"500\" class=\"post-image-share popitup\" title=\"Share Illustration to a journal article on the \u2018intelligence paradox\u2019. The caption reads: \u2018Evolution of intelligence will always lead to a drive for environmental utopia. Hence, many species may well get fat and spend much of their GDP on healthcare. Life may be everywhere, but due to obesity-related medical issues, it might have to pay a healthcare tax and simply not be able to afford space travel.\u2019 \u00a9 2014 Nunn et al.; licensee BioMed Central Ltd. on Twitter\"><i class=\"icon icon--inline icon--twitter--dark\"><\/i><\/a>Illustration to a journal article on the \u2018intelligence paradox\u2019. The caption reads: \u2018Evolution of intelligence will always lead to a drive for environmental utopia. Hence, many species may well get fat and spend much of their GDP on healthcare. Life may be everywhere, but due to obesity-related medical issues, it might have to pay a healthcare tax and simply not be able to afford space travel.\u2019 <a href=\"https:\/\/nutritionandmetabolism.biomedcentral.com\/articles\/10.1186\/1743-7075-11-34\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">\u00a9 2014 Nunn et al.; licensee BioMed Central Ltd.<\/a><\/figcaption><noscript><img decoding=\"async\" loading=\"lazy\" src=\"https:\/\/images.theconversation.com\/files\/408144\/original\/file-20210624-15-l7323d.jpeg?ixlib=rb-1.1.0&amp;q=45&amp;auto=format&amp;w=754&amp;fit=clip\" alt=\"Sketch of an alien above an insular domed civilisation.\" width=\"600\" height=\"448\" class><\/noscript><\/figure>\n<\/p>\n<\/figure>\n<p>Back in the countercultural heyday of the 1960s, the molecular biologist <a href=\"https:\/\/senate.universityofcalifornia.edu\/_files\/inmemoriam\/html\/gunthersstent.html\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Gunther Stent<\/a> suggested that this process would happen through \u201cglobal hegemony of beat attitudes\u201d. Referencing Olds\u2019s experiments, he helped himself to the speculation that hippie drug-use was the prelude to <a href=\"https:\/\/www.google.co.uk\/books\/edition\/The_Coming_of_the_Golden_Age\/jz5eAAAAIAAJ?hl=en&amp;gbpv=0&amp;bsq=gunther%20stent%20golden%20age\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">civilisations wireheading<\/a>. At a 1971 conference on the search for extraterrestrials, Stent <a href=\"https:\/\/www.google.co.uk\/books\/edition\/Communication_with_Extraterrestrial_Inte\/iXIsAQAAIAAJ?hl=en&amp;gbpv=0&amp;bsq=%22archipelago%20whose%20tenants%20are%20mostly%20concerned%20about%20their%20inner%20life%20and%20not%20anxious%20to%20communicate%22\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">suggested<\/a> that, instead of expanding bravely outwards, civilisations <a href=\"https:\/\/doi.org\/10.1016\/j.actaastro.2011.11.006\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">collapse inwards<\/a> into meditative and intoxicated bliss.<\/p>\n<p>In our own time, it makes more sense for concerned parties to point to <a href=\"https:\/\/www.edge.org\/response-detail\/11475\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">consumerism, social media<\/a> and <a href=\"https:\/\/pubmed.ncbi.nlm.nih.gov\/25089149\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">fast-food<\/a> as the culprits for potential collapse (and, hence, the reason no other civilisations have yet visibly spread throughout the galaxy). Each era has its own anxieties.<\/p>\n<figure><iframe loading=\"lazy\" src=\"https:\/\/www.youtube.com\/embed\/vLmuDmd9Ymk?wmode=transparent&amp;start=0\" width=\"440\" height=\"260\" frameborder=\"0\" allowfullscreen=\"allowfullscreen\">[embedded content]<\/iframe><figcaption><span class=\"caption\">Wall-E.<\/span><\/figcaption><\/figure>\n<h2>So what do we do?<\/h2>\n<p>But these are almost certainly not the <a href=\"https:\/\/theprecipice.com\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">most pressing<\/a> risks facing us. And <a href=\"https:\/\/qualiacomputing.com\/2016\/08\/20\/wireheading_done_right\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">if done right<\/a>, forms of wireheading could make accessible <a href=\"https:\/\/www.hedweb.com\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">untold vistas<\/a> of joy, meaning, and value. We shouldn\u2019t forbid ourselves these peaks ahead of weighing everything up.<\/p>\n<p>But there is a real lesson here. Making adaptive complex systems \u2013 whether brains, AI, or economies \u2013 behave safely and well is hard. Anders works precisely on solving <a href=\"https:\/\/foresight.org\/salon\/grand-futures-thinking-truly-long-term-anders-sandberg-future-of-humanity\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">this riddle<\/a>. Given that civilisation itself \u2013 as a whole \u2013 is just such a complex adaptive system, how can we learn about inherent failure modes or instabilities, so that we can avoid them? Perhaps \u201cwireheading\u201d is an inherent instability that can <a href=\"https:\/\/ai.objectives.institute\/blog\/8gwiqyoxcbuzfuc707vz0qb4zugp2g\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">afflict markets<\/a> and the algorithms that drive them, as much as addiction can afflict people?<\/p>\n<p>In the case of AI, we are laying the foundations of such systems now. Once a <a href=\"https:\/\/slatestarcodex.com\/2015\/05\/22\/ai-researchers-on-ai-risk\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">fringe<\/a> concern, a growing number of <a href=\"https:\/\/www.penguinrandomhouse.com\/books\/566677\/human-compatible-by-stuart-russell\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">experts<\/a> agree that achieving smarter-than-human AI may be close enough on the horizon to pose a <a href=\"https:\/\/futureoflife.org\/background\/benefits-risks-of-artificial-intelligence\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">serious concern<\/a>. This is because we need to make sure it is <a href=\"https:\/\/www.fhi.ox.ac.uk\/research\/research-areas\/#aisafety_tab\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">safe<\/a> before this point, and figuring out how to guarantee this will itself take time. There does, however, remain significant disagreement among experts <a href=\"https:\/\/futureoflife.org\/background\/benefits-risks-of-artificial-intelligence\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">on timelines<\/a>, and how pressing <a href=\"https:\/\/aiimpacts.org\/ai-timeline-surveys\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">this deadline<\/a> might be.<\/p>\n<p>If such an AI is created, we can expect that it may have access to its own \u201csource code\u201d, such that it <a href=\"https:\/\/www.google.co.uk\/books\/edition\/Superintelligence\/C-_8AwAAQBAJ?hl=en&amp;gbpv=1&amp;dq=%22can+short-circuit+such+a+motivational+regime+by+directly+changing+its+internal+state+into+the+desired+configuration:+the+external+actions+and+conditions+that+were%22&amp;pg=PA122&amp;printsec=frontcover\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">can manipulate<\/a> its motivational structure and administer its own rewards. This could prove an immediate path to wirehead behaviour, and cause such an entity to become, effectively, a \u201csuper-junkie\u201d. But unlike the human addict, it may not be the case that its state of bliss is coupled with an unproductive state of stupor or inebriation.<\/p>\n<p>Philosopher <a href=\"https:\/\/www.nickbostrom.com\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Nick Bostrom<\/a> conjectures that such an agent might devote all of its superhuman productivity and cunning to \u201creducing the risk of future disruption\u201d of its precious reward source. And if it judges even a nonzero probability for humans to be an obstacle to its next fix, we might well be in trouble.<\/p>\n<p>Speculative and worst-case scenarios aside, the example we started with \u2013 of the racetrack AI and reward loop \u2013 reveals that the basic issue is already a real-world problem in artificial systems. We should hope, then, that we\u2019ll learn much more about these pitfalls of motivation, and how to avoid them, before things develop too far. Even though it has humble origins \u2014 in the cranium of an albino rat and in poems about tapeworms \u2014 \u201cwireheading\u201d is an idea that is likely only to become increasingly important in the near future.<\/p>\n<p><em>Article by <a href=\"https:\/\/theconversation.com\/profiles\/thomas-moynihan-756504\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Thomas Moynihan<\/a>, Visiting Research Associate in History, St Benet\u2019s College, <a href=\"https:\/\/theconversation.com\/institutions\/university-of-oxford-1260\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">University of Oxford<\/a> and <a href=\"https:\/\/theconversation.com\/profiles\/anders-sandberg-107819\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Anders Sandberg<\/a>, James Martin Research Fellow, Future of Humanity Institute &amp; Oxford Martin School, <a href=\"https:\/\/theconversation.com\/institutions\/university-of-oxford-1260\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">University of Oxford<\/a><\/em><\/p>\n<p><em>This article is republished from <a href=\"https:\/\/theconversation.com\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">The Conversation<\/a> under a Creative Commons license. Read the <a href=\"https:\/\/theconversation.com\/drugs-robots-and-the-pursuit-of-pleasure-why-experts-are-worried-about-ais-becoming-addicts-163376\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">original article<\/a>.<\/em><\/p>\n<p> <a href=\"https:\/\/thenextweb.com\/news\/ai-addicts-experts-worried-syndication\">Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>In 1953, a Harvard psychologist thought he discovered pleasure \u2013 accidentally \u2013 within the cranium of a rat. With an electrode inserted into a specific area of its brain, the rat was&#8230;<\/p>\n","protected":false},"author":1,"featured_media":8013,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[1],"tags":[],"_links":{"self":[{"href":"https:\/\/www.londonchiropracter.com\/index.php?rest_route=\/wp\/v2\/posts\/8012"}],"collection":[{"href":"https:\/\/www.londonchiropracter.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.londonchiropracter.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.londonchiropracter.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.londonchiropracter.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=8012"}],"version-history":[{"count":0,"href":"https:\/\/www.londonchiropracter.com\/index.php?rest_route=\/wp\/v2\/posts\/8012\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.londonchiropracter.com\/index.php?rest_route=\/wp\/v2\/media\/8013"}],"wp:attachment":[{"href":"https:\/\/www.londonchiropracter.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=8012"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.londonchiropracter.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=8012"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.londonchiropracter.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=8012"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}