{"id":12130,"date":"2022-08-26T17:49:18","date_gmt":"2022-08-26T17:49:18","guid":{"rendered":"http:\/\/TheNextWeb=1390889"},"modified":"2022-08-26T17:49:18","modified_gmt":"2022-08-26T17:49:18","slug":"how-an-award-winning-ai-film-was-brought-to-life-by-text-to-video-generation","status":"publish","type":"post","link":"https:\/\/www.londonchiropracter.com\/?p=12130","title":{"rendered":"How an award-winning AI film was brought to life by text-to-video generation"},"content":{"rendered":"\n<p>If you\u2019re impressed by the <a href=\"https:\/\/thenextweb.com\/news\/google-takes-on-openai-with-flashy-text-to-image-generator\" target=\"_blank\" rel=\"noopener noreferrer\">recent spate<\/a> of <span><a href=\"https:\/\/thenextweb.com\/news\/openai-dalle-2-text-to-image-generator-shows-ai-could-transform-design\" target=\"_blank\" rel=\"noopener noreferrer\">text-to-image generators<\/a>, <\/span>get ready for the next step in <a href=\"https:\/\/thenextweb.com\/topic\/artificial-intelligence\" target=\"_blank\" rel=\"noopener noreferrer\">AI<\/a> artistry: text-to-video.<\/p>\n<p>While the huge compute costs and scarcity of text-to-video datasets have stunted the technique\u2019s growth, <a href=\"https:\/\/arxiv.org\/abs\/2205.15868\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">recent research<\/a> has brought the promise closer to reality.<\/p>\n<p>A computer artist called Glenn Marshall has given a glimpse at the potential.<\/p>\n<div class=\"inarticle-wrapper neural channel-cta hs-embed-tnw\">\n<div id=\"hs-embed-tnw\" class=\"channel-cta-wrapper\" readability=\"6\">\n<div class=\"channel-cta-img\"><img decoding=\"async\" class=\"js-lazy\" src=\"https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2022\/07\/neural.webp\"><\/div>\n<p><noscript><img decoding=\"async\" src=\"src='https:\/\/cdn0.tnwcdn.com\/wp-content\/blogs.dir\/1\/files\/2022\/07\/neural.webp'\"><\/noscript><\/p>\n<div class=\"channel-cta-input\" readability=\"7\">\n<h2 class=\"channel-cta-title\">Greetings, humanoids<\/h2>\n<p class=\"channel-cta-tagline\">Subscribe to our newsletter now for a weekly recap of our favorite AI stories in your inbox.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<p>The Belfast-based composer&nbsp;recently won the Jury Award at the Cannes Short Film Festival for his AI <em>The Crow<\/em>.<\/p>\n<p><iframe title=\"The Crow\" width=\"500\" height=\"281\" srcdoc=\"\n\n<style>*{padding:0;margin:0;overflow:hidden}html,body{background:#000;height:100%}img{position:absolute;top:0;left:0;width:100%;height:100%;object-fit:cover;transition:opacity .1s cubic-bezier(0.4,0,1,1)}a:hover img+img{opacity:1!important}<\/style>\n<p><a href='https:\/\/www.youtube.com\/embed\/5dvxY6vXHsA?feature=oembed&amp;autoplay=1&amp;mute=1&amp;modestbranding=1&amp;iv_load_policy=3&amp;theme=light&amp;playsinline=1'><img src='https:\/\/img.youtube.com\/vi\/5dvxY6vXHsA\/hqdefault.jpg'><img src='https:\/\/cdn0.tnwcdn.com\/wp-content\/themes\/cyberdelia\/assets\/img\/ytplaybtn.png' style='top: 50%;left:50%;width:68px;height:48px;transform:translate3d(-50%,-50%,0)'><img src='https:\/\/cdn0.tnwcdn.com\/wp-content\/themes\/cyberdelia\/assets\/img\/ytplaybtn-hover.png' style='top: 50%;left:50%;width:68px;height:48px;opacity:0;transform:translate3d(-50%,-50%,0)'><\/a>&#8221; frameborder=&#8221;0&#8243; allow=&#8221;accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture&#8221; allowfullscreen>[embedded content]<\/iframe><\/p>\n<p>Marshall had previously earned plaudits for an <a href=\"https:\/\/thenextweb.com\/news\/daft-punk-ai-music-video-tribute-voyager-discovery\" target=\"_blank\" rel=\"noopener noreferrer\">AI-generated Daft Punk video<\/a>, but he applied a different approach to <em>The Crow<\/em>.<\/p>\n<p>While his earlier technique turned text into <span>random visual mutations<\/span>, <em>The Crow<\/em> uses an underlying film as an image reference.<\/p>\n<p><span>\u201cI had been heavily getting into the idea of AI style transfer using video footage as a source,\u201d Marshall told TNW. <\/span><\/p>\n<p><span>\u201cSo every day I would be looking for something on YouTube or stock video sites, and <a href=\"https:\/\/thenextweb.com\/topic\/openai\" target=\"_blank\" rel=\"noopener noreferrer\">trying<\/a> to make an interesting video by abstracting it or transforming it into something different using my techniques.&nbsp; <\/span><\/p>\n<p><span>\u201cIt was during this time I discovered <a href=\"https:\/\/www.youtube.com\/watch?v=Pd2KM3qjcKk\" target=\"_blank\" rel=\"nofollow noopener noreferrer\"><em>Painted<\/em><\/a>&nbsp;on YouTube \u2014 a short live action dance film \u2014 which would become the basis of <em>The Crow<\/em>.\u201d<\/span><\/p>\n<p>Marshall fed the video frames of <em>Painted <\/em>to<span>&nbsp;<a href=\"https:\/\/openai.com\/blog\/clip\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">CLIP<\/a>, a neural network created by OpenAI.<\/span><\/p>\n<p>He then prompted the system to generate a video of \u201ca painting of a crow in a desolate landscape.\u201d<\/p>\n<blockquote class=\"twitter-tweet\" data-width=\"500\" data-dnt=\"true\" readability=\"4.5170068027211\">\n<p lang=\"en\" dir=\"ltr\">My AI film &#8216;The Crow&#8217; wins Jury Award at Cannes!<a href=\"https:\/\/t.co\/WHDsI7UzJM\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">https:\/\/t.co\/WHDsI7UzJM<\/a> <a href=\"https:\/\/t.co\/Ww1DGyBbxw\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">pic.twitter.com\/Ww1DGyBbxw<\/a><\/p>\n<p>\u2014 Glenn Marshall (@GlennIsZen) <a href=\"https:\/\/twitter.com\/GlennIsZen\/status\/1562362889377353728?ref_src=twsrc%5Etfw\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">August 24, 2022<\/a><\/p>\n<\/blockquote>\n<p>Marshall says the outputs required little cherry-picking. He attributes this to the similarity between the prompt and underlying video, which depicts a dancer in a black shawl mimicking the movements of a crow.<\/p>\n<p><span>\u201cIt\u2019s this that makes the film work so well, as the AI is trying to make every live action frame look like a painting with a crow in it, so I\u2019m meeting it half way, and the film becomes kind of a battle between the human and the AI \u2014 with all the suggestive symbolism.\u201d<\/span><\/p>\n<p>In the future, Marshall wants to add 3D animation to his AI creations. He\u2019s also exploring <span>CLIP-guided video generation, which can add detailed text-based directions, such as specific camera movements.<\/span><\/p>\n<p>That could lead to entire feature films produced by text-to-video systems. Yet Marshall believes even his current techniques could attract mainstream recognition.<\/p>\n<p>He says <em>The Crow<\/em> is now eligible for submission to the prestigious BAFTA Awards.<\/p>\n<p><span>\u201cI haven\u2019t got a speech prepared, but I fantasize about collecting an award, in the role of a herald of AI, and proclaiming to the star-studded audience that [for] each and every one of you, actor, director, set designer, costume designer, artist, composer\u2026 AI is coming, and you\u2019ll find yourself in a very different job soon \u2014 or out of a job all together.\u201d<\/span><\/p>\n<p> <a href=\"https:\/\/thenextweb.com\/news\/text-to-image-generator-creates-award-winning-ai-film-the-crow\">Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>If you\u2019re impressed by the recent spate of text-to-image generators, get ready for the next step in AI artistry: text-to-video. While the huge compute costs and scarcity of text-to-video datasets have stunted&#8230;<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[1],"tags":[],"_links":{"self":[{"href":"https:\/\/www.londonchiropracter.com\/index.php?rest_route=\/wp\/v2\/posts\/12130"}],"collection":[{"href":"https:\/\/www.londonchiropracter.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.londonchiropracter.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.londonchiropracter.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.londonchiropracter.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=12130"}],"version-history":[{"count":0,"href":"https:\/\/www.londonchiropracter.com\/index.php?rest_route=\/wp\/v2\/posts\/12130\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.londonchiropracter.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=12130"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.londonchiropracter.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=12130"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.londonchiropracter.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=12130"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}