Well things have moved on pretty fast.
Its hard to believe how quickly image to video has moved on in the last two years. To learn about it a bit more I decided to make a music video to test “image to video” generation with Krea.ai and Klingai.com, plus MidJourney for image making. I created and found images in MJ, then put them into Photoshop to create montages. Using Krea and Kling I generated loads of 5 second “image to video” from those images. Firstly I did it as a comparison to see examples of how the two AI apps would animate photomontage images. The video outputs I created were not 1080p to keep them affordable and easer to personally produce for this tryout. Each generation of ‘image to video” costs a few credits which I prepaid for.
Most generated very well, but others came out very wonky and deteriorated within 2 seconds of starting. So a fair amount of trial and error was needed. Im not sure they would do well with realistic, detailed character generated footage. Being able to make subtle changes and corrections is so important to finishing off an animation project professionally. Having the ability to make specific changes a client would want at the end of a video commission would be hard to do, currently. Those fast close to deadline changes I would be concerned about. Due to the fast progress I imagine this will change soon. I first tried “image to video” in 2023 with Runway and was not that impressed. Kling and Krea are amazing in comparison from as little as 2 years ago.
The eventual cutting of costs for the feature film industry will be colossal. The idea of some sort of live or near live AI graphics engine sounded like a complete impossibility 2 years ago, but soon may happen. This could rival Unity and unreal for example. I fully functioning live graphics engine that can respond to any form of input. For example code input, text prompts, a person talking to it, a conventional game engine input or live video stream plus style transfer. If this type of thing is even possible it would be a completely new form of visual engine, for any type of visual media.
Generating the videos for the test music video I made I found the two sites to be pretty powerful tools to use and very quick. I burnt through a month of credit/Generation minutes in 48hours. Even using the older Kling 1.5 model which is 5 times cheaper to use than the new 2.0. Krea has style transfer function which I will look into soon. I will defiantly look into training an AI generator on just my own illustrations and animations I have created in Blender and PS over the years.
The idea behind the video is about a dystopian montage of information overload, phones and being purged in data and personal information. Watch to the ending to see a possible antidote to it.
Enjoy
Watch to the end!
https://www.youtube.com/watch?v=edAZl1fQMOE
Music by the great, Future Sound of London. Called: Purged
⁃ Update: As I finished this blog Kling got updated to 2.1 adding 1080 pixels.
