so i kept running into this thing when prepping scenes for AI generation
id write the script, know exactly what i wanted. the framing, the mood, how the camera moves. and then sit down to prompt it and just… lose it. by the time its in text form its already wrong. the model picks something technically fine but its not the shot i had in my head.
took me a while to realize the problem wasnt the prompt. it was that i was skipping the storyboard. thats the step where you actually lock your decisions as a director – shot size, angle, blocking. without it youre just describing vibes and hoping the model fills in the gaps the way you would.
started building something to force myself through that step faster. uploads the script, spits out a shot list, generates rough frames i can use as image-to-image references before i touch kling or runway. honestly just using it to not skip the part i always used to skip
its called StoryBirdie — Professional AI Storyboarding Software
anyone else find this is where the breakdown happens, between the script and the first prompt?
so with our film treatment and our shot list on the edit bay desk we commence. And we talk and we review the shot. I have learned that AI in certain situations like narratives that need matched motion and reliable performance that A I can be a real pain in the ass. So after cranking out 12 projects over the last five months from music videos to epic poem adaptations, I decided to try my hand at this kind of narrative. And it was a fight the whole way. I would spontaneously come up with shots that might have been suggested by our film treatment/shot list but I had gave myself permission to come up with better things if inspired. I love the imagery from AI and I hate the unreliability of performance. Whether it’s hitting marks going in a particular direction saying a particular thing at the right time these models can be real crappy actors. But I’ve discovered a new hybrid approach to take back control from the models. And it uses a very innovative idea that seems to be slowly seeping into the industry. What is it in involves is first creating a 3-D animation. Using your set and your characters and your camera placement you get everything back when it comes to framing shots, blocking shot shots and getting the right timing on everything. Basically you set up a scene if you’re familiar with Blender or light wave. And all you are basically doing is doing simple low resolution animations of exactly the motions of the character and the timings you want. The cool thing is your animation will output low color information as well as depth map information which describes all the 3-D surfaces in your little movie. That is fed into control net, not Skynet :), and basically forces AI to follow your direction. In addition it allows you to wrap your AI imagery flawlessly around everything in your animation. So basically you are documenting all the moves and the timing and letting AI be guided by it as well as beautify it. You never see the animation it is just a guide and the control that we need to impose at times with models. I’m in the learning curve and as a professional editor for 30 years I’m loving the fact that this is powerful and complicated software like I’ve used in the past. None of this… Oh I’m a director because I just told the computer what to do and push the button for it to do it… Sorry that’s not directing. It’s making believe you are director face. Anyway I thought I would answer your question and am embellish on it a bit with something I am very excited about. Have fun!