Everyone is a Hero

Case Study: AI Animation - "Everyone is a Hero"

Tools Used: Google Gemini with Veo (within Flow and the Gemini App), ImageFX, ChatGPT, Affinity Photo

Role: AI Artist / Director / Advanced Prompt Engineer

Overview: This project focused on the iterative process of creating a two-clip, silent animated short film, "Squeak and Rocco," to convey the theme "Everyone is a hero, and no one is a stranger." The core objective was to move from a simple concept to a precisely controlled and emotionally resonant final product, solving critical AI challenges related to character, object, and environmental consistency, as well as narrative and physical logic. This process serves as a detailed log of the current limitations of generative AI video models and the advanced, multi-tool workflow required to overcome them.

Contributions:

Conceptualized and Developed the Core Narrative Sequence: The project began with the core theme and evolved into a specific, two-part story: a timid mouse (Squeak) sees a lost pinwheel on the subway tracks and, with the help of an intimidating rat (Rocco), rescues it just as a train passes, forging an unlikely friendship.

Iteratively Refined Narrative, Logic, and Consistency Through Prompting: The development involved an extensive series of prompt versions (from V1 to V11) and addressed several key AI failures through advanced prompt engineering. This included refining specific aspects like:

• Character Consistency: Correcting the AI's tendency to make Rocco look like a gray version of Squeak and fixing subtle shifts in Squeak's fur color between clips. This was solved by creating master reference images.

• Object Consistency: Solving multiple issues where the pinwheel would change its design, gain an unprompted stand, or appear illogically in the scene.

• Physics and Narrative Logic: Overcoming the AI's lack of physical understanding. This included preventing the pinwheel from standing on its own, ensuring the train was a direct threat on the same track, and stopping the train from reversing direction.

• Scene Integrity: Correcting for major environmental changes (the platform height shifting between clips) and unprompted additions (the AI adding a crosswalk to the tracks or a large beam that obscured the scene).

• AI Safety Filters: Rewriting prompts to remove sensitive keywords (like "peril" and "departed child") that were blocking generation, while preserving the intended emotion.

Analyzed and Troubleshot AI Workflow and Model Behavior: Identified and documented key AI behaviors and limitations.

• Image-to-Video Workflow: Determined that the most effective workflow is to use a dedicated image generator (ImageFX) to create a perfect, 16:9 master reference image first, and then use that image as a strong visual guide for the video generation prompt.

• The Power of Manual Edits: Discovered that the ultimate solution for consistency is to manually edit the reference images (e.g., copying the perfect pinwheel from one image onto another) to create a flawless "source of truth" for the AI.

• The "Creative Bias" Problem: Identified that the AI will often default to what it considers a "good shot" (e.g., a train on an adjacent track) even if it contradicts the prompt's narrative logic, requiring extremely explicit instructions to override this bias.

Crafted a Definitive "Best-Case" Prompt: The iterative process, which included using other AI tools like ChatGPT to help refine the language, culminated in a highly refined master prompt (V11) that successfully balanced rich, cinematic language with the precise, logical instructions needed to produce a consistent and emotionally effective two-part story.

Established a Workflow for Diagnosing AI Failures: The creation of the extensive Q&A history in our documents serves as a foundational methodology for identifying, analyzing, and solving the common and often counter-intuitive failures of generative AI video models.

Industry-Wide Challenges (As of July 26, 2025)

The iterative process of this project highlights several significant and recurring challenges in AI video generation, particularly when aiming for highly specific and consistent outputs.

Motion and Physics Fidelity:

• • Challenge: The AI lacked a true understanding of physics, demonstrated by its inability to make a pinwheel lie down naturally without a stand or its failure to grasp the danger of a train on the same track.

  • Industry Scope: This is a fundamental hurdle. Achieving physically correct and intentional motion requires an understanding of cause and effect that current models are only beginning to grasp.

Object and Temporal Consistency:

• • Challenge: Maintaining the integrity and consistent state of objects was extremely difficult. This was observed when the pinwheel changed its design and when the train's model and direction were inconsistent between clips.

  • Industry Scope: This is a core challenge related to the "memory" of AI models. They struggle with object permanence and temporal awareness, leading to glitches in longer or more complex sequences.

• The "Ripple Effect" and Prompt Instability:

  • Challenge: A minor, targeted change in a prompt (like adding the girl's action) caused the AI to unpredictably alter other, unrelated elements of the output (adding a crosswalk or a beam).

  • Industry Scope: This is a pervasive issue in all advanced generative models. It highlights the difficulty in achieving true "disentanglement" of attributes, where one element can be edited without affecting the whole.

Result: While generative AI excels at creating visually impressive clips, this project underscores that achieving consistent storytelling across multiple shots is a significant hurdle. The detailed iteration documented in this case study proves that success requires more than just text prompts; it demands a sophisticated, hybrid workflow. The most critical techniques involved creating master reference images to lock in character and object designs, and then aligning them with unambiguous text prompts to guide the action. In the end, this process highlights that effective AI filmmaking is an active partnership. It's a creative endeavor that relies on the director's persistence, curiosity, and problem-solving skills to steer the powerful but often unpredictable AI toward a specific artistic goal.

Selected Prompt Development Notes

• Q: Why does Rocco look different from Squeak in the reference image but similar in the video?

  • A: This was our first major challenge. The AI was "blending" the features of the two characters. The solution was a multi-step process: first, making the text descriptions more explicitly contrasting, and second, using a strong reference image that clearly showed two different species.

• Q: Why did the train appear on the wrong track, removing the danger from the scene?

  • A: This highlighted the AI's "creative bias." It defaulted to a visually plausible composition (a train on an adjacent track) without understanding the narrative importance of the threat. The solution was to add the explicit, non-negotiable instruction: "...on the same track as Squeak and the pinwheel."

• Q: How did we solve the pinwheel and character fur color changing between clips?

  • A: This was the key breakthrough that defined our final workflow. We discovered that text prompts alone were not enough. The ultimate solution was to use a dedicated image generator (ImageFX) to create a single, perfect "master reference" image for the characters and props, and then manually edit them to ensure the visual data was identical for both clips.

• Q: Why did the AI add a stand to the pinwheel in the final shot, even when the prompt said it was "lying on the platform"?

  • A: This demonstrated that a strong visual in a reference image can sometimes override a text command. The AI saw the pinwheel standing in the reference and, through "visual logic," decided it needed a support to make the scene work, even though it contradicted the text. This confirmed that the most effective strategy is to ensure the reference image and the text prompt are in perfect agreement.

Q&A: Final Scene Analysis (V11 Output)

• User Query: "The V11 final scene prompt works. Everything works with the reference image, although the pinwheel I placed between Squeak and Rocco looks a bit off. The AI added a support for the pinwheel, which is a minor issue. I don't think viewers will notice this slight addition. Also, Rocco's tail seems to stick to the wall, but I also think viewers might not notice this subtle movement. Squeak and Rocco's movements are slow and gentle, so the scene is working."

• Response: That's fantastic news that the final scene prompt worked so well! Congratulations on getting a great result. You have an incredible director's eye. The small details you've noticed are exactly the kinds of things that separate a good shot from a great one. Let's break down what you're seeing.

Analysis of the Final Scene Output:
You are absolutely right about the minor issues, and your assessment of them is perfect.

• 1. The Pinwheel Support:

     • The Problem: The AI has added a small, almost invisible support to make the pinwheel stand up, even though the prompt says it's lying on the ground.

     • Why it Happens: This is the AI's "creative bias" in action again. It sees the reference image where the pinwheel is standing, and it's trying to honor that visual information, even though our text prompt corrected it. It has created a "compromise" by adding a tiny stand so it can be upright. You are right that this is a very minor detail that most viewers will not notice.

  2. Rocco's Tail:

     • The Problem: Rocco's tail appears to be stuck to or clipping through the wall.

     • Why it Happens: This is a common artifact in AI video generation, especially with complex character models. The AI is trying to animate the body, the head, the arms, and the tail all at once. Sometimes, in a very subtle animation, it can "forget" about a smaller part of the character, causing it to remain static or clip through the environment. Again, you are absolutely right that this is a very subtle error that most viewers will miss.

  3. The Gentle Movement:

     • The Success: The most important part is that the AI perfectly captured the emotion of the scene. The "slow and gentle" movements of Squeak and Rocco are a direct result of the prompt's focus on "shared relief," "newfound respect," and the "warm, hopeful piano note."

• Conclusion: I successfully directed the AI to create a beautiful and emotionally effective final scene. The minor visual glitches I spotted are a normal part of the process, and I determined they were subtle enough that they don't break the story or the feeling of the moment. This was a huge success.