JSON prompts for AI video: what the vendors say, and what actually helps

What is a JSON prompt for AI video?

A JSON prompt is a video prompt written as a JSON object: named fields such as subject, action, camera, light, audio and dialogue, each holding a short piece of text. Creators share these on YouTube and Reddit, often with the claim that the model "reads" JSON better than a paragraph. This page checks that claim against what the vendors publish, and keeps what is useful about the idea.

We are fair to JSON here. It is a perfectly good way to organise a prompt. What we could not find is a vendor document saying the video model treats it as anything other than text. We also ran no generations for this article, so we cannot tell you that JSON beats labelled plain text, or the reverse. Nobody can promise views either, and nothing below does.

Do Veo, Seedance, Kling and Sora read JSON?

We read each vendor's own prompt guide on October 22, 2026 and looked for JSON. The short version: they document text. Some organise it with labels, shots or fields written in plain language, and none says a parser reads braces and keys.

Our reading, not a vendor statement: no guide we could read tells users to write prompts as JSON, apart from LTX for its own product. Nobody publishes whether JSON syntax reaches a dedicated parser or is simply read as text, so we cannot say a model "reads JSON as code" or that it "ignores" it. We did not find ByteDance's own Seedance documentation, so the Seedance rows rest on two distributors.

A claim you will see is that JSON "reduces randomness by about 70%". We found it on a forum guide and on an agency page, each with no method and no source, so we do not repeat it. Another is a user saying a Google engineer told a talk audience that JSON prompts were the way to go. We could not verify that, and Google's published guides do not say it.

Why does structure help?

What the guides agree on is that a prompt is better when its parts are separated and specific. Google's prompt anatomy lists subject, action, scene, camera angles and movements, style and audio. Higgsfield warns that skipping a section makes a clip "fail in a specific, predictable way", and calls a vague location "the most common reason" a multi-shot sequence drifts between cuts. A JSON object forces you to fill each slot or consciously leave it null.

• It exposes gaps. An empty light field is easier to spot than a missing sentence in a paragraph.
• It limits each shot. One camera_move field nudges you toward one move, which Runway and OpenAI both advise.
• It separates dialogue from description. OpenAI suggests dialogue in its own block below the prose.
• It can be reused and checked. You can copy a character block unchanged into every shot, and compare two versions line by line.

Notice that every benefit above comes from the fields, not from the braces. That is why a labelled plain-text prompt can hold the same information, as the next section shows.

What does one example look like, as JSON and as plain text?

Here is an original two-shot, eight-second shot list for a faceless animated story, written for this article. The same content follows as labelled text. We have not generated either one, and we cannot tell you which would look better.

{
  "project": {
    "aspect_ratio": "9:16",
    "total_seconds": 8,
    "style": "soft 2D storybook animation, warm palette, gentle film grain"
  },
  "characters": {
    "baker": "middle-aged woman, short grey curls, flour-dusted green apron, round glasses, kind tired eyes"
  },
  "location": "small neighbourhood bakery at dawn, wooden counter, trays of bread, window with blue morning light",
  "shots": [
    {
      "id": 1,
      "seconds": "0 to 4",
      "shot_type": "medium shot",
      "camera_move": "slow push in",
      "action": "The baker slides a tray of rolls onto the counter, then stops and looks at the shop door.",
      "light": "cool window light, warm oven glow behind her",
      "audio": "oven hum, soft clink of the tray",
      "dialogue": null
    },
    {
      "id": 2,
      "seconds": "4 to 8",
      "shot_type": "close-up",
      "camera_move": "static",
      "action": "She smiles and speaks quietly to someone off screen.",
      "light": "same as shot 1",
      "audio": "quiet room tone",
      "dialogue": { "speaker": "baker", "line": "You came back. The first batch is yours." }
    }
  ],
  "constraints": "no subtitles, no logos, no watermarks, no extra characters"
}

GLOBAL STYLE: soft 2D storybook animation, warm palette, gentle film grain. Vertical 9:16, 8 seconds total.
CHARACTERS: baker, a middle-aged woman with short grey curls, a flour-dusted green apron, round glasses and kind tired eyes.
LOCATION: small neighbourhood bakery at dawn, wooden counter, trays of bread, window with blue morning light.

Shot 1 (0 to 4 s): medium shot, slow push in. The baker slides a tray of rolls onto the counter, then stops and looks at the shop door. Light: cool window light, warm oven glow behind her. Audio: oven hum, soft clink of the tray.

Shot 2 (4 to 8 s): close-up, static camera. She smiles and speaks quietly to someone off screen. Light: same as shot 1. Audio: quiet room tone. Dialogue: the baker says: You came back. The first batch is yours.

CONSTRAINTS: no subtitles, no logos, no watermarks, no extra characters.

A few choices in it are deliberate:

• One camera move per shot. A slow push in, then static.
• The character and location are written once and reused. Repeating them unchanged is the practical way to try for consistency. It does not guarantee it.
• Dialogue is its own field and short. Two short sentences for a four second slot.
• No resolution or frame rate in the prompt. Set output format in your tool's controls unless the tool says it reads them from the prompt.
• Quotes differ between the two. JSON requires quotes around its strings, but Google's best-practice page says to avoid quotation marks around speech so the words are not rendered as text. In the flattened version we wrote a colon instead. Test how your tool handles it.

Where does JSON earn its keep?

Users in the r/VEO3 thread we read describe it as a workflow tool more than a magic input. One says it "keeps it nice and tidy in case another prompter needs to edit a certain part" but that they saw no creative difference against plain text, and called it a "pipeline deal". Another says it can be "more strict and consistent" and that "you can automate it". A third says they preferred the JSON results most of the time, which is a self-report, not a test. These are individual opinions.

That matches where the format has an edge, whatever the model does with it:

• Reusable blocks. Keep one character and one location object, then change only the action and camera per scene.
• Scripting. If a script or automation fills in scenes, a structured file is easier to generate and validate than free text.
• Handoffs. A teammate or an AI writing partner can edit one field without rereading the whole prompt.
• Review. You can scan each shot for the same fields before you spend credits.

LTX Studio's own guide takes a similar line for its product: explore in natural language first, then convert to JSON once you have a direction, and keep scene and subject fields constant while varying camera across shots.

What goes wrong, and how do you fix it?

Most failures are not about JSON at all. Each row below is tied to a source or labelled as our inference. We have not reproduced any of them ourselves.

When is plain text fine?

For a single short clip, or early exploration, a few plain sentences are enough. OpenAI says shorter prompts give the model creative freedom, and LTX suggests starting with natural language. Labelled text, like the example above, gives you most of the benefit with no syntax to break. Choose JSON when you reuse blocks, script scenes or hand prompts between people and tools.

If you already have footage and want an AI agent to cut it, that is a different prompt. See our guide to AI video editing prompts. If you are planning a whole animated story, our walkthrough of how to make AI story videos covers the wider process. Rushilo, our editor, imports clips and edits them with your own AI, but it does not generate video. We make Rushilo, so weigh this section accordingly.

We also sell a course, AI Video Empire, which lists JSON shot-list prompting among its contents for animated story videos with no face on camera. It is a one-time payment of $27 with lifetime access, and the tools it uses each have their own paid plan or credits. We sell it, so weigh this accordingly, and remember that nobody can promise views. For vendor details, read each guide in the sources below, because limits and models change. The Google Veo 3.1 guide is a good place to start.

Patel, O. (2026, October 22). JSON prompts for AI video: what the vendors say, and what actually helps. Rushilo. https://rushilo.com/blog/json-prompts-for-ai-video

About the author

Om Patel

Founder, Rushilo

Om Patel is the founder of Rushilo, a desktop video editor driven by the user's own AI, and of BigIdeasDB, a research platform that reads real complaints and reviews to help founders validate products. He writes Rushilo's research articles with the same method: primary sources, dated numbers and the words real users write.

• X (Twitter)
• BigIdeasDB