How to describe something visual in preparation of said visual?

Question

I'm working on a video game with visual novel style "cutscenes" inbetween, where you see characters talking, usually from the torso up. Most of the things you'd normally describe in text-only media can be shown by the expressions and actions of the characters onscreen, like their facial expressions.

Before you start actually creating the game and its visuals, how would you write these interactions down to describe such visual cues to someone creating the visuals, without it getting in the way of what is actually shown as text on screen?

Here's how I did it so far, just as an example:

EVENT: boy and girl hear someone angrily shouting outside of their house
boy: Sounds like someone's angry.
girl: *sarcastic* Noooooo! How could you tell?

Obviously things like the whole EVENT line and *sarcastic* won't appear like that in the text, they'd be turned into visuals, with boy and girl turning their heads to the window when they hear the shouting and the girl rolling her eyes or something.

How should you describe these kinds of interactions? Should I continue with this loose structure? Are there better alternatives?

Answer 1

You are writing a script (screenplay) for a visual display; I'd follow (roughly) the format of a script. For this particular question, you are looking for "Personal Direction" (of an actor) and the standard would be to specify what you want in parentheses after the name, before the speech. Although you might like a more concise format than the standard script format.

The standard script format is specifically designed with very wide margins and double spacing so a page of dialogue will require about 60 seconds to film; and overall the duration of the film in minutes will be approximated by the number of pages in the script. That may not be important to you in a game; and you can always stopwatch reading the lines out loud, or going through the motions of acting out your scene, to get an idea of how much animation will be required to render it.

Girl (sarcastic): Noooooo! How could you tell?

As you will see at the link, you can use more than a word, often whole sentences are used. As in a script, with no parenthetical, the expression to use is up to the actor and director; in your case that would be whomever is rendering your video. In your position I would make a point of deciding in each case; sooner or later someone has to decide.

boy (amused): Sounds like someone's angry.
girl (sarcastic, amused): Noooooo! How could you tell?

EDIT: Here is a list of 100 one-word facial expressions you might find useful.

Answer 2

I've been writing an actual visual novel for a few years, so I'm going to approach this from a game development perspective as much as a writing perspective.

To help me with this problem, I borrowed a technique from the VN Katawa Shoujo. The images for the character sprites are all named using a pattern something like name_outfit_pose_expression, with outfit being omitted if they're wearing their school uniform.

So what I've started doing is, whenever a character's pose and/or facial expression would change, I insert a line of pseudocode with a fake file name, telling me what the new pose/expression would be. Something like:

Boy: Sounds like someone's angry.
[show "girl_foldarms_rolleyes"]
Girl: Noooooo! How could you tell?

This also helps you keep track of how many sprites you need for each character when it's time to do the artwork. If a search for show 'girl_ returns 400 results, you'll need to do a rethink of whether you actually need that many sprites, because that's a lot of work for the art department.

Answer 3

I don't think it's going to be possible to translate every written interaction into something that's understandable in purely visual form.

In the example you provide, for example, I can't think of any kind of visual representation that would unambiguously convey sounds like somebody's angry. In fact, any kind of surprised expression would more likely convey something like a generic What was that noise?

I'm not saying that you shouldn't give up on some things, but I think you'd be more successful if you limited your vocabulary to a series of much more basic cues.

As another example, if somebody "jumping" and looking elsewhere is a cue for What was that? the other person shrugging would likely be interpreted as I don't know.

Meanwhile, eye rolling of one person would more likely be interpreted as some kind of irritation of that person with what the other person has said, not something more refined.

In other words, don't assume that visual cues can convey any kind of subtlety—stick to broad and "cartoonish" displays which are not likely to be misinterpreted.

Of course, if I've misinterpreted, and you actually mean to provided text along with the visuals, then that's something else. I would expect an animator to create the visuals appropriate to the text just as I would expect a director to manage the expressions of actors who follow the dialogue they are speaking.

Answer 4

Depending on your story engine there will likely be an embedded tagging system to signal instructions to the game engine via the script. The general idea is that you'd use a consistent tag followed by short commands that are parsed from the script before the words are printed to the screen, for example:

@BOY(LAUGH) Sounds like someone's angry.

@GIRL(EYEROLL) Noooooo! How could you tell?

The game engine parses each line as it comes out of the story engine. It sees the @ and reads the letters that follow to learn the character who is speaking, and their facial gesture that goes with that line.

There might also be tags for @AUDIO, @SCENE, @BKG @INCLUDES, etc. If you are not the one programming the game engine, you just need to keep your tags consistent and simple so they can be edited by the programmer with SEARCH/REPLACE.

The general idea is that there will be a limited number of facial gestures (typically 6-8). Your expression tags will be edited by the programmer to match the facial gestures that are available. In dynamic stories, characters will be swapped by the story engine but their face gestures will have the same names – ergo it won't matter if Linda or Bill says a certain line, the script will still provide them with the correct expression.

The story engine script will probably assemble parts of the story dynamically, so the actual script will be replaced by variables and function calls. Let's say a scene might have Linda or Bill as the friend who reacts. Bill also thinks it's funny and responds with sarcasm. Linda however knows who is yelling and her reaction is different. The story engine script will look something like:

@AUDIO("ManShouts.WAV")

@WAIT(5)

@{NPC1(LAUGH)} Sounds like someone's angry.

@{FRIEND(DO_REACTION)} {FRIEND(GET_REPLY)}

The story engine dynamically updates the script variables, filling in the character's names, gestures, and replies. The game engine parses the directions from the script and displays the appropriate sound and graphics, in this case the variable FRIEND is replaced by either Bill or Linda and the game engine sees:

@LINDA(UPSET) I'll see you later.

The more you learn about the underlying story and game engine, the more dynamic your narrative can become, but there will always be this bottleneck between the story engine and the game engine where simple directions are parsed from the script itself. Have a conversation with the programer and artist so everyone can be on the same page with what is possible.