Is there a good way to subvert expectations?

Question

In Star Wars, people were mad because of how the writers decided to subvert expectations by making Luke Skywalker behave in an unexpected way, and then there's how Chainsaw Man subverted expectation by making the final boss be pretty much irrelevant to the whole story even though the writer kept foreshadowing the final boss and lead the readers believe that the final boss was indeed the final boss and flipped the script by making the final boss completely irrelevant to the story. Is there a good way to subvert expectations, or almost anything can be done as long as you somehow predict how your readers will react to it?

Answer 1

If you're going to subvert expectations and have a positive audience response, you need to do two things: reveal that you're telling an even better story than your audience thought you were telling, and also be able to show that the new story is really the one you were telling all along.

A better story?

In the new Star Wars, when it turns out that Luke Skywalker screwed up, then gave up and ran away and has been waiting around to die, that is not a better story than "Luke Skywalker is going to save us!" At least, not for all the fans who fondly remembered Luke as the hero they idolized in their youth. If Luke had been guarding not a Jedi temple, but a Sith temple, and keeping back a threat which made the First Order look like small potatoes, and he was freaking Luke SKYWALKER, the amazing guy who has now sacrificed the past few decades for the sake of everyone, then Star Wars fans might have said it was even better than him being cloistered in a Jedi Temple when people needed him. Acceptable subversion.

Was this the story you were being told all along?

I haven't read Chainsaw Man, but I saw Frozen. In Frozen, it turns out that the handsome prince the younger princess has fallen for is really indifferent to her and is happy to let her die and to assume power. There was no foreshadowing of his betrayal. On the contrary, the last shot of his too-fast romance, when you are supposed to believe that he is in immature and puppyish love with Anna, is of him gazing after her with adoration, and no dissimulation is apparent. Was having him not be the one to save Anna later a better story? Possibly. Was having the too-fast romance be too-good-to-be-true a better story? Almost definitely. Was the twist fairly foreshadowed? Absolutely not. If the betrayer-prince had been torn by feelings for Anna, but ultimately overcome by greed, it might have been argued that you were still telling the same story you'd actually been telling all along. If there had been some sign the prince did not really reciprocate Anna's enthusiasm earlier, it would have been fair. But as Frozen stands, you're really not given fair warning.

It's fine to make a story surprising and unexpected, even in very unexpected ways. But you should respect your audience enough to fairly foreshadow, and you definitely shouldn't be telling them a worse story than you were promising before the subversion.

Answer 2

Foreshadowing is generally what separates a good twist from a bad twist.

A bad twist comes out of nowhere, makes no sense in canon and defies previously established lore or logic. It's all shock and no substance. Or worse, it's neither shock nor substance.

A good twist makes perfect sense in the established lore, takes the story in a compelling new direction, and/or recontextualizes the whole narrative.

Take the original Star Wars trilogy. The reveal Darth Vader is Luke's father. The "villain is the hero's parent" is a trope as old as time, but it works because it solves a few basic questions. Why does Vader even care about Luke? Why did Obi-wan say Vader killed his father? Etc.

It also paints Obi-wan in a different light. Makes Luke question his wisdom. Think he's a liar. The twist creates conflict and self doubt. Luke's fighting his own flesh and blood.

The Vader twist makes sense in the lore, adds compelling conflict, and recontextualizes past scenes. Good twist. Gold star.

Bad twist-Episode Nine revives Palatine for no apparent reason.

For starters, it comes out of nowhere. Until Episode Nine we get no indication Palatine is alive except the vaguest hints. No foreshadowing.

It makes no sense in context either. In the original trilogy Palatine died. Exploded. Vaporized. Left in the void of space. If that can't kill him, what can? More importantly, if he's that powerful, how could he have ever been defeated in the first place?

Bringing Palatine back also minimizes the importance of the sacrifices of the past.

Sorry, rebels who gave their lives to free the world from Palpatine's reign. He's apparently semi-immortal so you gave your lives in vain. The good you did now rings hollow because Palpatine seemingly can't die.

It even makes Rey's efforts seem fruitless because if Palpatine didn't die the first time, how do we know he's not gonna do it again? And again. And again.

A good twist is logical, well-planned, and presents meaningful new directions for the story.

A bad twist is illogical, poorly thought out, and takes the story in a boring path.

Answer 3

This isn't an easy question to answer.

Any attempt to answer it will have to address two aspects: the qualitative and the qualitative.

With the first, you're looking at how well expectations are subverted.
With the second, you're looking at how often the expectations are subverted.

Best practice would probably fall in a happy medium in each of these categories.
However, best practice is relative to the genre and work.

Reviewer Tom La Farge delights in the genre-specific subversion of all reader expectations of character behaviour in his review in The Marginalia Review of William Gillespie’s Letter to Lamont (Providence, RI: Spineless Books, 2005) , in which Gillespie describes the mysterious character of Lamont as an intriguing woman, an ingredient in a cup of punch, and a desktop object.

Of course, now I've set up the expectation for you to engage with the work on this basis, your expectations are far less likely to be subverted - and are far less likely to be subverted negatively.

What La Farge also points out - rightly or wrongly - is that (Mainstream) 'Publishers don’t care for this sort of thing', perhaps because the subversion of expectations is too extreme.

Every subversion, however, can arguably be prepared for so it either comes as a realisation or as a complete surprise, and the art of subverting and fulfilling expectations lies in this craft of laying the breadcrumb trail that sets up and expectation and have it either fulfilled or subverted.

One of the best ways to do this might well be to feel your way into it, rather than to intellectualise it - it's a sensed thing, first and foremost.