After much world building I begin to feel the need to write short stories set in my world. In other people's fantasy novels I particularly enjoy twists and surprises. I have started paying attention to those twists that manage to surprise me. Beyond that - and pestering your friends with requests to read your stories - how do you judge, if your twist is surprising? After all, you wrote it and know everything. Unless you invented the twist pretty late in the process and are still surprised by your own shrewdness, which I dare not expect.
How to test if a twist is surprising
There are three techniques that I use to test if the twist was surprising.
1. Order of inspiration
If the idea was one of the first things I thought of then, no it is probably not surprising but obvious. At least obvious to me.
The exception to this is when I find myself grinning like a loon and thinking "damn, that is clever". Of course, I might be deluding myself but gut reactions count for something.
2. Character test
Again this is just me, but when I imagine my characters (who tend to be fairly genre savvy) do I sense any surprise? If the antagonist or the protagonist would say something sarcastic about seeing it coming then, no.
However, if I can reasonably see all the characters being caught completely off guard...
3. Use a writers' group
I chair a charity that runs a writers' group. If I want to know how well my twist works I give it to some members. The back sheet will have 4 or 5 questions and one of those questions will be "did you see the twist coming and how surprised were you?"
4. (Bonus) Count the number of techniques (below) that you have used
There are a number of tried and tested methods for making a twist surprising. The more of them that have been used, the more likely your twist is to be surprising. If your story is full of twists and even half of them are somewhat surprising, then the whole story should be fairly surprising.
How to make a twist surprising
Even an obvious twist or plot development can be made to be surprising. It is all in the setup. After all, something is surprising when it is not what we were expecting.
1. Red Herrings
Feed your reader some red herrings. That is some clues that point in the wrong direction or turn out to be unimportant.
Red herrings are a vital part of a good mystery. The reader will enjoy trying to figure out which clues are really important.
Similar to the red herring technique a good story teller can make the most obvious ending by drawing the readers attention away from the magic just like a stage magician would. Red herrings are part of this approach. A big part.
Other methods include drawing attention to something only to have it pay off later, sooner, or not at all. (Be very careful with that last one).
3. Speed it past them
While the reader is caught up in the action, for example, a chase scene, this is the perfect opportunity to slip in a vital detail in an off-hand way. Casually mention it when the action is most frantic and then turn it up to eleven. Most readers will fail to realise the importance of the detail (if you did a good job).
4. Subvert expectations
One way to bring the surprise factor is to set up a common story trope (say a princess with a magical problem) and then as the expected conclusion approaches take things in a totally different direction. The princess refuses to marry the hero, the hero gets eaten by the dragon, the frog is happy to remain a frog... You get the idea.
5. The partial information method (use with care)
Sometimes we assume too much based on the information we have at hand. Use this. You can give the reader clues based not only on what you say but on what you clearly do not say.
Take, for example, a scene where a lady is walking down the aisle and thinking about her boyfriend and the day he proposed. Then the director yells cut. You never said this was the wedding but you kinda, sorta, implied it.
As I said, use with care because this can come off as cheap.
6. Same action, different result
This is the format of many jokes. An Englishman, An Irishman, and a Scotsman go into a bar...
The characters do something and there is a result. The characters (or some characters) do it again with the same result. Then they or another do the thing for the third time only now...
This trope is exactly why I love road runner cartoons. You know that no matter how many times Wile E. Coyote tests that trap, the Road Runner will not trigger it. No matter how often the Road Runner does something it will go badly the first time Wile E. Coyote tries it.
What not to do
Never pull out a fact that the reader and the characters had no way of knowing to pull off your surprise. While this will be surprising, the reader will feel cheated.
Pestering friends or colleagues is the only way to know whether something in a story of yours will work, or, as in you case, is surprising enough.
If you think you're annoying your friends too much, maybe it's time you joined a writers group (real life or online) where nobody knows you. Those can be the best critics, since friends will (sometimes) tend to lighten the tone of the review.
As the writer, you'll likely not get surprised at all, unless you leave the story to rest for some months, write other things, and them re-read it. If you give the story enough time, you might also be surprised, but there's nothing better than having other people read your story.
Kudos to Lord Matt who has provided a very robust and comprehensive answer. Firstly I'll continue to chip away at the 'everybody's equal' education theory. Twisted plots are born of a twisted mind - some people aren't built for this style of story-telling. It's very hard to teach.
If the twist is the major feature don't burn beta readers. You can only use them once. On third viewing "Sixth Sense" isn't a very good film. Watching it for the first time you are in shock at the final revelation. On second viewing you are amazed how obvious the outcome was - and you missed it! Watch it again and you'll realise - without the twist it ain't all that.
Setting up for a good twist or twists requires superior intelligence, confidence, arrogance, and an inherent cruel streak. I measure my best twists by how many times I can show what's going on knowing you'll never get it because you're just too stupid.
EMERGING TWISTS (The case against outlining).
I wrote a novel about a gangster and the cop who was trying to put him away. A mild twist was the gangster was too smart for cop. The cop became the villain and the gangster became the hero when the cop started venturing further over the line to catch the gangster. To add a little spice to the story and ramp up the conflict I had the cop's daughter ( a state prosecutor) fall in love with, and marry, the gangster.
As this story drew to a close I took a read through. The gangster who was, deep down, a good guy had ensured that his wife was well out of the way. If the shit hit the fan she had an alibi and could never be convicted.
After reading my own novel it occurred to me . . . he wasn't giving her alibis she was creating her own. Throughout the entire saga this secondary character had been the puppet-master all along.
FOR WHO THE PENNY DROPS (This is going to be long)
When writing a novel full of dark twists and turns you must be aware of different qualities of reader. e.g. The opening of "Little Miss Lightning" is blatant misdirection. I know what I'm writing but I know what the reader thinks I'm saying.
The group of young girls stood in the rain, stripped, ready, in a huddle.
Cold blue eyes made a cursory inspection of each individual. "Behind the line," he grunted, motioning with one hand. His gun, in the other, pointed downwards. Droplets of rain fell from the muzzle onto the ochre-stained ground. "You will be eliminated. Step back, behind the line." He pushed one of the girls. "Schnell! Schnell!"
The girl edged back, shaking, fighting back tears. Her bare thighs blue with cold.
Carlene looked over to the frightened girl; for a brief moment pity for the other momentarily threatened to weaken her resolve. She turned away. Now was not the time to be pitying others. It was every girl for herself.
The man strode back to his position, swinging his pistol as he walked.
Carlene watched, eyes filled with hatred, fixed on the gun. Her thoughts travelled to her grandfather and brother at home in England, and how she missed them since she’d been taken from her school three weeks earlier. She’d been held with the others at that awful camp for the first two weeks until she was transferred here.
Another voice instilled silence, "Fertig."
Carlene didn’t understand. She trembled, watching as he raised and pointed the gun. In the silence, she closed her eyes for a long second. The long second lingered, she peeked, just in time to see his finger squeezing the trigger. The moment she heard the shot, her mind told her to run. All around she could hear screaming, shouting, and mayhem. When her throat began to burn and her lungs felt as if to explode, fear told her not to stop - just keep running, as fast as she could.
- I know what you think that was. The heading "Berlin, Germany" put the blinkers on you, and after that you got played like a cheap fiddle. However, the start of the next scene catches up the smarter readers.
Three years later
"As they round the top bend and come on to the home straight, Salter is powering away. There is daylight between him and the German athlete . . . Oh my word!" The commentator’s voice buzzed with excitement. "A scintillating performance. That – is an amazing time. We knew he was in the form of his life but the clock’s stopped at 44.05 seconds, a new Commonwealth record, and the fastest time in the world this year. And that's maximum points for Great Britain." An athlete clad in the UK strip of sky blue Lycra bent over, hands on knees, breathing heavily, head raised, watching a replay of his own performance on the big screen.
From this point on I'm writing for two sets of readers. Set(A) have worked out that the opening scene was the start of a race in a track and field meeting. Set(B) remain adamant that my character escaped from a Nazi concentration camp. I must continue the story whilst appeasing both audiences.
- At some point I have catch everybody up. I go eight chapters before our rising track and field star appears on a cookery show and explains chapter 1.
The presenter read from a card. "Question number one from Helen in Essex. Why did you decide to become an athlete?"
Carlene leaned back as she thought. "I don’t know that I chose athletics. Although as a child I can always remember I seemed to be running. Originally, I wanted to play football, I suppose. One day I took the boys’ ball away because they wouldn’t let me play. They chased me all over the park but none of them could catch me. After that, my granddad would drop me off at Crystal Palace Athletics Club whenever the boys were playing. He says it did it to keep the peace and to give me a creative outlet for my excessive energy. It was all good clean fun until I turned seventeen. They thought I was good enough to go to the European Schools Championships in Berlin. I thought ‘great’, a free holiday and I’d get out of doing revision for my exams." She rolled her eyes theatrically. "Wrong on both counts! Holiday? The Third Reich never did close down those concentration camps – trust me on that. It was the first time I’d been away from home on my own, and it scared the . . . life out of me."
"And I bet you still had to revise."
"Didn’t we just, in solitary confinement, locked up inside the training camp. Because we were all still at school they had tutors and supervisors out there. But they were guards really – we all knew it."
"How’d you get on out there?"
"I won the title. I was so scared I did a bit of a Forrest Gump. The moment the gun went off, my one and only thought was ‘run, Carlene, run!’" She pumped her arms. "I was running as fast as I could waiting for the crowd to hold up a banner saying. "Stop, Carlene, stop."
Give it to a beta reader, or a group of beta readers, and ask them questions afterward.
I've never read as a beta reader (nor given my work to beta readers), but I'm given to understand that it's a pretty common practice. I don't know how those things are usually done, but I don't see why you can't give the beta reader(s) a form with space for general feedback, and also include a few targeted questions like "Did you find the twist surprising?," among other questions.