Writing dialogue for my novel has proved to be more of a challenge than I anticipated. It all seems clunky and unnatural. Any tips for writing natural sounding dialogue?
Natural sounding dialogue is a contradiction in terms. Well written dialog is more like the 'Best of Conversation' rather than like real people talking.
Tips for writing good dialog:
Reverse Engineering: read other writers you enjoy reading and try to see how they structured their dialog.
Eliminate the Negative: A common source of clunky dialog the writing trying to push exposition into the story through characters talking about stuff they'd already know about their backstory, or their common world, their situation, and so on. If you are doing this, then read through your dialog and scratch out every word that conveys information all characters in the conversation would already reasonably know about anything in their world. Interesting people don't tell other interesting people what they already know.
Minimization: People make idle chatter. "Hi. How are you? Fine. That's Great." Characters that do that are boring and dull. Have your characters only speak to move the story forward or create empathy for the reader or react to events (actions, dialog, movement, things happening in the environment). Ideally, every piece of dialog should do two or three things -- evoke interest or engagement with reader, communicate the character of the character, move story forward, etc. That's hard to do all the time, so at a minimum it should do at least one of those things.
Character's speak like they want to be heard: Just like vain people who speak in blustering praise of themselves or in humble brags, the characters reveal their nature or character in their lexicon. Each character should sound like a different person, unless they are raised and educated to the same level and blah blah. So dumb characters ought to sound dumb and highly educated characters should sound very well educated. Highly educated characters trying to communicate with dumb characters should sound like ... Well, I am sure you get it by now. One method to learn this is to use writing level tools to assess the grade level of your writing. Then try to raise and lower your writing.
Lastly, don't worry about dialog on your first pass of the story. Just get it on paper. Then re-read your work and think about the juiciest core of what your characters are trying to communicate in that bit of the story. Then distill it down to be the most effective and shortest dialog needed to communicate the character want, reaction and feelings in that moment of the story.
Tape record some real conversations. Transcribe them. Note that they are much more informal and interactive. There are very few full complete formal sentences being used. Sentence fragments abound. They interrupt each other with other topics at the same time.
Forget small talk. The dialog must add to the plot or the characters somehow. Use only the dialog that is necessary; don't overdo the dialog usage.
When it comes to dialogue, always remember, it is a way to characterize your characters. That is, part of what the reader understands about the nature of your character's comes from their dialogue (i.e., we infer traits about them through what they say and how they say it). That being said, how do you want your characters to be perceived by the reader? Are the characters highly educated? If so, then maybe craft their dialogue to be more academic and stringent when it comes to prescribed grammar rules. Are your characters from a specific subculture (e.g., South Boston vs. Up-state New York)? If so, then craft the structure of their dialogue to reflect how someone from that subculture might speak? As long as your dialogue conveys the same traits to the reader that you believe it conveys, then the dialogue is doing it's job (as far as the characterization part of it is concerned).
You have to really love conversation and the way people talk to write good dialog. So doing a lot more close listening is a good start.
One thing you'll notice is that people are very rarely direct in their communication. There's a ton of context and allusions, hidden meanings, repressed emotions and so forth. So you want to make sure your characters aren't too direct in their dialog, or it will sound clunky and too on-the-nose.
Conflict also helps a lot. Conversations where people agree too much sound fake. And real people tend to talk past each other. Realistic conversation has a lot of seeming non-sequiturs.
Her: It's such a beautiful day!
Him: Is it?
Her: Geez, when did you turn into such an old man?
Him: Careful, you're not that far behind me.
Her: That's a lie, you robbed the cradle when you met me.
That's what real conversation sounds like. Trivial on the surface, but there's a lot boiling underneath. Even in everyday smalltalk, people have hopes, fears, goals, schemes, vulnerabilities and so forth.
A tactic I use make human dialogue more realistic: write as quickly as I think, not paying too much attention to phrasing and grammar (in other words just as you'd speak to someone in the real world), and secondly, go back and make another pass, trying to take out as many words as possible (ie, unnecessary ‘that’ for example) while keeping the same meaning. I generally use contractions whenever possible and sometimes use informal and made up words like 'gonna' instead of 'going to' for example. Replacing longer phrases with slang is usually a good idea, as long as it's consistent with the character. For example, you could replace 'we have to leave' with 'let's bounce'.
However, one thing I almost never do is try to spell words and phrases to match a particular character's accent. For example, someone with an Austrian accent might pronounce 'get to the chopper' as 'get to za choppa' (ie Ahnuld in Predator) but if you do that then you pretty much have to be consistent the whole way through which can be a huge chore and probably not worth it.