I highly agree with @EDL's answer: read your work out loud. It takes a long time to hone your ear and understand how prose flows together, and the best way to practice is by reading out loud and seeing if anything sounds strange to the ear.
However, I also want to offer some concrete guidelines to follow to improve your rhythm, so here are a few I've found helpful:
Vary your sentence lengths.
Take the following passage:
Lindsay got up and walked to the door. She opened the door and stepped outside. She snatched up the Times and fled back inside. She stared at the headline and realized she had done this already.
Notice how we have four of the exact same type of sentence in a row - medium-length compound sentences connected with "and" - and as a result this sentence has horrible rhythm. It clunks hard to the ear, and you can hear how repetitive and dull it sounds when you read it out loud to yourself. It sounds like "bah dah, bah dah, bah dah."
Instead, try to keep your sentence lengths and types varied. We could improve this passage dramatically with just a few adjustments to our sentence types:
Lindsay got up, walked to the door - stopped for a moment with her hand on the doorknob. She swore she'd done this already. She opened it, hands trembling, and there was the Times, folded up and bundled neatly on her doorstep. She snatched it up and fled back inside, petrified. The headline was exactly the same.
Match the rhythm to the mood.
Slow, thoughtful scenes should have slow rhythm, while fast-moving action scenes should have fast rhythm. If you're having a relaxing, languid scene where your characters are lounging around and chatting, it's okay to have long, relaxed sentences. Drag out your rhythm and indulge fancies, to match the calm mood of the scene.
Action scenes, suspense scenes, and moments of high tension, on the other hand, are quick, fast-paced and frenetic, and your sentence rhythm should reflect that. Play with tight, tense fragments, and keep your prose fast and furious.
He cocked his gun with a sharp clack. Held still, listening. Dust spun in the air, stung his nose; warehouse floor was cold under his shoes. Nobody here - yet.
Shake up how your sentences start and end, and don't be afraid of fragments.
You probably heard this a lot in your English classes, but your sentences shouldn't all start the same. If you say "he said, he walked, he went" over and over, unless you specifically are doing it for a reason, it gradually bores the reader.
Don't be afraid of using fragments or starting with prepositions to make this happen - there are no hard-and-fast style rules, contrary to what your English teachers claim, and if it sounds good, chop off unnecessary parts of the sentence and start them in unorthodox ways to spice up your writing.
One of my absolute favorite pieces of writing, Fahrenheit 451 by Ray Bradbury, does this wonderfully sometimes:
And wasn't it this bright boy you selected for beating and tortures
after hours? Of course it was. We must all be alike. Not everyone born
free and equal, as the Constitution says, but everyone made equal.
Each man the image of every other; then all are happy, for there are
no mountains to make them cower, to judge themselves against. So! A
book is a loaded gun in the house next door. Burn it. Take the shot
from the weapon. Breach man's mind. Who knows who might be the target
of the well-read man? Me? I won't stomach them for a minute.
No sentence in this passage starts or sounds the same way, or has the same length or rhythm. And doesn't it just sing?
