In a novel-length work, there is almost always room for some humour. I'd say the trick is to choose the right type, and in the right places.
Be the right kind of funny
If you've seen it, think of the TV show Breaking Bad. Its subject matter was bleak and often gruesome; its emotional content was utterly brutal; but the writers sprinkled in plenty of humour. If they'd had Jessie Pinkman cracking hilarious puns while [spoiler for season one]...
liquidizing the corpse of a rival drug dealer
...or going off into wacky flights of fancy à la Family Guy, it would have sucked away the impact of the story. Instead, In that scene, the writers inserted no laughs, and when they wrote them into other scenes, the comedy was black. Gallows humour and irony. The laughs were bleak, just like the story. That meant they didn't detract from the story's impact. And in fact, they probably increased it, because (segue alert)...
Be funny in the right places
In a novel-length story, you need to vary your tone, your pace, your level of action and tension. You might open with a high-intensity horseback chase — great! — but if, thirty chapters later, the characters have spent every moment locked in high-octane action, then the impact will be lost.
Think of a piece of music — the power of the loud parts comes from the quiet parts. It's the contrast that makes them meaningful.
If you want your readers to feel it when you crank up the tension, then you need to give them moments of lower tension. That categorically does not mean bore them! But it means that if your story goes battle scene — gunfight — bandit attack, you can increase the impact of the final bandit attack by replacing the gunfight with something like campfire scene, worrying about medicine for a wounded character.
Tension is one of the things you, as the author, want to fine-tune from scene to scene, and from moment to moment. Humour, of the right type, is one of your tools to do that. You probably know lots of ways to build tension. Laughter, whether joyous, nervous, or otherwise, is a release of tension.
A belly laugh in the middle of a climactic action scene would probably release too much tension — your instincts are good, here, this is the immersion-breaking, emotion-breaking effect you want to avoid — but at the end of a completely different scene, you might want exactly that in order to let your readers feel relief. On the other hand, in a different moment, you might want a smaller release — picture your characters clustered around the door to the castle's throne room, near-certain death awaiting on the other side. They've battled to get this far and there's no turning back. Somebody's fingers hover an inch from the iron handle, when he stops and turns to the others, and says, 'I've just realised something...' A droplet of gallows humour here could be the set-up you want for your climax, releasing a whisper of tension in the last second before you pull your readers into the dramatic final showdown.
This is hard, I'm no better at it! But if you're worried, don't be. Nobody writes a perfect first draft. Try it, and when your story's ready, get some feedback. Your readers will be able to tell you if the humour isn't working right.
Short answer: It's definitely possible for a story that contains humour to be epic and dramatic. Humour is one of your tools. Like you've worked out, it can hurt the plot, if used badly. But if used well, it can make it better.
Hope that helps a bit!