Let's say two characters are conversing and one character interrupts the other in mid-sentence. How would that actually be shown in dialog? I have seen some use a hyphen at the end of the sentence but I have also seen an ellipsis used as well, but which is the most appropriate?
Not a hyphen. Use an em dash. In most word processors, you can insert an em dash by using two hyphens (it will be converted to a single em dash). In LaTeX, use three hyphens (which will be converted). Or, from a character map, copy and paste the em dash.
Interruption, either by being cut off, or by a sudden event, is always shown by an em dash. On the other hand, if the speaker is losing the train of thought, or is tapering off, that is shown by an ellipsis.
Note that some fonts show a long em dash (full em width), while other fonts show a shorter em dash (maybe 75% to 80% of em). If you find the true em dash to be unattractively long, you might cheat an use an en dash instead. This is technically incorrect, but few would notice in print. In any case, it must be noticeably longer than a mere hyphen.
Example, with em dash:
"Don't give me that rubbish. I'm tired of your constant complaints."
Another example of em dash, used as a stronger alternative to parentheses:
It was dark — especially for London — and stormy night.
Example use of ellipsis:
She looked out at the rain, and mumbled to herself, "I wonder if…"
EDIT: There is a difference between the en dash (EN) and the em dash (EM). The en dash is used to indicate a range of things (say, between times or dates). Alas, the most common word processor rarely inserts an automatic en dash, but you can insert it by other means. In LaTeX, it is two hyphens, automatically replaced.
The choice is stylistic. It's almost always your decision, as the author. Do what you feel would be appropriate and convey your meaning well.
However, it goes without saying that some publishers may have a certain way they want you to format your work. Be sure to fit their guidelines.
For me personally, I use hyphens. I feel like they work better than an ellipsis, even though I have seen some authors use ellipses.
I hope this helped you.
Standard punctuation for an incomplete sentence is ellipsis.
Don't have one character interrupt another at all. Dialogue is not speech and the page is not the screen. The page is an asynchronous media. Events do not unfold in real time but in read time. It can take far longer to read the description of a complex event that happens quickly than a simple event that takes a long time. So time based effects, like interruptions, are impossible to dramatize well in prose. Instead, the drama should come through what is said, not the way it is said.
As far as is possible, let each character have their full say. If one must interrupt the other, let it be a full and final interruption, a dismissal, not just two people talking over each other.