Writing Stack Exchange is a question and answer site for the craft of professional writing, including fiction, non-fiction, technical, scholarly, and commercial writing. It only takes a minute to sign up.
Sign up to join this community
The response to this question makes it clear to me that I haven't quite asked the question I had intended, the answers are useful but not quite what I'm looking for.
So different but related question; to what extent can one include events that can only be explained in-universe as "supernatural", falling outside the science of the setting before they're writing fantasy instead of sci-fi?
To directly answer your question: The role of the supernatural in hard science fiction is that it doesn't exist. Period. There are no shades of grey to the laws of physics. You can't say "this one location is the Space Bermuda Triangle" or "this one alien race defeats time with their brain" and still claim that you are writing hard sci-fi. This is the old immutable object meets an irresistible force brain teaser. The two cannot co-exist. Pick one or the other.
If Clark's 3rd Law states "Any sufficiently advanced technology is indistinguishable from magic," then the corollary would be something like: "Magic will be accepted as advanced technology as long as everyone keeps calling it technology."
Does that actually work? Unfortunately yes.
Star Trek Voyager is probably the all-time champion for technobabble. There are maybe 3 or 4 episodes that don't rely on jargon-salad to obscure the complete lack of a coherent conflict, but this is television. As long as the dramatic beats occur between commercials, an alarming majority of viewers happily accept gibberish if it is said with confidence.
This is the corollary of Clark's 3rd Law in action.
Since Star Wars was mentioned in another answer, I will say it is not even "soft" sci-fi. SW is a 7-genre-mashup but its core rules come straight from Fantasy Genre, not just because the force gives everyone limitless paranormal superpowers based on self-confidence, but also because it has princesses that need saving, villains with literal black hats, and most of all, Family Dynasty = Destiny – not just one guy's personal destiny but the whole galaxy's destiny and every living soul in it is tied to the bloodline of this one dysfunctional family – the only family in the universe that matters (waves at Dune). Dynastic families, end-of-world prophecies, and blood-born heroes are Fantasy's stock and trade. Yet people are still trying to call it some kind of sci-fi rather than fantasy. There is literally zero science there, it is all magic that people keep saying is technology.
While fanboys gushed over the alleged "science realism" of The Expanse, I seemed to be the only person who saw the plot was about magic-psychic-mushroom-people that defy all laws of physics (and one guy from a rural area with a destiny because reasons). Obviously that didn't matter to anyone else, if the sets are stylish and the computer monitors have edgy graphics and the "fire" shooting out the back of the spaceship is blue, then the majority of people will accept all this magic as "advanced technology". Fans will even argue online about how "scientifically realistic" the show is (see comments below). Don't get me wrong, the show is crafted with confidence and skill – that's not the same as science realism. Just like Lord of the Rings, The Expanse is a magical universe where the humans just happen to be un-magical. (They make up for it by having other superpowers, like being undefeatable in battle and having a destiny that decides the fate of the universe… for reasons).
These shows are entertainment, not textbooks. They have no obligation to teach accurate science. Sure, you can rate them on a spectrum of "soft" to "hard", but we can just as easily score them on hundreds of other metrics that we care about, like: Feminist Intersectionality (Voyager scores ridiculously high, the highest of any sci-fi show; Star Wars is pathetic with 6 movies yielding only one self-motivated female character who achieves anything more than being another character's mom), or the Bechdel test, or how many pseudo-indigenous people have some bogus culture based on talking to ghosts (Voyager: fail – but to be fair all 3 shows involve talking to ghosts).
It goes without saying the producers of these shows are measuring success by how much money they make, with themes and content being more or less "voodoo economics" – when it makes money the producers are geniuses who can do no wrong and everyone copies them. But if it fails financially those noble platitudes are deemed box office poison for everyone. In Hollywood, profit is the only metric that matters since it will decide whether a project is made in the first place. Our arbitrary metrics are only valuable to us (or more likely how we see ourselves within our tribe, whether we are gatekeepers of our beliefs and how vigorously we are willing to defend our values).
If science realism is the sword you feel like throwing yourself on, by all means make the science as realistic as possible but don't mistake that as a universal "truth" that all readers will judge it by. Coherent science or not, I can't stand any story where a rube whiteboy from a rural planet is destined to save the galaxy because reasons, that is just not sci-fi. Another trope is the hybrid baby with magical superpowers, once they appear I can't suspend disbelief and the story is over for me. But the sword I will throw myself on, in whatever genre, is narrative structure and character arcs, if the author is floundering in these aspects I won't bother to finish because it's unlikely there will be any payoff. It's just a series of fantastical events without a story to hold it together.
The next question to ask is why you feel your story about the Supernatural needs to adhere to some arbitrary sci-fi-fan metric that doesn't even seem to be any measure of success within its own genre.
It depends on how "hard" you want your sci-fi to be. I think Larry Niven is a boundary case of sci-fi hardness. Niven writes about things that do not exist and most likely cannot and will never exist, but he approaches the topics with just enough science to keep things "hard".
Some examples of Niven's suppositions:
In order to write hard science fiction, it is quite important to know as much as you can about actual science. Niven builds and maintains a lot of hard sci-fi "cred" by telling stories that involve relativity, tidal forces, orbital mechanics, and anti-matter in ways that are quite accurate, and he only uses the "supernatural" or impossible elements when necessary to create the story. His stories couldn't happen without these fantastic projections and semi-supernatural elements, but the stories are not about those things.
After reviewing your original question, I feel like my last bullet suggests an answer: Yes, there is plenty of room for the unexplained in hard sci-fi, just as there are many, many things that are still unexplained in the real universe. In fact, many suggest that the more mysterious something is, the more it is a good candidate for introduction into a story without explanation: "A plausible impossibility is better than an implausible possibility". In other words, humans spontaneously developing psychic powers for no known reason is a better story than a random string of unlikely coincidences occurring that happen to save the main character from death.
You can’t. What makes hard science fiction hard is the fact that it works within the bounds of what is known (or reasonably theorized) to be possible.
Movies such as Star Wars are not hard sci fi; they’re science fantasy — fantasy stories with the trappings of sci fi. The Force is, for all intents and purposes, magic. Space ships do not, in and of themselves, make something science fiction.
The closest example of what you’re talking about that I can think of (SPOILER ALERT) is a series of books by Neal Stephenson called The Baroque Cycle. It’s a trilogy of thousand page whoppers set in the 1700s. It’s “science fiction” in the sense that it’s fiction... about science. A main character is a member of the Royal Society and a friend of Isaac Newton. Solidly qualifies as both historical- and science-fiction... except...
Newton is experimenting with alchemy throughout the books, (which he actually did historically). In a surprise twist at the very end of what amounts to a three thousand page novel, the alchemy works. Once. It’s an odd (but not unsatisfying) end to a long complex story. I suppose it amounts to a version of the old “There are more things on heaven and earth that are dreamt of in your philosophy”; but it definitely breaks away from hard science fiction at that point
It is not that you can't add supernatural stuff, it is how you approach it. Take Solaris by Stanislaw Lem, for example; Solaris is a (small spoilers of the beginning of the novel ahead) living water planet that can communicate via electrical impulses, and even generate humans from the memories of members of the science vessel crew. This is, for all intents and purposes, unequivocably supernatural by our standards, but the absolute madmen of the science vessel keep trying their best to understand Solaris. They know what happens, they just have no idea how it happens. Nonetheless, they keep testing, even after several unfruitful attempts, because that is what science does. They know Solaris is real (because they are right there witnessing the things it does), that it is not just fantasy, and if it exists and it is not fantasy, it can't be magic, and therefore, they can attempt to comprehend it. Whether they succeed or not is completely irrelevant to our purposes.
Could Solaris exist in reality? It may be possible in theory, but so are elves and shooting fireballs out of your hand. In practice, however, we know it needs a hard suspension of disbelief, because such a thing would be impossible. Does this mean Solaris is a fantasy book? No! I would argue it doesn't even necessarily invalidate its sci-fi "hardness" for this specific reason, if it were any hard to begin with (which, according to Wikipedia, it is, but sci-fi hardness spectrum is a more or less bogus subjective metric). Point is, there will be unexplained (or, more accurately, "not fully understood yet") things even in the hardest of sci-fi novels, least they have managed to discover all there is to the secrets of physics and the universe in general. You can have unexplained, or "unexplainable even by the story's time frame standards", even in hard sci-fi, and you should have these things if you want to make a story about the discovery of something that baffles the humans of their time.
Whether an event is apparently supernatural or not is irrelevant, and more often than not the difference between sci-fi and fantasy just accounts for the world they are set in: sci-fi extrapolates from our world in the future, while fantasy builds its own world with its own rules that differ from our own. As long as your characters understand that what they are seeing clashes with their comprehension of the world, or at least it is implied the previous generations had to make sense of it, and doesn't "feel" like the world you are describing has gone in a completely different direction to what we were expecting (say, technically Adventure Time is a post apocaliptic setting based on our world after a catastrophe; it still is fantasy with just some sci-fi elements, like Shadowrun, because no amount of scientific breakthroughs or random misshappenings could foresee how wacky and alien and completely unexplainable the world is), you could still consider your story sci-fi
It depends on how you handle it in universe. Let me give you two reasonings why it can work, if done well.
First, let's look at Clarke's third law: any technology, sufficiently advanced, is indistinguishable from magic. I've seen examples used from our world where this makes perfect sense.
Take cell phones. If you go back to the 1950s and talk about cell phones readily available right now? A computer in your pocket? A computer that can call and be called like a land line? They'd brand it as witchcraft, and that's less than a century ago. Extrapolate that to a civilization (like often happens in sci-fi) thousands of years into the future? And everything seems magic-y. Star Wars capitalizes on this with their tech, as does Star Trek and many others.
So if you were to introduce a god-like being or simply god-like events (supernatural), like was done in both Star Trek and Star Wars? There will be 'Data' and 'Spock' like characters that will try to explain it away with technology and science, or simply become fascinated by it because it "must be done with tech we do not yet understand".
You can also use this to show differences with people. Like a logical person trying to find some rational explanation for it, or a religious person making the sign of the cross (for example) in awe of their "god's magnificent work".
Second, there's Brandon Sanderson's take on this. A 'soft magic' that is poorly defined (supernatural, in this case) can be used to instil a sense of awe.
Imagine a civilization that is thousands of years more advanced than we are, and they are still being confronted with things they cannot readily explain. They try, they study it, they analyse the data. And there's no explanation handy to 'science it away'.
You can use this to instil a sense of awe, of eternal curiosity in those trying to study this strange and wondrous happening. Give an otherwise stoic and jaded researcher a renewed vigour, because there's more out there. More to find, more to discover, more to learn.
And more importantly, you can take this as a moment to have that jaded person's mentor point something out, like the aurora borealis, and say: "Just because you can explain that, doesn't make it less beautiful. And just because you can't explain this, doesn't make it supernatural. There's rhyme and reason to this. So go figure it out."
My first instinct was to say "you can't" - the very essence of the science fiction genre is that things are not supernatural - they make sense within the in-universe rules, if not right from the start, then in the end, when we get to the bottom of the mystery.
But then I thought of some examples to the contrary. Look at Star Wars - what is the Force, if not magic? In fact, when the prequels attempted to make the Force less supernatural by talking about "medichloreans", the response was negative, because they took the magic away and replaced it with something mundane.
Similarly in Dune, you've got precognisance, lie-detection, rendering consumed poisons non-lethal, and a bunch of other supernatural abilities. The Bene Gesserit are even called 'witches' in-universe.
At the same time, there is one important caveat: while the Force, the Bene Gesserit abilities etc. appear supernatural to us, they are not framed as such within their respective worlds. They are framed as part of the natural order, and referring to them as "magic" is, in-universe, considered superstition. The setting in those cases is completely fantastical, if you think about it, but instead of calling it "magic", the author calls it "science". Then, the author doesn't even resort to technobabble to handwave away your claim that it's supernatural, but goes straight to "there are more things in heaven and Earth..."
I think the key to this question is to break down this phrase:
... writing fantasy instead of sci-fi
Genres are not universally agreed things with solid boundaries, and fantasy and science fiction have a long history of cross-overs, sub-genres, and conflicting definitions. The terms "SFF" and "speculative fiction" are sometimes used specifically to avoid such a distinction.
The question of "is it fantasy or sci-fi?" can therefore have a number of meanings:
Ultimately, you can include whatever elements you want, but it's sensible to choose where you want the limit to be, and be consistent:
If you explain some things and not others, readers are likely to be either bored by the explanations on the one hand, or frustrated by their absence on the other. If you're consistent, they will either enjoy the story on its own terms, or not because it's not to their taste.
Note that the question in the title is subtly different:
... in hard science fiction
Although still just a label with varying interpretations, "hard science fiction" generally implies an intent to put plausible science at the centre of the story. If that is your focus, then the question is something of a tautology:
If I'm trying to stick to plausible science, can I include things that aren't plausible science?
But again, the answer is only no because of the constraints you've set yourself, not because of any hard-and-fast rule.
+1 Galastel. Along the same lines, you can keep a "supernatural" element in the realm of science fiction by having characters refuse to acknowledge it as supernatural, and insisting (as scientists would) that just because we don't know how something works, and just because it seems miraculous, does not make it supernatural.
My crowd (professional scientists that eschew ALL supernatural explanations [which is not 100% of scientists by a long shot]) would say that anything that doesn't fit with our theory of physics is only proof that our theory of physics is wrong or incomplete, we'd say there just is no such thing as phenomenon that are not natural. We would rather say "I don't know" than accept any explanation that cannot be proven by logic and experiments. This is one reason we reject String Theory; it has been shown there are more possible solutions to String Theory than there are protons in the visible universe. IMO (and that of others) It has strayed into the realm of faith.
Thus even though there ARE no supernatural events, there are at least a dozen phenomenon (even now IRL) that do not fit the most advanced and current models of particle physics and gravity. Einstein's theory of general relativity tests very well, super accurately. The current theory of Quantum theory tests very well, even more accurately than general relativity. They cannot be reconciled and one or both of them are wrong. But no scientist should ever accept that the exceptions to these theories should be attributed to supernaturalism, or God's Will, or anything else.
For scientists, such attributions are a brick wall that shouldn't exist, a false explanation that, intentionally or not, thwarts any further real explanation, and thus interferes with or even prohibits scientists from finding a deeper or different model that will explain all the current models do, and explain some or all of the apparent anomalies in the bargain. That would be an advance in science.
So, given a setting of a complacent science that thinks it has all the answers, justified by the fact that what they do actually does work, any apparently supernatural event that is prohibited by their science should cause a hurricane of activity in the scientific community to see where their science has gone wrong, in prohibiting something so obviously possible.
Or trying to debunk what happened as an illusion, intentional or not. (No real scientist believes Penn and Teller can make either a coin or a woman vanish, we know there is a trick even if we don't know what the trick is.)
Accompanying that hurricane will be tornadoes of the non-scientific, of various stripes, pointing at the event as proof the snooty scientists have been wrong all along, there is a God, or magic, and by implication an afterlife and all those they loved still lived on another plane of being!
Like Star Wars, Star Trek, Dune, Heroes, and the current series "Colony" and "The Expanse" and many other scifi movies and series, you can include devices and events that are effectively impossible (e.g. FTL travel) by real-life current science, and even unexplainable by your in-world science (in the Expanse, the Alien Ring portal, slowing down time, etc).
The trick they use is to not call it supernatural, or magic, or anything but some version of "unexplained". For SciFi, as an author adopt the scientific attitude: Embrace the answer "I Don't Know," and let the non-scientific amongst your readers think and say what they will.
One useful way to conceptualize "hard" science fiction is structural, as being a story primarily about the ramifications of a plausible, rule-governed, future technology, one that largely aligns with our current scientific understanding of the world. "Soft" science fiction, on the other hand, can be any sort of story, in a setting that "looks" like science fiction. Finally, "science fantasy" is a soft science-fiction story with a technology that operates more like magic --which is to say, in a less plausible, less rule-governed manner, and no requirement of true alignment with current science.
With these as our guidelines, is there any way to create a hard science-fiction story that includes supernatural events? Perhaps. We can immediately rule out explicitly presenting an event as supernatural ("word of the author"-style, no room for disagreement) because that won't strike the intended audience as either plausible or in alignment with current science. However, there's not necessarily a problem with presenting an event that characters within the story perceive as supernatural, nor with presenting an event that the characters within the story are simply unable to understand at all.
If so, however, we still need to fulfill the genre promise of exploring a plausible future technology as the primary focus. So, for example, a seemingly supernatural occurrence could be the "inciting event" of a story, that then shifts to investigating the mystery with science and technology. (The audience will probably expect, in that case, to eventually reach a plausible scientific explanation of the mystery.) There are other possible story structures, but in all cases they will need to honor the basic genre promise of a technological focus. It's possible to be creative about this, however. For example, Delany's Babel-17 is basically (at its core) a work of hard science-fiction, structurally speaking (despite its "soft sci-fi" setting and stylistic sympathies), but one that revolves around a psychological technology, not a physical one (the same is true of Asimov's legendary Foundation series).
I don't believe there is any limit to the events that can be included in a science fiction story. One reason is that the simulation hypothesis eats all "supernatural" phenomena.
Another reason is that our scientific knowledge still doesn't explain everything. Extra particles outside the standard model are needed for current cosmological theory to balance its books. General relativity predicts infinite densities, infinite time dilation; how can that be right in a quantized universe? There's plenty of room for events to be categorized as having no known explanation rather than accepting the lazy assumption of the rules of nature having been altered by an external being.
And you don't have to necessarily explain the event in the story. In Neal Stephenson's novel Seveneves the moon explodes without warning. No explanation was given for the explosion. The novel is about how humanity dealt with the aftermath.
Or your story could be about your characters trying to explain the odd events. In The Day the Sun Stood Still three novellas are presented that deal with the seemingly supernatural event of Earth ceasing to rotate for a day.
There's an important class of things which science does not feel a need to provide a satisfactory explanation for. Events with a sample size of one are a major player in that class.
As an example, I recall a wonderful scene from a book. I want to say it was from "Manifold Time," but I might be wrong. In that scene, one of the people recruiting the protagonist to a task produced a small box. They said, "This box will produce a single marble when this lever is pressed. I can truthfully say that there are either 9 steel marbles and 1 black marble, or 99 steel marbles and one black marble in here." The narrator made a note that the box was large enough to justify either claim. The protagonist pressed the lever, and a black marble came out. With nothing more than an "hmm, interesting," they put the box back in their backpack, and we never again get to see the lever pressed.
The entire point of the scene was to create an event which had a sample size of one. This was a pivotal moment for the protagonist, where they were recruited to the task. And all of it hinged on the likelihood or unlikelihood of a black marble coming out, and the infuriatingly unscientific inability to repeat the experiment for statistical rigor.
Miracles can fit into hard science in a similar way. Remember the most important rule of probability: The probability that an event will happen if it happens is 1.0.
I'm assuming that you are asking about supernatural and not just technology we don't yet understand. After all, most SciFi has technology that the author cannot fully explain, or that assumes some breakthroughs in fundamental theories.
But supernatural is a different category. All of that science and technology deals with the natural world, and generally refuses the concept of a supernatural world altogether. To bring the supernatural into a hard SciFi story means questioning the basic assumption that there is only the natural world.
That does not mean going non-scientific. Science is not mechanistic or materialistic. Science is simply the approach of coming up with an explanation, testing it, and then refining it based on the results. There is no reason why this approach could not be applied to the supernatural.
However, "hard" SciFi means roughly that you expand upon current technology or theories and that your imaginary future tech does not violate known laws of nature, or explains the violations (i.e. expands not just on the technology but also on the scientific theories governing it).
The supernatural is literally defined as outside the laws of nature. To combine it with a hard SciFi approach would mean to build your world around realistic technology and then add an entire new dimension to it. The background story of Shadowrun is an example of this. A near-future setting with realistic technological progress, and also magic has reappeared.
Most people would consider this a crossover, however, not a hard SciFi setting.
The answer is its all made up. So write what you want, and let other people worry about what genre it falls into.
For me exhibit A of this is Anne McCaffrey's Pern series. The first book started out with people bonding with, and riding, fire-breathing dragons in a medieval setting. They periodically needed to do this to combat occasional death rain, falling from the sky.
100% Fantasy, right?
Later books in the series explained the dragons as a small native species with a really useful behavior (neutralizing the death rain), genetically engineered and bred over centuries, as the original colonists slowly lost their tech. The death rain ended up explained in detail as coming from a comet.
Now its Science Fiction.
Think about this; later books retroactively made the earlier books SF instead of Fantasy.*
If I can change the genre of a book simply by writing another book in the same series, without altering a letter of the original work, the genre distinction is not really self-contained. It basically doesn't really exist, except in some marketer's head.
More to the point, even if there is a line between the two, modern authors are tromping all over it with big muddy boots. Take the Monster Hunter International series. It pretty clearly comes straight out of the tradition of Military SF. To the point where I'd not recommend it to any reader who claims to like "Fantasy" but not Military SF. But all the antagonists, and some of the protagonists, are supernatural creatures. So which genre is it? The proper answer is, "Who cares?"
Another pretty enjoyable Military SF book I read a few years back (David Weber's Out of the Dark) was about Earth being invaded by aliens, from the POV of some of the bands of survivors fighting guerilla warfare against the invaders. Hard SF all the way.
Right up until the end that is, when the vampires decided enough was enough and took over the invaders' ships (a bit of a Deus, but I still enjoyed it).
Seriously, these days just write a good story. Let the 3-drink minimum crowd over in marketing figure out what it is.
* - David Freer's The Forlorn managed to pull off what I'm calling The Pern Trick within a single novel. It started Fantasy and ended up SF.
The difference, in my opinion, between Science fiction and fantasy is not in the difference between magic and science – after all, there is hardly a difference. The difference doesn't lie in whether or not an element is explained or not. The difference lies in this question: Will your readers remember the plot or the characters? Is the climax of your story felt or understood? Is your story primarily an emotional story, or primarily a mental story?
Why is The Twilight Zone typically considered Sci-fi as opposed to fantasy? It is often the case that the episodes involve science, but it is also very often the case that it involves the super-natural, ghosts, the after life, magic items and abilities etc. So why is it sci-fi?
Its because in most, if not all, Twilight Zone episodes, the 'magic' and/or 'science' is used to serve a plot that is interesting in and of itself, not to serve the emotions of the characters. We only care about the characters because we are empathetic towards people in general. In most episodes, the characters don't have anything to distinguish their personality from other people, and we could easily substitute them for someone else. We remember the plot twists of these stories, not the characters.
On the other end of the spectrum, we have Star Wars, in which all of the 'science' is used almost exclusively to serve the emotional/character aspects of the films. What we remember most about Star Wars is the characters. Luke, Leia, Obi-Wan Han Solo, and Anakin/Vader are iconic on their own, and interest in the plot is generated by their actions, not in the concept itself. The force, then, is simply a metaphor for emotional aspects of Luke and Darth Vader's character and the emotional struggle between good and evil; it is their emotions projected into the real world.
This highlights the real difference between magic and science in stories. Science is mental. Magic is emotional.
One of the best examples of this is Andrew Weir's short story The Egg. This barely includes science at all. It has no scientific element in almost any way, but it is still considered a Sci-fi story. Because the main character is really not the point, the plot, the idea, is the point, and so it is considered Sci-fi. We don't feel it; we understand it.
So, in your story, it hardly matters how much 'magic' you put in, what matters is how you use the magic. If the 'magic' is powered by emotional aspects of the characters in your story, then it will be considered fantasy or at the very least "Soft Sci-fi". If the 'magic' has logical rules and is there to serve some aspect of the plot, such as creating plot twist, or creating conflict, then it will be considered Sci-fi.
If it does neither, then it should probably be taken out.
This means that, more generally, you should be looking at the overarching narrative to decide whether your story is fantasy or sci-fi, the themes and the characters, not necessarily whether or not this or that element is 'supernatural'.
Some examples from Sci-fi:
Most people consider H.P. Lovecraft to be closer to Sci-fi than fantasy because the plot is what is important, even though there is never any science in any of his stories.
What we remember most about Alien is the Xenomorph. Ridley is somewhat iconic, but the climax of the story is not about how Ridley changes, its about how difficult it is to kill the Xenomorph, which is purely a plot element.
The end of 2001 space odyssey is pretty much magic. It is essentially the character meeting God. But there is very little, if any, emotional content, only 'intellectual' content.
The climax of Interstellar has a plot twist and emotional content. So it could be considered both, but in a case like that, setting overrides.
In Blade Runner, the difference is subtle. Harrison Ford has an intellectual change as a character, not an emotional one. He learns that Replicants have feelings, but he doesn't really change himself. We care far more about the idea of Replicants having feelings than we feel sad about the replicant dying (since he is a murderer after all). The fact that I remember Harrison Ford and not the main character's actual name should say something.
Some examples from fantasy:
Regardless of the technology they possess, people will still be people. There have always been people who have believed in magic and the supernatural, and there likely always will be, even once we've gone and colonized Mars using nuclear-powered rockets or whatever hard scifi technology your story's centered around.
Whether things like Christian faith-healers, "spirit cooking" Luciferians, or various varieties of New Age psychics count as genuinely supernatural is left for you (and/or your readers) to decide, but the people in question would certainly believe that they would have real, repeatable supernatural powers (or, at least, that God does, and He's acting through them).
How do you incorporate them into a scifi story? Charismatic revival conferences on Mars. Children disappearing on an asteroid habitat, ritually murdered by villainous Satanistic politicians who think it gives them supernatural prowess. A freighter captain who ships the volcanic crystals of Io to buyers who think they have healing properties.
Hard Science Fiction is essentially fictional circumstances using only technology that comes from our current understandings of science. All happenings are measurable and correlate to the known universe and our understanding of it.
Therefor, anything attempting to introduce concepts (such as faster than light travel) that have no scientific foundation are not hard. Any seemingly super-natural happening which is not explained by known science by the end of a story disqualifies the story from being hard-science-fiction.
Hard Science Fiction: The End.
Science Fiction vs Fantasy - Soft Science Fiction allows squishier "technology" as referenced elsewhere in the thread. So long as there are rules and those rules apply and "science" is the foundation explanation of everything that occurs, you're in the science fiction space. IE, you could be exploring a theory of the way a law of the universe might impact things. Hand Wavium is stronger in soft science fiction, but the pre-requisite is that some core "Idea" of the world (from MICE) revolves around an idea that is at least proposed to be part of the scientific understanding of the world. This doesn't mean space travel, or advanced tech, or even the future. You can write science fiction if it's about the first applications of the scientific method on some other world in the past. The point is that it's about Science (Rigorous application of the scientific method to acquire and understand the world around you) and how that affects problems that characters are dealing with.
Fantasy is different from science fiction, but not in the exclusive sense. Fantasy is largely an idea story as well; but it's more about Super-Natural/Magic etc forces.
Before we get into where the lines exist, think about this: If both types of stories are idea stories, then they must share something in common. That something is the "What if?" and the super-genre that holds them both is called "Speculative" fiction. This super-genre also holds mysteries and other fictional works that ask the simple question "What If?"
So, how you answer that "What If?", the prerequisites you define for the "What If?", and the ability to answer the "What-If" on the part of the characters all factor into whether you can describe the work as a fantasy or science fiction. I would urge you to think of "Fantasy" and "Science Fiction" as tools or descriptors which can be applied if the work meets specific criteria; not as a fallacious exclusive binary categorization, where the work is either A or B, but not both.
Does the work have anything to do with wish fulfillment? The inability to understand seemingly magical occurrences? Or, possibly spectacularly, but inexplicably powerful beings which influence the story? The author is using fantastical elements in their story and the Fantasy classification can apply.
Does the work have anything to do with the way things work? Are there explicit rules that are discover-able that perfectly describe the interactions of things in the universe? Are there people who investigate how things work via the scientific experiment, and do they codify their findings for others? Do elements, set-pieces, or characters revolve around the combination of such understandings into some sort of construction? If yes, then there are Scientific elements in the story. The story fits a science-fiction classification.
Fantasy & Science Fiction are 2 circles in a ven diagram. They live in the speculative fiction space and there can be overlap. Hard Science fiction is a term given to the section of science fiction that does not overlap with Fantasy's "inexplicable" zone at all.
There are plenty of works out there that already explore this concept. A world that starts out fantasy can be revealed to actually be a world of science, and though rarely recognized the opposite can often happen. Consider most modern religious works, they take our world of logic and reason and try to imply by the end that there's a greater mythology that governs the world, that is perhaps outside of science. This is hardly different than the fantasy book that starts with orcs vs elves and ends with the revelation that they're all descended from gene-modified humans, and that "late earth" is really and always just earth; that magic is really just manipulation of a nano-bot net that's infected a planet.
The point at which you cross the line from one space into the other is the point at which you use an element or concept which rules out the other. IE, explaining that magic was really science all along: welcome to exclusive science fiction. Explaining that science was really magic all along: exclusive fantasy.
Arguing that you can have your cake and eat it too? It depends. Note that just because you explain something doesn't mean it will hold up to the rigor of inspection if your explanation is flawed; this is where the line for hard sci-fi is drawn. Not only must you posit scientific truth, you must adhere to our understanding of scientific truth.
To answer your other question that inspired this one in the context of this answer. Is it ok to have things that aren't explained or explainable by characters in the story and still be hard-science-fiction? Yes, so long as an explanation can exist that is not ruled out by our scientific understanding of the way the universe works. IE, a character not knowing something or not being able to know something is entirely scientific.
There are mathematical proofs that show it is impossible to know everything. It would be fantasy to imply otherwise. There are bounds to the observable universe. You can know the speed of a particle, but not it's position; or you can know it's position, but not it's speed. Science does not grant perfect knowledge; Science leads to the acquisition of models that describe some portion of the universe and it's functioning which has been verified by legitimate, well constructed experiment. These models and discovered rules are the best we can reason based on observation, and we have reason to doubt anything that violates them and can't be re-produced. But, having a set of models and rules does not mean you know everything, it merely means you can likely explain the set of things which have models & rules.
IE, People can go missing, and you might not know why; but there still exist many understandable ways those people might go missing: Tidal Waves, Disease, Alien Abduction, etc.
A final note: Publishers shelve things by genre in order to sell books. This is very important. It also applies to tv, movies, radio and other forms of entertainment. So, while everything I said makes a sort of academic sense (and is frankly far more useful to the writing process), if you're trying to sell a book then you want to describe it in a way that the audience you think will read it will find it, even if it does not totally or neatly fit the definition. Sales-level-genre classifications are used to make short-hand promises to consumers about what they might experience if the individual attempts to consume the work. This means that if you say something is sci-fi, and there's a class of people who thinks that means "has spaceships", but they will buy anything with space-ships; then it would be correct to call the work sci-fi (even if you count up all of the fantasy/sci fi elements and there are more fantasy elements).
But if you're trying to understand components of books and the building blocks of them in order to write, or even analyze what's going on at a higher level for academic purposes, the above definition will suit you better. It's important to know how different communities map genre to works if you're going to sell something because they will expect you to live up to your promise.
The Hard-Sci-Fi community will not tolerate unexplained supernatural forces in a work classified as "Hard" Sci-Fi.
Alastair Reynolds this in the following terms. He said the New Weird could all manner of fantastic entities including ghosts and this worked fine. In fact, if China Mieville had ghost in one of his New Weird novels that was perfectly credible. But as a science fiction it didn't feel right to put ghosts into the sort of science fiction he wrote.
Essentially it is a matter of credibility. Ghosts, as a signature example fit right into supernatural fiction, many forms of fantasy, and the New Weird. The trouble is ghosts on a spaceship lacks credibility and plain doesn't feel right.
While it is arguable that various forms of the magical, the fantastical and even default supernatural are domesticated in science-fiction as the pseudoscientific. Typical examples are psi powers like telepathy and psychokinesis, psionic devices, and force-fields.
There is a subgenre of science-fiction is the pseudoscientific rationalization of the apparently supernatural, For example, finding ways of rationalizing werewolves, vampires, and other supernatural creatures. This takes what was supernatural and domesticates it into the fold of science-fiction. This isn't the same thing as the supernatural in science-fiction.
The inexplicable in science-fiction can and will appear, because current science doesn't know everything. However, the strictly supernatural lacks credibility to be a legitimate part of science-fiction especially with science-fiction attempts to maintain scientific fidelity.
I would propose that there is a simple yardstick to measure whether your "supernatural" elements are making your story a fantasy story.
A ghost story in space is not a fantasy story. It's still sci-fi.
A story with religious faith and maybe a miracle or two in space is not a fantasy story. It's still sci-fi. Example: Out of the Silent Planet
A story about wizards and unicorns in space is probably a fantasy story. Example: The Darksword Trilogy (On the other hand, maybe your story happens on a holodeck, maybe? But that's still probably going to appeal more to a Fantasy audience.)
(Star Wars is a famous example of a story that would clearly be a fantasy story if it wasn't in space and with robots)
"Supernatural fiction or supernaturalist fiction is a genre of speculative fiction that exploits or is centered around supernatural themes, often violating naturalist assumptions of the real world."
So i believe that the like is exactly at "violating naturalist assumptions" with the key word been "assumptions".
To keep your category at science fiction, presumably hard science fiction, and include supernatural fiction at a magnitude that does not alter from science to supernatural, there must definitely be a line.
When dealing with simple or soft science fiction, blurring those 2 categories is natural thus keep the line is more difficult and perhaps not required.
For hard fiction now, as long as the supernatural event or phenomenon remains unknown and unexplained because of science knowledge limitations, this i believe suffices. It is Science that defines Supernatural, not the opposite. This is the main key i believe. What i mean, assuming a supernatural event X:
The observant will not accept X neither as permanently unexplained nor breaking existing laws of science. It is presently unexplained and a great opportunity to enhance the laws of science.
The observant will try to identify and explain X using all means at his disposal. High tech, researcher specialists, think tank groups, anything his level has to offer.
The more advanced the observant is, the greater X is described as "research subject" versus "impossible to happen" and "enhancing current science laws" than "breaking current science laws".
A different observant may have partially or even fully understood X, so for him X is not supernatural.
X is happening so it must be explained by you within your story world, even if this explanation never reach the reader full part, or even not at all. However it is quite dangerous not to explain even a small but important part because that way you do not prove that supernatural fiction exists within the reference frame your science fiction setup allows.
X is not really happening but manifests within the minds of advance intelligent and obviously emotional beings. This is a dangerous approach for the line you seek because you probably go for a physiological thriller placed within a science fiction setup. Even if science fiction setup is the only setup to feed your thriller theme, you can hardly categorize this as science fiction.
Please not however that all these work for most readers. When merging fictions a percentage will always rank differently. By most you may succeed to science fiction with supernatural fiction embedded but some will still categorize as supernatural. That is not necessarily bad.
