It is most typical, in a fantasy setting, for the otherworldly creatures to come from another realm, or plane of existence --fairyland, or hell, or the outer realms, etc. If your otherworld creatures are from outer space, I would question what aspect of that specific origin is important to your story.
Assuming that it is important for them to actually be space creatures, I would look at three things. 1) What is their home like? Is it the actual cosmos that modern science teaches us about, or is it a more mythic region, "the Sky Country," for example? If it is clearly not mappable to what we actually know of outer space, it becomes easier to read as fantasy. 2) What do the earthlings know or understand about where the creatures come from? It takes a fair amount of technological advancement and scientific sophistication to know about the actual structure of outer space. If the story focuses too much on the technology, however, it feels science-fictiony, even if the tech is outdated. 3) How did they get here? If they came in a modern spaceship, your story becomes science fiction, by definition. But what if they arrived in an actual (oceangoing) ship? Or walked? Or used a magical ring?
The book Enchantress From the Stars is explicitly about a narrative that plays both as fantasy, from one point of view, and as science-fiction, from another. You might find it an interesting resource. And Raymond E. Feist's book Magician features alien magicians from another planet, who travel via magical portals, not technology.