The line between science fiction and fantasy is often blurred. And that's okay. Genre is often more about marketing than anything else. As a general rule though: SciFi has science and technology and fantasy has magic.
This excellent question explores the distinction: How to distinguish if a novel is science fiction or fantasy?
Time travel is usually considered SciFi, but can also be part of fantasy if the traveling is via magic and/or the rest of the story doesn't fit into SciFi. Many other sites focused on worldbuilding and literature and readers discuss this at length, but not from the point of view of writing.
Within the Writing.SE world, where (or why) would we draw the line in deciding the genre of time travel? What are the implications for authors and publishers?