However, is it really bad to include something just for fun or just
because it conveys the rare (exotic and interesing) idea?
It's not bad. Truth to be told, many successful authors do it to an extent.
What you are describing is akin to the process of worldbuilding:
As Matthew Dave said, sci-fi is a major example of it. A lot of short stories (I'm reminded of Asimov and Ted Chiang) are built around "exploring an idea" rather than exploring a plot, or a character arc.
But the same could be said for other genres of novels. Mainly it's something you'll find wherever the author is building a fictional world, so fantasy and it's many branches are all culprits, but you'll find worldbuilding efforts across a wide variety of genres (I'm willing to argue that horror, distopian, alt-history and historical novels all fall into the list).
Exploring exotic and interesting ideas is usually fun for the writer. And it can be fun for the reader too, if done well, because it engages the reader in an intellectual level ( No suprise there's a whole SE for that ).
Let's say you introduce FTL travel in your sci-fi novel. Maybe it's not a core element of your plot: it serves only to carry your characters from point A to point B. You may just tell the reader "yea, they got FTL" and move on. But most novels don't cut it so short.
Seeing how a writer takes an interesting idea and expands it into a working enviroment is engaging.
But, balance is key.
While it's true that the readers may enjoy your ideas, some will want to see the plot go forward. Delving too much on exploring facts and ideas risks to bore or alienate part of the audience. So, think about what kind of readers you want to keep in: the action-thirsty ones or the more speculative ones.
And (again as Matthew Dave already said) learn when and how to interleave plot and setting without making your novel worse.
I'm reminded of China Miéville's "Perdido Street Station": it's a really great book with a really original setting, but the author has the habit of starting almost each chapter with a long description of how the streets of the main city look like, where are the squares, how building looks. He's very good at doing it, but personally in some chapter towards the end I just wanted to see the plot unfold. On the other hand, the friend who suggested the book to me enjoyed those descriptions wholly, so to each their own.
One last thing:
I'm guessing without these things the novel might just become a non-artistic book, like the scientific (not a popular science) one, but just the one that describes something unreal. Unscientific science.
You may want to explore differents formats. I see those "unscientific science" descriptions that you talk about more suited to short stories (as I said, it's not unheard of in the sci-fi genre).
In a short story - almost like a scientific article - you could dissect an idea without boring the readers, in a format like "What if x - then y".
Longer formats, like novels, will probably require a plot able to stand on its own.
Some authors do include pieces of "scientific like" description of non-existant things. In the Thirteen lives and a half of capitan Bluebear, Walter Moers inserts encyclopedia pages describing creatures of the world. Are they relevant to the plot? Eh, not really. In Ensel and Krete, it gets even worse! I've also seen some italian authors do this (in parody and satirical genres, like Stefano Benni's works).