You have made a common mistake about world-building: believing that it all has to go on the page.  World-building is for *you*, the author, to help you craft a story in a setting that feels real and unique, even though fictional.  The actual details that make it to the page are only what the characters and reader need to know.

Knowing details like the actual length of the year or exact average temperature might help you avoid committing jarring inconsistencies, or writing descriptions that are so vague they feel insubstantial.  So it's important work that helps the story eventually.  But you absolutely don't need to shoehorn it into the narrative.

Some writers, like Murakami, or Diana Wynne Jones, get away with making up their worlds as they go along, at the price of a certain insubstantial, dreamlike quality to their settings.  Others, like Rowling, can sometimes get the odd detail wrong, or make questionable world-building decisions, despite putting a lot of advance effort into the planning.  But in general, world-building helps your fantasy world feel more substantial, even (or especially!) if you keep the details to yourself.