I've had similar experiences.
Boy Meets World
Boy Meets World seems to have more discovery potential. Anything could happen. The protagonist is an undeveloped personality with an incomplete understanding of this world, and probably many naive beliefs on what their role should be, how things actually work, and other false perceptions that become more nuanced along the way.
Every plot beat introduces them to greater complexities. They must un-learn everything they took for granted as they realize the world doesn't center them. The MC will need help, but other characters have their own agendas too. The goal may be clear, but the path to get there is uncertain.
As the protagonist 'levels up' through experience, the reader's awareness of this world grows. It's expected that a fresh character won't have all the answers. Their personal antagonists are the story's antagonists. But as their situation changes, so do their antagonists and conflicts. It's an organic progression with the world always offering more lessons and surprises.
World Meets Boy
World Meets Boy is the situation we see in Chosen One stories like Dune, StarWars, and The Matrix. Here it's the world that is wrong from the very start, and only one person is destined to fix it – if not for that Chosen One the world would continue on its 'wrong' path… forever, presumably.
These stories tend to have elaborate but 'false' worlds already established with pre-determined conflicts and power structures. There is no other 'better' world to aspire to; how it got this way is not important – it's more about the idea. This world cannot 'level up' to meet the challenge of a Chosen One. This world is a strawman designed to be toppled.
The 'Chosen One' is an overpowered, typically one-note archetype who is there to 'break' the world. Other characters are impotent, existing only to marvel at the Chosen One's power. Sure they have been in this fight their entire lives (as have their parents and grandparents), but for what? At best they can explain 'how things are' to the MC but of course they are proven wrong. There is nothing to learn from these other characters the MC won't quickly surpass.
It's hard to feel any joy of discovery (as writer or reader) since the 'boy' will just tear it all down, often through un-earned abilities they don't even know they have. The subversion of a Chosen One trope (accidental hero, you can't win) doesn't resolve whatever world-flaw is baked into the premise.
If we see remarkable places, but without a personal connection, the story might feel like fantasy tourism. If no character leaves an impact… did anything happen?
Complicated Worlds might need a foil…?
I'm trying to think of stories that are clearly world-building concepts that don't have world-breaking heroes: THX-1138, 1984, Star Trek: Deep Space 9. They tend to be 'idea' worlds.
The main characters are underpowered and compromised. They move through the world so we see its machinations. They bear witness to big events. I think maybe we don't love characters who are manipulated just to make a point. Their stories can be a slog.
As writers, we indulge our complicated worlds but they aren't stories in themselves. We can imagine great locations, but the interesting spark is still the ghost in the machine. We look for the different-beat drummer, the fish out of water.
That creates a sweet spot where simple 'heroes' can be a foil to complicated worlds, just by being themselves. Dorothy is a foil to Oz. John Carter is a foil to Mars. Mad Max is a foil to the carpocalypse. These characters aren't particularly deep and shouldn't be able to topple existing governments, but it somehow works. The take-away is 'escapist' entertainment rather than a ponderous epic, even when many of the story elements are the same.
It's having the cake and eating it. You get to show off the elaborate world while putting a fresh character at the center of the experience. Another way to think is that the MC-as-foil is having an ongoing romance with this exotic world, the way Flash Gordon gets 'involved' with all of Mongo – there's a mutual sexy/dangerous fascination. That probably means there's a certain chemistry in finding the right foil for your world, someone who can rub against the rules in all the right ways.