In Uprooted, Naomi Novik deals quite elegantly with this issue.
First, she doesn't dump all the information at once, but sprinkles it where it's relevant.
We lived in Dvernik, which wasn't the biggest village in the valley or the smallest, or the one nearest the Wood: we were seven miles away. The road took us up over a big hill, though, and at the top on a clear day you could see along the river all the way to the pale grey strip of burned earth at the leading edge, and the solid dark wall of trees beyond. The Dragon's tower was a long way in the other direction, a piece of white chalk stuck in the base of the western mountains. (Naomi Novik, Uprooted, chapter 1)
So we learn the topographic details (valley, mountains on one side, forest on the other, you can see from one end to the other from atop a hill.
Later, we gain more bits and pieces:
Our valley was at the very edge of Polnya (ibid)
The Dragon had been called to court that year (ibid)
He never came out of his tower to stand a drink for the men at harvest-time the way the Baron of the Yellow Marshes would. (ibid)
singers would come through over the mountain pass from Rosya (ibid)
So gradually we learn the name of the country (Polnya), and that there's another country (Rosya), that the court of Polnya is some distance from the valley, that there's a place called "Yellow Marshes" and it's close enough for villagers to be familiar with how their lord acts. All of this information is interwoven with the narration. Because the cartography information is interwoven with the narration, it creates a general picture that fits into the story, instead of requiring the reader to remember details that are not connected to anything.
Distances in kilometres are less important than "how long it would take to travel from point A to point B". If there's a mountain range between point A and point B, they're for all intents and purposes farther apart than if there were fields there instead. It used to be quite common to measure distances in days of travel (on horse, on foot), and that's information you can expect a common illiterate person to know; not from one side of the country to another, but from one village to another, to the nearest city, etc. A person would know quite well the area within which he routinely travels (whether it's a farmer who travels as far as the nearest market, or a peddler who does somewhat longer journeys, or a soldier who's being sent every which way). Beyond that, they'd have an idea of what's around, gleaned from what they head at the market or in a tavern. And further away it's names of important locations and "here be dragons". If you think about it, there was no reason for an average person to care about distance in km, but every reason to care about distance in travel-days.
tl;dr topographic features ("I can see the mountains from my village") and travel times give a general frame of things without breaking the immersion.