This is fairly standard practice for historical novels, actually. The important thing is to apply it consistently and carefully, and only when it won't hurt the reader's comprehension.
Using the older names for places is very common and harmless. But if you do that, you need to do it for ALL place names and all names of people. If your medieval knights are heading to the port at Lundenwic they'd best not run into anyone named Olivia or Miranda. This means sometimes doing an absurd amount of research to make sure your characters have names appropriate to their time and place of origin. No pre-contact native americans named Tim.
For non-specific nouns, like "horse", just use "horse" (assuming you're writing in English). And don't use "Thee" and "Thou" unless you're directly quoting something. Your characters and your narration are all using contemporary English (or whatever language). These just become stumbling blocks which annoy (or worse, confuse) the reader. When they call Japan "Iapam" they are calling it by its proper name, in their time and place. When they call a horse a horse they're just referring to a horse, and you can "translate" like you are for everything else they say and think.
There are grey areas, obviously. Nicknames are a good example. A young Mongol in the army of Genghis Khan might have the nickname "Horse" or it might be "Aduu". Whichever you think sounds better, it's unlikely to put off a reader either way. But don't refer to actual horses in the story as "aduu". Even in worlds with wizards and dragons where no one has ever heard of England or English, people usually ride horses.
In short, use proper names of people and places taken from the actual historical context (Londun for London, Iapam for Japan, etc) but do your research to keep this historical accuracy consistent because you WILL get letters for the smallest mistake. Do not use archaic language for common nouns. There are exceptions because there are always exceptions, and there's a frustrating lack of logic to when such exceptions are made. Avoid writing in dialect (thee, thou, ye olde) as it just irritates people, unless you're directly quoting an historical document or speech.
In the case of historical fiction you also have the bonus option of being able to include a short afterword or glossary to explain archaic terms or mention what you've changed in the events. When someone writes about wizards and makes a Tolkien-esque appendix to explain the history of imaginary countries at the end, it usually comes off as ridiculous. When you write about Napoleon and include an appendix explaining what REALLY happened and how you changed the order of events or whatever, it's considered thorough. If you write about a wizard who was at Waterloo I guess it can go either way, but you should probably err on the side of "what really happened" over "the history of wizards and why Napoleon was still an important historical figure despite them"