For French words, at least, it is mostly a pretentious way to show the author is cultured, thought funnily enough their "French" is often incorrect, though it will pass mustard for most readers.
For other languages, there are words or concepts that are not easily translated like the German gestalt, the Czech litost, or the Japanese bakku-shan
Though, as suggested, it is also sometimes used to give it a foreign flavor and greater realism.
I think that it targets speakers of that second language only if the book is mostly culturally dependent and the readership is assumed to be bilingual. For example a book using Spanish words whose MC is a Mexican descended teenager having a Quinciniera.