There are at least three kinds of issues involved in the kind of thing described in the question: Intellectual Property (IP) issues, defamation issues, and suspension of disbelief issues.
IP
Copyright
Names and short phrases are not protected by copyright. Characters and invented settings can be, if they are sufficiently detailed and distinctive, but brief literary references to such characters and settings will almost always qualify as fair use, fair dealing, or some similar exception to copyright, depending on the country involved.
Literary references are used very frequently, and do not subject the authors to copyright liability.
The idea that "there is a certain amount of words (about 24) that one is able to directly quote before needing permission from the parties in question" is a myth. There is no such number. In some cases quotes of hundreds or even thousands of words have been judged acceptable, and in other cases a few words have resulted in a judgement of infringement. Under the US doctrine of "fair use" there are four factors top be considered, and the amount of the work used is only one of them, and then in proportion to the size of the source work. One line from a poem may be "bigger" than 5 pages from a novel or textbook.
Trademarks
Titles of works, the names of characters and other identifying phrases can be protected as trademarks. For example, I understand that "Gandalf" has been trademarked by the Tolkien estate.
But the protection given by a trademark is limited. A trademark primarily is meant to indicate the source of goods or services, and secondarily to indicate a reputation for quality. Trademarks are only protected against being "used in trade".
Any use which might plausible confuse reasonable people into thinking that a work comes from the trademark owner, or is endorsed, approved, or sponsored by the owner may be trademark infringement (unless permission was obtained).
So if Author A has written a series about "Joe the Great" and has trademarked that name, author B cannot label a new work "A Fresh 'Joe the Great' novel" without permission. But a book can be described, even on its cover as "More thrilling than Game of Thrones" without fear of infringement, and author B can have a brief cameo by Joe the Great in the work.
Defamation
If an author writes a work in which a recognizable real living person C appears (whether under the person's own name or under an alias), and in the book the C character does bad things, such as committing serious crimes, C might sue for defamation. C would need to establish that readers would plausibly think the character represented the actual person C. C would also need to show that people did or were likely to think that the actions of the character were taken by readers as the actions of the real person: for example that the real C had committed such crimes. Given all that C might be able to win a suit for defamation. But such suits over a fictional portrayal are often hard to win. It must be clearly established that people are likely to view the real person less favorably because of the fictional portrayal.
In the past there were, I understand, a few cases where a disclaimer such as "No reference to any actual person is intended. All characters are fictional or are used fictitiously" helped avoid defamation claims, and it came to be common to include such a statement. I was amused when such a statement was on the copyright page of the SF novel Island in the Sea of Time by Stirling, and on the first page of the book was a character obviously based on the real person Harry Turtledove (an author of several somewhat similar works). But that character is shown in a generally positive light, and I doubt any defamation case was ever contemplated. Indeed I understand that Stirling and Turtledove are friendly. In any case, such a disclaimer will not overcome an obvious similarity, although it might help in a marginal case.
A realistic novel in which a character obviously based on a specific, clearly identifiable person, is shown in a significantly negative light, and in ways which might plausibly be attributed top the real person (for example a controversial politician being shown as corrupt) might be a plausible target for a defamation suit. But it is the effect on the real reputation of the real person which matters. There are other requirements for a successful defamation case, such as fact vs opinion, which I will not go into here. A question about this over on Law.SE might get a good response.
Suspension of Disbelief
We do not commonly use idioms and expressions from long ago, from other cultures that we know little of, or from worlds we have never heard of. For example "To the Crows with him" was (in Greek) an expression common in classic Greece, I understand, meaning something like "to hell with him" in modern English.
When in a fantasy or SF work, set in the far future or in some other world or universe altogether, a modern cliche phrased is used, it makes me wonder a bit and be pushed out of the story. If a Character in Game of Thrones said "Beam me up, Scotty!" I would think it out of place. Similarly if in a historical novel set in the 1700s a character started saying things like "way out" and "cool" I would be annoyed. (I recall being annoyed by a use of "OK" supposedly in the 1670s,)
Some readers are not bothered by this sort of thing. After all, the actual characters probably did not speak English at all, and we wouldn't understand whatever language they do speak. But use of overly specific terms and phrases does bother some readers.
Sometimes this is used for a joke. For example in The Case of teh Toxic Spell Dump people scared by an earth tremor in an alternate version of California are shown flocking to a church dedicated to St Andrew, or since it is a Hispanic neighborhood, "San Andreas". The narrator thinks to himself "Well if there is a quake it won't be San Andreas fault." But then that book is full of such jokes: the narrator examines what spells have been cast with a portable spellchecker.
I think it is usually better to avoid this sort of thing, but that is a choice of writing style, and what the author thinks readers might enjoy or dislike.
One can invent alternate versions of phrases or events in place of current references, although if one spends too much time and effort explaining them, the effect can be even worse.
