I'll answer this question from an uncommon perspective. The Publication Manual of the American Psychological Association (APA) recommends that you do not use the ™ or ® symbols but requires that you capitalize trade and brand names (2009, p. 102-103).
If a whole science can do without these symbols, so can you.
If you are unsure, look how respectable publishers like Addison Wesley or O'Reilly handle the case.
O'Reilly mostly does not use the trademark symbols on the covers of their books on Windows:
but sometimes they do:
Yet in the interior text of the Windows® 8.1 Bible O'Reilly does not use the trademark symbol that they printed on the cover. You can see a large part of the book online in Google Books (direct link):
What they do is print a trademark attribution on the copyright page:
Only if you sell or advertise trademarked products do you need to follow the trademark owner's guidelines. What you must make sure, though, is that it is clear that your publication is not official or endorsed by Microsoft in any way, and that you do not use their trademarks to market your publication (so, for example, do not make "Windows" bigger than other text on the cover).
If you are the trademark owner, you don't have to use trademark symbols. For example, Apple does not use trademark symbols with their hardware, software or brand name. They have no legal implications. Their only purpose is to quickly identify trademarked words and phrases, but their absence does not mean that a word or phrase is not trademarked.
I think that you should be safe if you do what other publishers do, but personally, I would consult a lawyer, especially if your publication is critical of Microsoft or its products. Paying for legal advice is cheaper than having Microsoft's lawyers destroying your life.