Am I allowed to use art (album covers) in my book? If not what sort of permission would I have to obtain? The album covers are not in the public domain.
You would need written permission from the copyright holder, and would generally need to pay for it, unless you can establish "fair use."
I am not a lawyer, so I can't give legal advice, but fair use generally involves uses of low-fidelity or excerpted reproductions, used strictly for reference purposes, and that cannot be held to compete with the original work.
Everything you put into a book that is not your original work must be done with permission.
This refers to both artwork and text.
Items in the public domain have blanket permission for others to use. That's what being public domain means.
Sometimes a creator will put a work into Creative Commons or another program. In this case there will be very clear usage guidelines. Usually you do not need additional permission to use the work in certain ways but you will need it for others.
Fair Use allows some use of copyrighted works without permission, but it's extremely limited. For example, you may use short quotes from a book or paper while reviewing it or while discussing it. But this never includes actual artwork and certainly not full pieces like album covers.
All other works have a current copyright which you must respect. When someone creates a work, they own the copyright (unless they created it as part of employment with a company that owns it). They sometimes sell the copyright to another person or company. If the creator is dead, their heirs own the copyright.
In these cases, contact the copyright holder, explain what you're planning to do and ask if you can have permission to use the work. Always have a written contract (email at the very least but paper with real signatures from both parties is much better).
Be prepared to pay for the privilege. Cost is almost always a flat fee up front with no royalties of any kind. That's true even if you commission artwork for your book.