It depends on how significant the use is. For just a line or two, attribution of who sang and/or wrote it in context would suffice:
"Crossroads...seem to come and go...."
To publish the full lyrical text, I would contact the copyright holder and obtain permission--there may be a small licensing fee, equivalent to if you'd purchased the sheet music.
In this "excerpt" case, the above qualifies as fair use because the quote is insignificant in comparison to the body of the complete work (a variable definition) and if your lyric-quotes are relevant to the original ideas being expressed. then you are sitting on pretty firm ground legally. (also variable, but if your criticism is ~95 percent original content and ~5 percent "lyrical quotations," I'd say you are well within the boundaries of US copyright law. along with a full citation in the appendices: I'm not sure of the exact form (it depends on your chosen style)--I normally use,
Composer(s), track name, album name, &c year Record label name
In the case of this post, I used the first line of Greg Allman's first song to illustrate the answer (and provide an example) to a specific question about using a quotation from popular music in the body of a larger critique regarding film--intended to publicly inform readers about one particular instance of US Media Law. Even the post itself meets the aforementioned criteria of being "mostly original content created by the author," (me) and the lyric-quote is insignificant when compared to the rest of the post AND my use of it is relevant to the information I provided. This meets the definition of fair use and doesn't violate copyright.