The reference is to buying out of a publisher's exclusivity clause, in order to regain the rights required to republish your own work elsewhere. Writers often refer to "getting your rights back" for the time frame for this clause to expire (it's commonly a year from date of publication).
There is sometimes (often?) a "penalty clause" that effectively allows the author to buy out of the publisher's exclusivity (which would otherwise prohibit the author from selling the work to any other publishers, or republishing it themselves). That would be "buying back" the author's rights. It's mainly of concern when a publisher goes out of business, and the author may have to buy back their rights in order for their work to ever see the light of day, or when the publisher fails to promote the book – leaving the author with no income from sales and (short of buying back their rights) no way to rectify the situation.