You don't need a valid/justifiable reason to dislike someone.
Maybe the judge didn't even care.
Maybe he was just watching porn on his smart phone and decided to reject the appeal, because that's his default answer...
The point is, it is hard work to explain why the judge dislikes him.. ;)
All you want to do is to have the guy stay in prison for the wrong reasons, and have the judge be the 'bad' guy...
So why not just go for it?
You can throw in whatever you want the readers to not like about him in the story.. And it didn't even matter ...