I know it is important for main characters to have flaws to make them believable and real. But should an antagonist have redeeming or good qualities, to make him more real?
I think it depends on whether you want the villain to be liked or not.
I think it is hard for people to believe in a good character who has no flaws. But it is easy to believe in a villain who appears completely bad. A bit of a double standard...
So if you are creating some sort of anti-hero or just want a likeable bad guy then give him some redeeming qualities. If you want him to be disliked by the readers, then it's probably best to keep him all bad.
Everybody has redeeming qualities and everybody has flaws. Any character that doesn’t have both is unrealistic.
I don’t think it is a good idea to separate antagonist and protagonist when doing character work. Those are plot roles. Character-wise, everybody sees themselves as a protagonist. Everybody thinks they are the good guy. Hitler thought he was the good guy. Generally speaking, the only thing that makes someone a villain is they are willing to hurt other people to get their way. But they typically have all kinds of excuses for that which enable them to see themselves as a protagonist in their own life story. So a villain might still be nice to his dogs or give to charity or be a great baker, even while he is having somebody killed to achieve his professional goals.
In real life, very few people are "pure evil". They have history that makes them the way they are and motivations for their actions. So it would definitely be more realistic to have a few redeeming qualities or at least a background explaining why they act the way they do.
A character who is evil with no explanation as to why isn't very realistic. People aren't just born evil (unless they have some sort of mental illness) and oftentimes something happened to them to make them the way they are, whether it was just how they were raised or a traumatic experience.
A few possible scenarios:
1) Your antagonist was once a good guy but someone did something to him (killed one of his family members for example) and he wants revenge.
2) Your antagonist was traumatized early in life. A lot of people who were beaten as children also beat their kids when they get older for example (obviously not always the case). Some people also want others to feel the pain they feel, which is a good motivator.
3) Your antagonist is insane. They have something wrong in the head and are literally psycho. This can make for an interesting story because the character doesn't necessarily have to have motivations for his actions. They can be completely random. (Although a little direction or purpose is always good, even if the insane character seems like his actions are random.)
So, you don't necessarily have to make the antagonist "good" or have redeeming qualities but I do think a good backstory and motivation is required to make the bad guy more realistic (unless he is psycho).