I shall go against the current.
Every person has a different approach to writing. For instance, I tend to draw a generalist plot (often a diagram of ideas and main events) first, then I start writing and, as the characters grow and gain flesh, and as I come to understand the characters and their motivations better, the general plot becomes more detailed (sometimes it is even changed and adjusted to the ideas that popped up as the writing proggressed). If, for some reason, I become disappointed with how I wrote some events or chapters, my motivation drops. I must go back and fix the problem or I'll go into an angsty writer block.
If the OP feels that he's so unhappy with the POV that it keeps him from moving on, then I would strongly advice to go back and revise. First, though, as I commented above, make sure you want to change the POV for the right reasons. A first person POV is not easy (and yet, when well used, even if only for certain chapters, it can be brilliant), but if one doesn't practice it, one will never get a hang of it.
I know my advice goes against the grain. I know that, for most people, re-reading and re-editing written chapters (searching for that ever elusive perfection) is a way of not writing more chapters. But a writer must take all types of advice to heart, understand where they're coming from, and choose their own path. My written chapters are usually not rough drafts; they're as close to the finished product as I can. That does not meant that, if something happens that requires foreshadowing, I cannot go back and add the necessary foreshadowing or hints or whatever to make sure everything flows as it should. It also doesn't mean that I am not able to go back and get rid of whole chapters if I feel they're really not necessary.
Assuming you have all the right reasons to change POV, think about how you write and what makes you feel comfortable: are you able to carry on with an entirely new voice, causing a break with what was previous written? Then do so. Do you feel like the not-fixed chapters are holding you back? Then rewrite it.
One last point: will rewriting everything now give you new insight into the characters, thus changing how you write them and how they react to events? If the answer is yes, go back and rewrite now. If you keep on writing only to later realise that no, A would never have said that and, therefore, event C should never have happened the way it did... that can be more time consuming and frustration inducing.
So, whatever you do, make sure it's for the right reason and that your 'creativity-productivity' is comfortable with it.