Given that you think about outlining each chapter, I can encourage you that there are writing teachers and writers who hold that a writer should have absolute (rational) control over their writing and not let their urges drive their writing. Writing, for these writers, is a job that they perform in the same way a carpenter builds a kitchen: just as the carpenter has the dimenions of the kitchen and the specifications of his client to follow, the writer (of popular fiction) is writing to the taste and needs of his audience – not his own. (If he is lucky, these overlap.)
With this perspective of quasi-commissioned, rationally planned writing, it should be perfectly possible to write the different parts in any order suggested by practicality, not chronology or sequence.
I can testify that this kind of writing works, at least in non-fiction writing, where it is often the norm. It does need a final sequential rewrite, that is, when you are finished you will need to go over your whole work from beginning to end to verify that it all works in the intended reading order and polish the edges.
I have never tried this for fiction, but I am positive it can work, if you are the kind of writer who writes in a very planned and controlled fashion.
You need to try.
I am convinced that you will find that although you think that you have most of the plot planned out, enough of the narrative will change in the course of the writing, forcing you to rewrite the later parts you already wrote.
Dale Hartley Emery mentions the writer Kristine Kathrin Rusch in his answer, and in a comment gives a link to a blog post of hers, where she describes how it goes for her with writing out of order. Apparently it works, because she has been a successfully published writer for decades now, but if you read her blog post you will find that she suffers just the problems I have described: having to read and rewrite in sequence to fill in missing and cut redundant passages.
And for that it helps, as Mark Baker has pointed out in his answer, if you do not fall in love with the results of your writing so that it hurts you to delete what suffocates the story.
But, as is endlessy repeated on this site, people are different and you need to experiment and find out what works for you.
After telling you that non-sequential writing can be done, I'd like to explain to you why I don't do it (and have never before your question considered the possibility). You are not me, and what you need to do will be different. But maybe this will give you some idea of what to watch out for: getting lost; and not having a hook.
Personally, I always start with the beginning and let everything follow, usually in reading sequence, from there. There are two reasons why I work that way. One, I am a discovery writer. I usually do have some points on the map that I want to visit (and I mostly manage to get there), but I need the surprise of what I discover on my way there to keep me interested in writing. I am in the process of learning to outline more, but this only means that now I am a discovery outliner, and what was once the rewriting is now becoming the writing.
Two, the beginning of a novel is the most important part when it comes to selling it. Readers, if the cover and blurb make them open the book at all, usually look at the first page to decide whether they want to buy the book. I have bought books because of the first sentence alone. Therefore, getting the beginning right is vital for any novel. Maybe others can go back and polish the first chapter after they wrote the whole book, but personally, like the reader, I need the first chapter to set up the vision that will drive me through the writing and interested in finding the end. I think I spend 90% of my writing time on the first few paragraphs. It really is a painful process sometimes, but once I have the outset, I'm in the world and ready to explore. Without this "beginning place", how could I possibly know what the story is about and where it needs to go?
