The purpose of all writing is to communicate. The target of the writing may be the author (to write is to think, to think is to write), a few people, or the masses. The time frame may be the-things-to-do-this-afternoon or timeless philosophy. But it is always to communicate.
What has to be in place for that to happen? A common understanding must exist between the author and the reader. That might be language. It might be expected sentence structure. Perhaps the organization of the material. Forms of punctuation. We define these rules, not because they represent some Platonic ideal, but because we have found that a common understanding of these rules fosters communication. We have to have rules because human language is plastic. Without the rules, it is chaos and incomprehension.
But why do writers violate the rules? Well, there are thousands of rules, too many for any one person to learn, understand, and apply. So, perhaps ignorance is an excuse. But perhaps it goes back to that overriding desire to communicate. Writing the same thing in the same way, time after time, turns the reader off. Blah, Blah, Blah! The writer knowing this, changes things up. Different words. Different sentence lengths. Different metaphors. Different phrasings. And so on. Something new and shiny to catch the attention of a reader drowning in a sea of blandness.
Can that be over done? Oh, my, yes, it can and is and almost certainly will be done again and again throughout the annuals of time. I could go on but you get my point.
A poor writer never fully masters the rules. A good writer masters the rules and follows them. A great writer masters the rules but knows when to break them in pursuit of more effective communication.
Learn the rules. Read widely and critically. Ask questions? Did this writer break a rule? Did it help or hinder the flow of communication between the writer and the reader? Could you have re-written the material to follow the rules? Would the re-write be as effective? Is there a way to break the rules in a more effective manner? Is there a better way to communicate? Find that way.