From your question, I assume that you want to write essays, not novels or popular science books. You want to express your thoughts on a topic and lay out your arguments for and against a certain idea.
Maybe it will help you to understand that there are two schools of essays, the earlier French tradition, epitomized by the Essais of Michel de Montaigne, and the later Anglo-American tradition which has been unfortunately fossilized into the essays we all have to write in school.
The essay, as we are taught to write it, is clearly structured, and it is this structure that you – if I understand you correctly – find inflexible and stifling. Your mind does not follow the principle of topic, thesis statement, arguments, and conclusion – it meanders. You feel yourself unable to stay on topic and find that your mind wanders to other, related, themes.
Maybe it will relieve you to learn that this meandering structure is the fundamental principle of the essai – I use the Frensh spelling to distinguish it from the academic essay – as it was conceived by Montaigne. Without going into too much detail, the basic difference between an essai and a well-structured scholarly treatise is that the essayist allows his writing to follow his thought, instead of ordering his thought into a structrue prescribed by academic tradition.
So maybe all you need to do to resolve your "problem", is to think differently about what you are attempting to do. You are not writing a paper for an academic journal. You are expressing your thoughts. So allow your thoughts to express themselves. You control your thoughts, if you think you are drifting too far, or you can revise what you have written, but I am sure that even if you don't, you will find that your writing is not as random and unstructured as you worry it might be. It will have structure, and that structure is the structure of your mind. And as you have found, others find that structure pleasant to read and quite publishable.
What I suggest is that you undertake an experiment: Try to write a few essays without restricting yourself. Maybe revise the results a bit, but not much. Then find a few beta readers who will give you feedback, and in the meantime let the texts rest a month or three to gain some distance. After the feedback and rereading, decice how you want to proceed.