Soap Operas are like the Ouroboros, the snake that eats it's own tail and has no beginning or end. In this context, opera is a pejorative indicating too much of one thing (an opera being several hours of "just" singing).
Soaps are endlessly unfolding stories without much narrative structure or substantial character development. Soaps have multiple scriptwriters who are constrained by production decisions outside their control: shooting schedules, existing sets/locations, choice of actors involved in any particular day's shoot, and fan service for the favorite characters who end up in similar situations again and again – not to mention all the plot threads left behind by other writers which are usually abandoned instead of woven back into the story to fit within the various other plots.
Narrative arcs and pacing become secondary to the show's production and broadcast schedule. A soap's version of a climax or cliffhanger might be scheduled for the end of the week, but not all characters appear in every episode, so it's typical to drag out a scene with a main character weeping over someone lying comatose in a hospital bed, meanwhile the audience knows they will "wake up" on Friday and still not say anything interesting until the following week, or when the unconscious person finally wakes they will have convenient amnesia, or some other writing cliché that allows the situation to continue without resolution.
To quote wikipedia:
One of the defining features that makes a television program a soap
opera, according to Albert Moran, is "that form of television that
works with a continuous open narrative.
"Quantity over quality" is a truism for pulp magazines, genre novels, and adventure stories which are plot-heavy and plot-driven, however soap operas are not plot-driven despite the preponderance of concurrent storylines. The "plot" of any particular episode is moronically simple, with clear character archetypes whose obvious motives are repeatedly spelled out for the audience. It's only after stringing these stories together in an attempt to tell a "show history" that the narrative becomes silly and incoherent.
Soap opera scenes are a series of vignettes that lead to emotional fireworks: the town gossip overhears a private conversation, an unrequited lover is friend-zoned, the young couple cannot be together, a villain returns for revenge, a wife makes a tearful confession to her unconscious husband in a coma, and Grandma Matriarch has the wisdom to tend-and-befriend. These archetypes never change and the outcome is always predictable, but the audience loves watching their favorite characters go through an emotional wringer – emphasis on emotional.
TV soap operas are the direct descendants of 19th Century melodrama, populist entertainment that was intended to provoke strong emotions in the audience. The word "melodrama" (music + drama) indicates a play that had incidental music that enhanced the action (not a musical). Characters were clear archetypes; plots were formulaic or based on well-known popular novels. Physicality was broad gestures and facial pantomime while the spoken text was bombastic (we'd call it "overacting" today). Actors frequently broke the fourth wall to engage the audience directly, and the audience was encouraged to boo the villains and applaud the hero – all in the interest of amplifying the audience's shared emotional state. There's no room for plot subtlety or character ambiguity, those would just muddy the group emotion.
Melodrama evolved through the 19th Century's obsession with exploring emotional states, part of the larger trend of Romanticism. Whereas today we have many expectations from a narrative including character development and maybe a surprise ending, melodrama was all about reveling in emotional states as group entertainment. Plot development and characterization became less important, meanwhile while on-stage situations became more sensational, or compounded (kidnapping AND incest) to provoke bigger emotions. C19th melodrama is contrasted with "problem plays" during the same era where social issues were debated intellectually through proxy characters. By comparison, melodrama is as uncomplicated as a monster truck rally.
Often ridiculous soap opera plot "twists" derive from cast changes and actor popularity. These changes are just an excuse to give a character more on-screen drama, often putting them in an unlikely situation of discovering an unknown relative, a doppleganger (played by the actor in a dual-role), or the revelation of a "dark past" that is completely out-of-character but gives an existing role new range.
American soap star Deirdre Hall with reallife twin Andrea who sometimes played her "evil" twin on the show Days of Our Lives, later
both roles were played by Deirdre after her sister retired from
Sometimes these "twists" are so overused they are baked-in at the beginning of the show. At least one character is searching for a birth parent, meanwhile a villain who is too powerful to jail is consumed in ruining a protagonist's marriage – in real life these might be minor character motives but not to the degree that someone would disguise their appearance, adopt a false name, and move to another town under an elaborate hoax just to get closer to their person-of-interest.
Since numerous soap operas ran successfully for decades with a (more or less) contiguous cast, I don't think you can call this kind of writing a "failure". Many plot issues arise from the fact that they are ongoing – essentially generative, rearranging a stock cast to retell the same archetypical stories over and over. Like generative novels they tend to collapse into formulaic gibberish when viewed as a whole.
But clearly there is audience appeal in watching sympathetic characters navigate sensational situations. The OP points out that "bad" writing is generally the fault of rushed production and diluted storytelling, but that dismisses the fact this form of entertainment isn't trying to be intellectually highbrow in the first place. Like a pop song that foregoes musical complexity in favor of familiar hooks and simplistic chord changes, it's more about how it makes the audience feel, than what it makes them think.