I have seen, IRL, a solution to this problem.
I would say, make one (or both) of the characters unavailable; already coupled with or wanting a different romantic interest, and make it clear that to their future partner, at least, they seem unavailable.
Under those circumstances, make them increasingly good friends. And then something changes, the attached person is freed up, not to pursue the other, but eventually that is what happens.
A single male friend of mine became good friends with a married female colleague for three years; they went to lunch together nearly every day. He says he never expected any change in that relationship; but then something happened. Her husband lost his job, became increasingly unhappy, and within a year they were divorced. Nearly a year after that, she asked my friend if he would go see a particular movie with her, they began dating and now they have been married for over ten years. To me they seem happy.
For writing purposes, I'd make the parting incident uncaused by the attached party. In the real-life example, it was not this woman's fault her husband lost his job or had a life crisis because of it.
A death by accident or disease or terrorist attack would be similar; perhaps an impulsive infidelity (but not by the attached character). Or her spouse might have fallen in love with another willing person, that happens. Or her spouse realized he's gay.
I would make some externality bring about the separation, so the once-attached character is not seen as ditching their partner for somebody else, and certainly not an incident caused by the single party; that could be interpreted as intentional causing a break-up out of self-interest, even if you write it as an accident.
And you can make the single person a moral and reasonably enlightened person that can have a friendship that they do not ever expect to turn physical or romantic, even if their friend meets all the physical criteria necessary for their romantic attraction.
Also in writing, there is a lot of good tension to exploit in making a decision to risk ruining a great friendship by making it a romantic relationship. I wouldn't gloss over that too quickly for either party, I imagine those decisions can be mind-rending. And fun to read about!
EDIT: I admit I overlooked the master/slave relationship in this question, but I don't think it makes my answer unsalvageable; it just adds a complication. The solution is pretty simple: Like above, the honorable master (banished prince) is friends with his servant; that is not unusual. Being friends, at some point something changes. Maybe she risks her life to save his. He decides he can't really be her friend unless he frees her. It is a fraught decision; he is smart enough to know he risks losing her if she chooses to leave him. But out of friendship he goes ahead, hands her her freedom, and tells her she is free to leave. She chooses to say. That converts the relationship into a true friendship; and the prince thinks that is where it will stay. Then at some point, she echoes the risk of losing her that he took in giving her freedom: She risks losing HIM as a friend by trying to convert this friendship into a romantic relationship. She feels a risk because she still thinks of him as higher born, and perhaps not interested in romance with a commoner and former slave; she is worried his romantic sights are set above her station. But she is in love with him, and takes the risk. And her trepidation is unfounded, because thanks to her HE is no longer a slave to his former royal station; and he is happy to be her mate, consequences be damned.