I voted for Erin's response, as it echoes my gut feeling regarding the inferred intent of the speaker. Since I can't add comments yet due to rep, I did want to go ahead and add some additional thoughts here regarding the question itself.
In general terms, I'd suggest that the author should consider whether such an emphasis/distinction is actually meaningful to your speaker. That is, ask yourself whether you are trying to convey some additional clues or subtle signals to convey additional meaning to their statements. Why would the speaker deliberately use the contraction in one instance, while using the full form in the another?
If your speaker is fairly passive about the acknowledgment that they had never talked with her, then "I had never talked..." is perfectly fine as-is. However, this doesn't really seem to fit, given the other provided context: "Not even [...] Kiyoshi deserved an angel like Kobayashi." That seems to imply at least some degree of interest / desire / longing.
Together, with the context, using "True, I'd never talked..." (when contrasted with later usage of "But I had watched her."), would come across as an almost confession-like, throw-away acknowledgment (does the speaker actually regret this, or is it merely a statement of fact?). Then, the speaker would contrast that admission with emphasis on what they did do - they watched / observed her. This emphasis can be conveyed by the deliberately inconsistent use of the contraction in one situation, versus fully expanding the other.
Perhaps, among other things, the speaker is attempting to justify their familiarity with Kobayashi, despite having never actually spoken (or interacted) with her? It could at least be possible that the speaker has a natural tendency to minimize (here represented by use of contraction) things they don't like, while emphasizing (via use of the expanded form) what they do.
Anyway, if those ideas matter to the concept you're attempting to convey via the speaker, then I'd absolutely lean towards the "True, I'd never talked [...]. But I had watched [...]" form.
While this may not strictly relate to the actual flow, I think it can add some nuance and convey subtle clues which reinforce the speaker's underlying motivations.
Side note, related to another answer:
I'd + past participle* = "I had", in all situations. It is incorrect to assert that "I'd" only expands to "I would". When paired with that past participle ("talked", "watched"), "I'd" should easily be parsed as "I had". Unfortunately, this is just one of those crazy, context-based English rules.
* -- Garden path only occurs if "I'd" is followed by a single [written] word which can serve as the bare infinitive, as well as the past participle (such as "read" [as in, "I am going to read that book today"; pronounced as reed] versus "read" [as in, "I read that book yesterday"; pronounced as red]).
Example: "I'd read that book." ("I would read that book [in the future].", versus "I had read that book [in the past].") For this, additional context would be needed to narrow down the actual usage of "read" in this case.