As I interpret that passage, there is a question, followed by an explanation of that question. That is, what the speaker is asking could be paraphrased as:
[Question:] Do you remember when we were (or what happened) in Ashcroft Apartments? [Explanation:] We were (or it happened) in Ashcroft Apartments two weeks ago.
If that is what you want to say, I would segment that passage like this:
"Do you remember? Not this past Sunday, but the Sunday before that--"
The explanation isn't a question and therefore doesn't need a question mark.
On the other hand, you could think of the second sentence as a question as well. That is, the passage could be paraphrased as:
[Question:] Do you remember when we were (or what happend) in Ashcroft Apartments? [Question:] Do you remember that it was two weeks ago?
Then I would add a question mark after the em dash to indicate the intonation of the interrupted sentence:
"Do you remember? Not this past Sunday, but the Sunday before that—?"
As a side note, the sentence "Not this past Sunday, but the Sunday before that" seems complete to me and not broken off at all. I don't understand how that sentence could have continued after the em dash.
If you want the sentence to be broken off mid sentence, then actually break it off, e.g.:
"Ashcroft Apartments. Do you remember? Not this past Sunday, but the Sunday bef–"
A sentence cannot be interrupted at its end, so your example, as you give it, should be written:
"Ashcroft Apartments. Do you remember? Not this past Sunday, but the Sunday before that./?"
— with a full stop or question mark, depending on how you want the second sentence to be intonated.
