That's not so difficult to unravel. Bringing us here would require some sort of mechanism, wouldn't it? Then perhaps it's as simple as stripping us of the memories we created here upon our departure, and re-encoding them into us in the event that we arrive again. Or a fortunate few of us, as the case may be.