Got finally around to package it, and on first glance it does what I kind of suspected, the teleportation inverts the selection rather then continue, at least on my system. For some operations it's actually nice, but on others not so.
Think Blender has this built in, thats why it works as intended, it's been a while.
The only way I can trivialy speculate on this is something like :
while on trigger event (left/right mouse button etc) and on first teleport, cross into a virtual greater geometry which somehow syncs with the real pointer location on release.
Sorry I'm trying to torus my head around this early in the morning and I'm not that helpfull :D