You can find the schematic for the NanoVNA V2 on github: https://github.com/nanovna-v2/NanoVNA2
How does the VNA measure S11 return loss?
With an RF Bridge and a balun, brilliant.
You can try it for yourself in spice as follows:
When the DUT test impedance is equal to the 50 Ohms, the bridge is balanced and there is no differential output signal. The balun attempts to reject the common mode signal.
Kudus to the designer, clean, simple, cheap... love it.