In 2013 I thoroughly tested the first rPi 256MB with four NiMH AA rechargeables. Finding was that "5V" input won't boot at 5.1V, which is inside USB specification, but after running down the batteries a bit it would boot if 4.9 to 5.0 Volts was supplied.
Suggestion is to buy a buck dc-dc converter, wind its screw until it supplies 4.98 Volts to your test resistor, and then solder to the power rails near to the micro-usb power connector on your rPi. A pwm buck will accept anything from 6x AA or 2x Li or 6V Pb battery and by regulating that down to 4.98V it won't care when the battery has gone down, and nor will it break your pi when recharging.
Many, many PIs have been running just fine on 5.10V or 5.25V. So even though your experiment shows that in your case your PI had a problem with "5.1V", that is not the case in general. Officially the pi will take up to 5.5V, in practice I've tested it down to 2.8V (for the pi2), and it kept on working, but I am not willing to "sacrifice" a pi to see where it will break if I increase the voltage. I don't expect this to happen at less than 6V, but I could be wrong.
When I was testing the lowest-possible-voltage, it became clear that part of the required voltage is to have a little margin to allow for voltage dips. If the CPU is idle and then gets an interrupt and suddenly starts to process again, the current consumption can rise so quickly that the powersupply (what/whereever it is) cannot provide the additional current quickly enough. A buffer (capacitor) close to the pi will help getting enough power to the pi on short notice. So, maybe the effect you were seeing was related to "dynamic effects".
Gerrit's suggestion to use a powerbank holds up: those provide the right voltage at enough current to allow a pi to function perfectly without odd stuff like having to "drain the batteries first before you start using them". The come with an included boost converter to provide a stable 5V (usually a little more).
