Press ON - Hold OFF Latching Circuit

A couple more ideas to add to the list!:

I wasn't sure if by "press ON" you meant "long press" or "short press" was OK for switch-on. Long-press is worth considering sometimes, to reduce the chance of accidentally switching on portable equipment (which may then need auto-power-off). In other words, long-press-on, and long-press-off could be worth considering.

A lot of circuits on the Internet have small idiosyncracies, such as capacitors which need time to discharge, sometimes resulting in the user being unable to turn on the circuit quickly after switching it off or turning off when the button is released rather than when the button is pressed, which may be unintuitive, and so on.

So, idea (1) is to use logic, e.g. 74HC can be pretty low power. An example of a circuit with long-press-on and long-press-off that needs one counter IC (could be replaced with other 74HC parts too, depending on what's cheaper), and one single-gate inverter, is here:  Building an RF Switching Unit (in the section titled power on/off circuitry). That circuit allowed the complete elimination of the microcontroller (I needed low noise for that application), with the same button also detecting short-press in theory either with a microcontroller as you may want, or in that blog post, by another logic chip that allowed a selection of four settings, and a long-press would power off, and everything was reset on the long-press-on. I don't know of a negative idiosyncrasy with that circuit, it's been working since I made that board, since April 2021 on the same batteries, so it's low power.

Such a circuit can be large, but small size isn't always necessary, and it's quite cheap.

Since lots of functionality was given to a single button, I decided to describe it simply as "HOLD ON/OFF" and "1/2/3/4" on the silkscreen, to try to show that short-presses did functions 1-4. and long-presses did the power control. I liked that everything felt intuitive, with all operations changing on button-press, and not on button-release.

Idea (2) is to have a more general-purpose solution like Gene's, i.e. to implement power management IC (PMIC) capability in a microcontroller, because requirements can change, and logic gate circuits such as (1) cannot be quickly modified. With the microcontroller you can also do fancy stuff if it were ever needed, like auto-power-off. MSP 430 is one of several decent options, and the batteries would last years. It can also be low-cost if you need extra features that you could integrate into the same code. An example of MSP 430 power management is here:  Energy Saving: Building a Microcontroller-based System Power Controller 
I didn't implement a push-button (because I didn't need it) but it would have been easy to do that in several ways, for instance allowing the main microcontroller to apply a KEEP_AWAKE signal (which is in that circuit) to the MSP 430 chip if a push-button applies power to the MSP 430, or alternatively modify the MSP 430 code to do the latching standalone since there are plenty of available pins on the MSP 430.

That MSP 430 PMIC was used in this project:  Pico IoT LoRaWAN EasyNode   where the MSP 430 internal software RTC was used to periodically wake up the more power-hungry RP2040 chip, and keep it powered up for as long as necessary. The MSP430 chips can be low-cost, although prices have gone up in the past year or so. Since it can incorporate functionality like RTC and NVRAM in software, it might save costs overall if those features are needed. Plus it's easy-to-interface, by using I2C. Effectively it becomes an external peripheral for power management and additional functionality.

A couple more ideas to add to the list!:

I wasn't sure if by "press ON" you meant "long press" or "short press" was OK for switch-on. Long-press is worth considering sometimes, to reduce the chance of accidentally switching on portable equipment (which may then need auto-power-off). In other words, long-press-on, and long-press-off could be worth considering.

A lot of circuits on the Internet have small idiosyncracies, such as capacitors which need time to discharge, sometimes resulting in the user being unable to turn on the circuit quickly after switching it off or turning off when the button is released rather than when the button is pressed, which may be unintuitive, and so on.

So, idea (1) is to use logic, e.g. 74HC can be pretty low power. An example of a circuit with long-press-on and long-press-off that needs one counter IC (could be replaced with other 74HC parts too, depending on what's cheaper), and one single-gate inverter, is here:  Building an RF Switching Unit (in the section titled power on/off circuitry). That circuit allowed the complete elimination of the microcontroller (I needed low noise for that application), with the same button also detecting short-press in theory either with a microcontroller as you may want, or in that blog post, by another logic chip that allowed a selection of four settings, and a long-press would power off, and everything was reset on the long-press-on. I don't know of a negative idiosyncrasy with that circuit, it's been working since I made that board, since April 2021 on the same batteries, so it's low power.

Such a circuit can be large, but small size isn't always necessary, and it's quite cheap.

Since lots of functionality was given to a single button, I decided to describe it simply as "HOLD ON/OFF" and "1/2/3/4" on the silkscreen, to try to show that short-presses did functions 1-4. and long-presses did the power control. I liked that everything felt intuitive, with all operations changing on button-press, and not on button-release.

Idea (2) is to have a more general-purpose solution like Gene's, i.e. to implement power management IC (PMIC) capability in a microcontroller, because requirements can change, and logic gate circuits such as (1) cannot be quickly modified. With the microcontroller you can also do fancy stuff if it were ever needed, like auto-power-off. MSP 430 is one of several decent options, and the batteries would last years. It can also be low-cost if you need extra features that you could integrate into the same code. An example of MSP 430 power management is here:  Energy Saving: Building a Microcontroller-based System Power Controller 
I didn't implement a push-button (because I didn't need it) but it would have been easy to do that in several ways, for instance allowing the main microcontroller to apply a KEEP_AWAKE signal (which is in that circuit) to the MSP 430 chip if a push-button applies power to the MSP 430, or alternatively modify the MSP 430 code to do the latching standalone since there are plenty of available pins on the MSP 430.

That MSP 430 PMIC was used in this project:  Pico IoT LoRaWAN EasyNode   where the MSP 430 internal software RTC was used to periodically wake up the more power-hungry RP2040 chip, and keep it powered up for as long as necessary. The MSP430 chips can be low-cost, although prices have gone up in the past year or so. Since it can incorporate functionality like RTC and NVRAM in software, it might save costs overall if those features are needed. Plus it's easy-to-interface, by using I2C. Effectively it becomes an external peripheral for power management and additional functionality.