Custom Bootloader

You'll either need a Bluetooth or wifi module to get the file from raspberry pi. If you have a gsm board, you can connect to atmega2560 using uart lines, download the files directly from any server and update your board.

the atmega2560 with the current sketch it can connect to the internet to download the firmware but in order to install it to it self it needs the bootloader to be customised 

Alternative way is to add cheap controller with two uart and delegating gsm related works to it. Let the new controller download file from server, once the download is done, send message to stm32 using uart to enable writing to memory. 

https://github.com/JAndrassy/ArduinoOTA/blob/master/examples/Advanced/OTASketchDownload/OTASketchDownload.ino

The above example downloads the file to Arduino memory and then updates to latest version using that file. 

Implementation to apply the update is

here github.com/.../InternalStorageAVR.cpp

It's no different coding the bootloader, than it is coding application code.

You can see any existing bootloader, and edit it to include your new interface:

https://github.com/dcnewman/MightyBoardFirmware-2560-bootloader

I'm not sure anyone is going to offer to do this work for you. Perhaps they might, but instead of clutching at straws, I think it could be better to go through the code, learn it, and then add your interface,, same as if you were developing an interface in normal application code. If you get nowhere, consider an option such as ESP32, which may have plenty of OTA examples using what is possibly a more popular interface for this (WiFi).

Arduino comes with its own bootloader format for AVR devices right? Is it possible to edit that file?

Yes, it's just code, no different from application code. The example linked above is pretty much just a single file, and it can be followed to see how the firmware is accepted over the serial connection and programmed into the Flash. Usually bootloaders are very simple, since for some microcontrollers there isn't a lot of space for the bootloader, so code is kept short. I've not written a bootloader for that Atmel chip, but have certainly written bootloaders for other microcontrollers, and it doesn't have to be difficult. Often there was a big incentive to write a bootloader (save on buying expensive programmer tools!). Nowadays programmers are cheap so that particular use-case might not be as important. 

Yes, it's just code, no different from application code. The example linked above is pretty much just a single file, and it can be followed to see how the firmware is accepted over the serial connection and programmed into the Flash. Usually bootloaders are very simple, since for some microcontrollers there isn't a lot of space for the bootloader, so code is kept short. I've not written a bootloader for that Atmel chip, but have certainly written bootloaders for other microcontrollers, and it doesn't have to be difficult. Often there was a big incentive to write a bootloader (save on buying expensive programmer tools!). Nowadays programmers are cheap so that particular use-case might not be as important. 

I guess it is upto the original poster to select whichever one is convenient for him and for OTA as well.