Group Actions
Engagement
  • Author Author: Former Member
  • Date Created: Date Created
  • Views 4207 views
  • Likes 1 like
  • Comments 13 comments
Related
Recommended

Arduino: Dynamically allocating and freeing memory

Former Member

Everything that happens on a micro-controller has to be known. This includes dealing with memory. Many people (myself included) have developed a wonderful set of code that can dynamically use and free memory through the standard malloc and free functions. Only later do we discover that our available memory is getting reduced as time goes by -- followed by undetermined and random failures.

 

This is because no microcontroller (the ArduinoArduino included) has the ability to garbage collect.

 

For instance, let's say you malloc A and then malloc B. You then free A. There is now a memory hole (the size of A) that will never be closed again. Essentially a non-existing A is now taking up memory. If you now malloc A again, it will not be stored in that hole -- new memory will be taken up. You can see how this can become a problem.

 

To solve this problem, I developed the library ReMem (Reusable Memory), located in my usertools library, which among other things contains a full featured User Interface.

 

Documentation of ReMem can be found here. Basically ReMem creates an object (with a specified size) that has two functions: rmalloc and free. These work exactly the same way as the conventional malloc and free except when you free data, it will be used again if the same size data is requested. (The disadvantage is that it can be slow, and takes an extra byte of data per malloc).

 

 

Upcomming

The code is currently stable, but I'm going to be adding the ability to reclaim data that has been freed at the end, as well as cannibalize large allocations. Stay tuned.


Update

I have a new, more tested library for this purpose called tinymem. Check it out here: https://github.com/cloudformdesign/tinymem

  • in reply to clem57

    sorry about that. Some weird glitch on my phone. Fixed now!

  • in reply to clem57

    It could be a RTOS. In fact, I intend to get the Contiki Developers on board as soon as tinymem is out of beta.

     

    tinymem is not yet finished, but the final implementation will not take a few milliseconds of continuous processing to defragment. The final implementation will be threaded and re-entrant (using protothreads), the current target is at most 100us per cycle (most of the time it will just be sleeping).

     

    I think quite possible we just have a disagreement on where the technology is and where it is going. The internet of things has the following requirements in my view:

     

    • Robust Many to Many communication that is easy to set up and can communicate all the way from a microcontroller on a CAN bus to a server on a public IP. I am exploring adapting ZeroMQ to work on microcontrollers and a range of socket types (uIP, TCP/IP, ZigBee, CAN). Unsurprisingly, this practically requires a memory manager that can defragment, as communication protocols like ZeroMQ require dynamic memory allocation.
    • Rapid deployment of a huge number of and kind of devices. This will require portable code and the ability to reprogram over the air. Certainly you can create C libraries that have this portability, and it is possible to compile locally and program over the air -- but it is much easier to simply send text files and have the target system run python. That is extremely portable and extremely simple.
    • Simplicity simplicity simplicity. I'm surprised you love Arduino so much if you are so gun-ho about being right on the silicon. Arduino is an abstracted language similar to C. It makes it easier and faster to develop for microcontrollers. Micropython and things like it such as the SparkCore and ContikiOS are advancements in the same vein as Arduino -- they make programming microcontrollers easier and more accessible. Most important, simple is often more robust.

     

    So ya, it's fine to disagree and believe that microcontrollers should never do threading or dynamic memory management. However, I think you will be proven wrong as time goes on image

  • Everything that happens on a micro-controller has to be known.

    Why? This is not Linux with multiple users or processes. This is not even an RTOS. BTW, why would I want real time event processing take an extra few milliseconds while garbage collecting goes on? Kind of defeats the advantage of processing hardware in finite intervals. Arduino is meant as simple get it done environment with predictable results. And micro-python or whatever? Better create a python cross environment and avoid interpreter overhead! I have been doing this for too many years to believe that this is a true solution.

    Clem

  • in reply to Former Member

    Large font does not make a good message! Please respect others thank you....

  • in reply to shabaz

    tinymem  is definitely intended for devices with >10k of RAM,  and was actually is being developed for micropython, whose target implementation has 190k of ram.  At only 50 bits per pointer though,  it could still be useful on embedded systems that are trying to do a lot of things; like take data,  analyze it,  and send a long message to a master node.

     

    Edit: also with some pretty basic preprocessor commands it could be brought down to 34 bits per pointer (as long as you had less than 256 pointers... which would be true on something like an arduino). So ya, it's not THAT much memory overhead!