Python 2.7 code profiling package

Introduction

Python is one of the most used programming languages adopted as the high-level application development environment to develop in the embedded environment. This is due to its portability and how it is easy to program, together with its wide diffusion in all the environments, from the desktop OS up to the embedded SBC: Beaglebone, Raspberry and many other. Python rapidly affirmed between professionals, engineers, makers and hobbyists as the privileged language to make easy and complex applications on small devices.

Just working with embedded devices, SBC limited resources maybe useful to keep under control memory usage and application speed. As a matter of fact, it is true that Python is a nice language, easy to understand and fast to develop but it is an interpreted language and in some cases language optimisation for speed and resource usage maybe a critical factor to keep under control.

Easy to use

Python itself makes available a profiling class but its usage maybe almost complex especially when we need to profile only specific parts of the code. Then the Python Profile class is limited to an in-depth analysis of the code execution time while it is interesting also the control the memory consumption. Better if these two kind of code profiling can be done together or independently.

To make both features available an external memory profiling package has been added to a profiler package; This exposes a series of easy to use APIs simplifying the original class usage.

Profiler architecture

Profiler package consists of a main Profiler class exposing the API to profile timing and memory consumption used in the Python code. When the profiler is instantiated the initialisation class accept a boolean parameter. The Profile class APIs uses the EnabledProfiler class if the parameter is True or the DisabledProfiler class if the parameter is False. This permit to disable the code profiling without changing the specific profiler methods.

External references

The profiler package uses the sProfile Python profile class simplifying the calls and the creation of the output report.

The profiles packge needs the memory_profiler 0.41 package (documentation is here: https://pypi.python.org/pypi/memory_profiler To run properly the Profiler package you should install memory_profiler with the following command:

pip install -U memory_profiler

Memory profiler sources are available on GitHub: https://github.com/fabianp/memory_profiler

To run under the Windows environment, also the psutil module should be installed. Psutil (python system and process utilities) is a cross-platform library for retrieving information on running processes and system utilization (CPU, memory, disks, network) in Python; For more details on how this module works and last sources and documentation the link is here: https://pypi.python.org/pypi/psutil
To install this package use the following command:

pip install psutil

The memory sampling mechanism can be called once. Multiple calls of the memory sampling api has no effect after the first call. If the mem_used() API is called but the memory sampling has not been initialised the call has no effect and no output is generated.

Profiler usage

The following scriptlets shows and example of the usage of the profiler.

# Import the profiler and initialise the instance
from profiler import profile
# Profiler instance
profiling = profile.Profile(True)

Then in the blocks of code that should be profiled need to be included in the profile enable / disable APIs

def method_to_profile()
     profile.enable()

     [function code]

     profile.disable()

In the same source file can be created multiple blocks of profiled code to focus the timing profile just in the parts that are needed.

Exiting from the profiling, the profile_module method generate the the output report sent to the console.

     profile.profile_module("Function name", "Comment")

The following example shows a small report that can be produced by the profiler.

------------------------------------------------
open_devices()
------------------------------------------------
  5850 function calls in 0.037 seconds

Ordered by: function name, call count, internal time
List reduced from 82 to 3 due to restriction <'_init_()'>

ncalls  tottime  percall  cumtime  percall filename:lineno(function)
  1    0.000    0.000    0.000    0.000 __init__.py:375(__getattr__)
  1    0.000    0.000    0.000    0.000 __init__.py:382(__getitem__)
  1    0.000    0.000    0.000    0.000 collections.py:38(__init__)

The profiler package is distributed under the GNU GPL 3 open source license and is available on GitHub

Credits

The profiler package was initially part of a wider project developed for NXP PLMA Audio, Nijmegen, then this tool has been release as open source software, as well as it will be maintained in its future versions.

Attachments:

Profiler-0.1.3-Python-2.7.pdf