element14 Community
element14 Community
    Register Log In
  • Site
  • Search
  • Log In Register
  • About Us
  • Community Hub
    Community Hub
    • What's New on element14
    • Feedback and Support
    • Benefits of Membership
    • Personal Blogs
    • Members Area
    • Achievement Levels
  • Learn
    Learn
    • Ask an Expert
    • eBooks
    • element14 presents
    • Learning Center
    • Tech Spotlight
    • STEM Academy
    • Webinars, Training and Events
    • Learning Groups
  • Technologies
    Technologies
    • 3D Printing
    • FPGA
    • Industrial Automation
    • Internet of Things
    • Power & Energy
    • Sensors
    • Technology Groups
  • Challenges & Projects
    Challenges & Projects
    • Design Challenges
    • element14 presents Projects
    • Project14
    • Arduino Projects
    • Raspberry Pi Projects
    • Project Groups
  • Products
    Products
    • Arduino
    • Avnet Boards Community
    • Dev Tools
    • Manufacturers
    • Multicomp Pro
    • Product Groups
    • Raspberry Pi
    • RoadTests & Reviews
  • Store
    Store
    • Visit Your Store
    • Choose another store...
      • Europe
      •  Austria (German)
      •  Belgium (Dutch, French)
      •  Bulgaria (Bulgarian)
      •  Czech Republic (Czech)
      •  Denmark (Danish)
      •  Estonia (Estonian)
      •  Finland (Finnish)
      •  France (French)
      •  Germany (German)
      •  Hungary (Hungarian)
      •  Ireland
      •  Israel
      •  Italy (Italian)
      •  Latvia (Latvian)
      •  
      •  Lithuania (Lithuanian)
      •  Netherlands (Dutch)
      •  Norway (Norwegian)
      •  Poland (Polish)
      •  Portugal (Portuguese)
      •  Romania (Romanian)
      •  Russia (Russian)
      •  Slovakia (Slovak)
      •  Slovenia (Slovenian)
      •  Spain (Spanish)
      •  Sweden (Swedish)
      •  Switzerland(German, French)
      •  Turkey (Turkish)
      •  United Kingdom
      • Asia Pacific
      •  Australia
      •  China
      •  Hong Kong
      •  India
      •  Korea (Korean)
      •  Malaysia
      •  New Zealand
      •  Philippines
      •  Singapore
      •  Taiwan
      •  Thailand (Thai)
      • Americas
      •  Brazil (Portuguese)
      •  Canada
      •  Mexico (Spanish)
      •  United States
      Can't find the country/region you're looking for? Visit our export site or find a local distributor.
  • Translate
  • Profile
  • Settings
Sensors
  • Technologies
  • More
Sensors
Blog Time-of-Flight Imaging: Low-cost sensing with ams/OSRAM TMF8828 and Pi Pico
  • Blog
  • Forum
  • Documents
  • Quiz
  • Events
  • Polls
  • Members
  • Mentions
  • Sub-Groups
  • Tags
  • More
  • Cancel
  • New
Join Sensors to participate - click to join for free!
  • Share
  • More
  • Cancel
Group Actions
  • Group RSS
  • More
  • Cancel
Engagement
  • Author Author: shabaz
  • Date Created: 11 Dec 2024 2:09 AM Date Created
  • Views 3133 views
  • Likes 15 likes
  • Comments 13 comments
  • ams/OSRAM
  • osram
  • TMF882x
  • time-of-flight
  • rp2040
  • ams osram
  • TMF8828
  • tof
  • time of flight
  • sensor
  • pi pico
Related
Recommended

Time-of-Flight Imaging: Low-cost sensing with ams/OSRAM TMF8828 and Pi Pico

shabaz
shabaz
11 Dec 2024

Table of Contents

  • Introduction
  • Quick Overview
  • How Does it Work?
  • I2C Registers Overview
    • Reading and Writing Registers
    • Application Mode Pages
  • USB UART Interface
  • Browser Based App
  • Summary

Introduction

Time-of-Flight sensors are fascinating. They measure the time it takes for light to travel to a target and bounce back. With an array of such sensing elements, it becomes possible to obtain a three-dimensional impression of the environment, and ‘see’ how far things are, and the directions they are moving in.

This blog post (inspired by an article from CorinnaR.   - Engineer-to-engineer: understanding time-of-flight sensing ) is about a tiny (grain of rice sized) device from ams/Osram which contains both a camera-like grid of optical sensors, and a laser! It is called the TMF8828. It’s that tiny two-aperture device visible in the photo below, soldered to a board which is piggy-backed onto a Pi Pico.

The device fires a wide spread of light (using its tiny laser and built-in lens) and then the grid of optical sensors measure the time it takes for that light to reflect back from a target (or multiple targets in the area!).

image

You could use it for occupancy or person or object movement sensing, distance measurement to a reflective surface, or to perform gesture sensing by examining which sensors in the grid are triggered and in what sequence. You could make interesting theremin-style musical instruments.

In this blog post, the hardware is discussed (you could self-assemble the board as I did, or purchase an alternative board from the manufacturer, which is functionally compatible but uses an Arduino). I then pass the measurements to the PC over a USB UART connection, so that they can be observed graphically using software, or captured using Python code.

See the following three-minute video summary:

You don't have permission to edit metadata of this video.
Edit media
x
image
Upload Preview
image

Quick Overview

In short, a TMF8828 board is needed. You could either assemble a DIY board as I did, or purchase a different one. There are a few development boards available for the TMF8828, they cost about £50 ($60 USD), although there could be slightly lower-cost options in the future (Adafruit has one listed as “coming soon”).

I decided to build my own development board and the PCB files are published. The main difference between my board and the other ones, is that mine will directly attach to a Pi Pico board if desired. If that’s not what is required, then it could still be wired to any other microcontroller of course. See the GitHub page for all files and code.

image

The board interfaces using I2C (and a couple of GPIO pins used for enabling the device and for device interrupts), and requires a 3.3V supply. The board has space for an optional 5V to 3.3V regulator, but that’s not needed when using it with a Pi Pico. The circuit diagram is straightforward, shown below.

image

If you wish to replicate this project, it will cost about £15 ($20) for the parts, and perhaps $2 for the PCB to be manufactured.

image

After that, the board is connected to a microcontroller (in my case a Pi Pico as mentioned above).

Then, set the Pi Pico into a bootloader mode (see the video below to see how to do that and transfer the firmware to it. After that, the firmware will start to automatically run.

You don't have permission to edit metadata of this video.
Edit media
x
image
Upload Preview
image

Now you can install and run ready-made software called TMF882x EvalSW Arduino Demo GUI (available from the Software section on the ams OSRAM TMF8828 Eval Board Product Page). on your PC (Note: If you're using a Pi Pico instead of an Arduino, then you’ll need to make a small change to a config file that will be installed in your C:\Users\username\AppData\Roaming\ams-OSRAM AG folder, called TMF8828 ARDUINO DEMO.ini – edit the line beginning with COM_PORT_DESCRIPTION so that it says COM_PORT_DESCRIPTION=USB Serial Device).

The software will automatically connect to the Pi Pico, and display in real-time an animated grid showing the measurements. There’s also a 3D view that can be enabled in the software, which shows an isometric view with the distance on the vertical axis.

How Does it Work?

As mentioned, the TMF8828 contains a tiny laser, covered with a frosted lens. It provides a small flood of light, pulsed at over 17 million times a second (confirmed by using a light sensor. Click here to see how:  Fast Response Light Sensing with Photodiodes: Building a DIY Radiometer! ). 

image

Reflected light enters another aperture on the TMF8828, and is focussed onto an array of Single Photon Avalanche Diodes (SPAD). These are like a silicon equivalent of a photomultiplier tube; when a photon hits a SPAD, there is a likelihood that it will cause a large detectable amount of current to flow. The TMF8828 measures how long it takes from the moment the laser is pulsed, to the moment when each SPAD detects it. There is actually some crosstalk from the laser to the array of SPADs, so actually it measures from the moment the crosstalk is detected by the SPADs, until the reflected photons arrive. The way SPADs work, they need to be reset after each successful photon detection, so the TMF8828 internally rapidly resets the SPAD after the crosstalk event, and after a reflected photon arrives.

Since there’s a chance that a single photon might be missed and a subsequent photon might be detected, the TMF8828 will reset the SPAD repeatedly, hundreds of thousands of times, and re-fires the laser each time, so that a more accurate time measurement can be made by using stats (i.e. by binning the results and looking for the bin with the largest amount) along with a confidence level percentage. By default, the binning occurs for 550 thousand laser pulses.

There are about 180 SPADs behind the lens, and only some of them are used, grouped in areas or ‘zones’. In the case of the TMF8828, there are six SPADs per output pixel (known as a zone in ams/Osram terminology. The TMF8828 captures a spread-out 4x4 array of these zones (i.e. 16 zones), and then repeats three more times for different slightly shifted SPAD groupings, to achieve a total of 64 zones (i.e. 8x8 array combined).

The diagram below (taken and slightly edited from ams/Osram documentation) shows the 64 zones, each has a number. The different colors represent the four 4x4 sub-captures, each of which contains 16 zones, and they are meshed to produce the final 64 zones result. The diagram can be used as a reference for performing the meshing.

image

An interesting thing to note is that because multiple SPADs are behind each zone, and results are binned, it becomes possible to detect more than one object in a zone. For instance, there may be a smaller object at a closer distance, and a larger object further back, both detectable in the field of view of a zone. The TMF8828 can actually report back the distances for up to two objects! It will report the nearest object distance first.

All of the above is fairly complex to co-ordinate, so the TMF8828 happens to contain a pre-programmed microcontroller internally, with an I2C slave interface, so that any external host microcontroller doesn’t need to do as much.

I2C Registers Overview

Reading and Writing Registers

The TMF8828 supports registers for reading and writing. The registers are addressed with a single byte.

Usage is simple. You can read or write registers individually, or sequentially.

To write a register (or registers) just perform an I2C write operation with the first byte being the register address, and all subsequent bytes will be written to that register address onward.

To read a register (or registers) you simply need to do an I2C write containing the register address. Then, when you perform an I2C read operation, the TMF8828 will provide the contents from that address onward. An I2C ‘repeated start’ is optional between the register address and the read operation.

Application Mode Pages

The register addresses can serve multiple purposes, because the registers are paged in.

The TMF8828 runs in two modes; the normal mode is application mode, and the other mode is bootload mode.

In the table below, anything shaded black is accessible in all pages (all application mode pages and the bootload mode page). Anything shaded gray is available in all pages too, apart from when in a bootload mode. You can tell if the bootload mode is in progress, if register 0x00 (APPID) contains 0x80. If it contains 0x03 (which is the only other option supported) then you’re in the normal application mode.

I’ll focus on some of the key application mode pages. There is far too much detail, so I’ve summarized things into the following register map diagrams, and for the detail, you’ll need to dig into the datasheet.

The pinkish-magenta shaded content below shows the most important page, which is the measurement results page. You’ll know if the page is set to measurement results, because the register 0x20 (CONFIG_RESULT also known as cid_rid) will contain the value 0x10.

If you look at the pink-magenta content, you’ll see most of it contains 36 groups of three bytes (i.e. 24 bits). Those three bytes contain the confidence value (8 bits) and the remainder 16 bits contains the measured distance.

As described earlier, each 8x8 result is really composed of four 4x4 sub-captures that mesh together (see the earlier diagram). The measurement results page is read four times, to obtain all four sub-captures. The four sub-captures are enumerated 0-3, and that is encoded in the two least significant bytes of the RESULT_NUMBER register (0x24).

It was mentioned earlier that the TMF8828 can detect up to two objects, a near one and a far one, within each zone, since there are multiple SPADs per zone, and sometimes an object behind a nearby object may be within the field of view per zone. In the diagram below, the measurements for nearby objects are shown in white. The values in yellow are for optional further objects whenever more than one object is detected.

image

If the CONFIG_RESULT register contains a value of 0x81, then you’re in the Raw Data Histograms page. I have not explored this page yet.

image

The configuration page can be used to set the number of laser pulses that are used to perform the measurements.

image

USB UART Interface

The Pi Pico exposes a USB UART interface. I copied the manufacturer’s Arduino code (I stripped off all Arduino requirements, so that the code can be built for any microcontroller in theory) so that the interface is compatible with the PC software available on the ams OSRAM website. It is possible to explore it using serial console terminal software. Alternatively, check out the Python demo code.

The key thing to note is that when data is being streamed to the PC, it is in the following format:

#Obj,A,R,T,N,S, followed 72 values and then a newline character.

A = I2C address, usually 65

R = result number, of which the two least significant bits indicate the sub-capture

T = temperature in degrees C

N = number of valid results

S = sysTick value (a timestamp)

The next 72 values are actually 36 pairs of zone measurement results, where the first in each pair is the distance in millimetres, and the second in each pair is the confidence value (higher is better) for the measurement. See the earlier Measurement Results Page diagram, to see the zone numbering, and the earlier diagram which shows how to mesh four sets of those results into an 8x8 array.

Here is an example output line as sent to the PC.

#Obj,65,185,33,24,61353057,653,48,643,35,660,77,647,22,32,140,693,80,648,26,675,61,0,0,666,46,652,36,704,80,645,24,620,98,694,43,615,38,698,43,0,0,0,0,0,0,0,0,827,51,1133,59,1039,52,0,0,954,34,0,0,0,0,0,0,0,0,1186,17,1064,16,1090,50,0,0,995,31,0,0

Browser Based App

I decided to use the Web Serial API, to create a simple browser-based application. The advantage of this is that it should be feasible to connect the Pi Pico to a mobile phone or tablet, and not just a PC. This could be useful when trying out the device in different locations, where it could get awkward to bring a larger desktop or laptop. I have not tested with mobile for now. The Web Serial API works with Chrome on desktop machines, but is only supported on certain browsers on some phones. See the earlier video to see the web app in action.

Summary

The TMF8828 is an interesting device. It’s not the easiest part to work with, due to the optical nature of it, since it will ideally require a transparent cover window. It is also down to the developer to code any useful algorithms to utilise the array of measurements. It has a lot of potential however, for those willing to put the time in to explore it further. Hopefully the information in this blog post can partially help with that.

This blog post covered how to connect the TMF8828 to a microcontroller, and how to capture or view measurements using Python and JavaScript applications on the PC. This will be useful for learning how to best make use of the data, before coding a standalone microcontroller application.

For standalone applications, the microcontroller source code on GitHub can be easily modified to not use the USB UART, and pass the data to your own code. GitHub link: https://github.com/shabaz123/TMF8828 

Thanks for reading!

image

  • Sign in to reply

Top Comments

  • shabaz
    shabaz 8 months ago in reply to obones +2
    Hi, I ordered from JLC, and went for the cheapest shipping option. This works out to $10.52 for a quantity of 5, including all taxes and shipping. Qty 5 is the minimum they will produce as far as I can…
  • shabaz
    shabaz 8 months ago +1
    I was thinking about applications. I think I have a genuine use-case; I wish to make a gesture sensor to remove the need for a cord switch in the bathroom (I'm renovating that, so it's the perfect time…
  • shabaz
    shabaz 8 months ago

    I was thinking about applications. I think I have a genuine use-case; I wish to make a gesture sensor to remove the need for a cord switch in the bathroom (I'm renovating that, so it's the perfect time). Maybe swipe up to turn on the lights. I'll definitely need a cover glass! And maybe even warm the glass a little bit (e.g. from the internal electronics naturally running warm) to prevent condensation on the surface.

    It is possible to buy non-contact switches (some look very basic!), but this would still be an interesting challenge, and if it fails, I can always swap out to a ready-made switch.

    The 8x8 array seems great for (relatively speaking) high-res sensing of the environment, but not so great for fast gesture sensing, because of the four separate sub-captures which need to be meshed. Each sub-capture occurs at a slightly different time of course, so you're not getting an 8x8 snapshot at a single precise time, but rather it is blurred in time.

    Reading some more, I decided there's no need for such granularity for the simple gestures I'm thinking of. I'd rather reduce the number of pixels ("zones" in ams OSRAM terminology).

    According to the user docs, it is possible to switch the TMF8828 into a different mode where it behaves like a TMF8820 or TMF8821 device, to run in 3x3 or 4x4 modes (which are single snapshots, with no meshing). I wasn't sure if that meant uploading different firmware into the microcontroller inside the TMF8828 device, but I found the latest firmware files (Intel Hex format), and spent some time converting them into C arrays that the Pi Pico can automatically push to the TMF8828 RAM on startup each time. And initially failed miserably, I thought I had damaged the TMF8828 by using the TMF8820/21 firmware! 

    But, it turned out I had made mistakes in the Intel Hex to C array conversion. Once I fixed that, I ran it, and now I'm in a very fast (I believe 30 Hz) mode with a 3x3 array, which I think will be more than sufficient for the gestures I'm thinking of. The web app now shows the grid very smoothly.

    Next step: try to implement a simple gesture detection in code! I'm going to aim for it to detect a vertical swipe from up to 50 cm away, just for want of a hopefully achievable goal.

    Maybe this could be useful for others too, if they wish to operate some electronics in kitchen/bathroom etc. 

    All ideas/suggestions/cautions welcome, since I'm just brainstorming currently.

    • Cancel
    • Vote Up +1 Vote Down
    • Sign in to reply
    • More
    • Cancel
  • shabaz
    shabaz 8 months ago in reply to DAB

    Hi DAB, 

    Thanks!

    • Cancel
    • Vote Up 0 Vote Down
    • Sign in to reply
    • More
    • Cancel
  • DAB
    DAB 8 months ago

    Very interesting post.

    Well done.

    • Cancel
    • Vote Up 0 Vote Down
    • Sign in to reply
    • More
    • Cancel
  • shabaz
    shabaz 8 months ago in reply to genebren

    I should try that! My JavaScript knowledge is very limited, so I get in and out as quickly as possible and did the simplest thing (grayscale), but I should get back in there and implement a spectrum conversion.

    • Cancel
    • Vote Up 0 Vote Down
    • Sign in to reply
    • More
    • Cancel
  • genebren
    genebren 8 months ago in reply to shabaz

    That is very cool.  This is sort of like the Grid Eye Heat sensor imaging.

    image

    Here intensities of the pixels were mapped into colors.

    • Cancel
    • Vote Up 0 Vote Down
    • Sign in to reply
    • More
    • Cancel
>
element14 Community

element14 is the first online community specifically for engineers. Connect with your peers and get expert answers to your questions.

  • Members
  • Learn
  • Technologies
  • Challenges & Projects
  • Products
  • Store
  • About Us
  • Feedback & Support
  • FAQs
  • Terms of Use
  • Privacy Policy
  • Legal and Copyright Notices
  • Sitemap
  • Cookies

An Avnet Company © 2025 Premier Farnell Limited. All Rights Reserved.

Premier Farnell Ltd, registered in England and Wales (no 00876412), registered office: Farnell House, Forge Lane, Leeds LS12 2NE.

ICP 备案号 10220084.

Follow element14

  • X
  • Facebook
  • linkedin
  • YouTube