Xsens MTi-680-DK: Experimenting with performance - Part 2

In the previous blog post I performed several experiments to assess some performance parameters of the MTi-680 from Xsens. In this blog post I will continue with some more experiments with the MTi-680 to see how well it performs. I will be mainly focusing on the magnetometers, as they are one of the trickiest sensors to work with.

Let's get started...

Compass Calibration Consistency:

The purpose of this test is to see how well the compass calibration works. The MTi-680 datasheet specifies the heading accuracy at 0.5 degrees RMS. However, you can imagine that if the error due to compass not being calibrated properly is more than that, then you are not going to hit the 0.5 degrees accuracy. Personally, I have seen errors of >10 degrees on digital compasses due to lack of calibration. You will be surprised how many metallic objects in our typical environment are "magnetically significant" . Most ferrous materials, even when they don't produce a magnetic field, will still bend the magnetic field passing through them, causing issues for magnetometers in their vicinity.

In terms of testing accurately, ideally, if I had a way to determine the heading information with accuracy at-least as good as the MTi-680, then I could have simply compared the readings from MTi-680, after calibration, with the reference. However, I do not have a reference magnetometer with such high accuracy, or a set of surveyed points available. So, I will have to settle for testing repeatability instead of absolute accuracy by using an arbitrarily oriented reference surface (that doesn't move across experiments).

Here is the plan: First, I will establish a reference position away from any ferrous/magnetic body. Then, calibrate the compass using Magnetic Field Mapper calibration tool GUI, and then note the calibrated heading again after bringing to initial location. Then I will repeat performing compass calibrations several times and noting down the calibrated heading readings at the same position, without any change to the sensor environment (magnetically speaking). We can then look at the numbers to see how consistent the calibration is.

While attempting to establish a space without significant magnetic distortions, I found a very interesting feature inside the MTi-680. If you turn on the Magnetic Field outputs, not only does it output the magnetic fields in the x, y and z directions, but it also gives a "MagNorm" output in the MT Manager, which is a root-of-sum-of-squares of the z,y,z magnetometer values. This can be used to assess the MTi's environment for magnetic distortions. More details can be found in the Base article:

https://base.xsens.com/s/article/Using-the-magnetic-norm-to-identify-magnetic-distortions

In short, if no magnetic distortions are there, the value of this Mag Norm parameter will stay more or less constant when you rotate or move the device around. This brought me to a very interesting realization. The place in my home-lab where I was initially planning to do these experiments, and which I had thought was away from any major magnetic fields, was actually quite bad, magnetically speaking. To illustrate this look at the following screenshots of MT manager's "Inertial Data view". The first two graphs are from where I was sitting initially:

The trace to look at is the "Mag Norm" trace in black. Following is just the "Mag Norm" trace (and auto-scaled accordingly) to show the magnitude of changes:

You can see the Mag Norm trace varies from 1.62 to 1.85 (a span of 0.23). Now, at the same traces when I moved to the middle of another room which was mostly empty:

The Mag Norm in this case varies between 0.98 and 1.02 (a span of 0.04). The deviation is ~6 time smaller than before.

So why was this happening? Well, I used the same MTi Device and its live magnetic Norm graph in MT Manager to scan around for the culprit. It didn't take long to realize that I had 2 audio speakers placed about 3 feet from where I was sitting. And while ~3ft may seem like a relatively big distance, the magnetometers inside the MTi-680 seem to be so sensitive that the outputs were getting influenced by the magnets inside the speakers. This also gave me an idea that one could perhaps scan around on the floors/walls of a building to find where ferrous pipes/rebars are passing through them under the surface, using the same (should make for a cool project!). 

Keeping the above in view, for the rest of the experimentation, I left the comforts of my home-lab and moved to the empty room. The things we do for science!!!

On the MTi, I start with default settings before each test (by pressing "Revert" button inside the Device settings window in MT Manager). Then, I switch to the "GeneralMag_RTK" filter, so that the calibrated magnetometer is used for heading once we save each new calibration. I had to close the MT Manager each time before opening the "Magnetic Field Mapper (MFM)" and vice versa.

The calibration process itself was very straight forward and I found the user interface very straightforward and intuitive. Without even reading the documentation, I managed to perform my first calibration. Although later I did read the MFM manual and found it to be a wealth of information! At the end of each calibration attempt, I wrote the calibration to the device. Then I closed MFM, and opened MT manager, power cycled the device and noted down the yaw heading after 1 minute of power-up while the device was placed aligned to my reference surface. The 1 minute wait was for the internal filters to stabilize, something I learnt in my previous experiments. For each attempt, I targeted "painting" the full sphere to my best abilities and tried to gather around 10,000 points (updated live in the GUI).

Screenshot while collecting data. Notice the roundness and offset of the datapoints sphere.

Calibration results summary:

Detailed Calibration results showing the data before and after mapping to a sphere. Notice the change in scale.

After saving the calibration to the device, I closed MFM, power-cycled the device and opened the MT manager to note down the heading output one minute after power up. The aluminum plate in the picture below was used as a reference surface, for consistent heading readings.

And here are my results from all the experimental test runs:

Note that whenever you hit "Revert" (Revert to factory defaults), the module erases the stored magnetic field calibration, so you will need to redo/re-upload the calibration afterwards. You can upload a previously-generated calibration as well. 

Conclusions: There are few things that I have concluded from my experiment above:

1. The calibration results seem to be VERY consistent. Frankly, I didn't expect the reference heading readings to be this much consistent. (Magnetometers are generally not easy things to use or calibrate!). So, the Calibration process seems to work very well!

2. I was surprised to see the difference between the calibrated and uncalibrated magnetometer values. You can see above that the difference can be as big as few tens of degrees! This experiment should REALLY convince anyone how important it is to calibrate the magnetometers in your environment and use case, even if you think there are no magnetic distortions around.

3. The Magnetic Field Mapper software is very simple and easy to use and provides very nice visualizations and derived statistics. I was able to use it without reading the manual for the first time. That said, the manual had tons of information that I didn't know before and so it was a very good read!

4. I also realized while doing these tests that it is actually not too easy to find an environment with no magnetic distortions. Even the walls and floors seemed to cause magnetic distortions, probably due to the embedded iron rebars and wires inside.

Compass Calibration in Magnetically challenging environments

In the previous test I tested the consistency of calibration (to my best abilities) in a magnetically interference-free environment. In the real-world, a lot of the times it isn't possible to find such a location to mount the device/compass. This is where the calibration feature really needs to shine by calibrating out the deviations due to soft/hard iron effects. So, the setup for this experiment will be similar, except that I will add some magnets to the MTi's environment to act as magnetic sources. Note that these sources will move with the MTi device (a.k.a "Soft iron" and "Hard iron" distortion sources).

For this test, I needed to mount the MTi device to something that was ferrous, and I could align the module to it, while it being small enough that I would be able to move it around for magnetic calibration. After looking at various items I had lying around, I finally chose a 30cm metallic measuring scale. This was made of ferrous steel material, which I confirmed by sticking a magnet to it. I also could see the "Mag Norm" value change by ~0.5 when the MTi was brought next to it, which gave me a good indication that this is a decent choice for a soft-iron mount. I fixed the device to it using 5mm wide Kapton tape, while keeping the edges aligned (to my best abilities).

I had also procured two small but very strong rare earth magnets to add to this setup and simulate different "hard iron" distortion situations by sticking these to the metallic scale at different random locations. In this way, the magnets moved WITH the MTi device during calibration and afterwards. In all cases, I made sure (using the Mag Norm live value reported by MT Manager) that I wasn't mounting the magnets so close that the magnetic values should start to clip, while still close enough to give very strong distortion to the MTi's magnetometers. 

The full setup, including the magnets, can be seen below. The magnets are stacked at the 8cm mark. The Aluminum plate serves as orientation reference surface.

Of-course I had to lift the MTi device with the metal scale and rotate/orient it in different directions when performing calibration and return to the reference location to measure reference heading after the calibration.

Below are screenshots from the Magnetic Field Mapper application during the tests.

Calibrating with the device attached to ferrous metal scale:

Calibrating with the device attached to ferrous metal scale AND magnets attached to the scale:

The offset in the data as well as the oval-ness can be very clearly observed. These are due to the soft/hard iron distortions introduced by our setup. In general, soft Iron distortions shift the center of the data away from the origin, while hard iron distortions change the data shape from sphere to more of a rugby shape. You can compare the below images with the previous case when no major distortions were present (first experiment).

Here are the "Advanced results" for the case with magnets attached. Notice the shape of data before and after calibration (top-left).

And finally, here are the results from my tests:

One thing I want to mention is that it was relatively harder to move the device in all the orientations due to size and shape of the metallic scale I had mounted the MTi device on. This may (or may not) have had some detrimental effect on the calibration accuracy.

Conclusions:

1. If we look at the values after calibration, we can see variation of up to 2 degrees between the different cases. This is certainly more than what we got in the previous experiment, when the environment was more magnetically neutral. However, if we compare the calibrated values with the uncalibrated values, then we really start to see what a HUGE difference a good compass calibration makes. The variation in uncalibrated heading values is more than 100 degrees across the different use cases. The calibration seems to be bringing it to within 1-2 degrees of accuracy, which is two-orders-of-magnitude better! I was quite impressed after seeing this comparison. It also might be possible to improve the calibration results further with some more experience (I noticed the post-correction distribution for my data didn't follow the bell curve too well - something that was suggested in the documentation). Also, I did try to cause as much interference with the strong magnets as I could, while still staying outside the clipping range of magnetometers.

2. The test results show very clearly that if we're using the Magnetometers for a North-reference heading output, we must calibrate the Magnetometers using Magnetic Field Mapper (Application or SDK) for decent results!

Heading resilience to magnetic interference: 

Magnetometers have the advantage of giving us a drift-free North reference. However, magnetometers are generally very sensitive devices (which makes sense because the earth's magnetic field is very weak). If you have seen the live output of a magnetometer/compass in a magnetically noisy environment (think of motors, current carrying wires, moving ferrous metallic objects, etc.), you would have noticed how much it jumps around. This would not be a good thing if you're relying on this type of noisy data for your Inertial Navigation System.

The MTi-680 outputs North-referenced outputs if the "GeneralMag" filter variant is used. The MTi-680 solves the issue of noise in compass data due to distortions by fusing this data with the gyro outputs. Gyros, while experiencing the inevitable "drift" issues, are NOT affected by magnetic noise. So, the MTi-680 uses its internal Extended Kalman Filter to fuse the data from the two and produce the best estimate of the real heading orientation. Intuitively, it works like this: If the filter detects that the magnetometer is showing a rapid change in heading, while the gyros show little or no change, then it means the Magnetometers are experiencing magnetic disturbances. And vice versa, if the heading gyro is showing a slow rate of turn while the heading output is staying constant, then it means the gyros have a bias/drift in the output that can be cancelled. The Kalman Filter performs this fusion of data through some clever mathematics. 

I had already noticed the behavior of the MTi in terms of resilience to magnetic interference during my tests for my previous blog post. So, for this time around, I instead decided to make a short video to demonstrate this visually. Here it goes:

Edit media

Dimensions x Small Medium Large Custom

Subject (required) Brief Description Tags (separated by comma) Video visibility in search results Visible Hidden

Parent content

Xsens MTi-680-DK: Experimenting with performance - Part 2

Poster

Upload Preview

Ok

Cancel

In this video I use two small but very strong magnets to simulate an environment/use-case with very strong magnetic distortions. We can see how well the Xsens MTi-680's algorithms perform in terms of keeping the heading output steady. Note that the device is using the "GeneralMag" filter which DOES use the magnetometers to output North-referenced headings in addition to using the gyro data.

NOTE: As we learnt in the previous tests, make sure to calibrate the magnetometers using the Magnetic Field Mapper application/SDK if you're using the Magnetometers.

“Field” Calibration options

In many field-use cases, a laptop with GUI cannot be used for calibration. Rather a calibration system has to be built-into the system. Before I got the device, this was one of the things I was concerned about. My concerns came from some experience in the past with another similar device whereby we had to write our own algorithm to generate the coefficients in-field, which was quite some work.

When I looked at the "Magnetic Calibration Manual", I was pleasantly surprised to find that the MTi has not one but several ways to do calibrations. Refer below to the image from the manual:

In short, the various options are:

1. AHS (Active Heading Stabilization) -> For dynamic use cases when there are lots of magnetic distortions in the environment that don't move with the device. If turned on, this runs continuously in the background and improves on gyro drift, but the output is no longer North-referenced. However, I could not find AHS in the Device settings. (Perhaps it's not relevant to this particular device?)

2. Magnetic Field Mapper Application/SDK -> The best way to do offline/initial calibration, assuming you can connect the device to a 32/64-bit computer. (More on this later)

3. Representative motion: This can be used to do quick in-field calibrations when you can rotate the device in all the expected orientations in the field. Does not require a computer. The command can be sent using the binary interface over UART.

4. In-Run Compass Calibration -> If selected this runs continuously in the background and keeps collecting data and improving the calibration. Xsens documentation recommends doing this in addition to a prior calibration using Magnetic Field Mapper to improve compass accuracy while running. This should cater for any changes in the magnetic environment since the initial calibration was performed.

The above are the more conventional ways offered for calibration. I think they pretty much cover most practical use cases. In addition, you can also generate calibration parameters from a previously dumped calibration raw data file and process it to produce a binary message that can be sent to the device over the binary interface (using UART) to restore the calibration. This way, you can restore the calibration using your connected embedded microcontroller, or maybe keep several different sets of calibration parameters for different situations, and then update the device as per the current situation. All in all, MTi gives one of the most extensive sets of magnetic calibration options that I have seen in a device like this.

Regarding the MFM SDK, I went into the Installation folder to check what all is available. It appears that the core functionality is provides as compiled dll files (probably to protect the Intellectual property). Currently, dll files are only available for "win32" and "x64" platforms. Supported Languages include Matlab, C#, C/C++. I hope binaries for more platforms are made available in future, especially the ARM platforms which are very popular these days.

Xsens has made available example (MFM) SDK projects for Matlab, C++ and C# (last two as visual studio projects), which should make it easier to start and build upon. I tried the C# examples, since I had visual studio already installed and was up and running in no time:

It took me less than 2 minutes to open, compile and run the provided examples without any issues. 5 stars for the hassle-free experience!