Introduction

This post shows the result of some experiments with noise generation. The first stage was covered here, and the circuit is now extended to become a useful test tool.

If you’re a beginner in Engineering, don’t let the graphs below scare you away – the information is not necessary to assemble and learn about programmable logic, nor to make use of the tool. The image here shows some example output.

What is Noise?

In plain English it is a signal which is unwanted. Often it may be random which makes it difficult to remove. There are ways of characterising and measuring noise. Most people will have heard of ‘white noise’ and it means a signal that, when observed in the frequency domain, has a flat spectrum across a useful range. The flat spectrum means that the power spectral density is flat, i.e. there is an equal amount of power is contained for any same-width portion of frequencies of interest, across the band of interest. A flat spectrum is highly desirable for a noise generator test instrument, because it allows you to make measurements easily as will be described later.

The flat spectrum doesn’t tell you everything about the noise signal. It only partially describes the noise.

Another way to look at noise is in the time domain, i.e. a normal oscilloscope view as shown here; you’ll see a mess but it will look to be of a certain amplitude for most of the time, and with larger peaks occasionally. Here is an example:

You can see that for most of the time the noise occupies two divisions-worth of vertical space, but it does extend into the adjacent vertical divisions occasionally. At this point, you could turn on measurement capability on an oscilloscope and gain some very valuable information (again described later). If you plotted the frequency of occurrence for ranges of amplitudes (e.g. number of times that the voltage is between 0.1 and 0.2V, and the number of times the voltage is between 0.2 and 0.3V and so on), you will achieve a distribution plot, and it might be bell-shaped (also known as Gaussian distribution of values, or Normal distribution to mathematicians). Naturally-occurring noise sources tend to have such a shape, but it is not an essential characteristic of noise in general. It could take other shapes.

Gaussian Distribution:

Note that in the frequency domain, a flat spectrum does not imply what the signal will look like in the time domain, and therefore does not imply what the distribution of the signal will look like (this is easy to prove by summing two sine waves, and then changing the phase of one of them).

If we want to simulate a real scenario then it is likely that we will want to have noise that behaves like real world noise, and so a signal with a Gaussian distribution is highly desirable.

What is pseudo-random noise?

Pseudo-random noise is a signal that looks as if it is a random noise signal, but actually repeats after a certain length. It can be achieved by math formula. In other words, it is actually well defined. This might seem a very bad thing compared to ‘real’ noise, but it can be highly advantageous for test applications. One reason is that it becomes possible to get completely repeatable test results that can be retested again and again to achieve the same result. Pseudo-random noise can still have all the benefits of random noise such as keeping the same frequency domain and time domain views mentioned earlier. If you’re worried about it repeating too often, then the length of the sequence can be extremely long, often lasting for many years unless reset.

How can noise be useful?

There are many things to do with noise; here are some examples that you will be able to use the tool for.

1. Simulation

If noise can be created with a test tool, and if it looks very much like real-world noise, then it can be used to test a design that would be awkward to test or prove in the real world. As an example, it would be possible to insert noise in a received signal to see how much noise will cause the circuit to no longer be able to successfully decode the desired signal, before spending any time or money trying it for real. As another example, noise could be used to simulate what a practical sensor will see (e.g. a light sensor) so that you can test the rest of the circuit (and any decision based it), such as a line-following robot’s light sensor to ensure that it behaves well (without crashing the robot by trying it for real).

2. Measuring the bandwidth of a filter

This is interesting; it is possible to determine the bandwidth of a filter using a single measurement, without requiring a spectral analyser. This relies on having a white (flat power spectral density) noise source (so that the calculation is easy!) and a known bandwidth filter. By applying the known bandwidth filter to the noise source, the output rms voltage is measured (but assuming a test instrument is capable of measuring this!).  Then, the filter is replaced with the unknown one and another measurement is taken. The ratio of the two bandwidths is equal to the square of the ratio of the two voltage measurements. So, BWunknown/BWknown = (Vrms_unknown/Vrms_known)^2.

Note that the rms voltage can actually be measured with a modern oscilloscope. A normal multimeter will not be ideal.

3. Confirming RMS meter behaviour

A known noise source would be useful for seeing how well an RMS meter performs, or how well an oscilloscope measures RMS voltages. Therefore it is useful as a calibration device. Incidentally the Tektronix MSO oscilloscope (which was used for the design) was measured to be very good at rms calculation – it was compared with a computer calculation of rms on captured sample data using GNU Octave (see here for information on using Octave). I didn’t record the results but I can rerun and record the detail if anyone is interested.

High level design

The approach taken is very classical and is a design from one of my favorite engineering books – The Art of Electronics. The implementation is almost entirely based on it (but with more modern components), but is different enough to publish as fair use of that design. The same approach has been used in commercial test equipment, although now more modern methods exist.

The design relies on a math formula that is simple to construct in hardware. The design is known as a maximal length linear feedback shift register (LFSR) and it consists of a multi-tapped shift register where two of the terms along the length of the register are XOR’d together and fed back to the input of the shift register. The entire shift register is clocked at a desired speed (32MHz in this example). Other methods are possible, but they consume more hardware (there is limited hardware in the CPLD).

The output is a logic level that seemingly flips at random. It is known as a pseudo-random binary sequence (PRBS) and it can be directly used if this is all that is required.

The CPLD contains the entire shift register (32-bit length in this example) and the feedback XOR’d signal. So, the only external hardware that is needed is a clock signal which will be supplied by a small oscillator. The entire circuit can run from a 3V battery (or use a 3.3V regulator).

The circuit is shown here, all parts are available from Farnell/Newark. U1 is the Digilent CPLD board which is a 40-pin device but only 5 pins are used in this circuit. U2 is the voltage regulator, but the circuit was tested and will happily run from two AA cells without it (although 3.3V is preferred; two AA cells is slightly below the specified voltage). X1 is a surface mount device but is large (7.5 x 5mm) and no harder to solder than square pads on a normal 0.1inch spaced through-hole board. Through-hole 3.3V oscillators are rare, so the surface mount device is used. For experimental purposes the oscillator could be replaced with a 7555 IC (not 555, since we need to run it at 3V) to operate at a slower speed, to view the PRBS more slowly. All this is described in the earlier post.

Next, the circuit is extended. The extended design is also closely based on the Art of Electronics design. You can see it here so that you can compare (image quality reduced to use this fairly). The book is highly recommended – it is full of extremely valuable information.

Clearly the CPLD will replace the logic gate ICs, and this will bring improvements in the maximum bitrate since the CPLD can run as some fairly high speeds.

As well as a digital PRBS output, an analog output is also highly desirable for a noise generator. This is so that we can have a signal that closely represents a naturally-occurring noise source, and therefore a Gaussian distribution of values (rather than a binary distribution) is desired. The PRBS signal can be low-pass filtered to generate an analog output. This is an important part of the design and is described next.

Digital Filtering

As mentioned, the PRBS signal coming out of the LFSR is a pseudo-random signal, but its probability density function is not Gaussian, since it has only two levels (high and low).

It can be converted to Gaussian by filtering it. The easiest non-math way to think of this is to think of sharp edges as containing high frequency content. A square wave has a spectrum that stretches out to infinity. To get a signal with content that is non-binary for any reasonable length of time, the signal needs filtering to remove the high frequency content and thus remove the sharp edges in the time domain.

We require a filter with a cut-off well before the clock rate of the LFSR.  It could be implemented using conventional circuits such as an RC filter. However, this would be difficult to maintain if the clock frequency changed, because then the R and C values need to be changed too. A much smarter way is to perform digital filtering.

If you have a stream of data that you would like to smooth out, then the trick to doing this is known as convolution with an envelope. If you’ve ever done a ‘simple moving average’ then you’ve done convolution with a rectangular envelope. For a simple moving average, you took several points and averaged them, and then looked at a different (overlapping) set of points and repeated the exercise. The envelope was rectangular in that each point was given an equal weight. Plotting the result gives a filtered response.

For a more general digital filter, the same thing occurs but the envelope has a special shape. It turns out that the ideal shape is sin(pi*x)/(pi*x), also known as a ‘sinc’ function. The reason for this envelope shape is because the ideal filter has a rectangular shape in the frequency domain, but the inverse fourier transform of it gives the sin(x)/x shape and convolution of that in the time domain ends up performing the filtering in the frequency domain.

The neat trick is to realize that convolution is very easy when you have access to the signal as it passes through the shift register. It can be convolved (i.e. like a moving average) by summing the signals (it is after all the same as what you would do with a simple moving average).

The sin(x)/x shape can be implemented by having a weight to each term that gets summed.

Also, notice on the y-axis that that some parts of the sin(x)/x shape are negative, and those terms will need to be subtracted. If you look closely at the Art of Electronics design, this is exactly what the authors did. Notice that the shift register outputs go to banks of resistors, which are grouped into two sets. Each resistor provides a different weighting. One set are the values for the positive terms for the convolution and they correspond to the central hump of the sin(pi*x)/(pi*x) curve. The negative terms correspond to a dip on either side of the central hump. Truncation occurs beyond that (the sin(pi*x)/(pi*x) shape extends to infinity).

The circuit is shown here (using the same circuit as before, but now extending it by soldering in a vast quantity of resistors):

The positive terms are summed by using an op-amp; the gain of the inverting amplifier is –Rf/Rin  for each term. The negative terms are summed in a similar manner by a separate op-amp. It is the subject of the next section.

Here you can see the 14 resistors for all the negative terms connected up (all are 0603 sized surface mount resistors):

Here you can see the positive term resistors soldered up. In many places two or three resistors in parallel were needed to get the right resistance:

A table of all resistor values and order codes is attached, also detailing which ones should be paralleled up. They are all fairly unusual values. You will end up with a box of enough resistors to build about 50 pseudo-random sources. Luckily they are all cheap.

Incidentally a quick way of determining how to achieve a desired resistance is to use Wolfram Alpha’s site, e.g. querying for ‘2740 ohm resistance’ gives the result shown below.

Op Amp Stages

The analog portion of the circuit looks like just a couple of inverting amplifiers connected to a buffer amplifier, but it is much more sophisticated (not my design – it is from The Art of Electronics – mine just uses different op-amps and a few modified values).

The first two op-amps are inverting amplifiers, but notice that their non-inverting inputs are tied to exactly half of the FPGA’s logic levels. This then means that both of the outputs sit at this voltage and vary up or down about this voltage. The op-amp is performing multiple functions; a level shift and amplification (gain less than one most of the time) and the resistors all connected together at the inverting input create a summing amplifier but with a reference of half the FPGA logic level. In brief it means that the more pins on the FPGA are at logic level ‘high’ (3.3V), then the lower the output voltage from the op amp will be below 1.65V, and the more pins on the FPGA are low (0V), then the higher the voltage will be above 1.65V.

The third op-amp is a difference amplifier, where the output voltage is equal to the voltage at the non-inverting input minus the voltage at the inverting input. The effect of that stage is to remove the 1.65V level (so that the output sits at 0V), and then inverts the terms that should have been negative and sum it to the other terms. The final output is a noise signal that has zero DC offset.

The actual rms voltage has a formula that is dependent on the ratio between the filter cut-off frequency and the bitrate. Since they vary together in this implementation, it means the rms voltage will remain the same regardless of the clock rate. That will be an interesting test by replacing the oscillator.

Note that the actual output is inverted overall, since the positive terms (through the lower op-amp) went through only one inversion and then into the non-inverting input of the difference amplifier portion. This means that the more of those terms are at +3.3V then the more negative the output goes, and the more terms that are at 0V then the more positive the output goes with regards to the lower signal path. This has no zero impact of course on the desired noise output, and matches the Art of Electronics topology for ease of following both circuits for those who have the book.

Since the circuit is being modernized, the op amps were replaced with high speed ones to allow a far higher noise bandwidth. In theory it should be possible to further extend the bandwidth by many multiples by increasing the oscillator clock rate and the selected op amps should support it.

The op amp stage was created using TI prototyping boards from their eStore. These are very good value and allow high-speed op amp circuits to be created (it would have been hard to do this with a DIL package op amp on prototyping board).

The first stage is shown here. The IC contains dual op amps, and one is used to sum the positive terms (inputs/outputs marked PT) and the other for the negative terms (marked NT). A 3.3V rail is divided down using a potential divider to create the reference voltage for the non-inverting inputs. The op amps are powered from dual rails (I used a DC-DC charge pump so that the entire circuit could operate from a single 9V source).

The gain was measured up to 20MHz and was found to be flat. This was easy to do on the oscilloscope by using the automated measurements capability and feeding in both the input and output signals into the oscilloscope and plotting the measurements while changing input frequency. The image here shows the input and output signals captured with an MSO oscilloscope.

The difference amplifier stage is shown below. A coax connector was used to make it easier to get the signal into the oscilloscope. Copper tape was used to create some pads at the top-left.

The variable resistor is used to remove any small offset (tens of mV) from the output.

In a similar way to the previous stage, the frequency response of the difference amplifier was measured with the oscilloscope too – again by taking automatic measurements and manually plotting them. The graph below shows the measured response in blue. The purple line is sloped at -20dB per decade for comparison purposes.

Since this implementation uses a far higher clock frequency than the original design, some better filtering is required, and an LC filter was used at the output. It is hard to create many-poled filters accurately without a lot of time and a lot of test equipment (and I know my limits :-) so an off-the-shelf 1.5MHz filter was used from Coilcraft (7 pole filter, code P7LP-155L) and as expected from such an item it gave excellent results despite the difficulty of measuring such a sharp response beyond the cut-off. The result would be better with a decent build and measured at low levels with a spectrum analyser. The red line shows the response of the difference amplifier with the 7-pole filter.

(Note that the graph above is a plot from the real measurements (measured with the MSO at different frequencies, stored in an array and plotted with Octave) but with an artificial 3dB shift added  – the true measured values were 3dB lower because the difference amplifier stage had a 50ohm series resistance inserted (marked R5A in the silkscreen in the photo above) and expects the oscilloscope end to have a 50 ohm termination).

Now that the amplifiers are constructed, they can be connected to the rest of the circuit. First, some sockets were soldered on the board:

The small green board on the left is a 3.3V regulator to provide a supply to the CPLD board (in the simple PRBS version of this project the board could have been powered from two AA batteries; now a 3.3V supply is needed.

The other side of the board is shown below. In the photo below, the circuitry to the right is not essential (it is a DC-DC charge pump and 7905 regulator to provide a -5V supply from a single supply; instead of all this, two 9V batteries could be used to power the circuit).

Then, the op amp boards can be plugged in and wired up:

Then, as indicated at the start of the post, analog noise can be observed on an oscilloscope, and the spectrum is white from DC to 1.5MHz.

There are still many measurements and tests to do (much more than I have time!) but I wanted to type up while it was still fresh in my head; there were a couple of tests that were worth doing immediately however to see what the quality of the noise was like. The first was to measure the crest factor, which is the ratio of the peak voltage Vp over the rms voltage. It was done by turning on two measurements on the MSO and performing the division. The behaviour of the MSO for rms accuracy was proved through confirming with Octave as mentioned earlier. Gaussian noise has a probability density function that spreads out forever, but the circuitry has voltage limits. The crest factor was measured to be about 3.78 which is reasonable for many use-cases (and compares favorably with some commercial older noise generators.

This was great, but sometimes visual confirmation is better, and it was worth trying to generate a probability density function to see if it really was Gaussian-like. A million samples were captured using the MSO with Octave, and plotted. The result is shown below.

It can be visually seen that it is Gaussian-like (although I didn’t try to overlay a curve to compare), which was a relief to finally see!

As mentioned, although comparable with previous commercial pseudo random generators, it doesn’t compare with modern waveform generator instruments which have in-built noise generation capability (with selectable profiles) as a subset of their full functionality, but this project could help you get practice, experiment and express your needs for a more complete instrument.

Digital Filter Summing Resistor Reference

The table here has the important bits highlighted. The blue and red are the pin numbers on the Digilent CPLD board, and the yellow values are the resistors that are needed (up to three in parallel for some of the pins). The green cells are the order codes for 1% accuracy 0603 resistors.                                                                                                                                                                                                                 

Final Notes

All graphs were drawn with Octave – well worth trying out.  All measurements and oscilloscope traces were obtained using a Tektronix MSO model 2024B, which made this project possible. If you wish, check out the review here partly if you’re interested in oscilloscopes or a refresh, but mainly just out of interest because if you’ve read this much, then you may be interested to hear about some other techniques to investigate different analog and digital scenarios in the time and frequency domain that I was able to collate as part of the review exercise, and some of it may be of interest.

Thanks to Element14 and Tektronix for enabling this, and I hope this project provides some ideas– personally I think a a noise source is a fascinating tool, and thanks to the original authors of The Art of Electronics it is a design that can be easily created. I’m still learning how to use it and I’m tempted to create a PCB for it.

There may be errors above since this post just seemed to grow more than I could review easily! Apologies if there are any mistakes. If you spot any, let me know, and I'll correct it. Thanks for reading this very long post!

<Parts list now attached below>