TCP/IP commnunications while GPIO interrupt

Hi Sam,

You'll see this issue because while you may be using an ISR in the WiringPi library, your code is still running in user space (as I understand) and thus has to still battle with other processes. I'm not familiar with WiringPi however so I could be wrong. You could run as root user to see if this makes a difference. Linux user apps are not suited for such responsiveness where jitter is a concern - you will need to either do this in a custom driver, or use an external microcontroller perhaps.

You are also not "double buffering". So, when your 1600 sampes are done, the main process starts to transmit the buffer to the server, while the ISR is starting to refill the buffer with new samples...

When you have interrupts at 16kHz, the interrupt comes every 60 microseconds. This is quite fast. A microcontroller can, with a little care probably achieve this performance. But if you want to do this on a normal Linux system you have to make sure that no system device driver disables interrupts for a period like this. From your graph it could be that the interrupts are disabled for a period like 500 microseconds, meaning you are losing around ten samples.

I looked up how the GPIO interrupts work. The interrupt fires, and then tries to schedule your function to run in userspace. If the system decides that doing "something else" is preferable above running your program then that will happen. It will run your function within miliseconds, but sometimes something else will have to be done first. In your case, you will definitvely miss interrupts if you schedule them to happen every 60 microseconds.

To solve this you could write a kernel driver that handles the ISR. Or you could use a DMA buffer to store the samples (is the 16bit ADC part of the BCM2835?)

shabaz and @rew

are both right. In computer parlay, we call this "two ships passing in the night". With the interrupts coming in at 16 Hz and being higher priority than the TCP/IP task(s) this can be trouble. Keep in mind that TCP/IP is layered and heavy in code. It will under the covers break up the stream of bits into packets of around 1500 bytes on average. So if you look at the gaps, I can imagine they are about this much in length. Another issue is reliability since IP level is not reliable. If the transmits get garbaged, a retransmit occurs. You are at the mercy of these things.

  So what can you do? Double buffering would be great. Or barring that, start your code with a memory copy from receiving buffer to a separate transmit buffer before sending. This would have a better chance of being done without an overwrite (99 44/100%).Could leave a flag set for the interrupt routine to know the sending buffer is not yet flushed to log how many times this occurs. See this, maybe play with buffer size to optimize the code.

Clem

Thank you Clem !!

First, I want to tell you that I am developing sample acquisition using

external A/D chips and transmission of the converted samples through the Internet.

But, I already applied the "double buffering" scheme you have told.

That is, when the number of  ADC samples are 1600, the buffer is

copied to another buffer for TCP/IP transmission.

Just now, I modified my source codes to use UDP instead of TCP/IP.

Although the sample missing is reduced, it does not disappear perfectly.

In my opinion, one strong candidate solution of my problem is to use DMA

for SPI communicatrions.

Could you tell me where I can get the related source codes ??

Your current "architecture" has userspace code running in places that are VERY time critical.

Just because I say that "using DMA" may solve your problem. doesn't mean that your problems go away if you still have the userspace triggeriung the SPI transfer, and then having to store the results.

As long as you're storing the results in userspace you're going to miss samples.

To make this work perfectly you should set a timer to go off at 16 kHz,  start the SPI transfer in the interrupt routine (i.e. in kernel space) and of course use DMA for input and output of the SPI.

But... If you are sampling AUDIO at a rate, between "good for voice" and "good for music", why not use an AUDIO ADC and connect it to an I2S port? I2S is sufficiently like SPI that the module in the chip is usually combined. Maybe the SPI module of the BCM2835 can do I2S?

Before I start I should point out that I'm not that familiar with RPI or Linux but I have done a lot of data acquisition.

Any full size non-real time operating system (like Windows or Linux) will have a variable response time to interrupts. It may be pretty good on average but there can be large variations. It will also sometimes block a user task for what may be a significant time (in Windows 200mS is possible).

In your application I think you have  a new sample from the ADC every 62.5uS and very little, if any, buffering on the ADC side of the SPI interface. So your system will break every time the RPI takes longer than 62.5uS to service your interrupt and that is likely to be often.

If you use DMA correctly it can get round your problem. You will need to set it up to put the ADC samples into a circular buffer and periodically read as many samples as are available from the buffer. You should check that the buffer has not overflowed. The buffer must be long enough that it won't overflow if your periodically called function is delayed by the maximum OS jitter. (which you'll need to find by experiment.)

What you can't do is let the DMA fill up a linear buffer and interrupt the processor because any delay in servicing that interrupt will cause problems.

I have no idea how easy it is to set up and control a circular buffer with DMA in this way on an RPi.

The other way to do things is to buffer the data on the ADC side of the SPI interface - this will need more hardware but will keep your RPi software simple.

Using UDP is fine but don't forget that there is no guarantee that the receiving node will see your message. Any robust system will need some kind of handshaking to cause missing message to be transmitted again. TCP/IP does all this for you automatically but a good UDP based system can perform better (well, faster anyway).

MK

@Michael, the problem is that with his current setup he really needs a software interrupt to trigger the SPI transfer. Run that in kernel-space, and you have a chance of getting it working. You have to be careful, and willing to take on any place that disables interrupts for a long time.

When I built a system, with real-time-requirements and running full-blown Linux, in 1996, we had to swap out the IDE disk for a SCSI one because the IDE driver was disabling interrupts for too long. On the raspberry pi, you have only two drivers that I suspect might turn off interrupts long enough that samples might be missed: USB and SDcard. The problem is you can't do without those, so an easy swap won't be possible.... (in my case in 1996 the cost of hardware was insignificant. The machine cost about 2M, the plan was to build "one, maybe more". So far six have been built.)

Just to clarify - are you thinking that this would go fast enough even without using DMA ?

MK

DMA is NOT the issue. The issue is that the CPU needs to be involved to start an SPI transfer.

Suppose the SPI clock can run at 30MHz, you send a byte in under 300ns, so transferring 3 bytes (command byte, 16 bit result/answer) takes 800ns. If you manage to do that inside the interrupt, all is fine, even without DMA. As long as you get that interrupt to run each and every 60 microseconds.

However, the current implementation probably uses the userspace SPI driver. That automatically uses DMA when the transfer is long enough (which it probably isn't). But the SPI transfers cannot be started by hardware, so even if you use DMA, you still need software involvement to start that DMA transfer. And missing just one of the "trigger" moments is "fatal". So "saying you need to use DMA" is not solving the problem. What is needed is "enough hardware that the hardware can DMA the samples into a buffer. This is usually possible for say an "internal ADC", but not for an external SPI ADC.

The Beaglebone black has IO processors. Those you can program to do things like this: Every XX microseconds send YYZZ over SPI to the ADC, extract the right bits out of the result and make those available for DMA. Those io processors are built to be hard-real-time.

The STM32 has loads of timers. They can be cascaded and put into very interesting configurations. There it might be possible for two timers to be put into a configuration that one triggers every 60 microseconds, and then the other triggers say 3 times to provide a "trigger" for the DMA to transfer a byte to the SPI module. The SPI module would be in "starvation" mode: it can't transfer any data because it doesn't have any data to transfer.

On the other hand, maybe sending an SPI byte every 20 us will make the ADC do what you want: convert every 60 microseconds and report the results... That should be possible with most SPI modules, even the one in the raspberry pi. This depends on the ADC: Does it need a deselect/select phase on its slave select pin.

Interesting just how much at cross purposes we are here !

I'm assuming a  scheme where you set up and start the DMA once to operate with a circular buffer (easy on an STM32xxx - is it possible on an RPI ?).

The RPi SPI needs to work as the slave and the external ADC needs to do it's own timing (essential anyway because of sample time jitter issues if not (at 62.5 uS sample interval for even 8 bit performance you should be looking at less than 600nS sample time jitter) ).

With this scheme the RPi is never interrupted at all so it's interrupt latency doesn't matter.

The ADC just clocks the data in when it's ready, the DMA sticks into a buffer and the user code checks the buffer from time to time (not critical times).

Some DMA systems support alternating buffers as well as circular buffers.

It would help if the OP could post his ADC set up.

I wonder if he could use an STM32xxx as the ADC - it could then do the timing and buffering for him

MK