A discussion on API latency

John G3WGV

I've recently had some fun introducing diagnostic and statistical analysis into my API code. One of the analyses is command/response latency. What follows is in no way a criticism: it is a discussion of
the latency aspects of the API, notably on tuning.

Definition
API latency is the time between my API interface code sending a command to the API and receiving the response back. The radio and the computer are both wired directly into the same router/switch at 1Gb/s.

Observation
Latency averages around 21ms for frequency changes within band, as would be the case for normal VFO tuning but varies quite widely. I have seen <8ms and >80ms. Although it is possible to send tuning commands more frequently than about once per 20ms there is no value in doing so, as the radio does not respond more quickly.

You can see this clearly in the following time-stamped set of exchanges. The latency value is calculated from the instant that the command is sent to the radio to the instant that the response is received. By comparing with the time stamps, you can see that my application is consistently adding around 3ms.

12:39:36.801 | [TCP>] C377|slice tune 0 18.084950
12:39:36.804 | [<TCP] R376|0|          |Latency: 18ms
12:39:36.822 | [TCP>] C378|slice tune 0 18.084955
12:39:36.825 | [<TCP] R377|0|          |Latency: 21ms
12:39:36.838 | [TCP>] C379|slice tune 0 18.084960
12:39:36.858 | [<TCP] R378|0|          |Latency: 33ms
12:39:36.882 | [TCP>] C380|slice tune 0 18.084965
12:39:36.888 | [<TCP] R379|0|          |Latency: 48ms
12:39:36.904 | [<TCP] R380|0|          |Latency: 16ms
12:39:36.942 | [TCP>] C381|slice tune 0 18.084970
12:39:36.985 | [<TCP] R381|0|          |Latency: 39ms

This has led to the received wisdom that one should send cumulative frequency changes to the radio on about a 20ms time slot basis. I do this in my controller and it works fine.

Discussion
Is an average tuning latency of around 20ms but up to 80ms OK?

When I first started playing with the API I quickly noticed that when tuning across the band reasonably quickly I seemed to miss stuff that I wouldn't have missed with a standard radio. This was on CW, with bandwidths in the 500Hz region. I now realise that the reason might, in part, be latency.

If one tunes at, say, 2kHz per second then with a 20ms time slot that makes each step 40Hz. On the face of it, apart from the tuning sounding "jerky" on CW because 40Hz is quite a big change in audio pitch, this shouldn't of itself result in signals being missed except on very narrow filter settings. If we take the worst case latency, 80ms, then the tuning step becomes 160Hz, which is really rather a lot.

Even so, we shouldn't be missing stuff, so something else must be going on as well. Perhaps it takes some time for the radio to settle after a frequency change?

Questions
The latency is high enough for tuning to sound jerky at anything other than slow tuning rates and the wide variation in latency exacerbates this effect. Can the latency be reduced to improve the "feel" of tuning?

Why is the latency figure so variable?

Apart from latency, is there some other artefact that limits the ability of the radio to hear stuff as it is tuned through?

I emphasise again that this is not a criticism of what we currently have. It is a desire to more fully understand how the API works and how to get the best out of it.

73, John, G3WGV

John G3WGV

Somewhat surprised that this topic hasn't garnered any responses or opinions whatsoever. I'd have thought that API latency would be of great interest to anyone working with the API.

Stu Phillips - K6TU

You don't provide any details of your test configuration;

• What OS, compute power?
• Any other operations using the radio (DAX, DAX IQ etc)
• Resolution of the clock you are using to measure the latency
• What other clients accessing the radio
• What else going on on the client side machine?  If Windows, long DPCs could easily cause the variation you are seeing.

If you use the in-band ping command within the API, what is the latency for its turnaround?

With literally 1000's of hours using the radio and tuning via FlexControl, Mouse Wheel, Maestro etc, although from time to time there have been releases that generate pops/cracks on tuning (and not for a while in my experience), I've never felt the tuning was "jerky".

Human reaction times tend to notice events that are separated from point of action to event by more than 50 or so milli-seconds.  So these timings (for the most part) are on the edge of perception.

Stu K6TU

John G3WGV

Hi Stu, thanks for the comments. To answer your questions:

• OS is W7 Pro on a 4GHz i7 with 16GB RAM, SSDs and 1Gbit/s wired Ethernet direct to the same gigabit switch as the radio. No DAX/DAX IQ. I normally stream meter data on the UDP port but I just turned that off and it made no difference.
  
• Resolution of the clock is 1ms, just because that's the way I wrote the timing thread. It could be made much finer but as I'm dealing with numbers in the 10s of ms there seems little point. I might change the granularity to 100us to see just how quick the pings are (see below).
  
• SSDR was running as well, on a separate PC. I turned that off. It made no difference.
• Nothing else of significance is  running on the PC, which has plenty of processing power, etc. CPU utilisations are negligible.
  

Pings are extremely quick, generally around 1ms or less:

14:37:02.964 | [TCP>] C9239|ping
14:37:02.967 | [<TCP] R9239|0|          | Latency: 0ms
14:37:04.974 | [TCP>] C9240|ping
14:37:04.976 | [<TCP] R9240|0|          | Latency: 1ms
14:37:06.979 | [TCP>] C9241|ping
14:37:06.983 | [<TCP] R9241|0|          | Latency: 1ms
14:37:08.985 | [TCP>] C9242|ping
14:37:08.991 | [<TCP] R9242|0|          | Latency: 1ms
14:37:10.995 | [TCP>] C9243|ping
14:37:11.002 | [<TCP] R9243|0|          | Latency: 0ms

The latency figures are at my API library's interface to the API, from the instant that the command is sent to the instant the response is received back. As you can see, the application, which talks to the radio via my API library and provides, inter alia, the above display, adds a few more ms.

I believe that my measurements are a fair reflection of actual radio/API latency. As to whether 50ms is perceptible or not, I think the best way to demonstrate what I am experiencing is to post an audio recording. I'll try to do that in the next day or so.

It's also worth noting that my tests have been on CW, where small changes in pitch are readily apparent. On SSB this is much less of an issue.

73, John.

PS: Concerned, in hindsight that my measurement methodology might be inadequate, I have spent the afternoon since writing the above reply rewriting the API library code to use the hardware clock directly for latency timing. This gives, on my PC, a clock resolution of 0.293 microseconds. The new code reports latency in ms to two decimal places and confirms that the old code was accurate within its 1ms granularity. I'll post more on the results shortly when I've gathered some more data.

Kevin

Wireshark has some very nice tools for recording timing between client and server. I'm wondering if any of the tools could be applied to your testing. Comparing your app's performance on the wire to SSDR's wire performance might give some insight. This might eliminate your app from the timing chain and give a clearer picture of the server's or switch's latency.

The concerning piece for me is the variance in latency.

Kev

John G3WGV

Hi Kev,

Yes, indeed, Wiresharking is next on my agenda, as a way of finding out what, if any, differences there might be between network traffic timings and those that I'm seeing through my API library.

To be honest, I don't expect much difference. The timing system is hardware driven and is right at the TCP/IP port. The clock starts immediately after the command string is sent to the TCP/IP port and stops as soon as I've recognised the response. The stop point is a little less easy to pin down, as I have to find the command number in the commands sent array, set that command as response received, then stop the timer. The array only has 128 records, so that won't take more than a few microseconds to scan and it will be a consistent delay.

I'm also encouraged by the values I am seeing for ping commands. These are generally very consistently around 700us, with rare excursions down to 500us and up to 1.2ms but never outside that range. That suggests to me that the wide variation I am seeing in tune command response times is not an artefact of my measurement system.

This is still a work in progress - I've come up for air from coding and testing to respond to your message. As is so often the case, forum replies have spurred me on to improve my code and better understand what's going on.

I too would really like to understand the variability in latency.

73, John.