Generating a pan adaptor and waterfall display

Stu Phillips - K6TU

Using the SmartSDR API a client program can instruct the radio to generate a panafall display. Once created, the radio streams data to the client enabling a pan adaptor and waterfall to be displayed.

FlexRadio Systems provides a C# implementation of the SmartSDR API called FlexLib - this is available in source form and is the definitive description of using the raw API over Ethernet.

From time to time questions appear on the community about how to generate the pan adaptor and waterfall display.  These have been answered (in part) by myself and others who have used FlexLib to build their own client or who have implemented interface libraries for other languages/operating systems.

Hopefully the following description of the data streams for both pan adaptor and waterfall will help others get up the learning curve faster.

If you find this description helpful, please "like" it so I get some feedback!
Thanks
Stu K6TU


Data Streams for Pan Adaptor & Waterfall

Creating a panafall for display requires use of the command API to instruct the radio.  You can generate the API commands either by using FlexLib or by sending a command string to the radio over the TCP socket connected to port 4992 on the radio.

Data describing the panafall is streamed to the client via UDP on a port designated by the client and is wrapped in a VITA-49 protocol header.  This stream is decoded directly by FlexLib or some of the other interface libraries available to authors of new clients.

Creating a panafall results in TWO data streams each with a unique stream id.  The stream id for the pan adaptor and waterfall streams are sent to the client as status messages after the radio creates the panafall.

Here are descriptions of the data contained in each stream.


Pan adaptor

Lets say that you have a window in which you want to display a pan adaptor - and that its 1024 pixels wide by 700 deep.  You create a panafall using the command:

display panafall create x=1024 y=700 fps=24 center=14.100 bandwidth=0.2 min_dbm=-130 max_dbm=-50

This tells the radio to create a panafall centered on 14.1 MHz, 0.2 MHz wide, that is updated 24 times a second and with a scale from -130 dBm at the bottom to -40 dBm at the top - and that you want to render this in a window 1024 pixels wide and 700 deep.

The radio will begin to send you updates to display from the origin of 0,0 (top left of the pan adaptor).  The data is sent you to you as an array of 16 bit unsigned integers that has 1024 entries in the array - one for each X value in the display starting from 0.

Each unsigned 16 bit int is the Y offset in the display where the radio has already:

• Run the FFT for the pan adaptor display
• Scaled the output so that a Y value of zero = -40 dBm and a Y value of 699 = -130 dBm

So to draw the pan adaptor you must (here in pseudo code)...

moveToPoint(0, panvalues[0]);    for (i=1; i < 1024; i++) {      drawLineToPoint(i, panValues[i])  }

Its that straight forward.  The radio sends you an array of points to draw - each entry in the array is the Y value (from 0 to 699) of the point to be drawn.  The radio pre-scales the data so that all you have to do is draw the line on the screen every time your get an update from the radio.

Waterfall

This isn't so straight forward...

The radio send you the waterfall information one line at a time - think of the waterfall being a bitmap that is X pixels wide on the screen and occupies some number of lines (Y) depending on how long you want the waterfall to present data into the past.  The waterfall bitmap is drawn so that the line at the top (NOW) is added and all the lines below (the PAST) are scrolled down.

FlexLib sends you this "line" of information as a WaterfallTile that is defined as an object in Util/WaterfallTile.cs with the following properties:

VitaFrequency FirstPixelFreq;  VitaFrequency BinBandwidth;  uint LineDurationMs;  ushort width;  ushort height;  uint Timecode;  uint AutoBlackLevel;  ushort[] Data;  

If you are processing the raw payload in the Waterfall VITA-49 frame, the items with the VitaFrequency type are sent as 64-bit integers in Big End format.  To convert between the raw 64 bit int and a VitaFrequency (which is held as a 64 bit floating point number), you need to scale the integer by 1.048576E12.

Float64 FirstPixelFreq = ConvertInt64BigToHost(value) / 1.048576E12;

FirstPixelFreq is a misleading term - FirstBinFrequency would be a better description.  Here's why...

The last item in this "tile" is an array of data values of length (width * height) laid out as rows of width "bins" end to end.  The current SSDR implementation currently sends a "tile" that always has a height of ONE.

The width value will be GREATER than the width of you panafall.  What the radio is doing is sending you data to enable a waterfall to be displayed for a RANGE of frequencies that is larger than that displayed in the pan adaptor and overlaps it at both ends.

In other words, the Waterfall "bins" start before the first X value of the pan adaptor display (in our case above the first frequency would be 14.000 (center - bandwidth/2)) and continue BEYOND the frequency of the right hand edge of the pan adaptor (14.200).

FirstPixelFrequency tells you the frequency corresponding to "bin" zero in the array of Data (Data[0]).  BinBandwidth tells you the width of each "bin" in MegaHertz (it will always be a fraction).

So we have a series of "bins" set out as:

Data[0] has frequency of FirstPixelFreq  Data[1] has frequency of FirstPixelFreq + BinBandwidth  Data[2] has frequency of FirstPixelFreq + (BinBandwidth * 2)  Data[3] has frequency of FirstPixelFreq + (BinBandwidth * 3)  ...

Each "bin" has a unsigned 16 bit integer (range of 0 - 65535) which represents the magnitude of the signal power that the FFT found in each frequency bin. 

LineDurationMs is what it sounds like - the duration of the line in milliseconds.

Timecode is a sequence number relating this "tile" to the last and is incremented by one for each "tile" sent from the radio.  This allows late tiles that arrive out of order due to network issues to be dropped (in general, its too late to do anything with them).

AutoBlackLevel is essentially the noise threshold of the waterfall - its a value for the Data bin that can be considered as the background color of the waterfall - all values in the Data array that are this value or less can be rendered as the "black" color in the waterfall.  "Black" because depending on the color gradient of the waterfall, its generally the first color in the gradient.  This value is pre-calculated for you by the radio.

IMPORTANT NOTE:  The "bins' in the waterfall "tile" DO NOT CORRESPOND to "pixel" frequencies in the pan adaptor and the bandwidth of a "bin" may be smaller or larger than the offset of adjacent pixels in the pan display.

So to render the waterfall, you need to:

1. Map the pixel offset in the display to a frequency using the pan adaptor settings as reference.  In the above example, pixel zero in the pan adaptor is 14.000 and pixel one is 14.0001953125 etc.
2. Find the nearest "bin" at or below this frequency, interpolate the "bin" values of adjacent bins to determine the appropriate magnitude for this pixel frequency.
3. Map the 16 bit unsigned integer value into an appropriate color space.
4. Set the corresponding X pixel in the bitmap to this color.

This needs to be done for each line in the bitmap which is rendered from the list of tiles received - with each update you draw the top line of the bitmap and scroll all the older lines down by one pixel vertically.

A note on color gradients

Color gradients are not complicated but it helps to have an example.  The color gradient is a mapping algorithm that takes a value (in our case the uint16 of the "bin" value) and maps it into a color range represented by appropriate values of Red, Green and Blue.

This color is used to set the appropriate X pixel in each line of the waterfall bitmap.

This link:

http://www.andrewnoske.com/wiki/Code_-_heatmaps_and_color_gradients

has a good description of how color gradients work and also provides a C++ class implementation that can serve as a template for other languages.

James Whiteway

Thank you Stu! I appreciate the info. James

IW7DMH, Enzo

Thank you very much Stu.
This and other posts like this should be in a sticky area of the community.

73' Enzo
iw7dmh

James Whiteway

Stu,
Thanks again for the pointers. It has really helped. Finally, starting to get some real results.
Not the prettiest panadapter I've seen. But, it is working!
james
WD5GWY

Stu Phillips - K6TU

James,

Glad that you are making progress and happy to help!
Stu K6TU

Mark_WS7M

Hi All, I wanted to add into this wonderful thread from Stu and add what little knowledge I have gleaned in my recent efforts... First, like mode complex instruments, sometimes things just don't work they way you expect. Case in point: I have no been creating pans in my software but I've been subscribing to them and displaying them. When I first got everything setup and started to receive FFT packets from the radio the data was all zeros! It took me a fair amount of time to figure out that this behavior occurs if you have not sent your x/y pixels for the pan. So here is my guess: I go into SmartSDR, setup a pan or two, then exit. These plans still exist but since there is no GUI active somehow they reset to having no size. So then when my software connects and if it doesn't send xpixels, ypixels then the data that is returned is still based on a pan that internally thinks it has no size. The moment I modified my code to get the size of my pan display area and send it to the radio for the pan xpixels and ypixels, magically the FFT packets started to populate with real data. So just one more note for the notebook!

James Whiteway

Mark, that was an issue that I ran into as well. If you do not set the pan.Width and pan.Height properties in the Panadapter.Added event, the return values in the bin will all be zeros. The problem exists in the radio's firmware and Persistence Database. While the database stores and restores, "most" of the data from a previous session, it does not keep the height and width values for the last pan's when closing. This is a known bug in the software. SSDR solves this internally. My guess is they keep a local database (on the user's hard drive) and restores the pans from that. Hopefully, FRS will correct the problem in the radio's firmware some day.
james
WD5GWY

Oh, forgot to mention, nice job on the multiple panadapters!!!

Walt - KZ1F

Why do you say it is a bug? How is the radio suppose to know what UI it is running against, much less the size of the window you are using? Generally the UI, be it Windows or X or Motif or JavaFX tells the graphics system what size window to create. After that is done the user may (or may not) be allowed to resize it. The radio is on the other side of the equation and has no concept about the drawing system or system at all, for that matter. The problem with the radio persisting the size of the UI, or even the type of the ui, let's say at home you have 3 monitors with an aggregate width of 12,000 pixels. You close down your version of ssdr and later, you reconnect, same IP same everything, except you use your 5x7" in Maestro. The window likely will never open and Maestro will be broken as it can not recreate the window you previously had. The radio does not know this so until you reconnect from the system with the 3 monitors, you can't run SSDR. The SSDR running in the radio is a server, it is not the client and doesn't do UIs. You'd have to connect with your 3 monitor system and create a small enough window that Maestro can open. Doing what you suggest would potentially break the user experience.

Mark_WS7M

I'm not completely sure it is a "bug" as I agree with you Walt that when a client disconnects the radio has no idea about the GUI it is servicing. 

I guess the part that was troubling was this:

So I connect in SmartSDR, set up a pan, everything looks good.  I disconnect.

My client connects and asks for the pan data stream.  Because the x/y have not been set the data stream is sent but almost every time it was 50 samples of zeros.

It was just enough data to sort of look real, but it was not.  It would seem to me that if the radio didn't know the x/y size why not send back a vitapacket with zero payload?  Why the 100 bytes?

Lastly, I posted a while back about getting the Wiki up to date.  I was told we can contribute but I have not had time to get an account going.  

This however is exactly the kind of note that would have saved me about an hour had I been able to go to the Wiki/Pan and see a note saying after you access pan, be sure to tell the pan your x/y pixel size for the data stream to be correct.

The other thing about the Wiki is that I'm finding that the radio is not consistent in command structure.

For example to set a property on a pan it appears you send:

display pan set 0x40000000 xpixels=1024 ypixels=768

Logically to set the RF_frequency of a slice I'd think it would be something like:

slice set 0 RF_frequency=7.068

But it is not.  It actually is:

slice tune 0 7.068

While I have no problem with things not being consistent and I've created the same situations at time in my projects, documentation can save hours.

Right now to figure these things out I have to use wireshark which takes time.  I setup SmartSDR, turn on wireshark then make the changes I'm interested in SmartSDR then I have to scan the packets in wireshark.

Personally I'd pay to have an up to date command set and wiki.  IE if FRS made the decision to day that access to the Wiki API and documentation would be $99 a year but it was up to date and correct I'd have my CC out in a second as the time savings would be HUGE on a project like mine.

I have not considered asking others for these discoveries but maybe I should.

So here is my formal request:

Does anybody have a fairly complete set of command syntax to the radio that they would be willing to share?

Thanks in advance

Walt - KZ1F

Let me try that again...

binding panadapter center frequency to slice A

<code>
        slc.getDAO().getFreqProperty().addListener((observable, oldValue, newValue) -> {
            activePan.setCenterFreq((double)newValue);
        });
</code>

part of onPanAdded

<code>
        PanDataReadyEventHandler tempVar1 = (Panadapter panx, short[] data) -> onPanDataReady(panx, data);
        activePan.addDataReadyEventListeners(tempVar1);
        WaterfallDataReadyEventHandler onWaterfallDataReady = (Waterfall waterfall, WaterfallTile tile) -> onWaterfallDataReady(fall, tile);
        fall.addWaterfallDataReadyEventListener(onWaterfallDataReady);
        activePan.setSize(new Size((int) spectrumCanvas.getWidth(), (int) spectrumCanvas.getHeight()));
        fall.setSize(new Size((int) waterfallCanvas.getWidth(), (int) waterfallCanvas.getHeight()));
        fall.setAutoBlackLevelEnable(true);
</code>


There was a third samplet but, in general, I can likely add clarity on the api, so ask away.

Walt - KZ1F

The other thing to remember is to the radio any client connecting on 4992 is JADC (just another dumb client). That currently it can only handle one client gui will change so the radio has NO IDEA the form factor of the client. Also, the waterfall and spectrum can and often will be different sizes.

Walt - KZ1F

In the onPanAdded....the registering of the eventHandler can occur after setting the size so when panadapter data arrives the size is known. It just has to be done by the client.

Walt - KZ1F

Off topic...I am watching the Fernando de Noronba pile up in slow motion and the station answering the report just jumps right out at ya.

Walt - KZ1F

maybe third time is a charm.

enabling click tune
 <code>
waterfallCanvas.setOnMousePressed(me -> {
            log.debug("Mouse clicked at " + me.toString());
            int tmp = (int)me.getX();
            double freqClicked = x2Freq.get(tmp);
            log.info("active slice = " + activeSlice + " pixel x = " + tmp + " freq = " + freqClicked);
            activeSlice.setFreq(freqClicked);
        });
</code>

Walt - KZ1F

this actually dovetails to another conversation about display geometry and what frequency spectrum is represented by a pixel. x2Freq is a map of x coordinate to the freq at that location.

James Whiteway

Here's why I said it is a bug: https://community.flexradio.com/flexradio/topics/add-panadapter?topic-reply-list[settings][filter_by]=all&amp;topic-reply-list[settings][reply_id]=16171410#reply_16171410 Read the last two posts, mainly the one from Eric. He says the issue is a bug. I know the radio does not know which GUI is requesting a panadapter, but, as you have found out, the radio will not presist the panadapter height and width from one session to another. Currently, that is up to the client. Not a big deal. But, it makes taking advantage of the presistance database, impossible to a non-SSDR client. Eric says it's a bug, but not impossible to get around. James WD5GWY

James Whiteway

I should add, his post does not explictly state it's a bug. But, an issue they are looking in to. James

Mark_WS7M

Hi Stu, or anyone...

Does someone have a hex listing of a waterfall UDP packet so I could test my conversion? 

I'm getting strange numbers for the first pix freq and band width.  Ideally if you had a hex byte list and could tell me that this packet has these values for the first pix freq, bw, time code etc that would be very helpful.

Thanks in advance.

Stu Phillips - K6TU

I don't have a hex dump of the packet but the frequencies are scaled:

See the following comment in my original post...

If you are processing the raw payload in the Waterfall VITA-49 frame, the items with the VitaFrequency type are sent as 64-bit integers in Big End format.  To convert between the raw 64 bit int and a VitaFrequency (which is held as a 64 bit floating point number), you need to scale the integer by 1.048576E12.

Float64 FirstPixelFreq = ConvertInt64BigToHost(value) / 1.048576E12;

That should set you straight I believe.
Stu K6TU

Mark_WS7M

Stu. Im doing that but still getting start of 0.0001 and bin is tiny.

Walt - KZ1F

what is your window size. I sent you some code you can use as a template.

Netty does its own RMR.

Mark_WS7M

Ok.. off by one error... sigh... Now I'm getting frequencies I would expect.  Also got bit by the window size problem again.

Walt asked me what my window size was.  I ran the app and looked at the WF packets before the window pan size was set... same issue as I reported before... bunches of zeros. 

Once the window size was set incomes data.  I shouldn't call this an issue, more of a gotcha.

Walt - KZ1F

BUT....if you do not register your pan data received ready event callback until after you set the window size, it just doesn't matter. (Bill Murray...what was that camp counsellor movie...."it just doesn't matter, it just doesn't matter because the other guys will always get the ...")
In that snippet I sent the other day, you can see the order of actions is important.

That all assumes you do segregation (and isolation) of  concerns.

I'm glad you got past it Mark!

Mark_WS7M

Well currently I'm starting up my UDP thread which can start to get various packets before the Window might be ready to receive them AND before the pan might send it's width.

So I just have to spot check the incoming data and ignore it until it is valid I think. 

I think I should have waterfall working tomorrow I hope. I got all of my data structures mapped out.  

My plan is to keep a very fast array of the line data.  I'll have an off screen image that matches the width of the pan and is the correct vertical size.  Upon a new line I will blast the interpolated line data to the off screen image.  When done that get copied to the display.

I prototyped this and there is no flicker just a smooth update.  Now we'll see if thats true with everything else going on.

Walt - KZ1F

One suggestion, before you proceed much further, as this is way easier now than it will be later. separation of concerns. Your IP interrupts should be handled by a separate thread. You absolutely do not want to be doing graphics in the same thread as processing the tcp/ip stack activities. So each packet arriving, particularly UDP and tcp inbound, should be processed by a separate thread. Maintain a list of registered interrupt sinks. Once the interrupt is processed and the relevant data extracted, it can serially drive the registered sinks. Better yet. Have a single thread process the stack interrupts and that's all it does before placing the packet in a FIFO blocking queue. Have a separate thread that consumes that blocking fifo queue and drives all the registered sinks. In this way the system is processing each 'interrupt' as unimpeded as possible, no user level processing in this thread. The thread that is dispatching the registered sinks can afford to process it serially as you can't draw frame two before you process frame 1. Make sense? The reason I suggest doing that now is it is way easier now as opposed to have to retrofit all of that later. The listener for UDP should be different than the listener for the TCP inbound. Don't think in terms of 1 pan..think in terms of the entire system a full load.
FWIW.

It certainly is your design, your project. I merely offer ideas you may yourself think of when it becomes more difficult to implement them.

Mark_WS7M

Got it Walt, I am well verses on the single responsibility principle. My code is organized as follows: Radio object - Handles TCP send/receive. There is not an overwhelming amount of this so currently it is foreground but protected by a mutex encoded queue. So all commands seem to execute instantly but then a queue signals the socket to send the data from the queue. The radio object spawns a UDP thread. This thread handles all incoming datagrams and vectors them to various objects like pans, waterfalls and soon audio etc. Upon receipt of a new packet a after the UDP thread processes and packetizes it a signal is sent up the chain for the foreground display oriented routines to grab the data and paint it. So there is a clear separation. Maybe not perfect but I have things pretty well segregated.