Waterfall and tile data

Walt - KZ1F

I had previously punted on the waterfall as I anticipate that would be the most challenging thing to convert to a graphic. The slice reticle and flag, not a problem...not a big problem.

The difference between what I am doing and what William did, which conceptually seems far easier to model, is I am using the waterfall and waterfall tile data vs taking the pan data with a strength of 0 (max strength) to view height (min strength) and mapping that to a shade of grey or a color from violet to red to white, roygbiv-> vibgyorw.

Here is my data:
Waterfall {streamId=0x42000000, bandwidth=0.016, centerFreq=18.0780006, size=1500,800}

WaterfallTile{FirstPixelFreq=18058301, BinBandwidth=11.0, LineDurationMS=80, _width=3360, _height=1, Timecode=5, AutoBlackLevel=11570, DateTime=Tue Jun 16 10:35:17 EDT 2015}


Waterfall {streamId=0x42000000, bandwidth=0.016, centerFreq=18.0780006, size=1500,800}

WaterfallTile{FirstPixelFreq=18058301, BinBandwidth=11.0, LineDurationMS=80, _width=3360, _height=1, Timecode=6, AutoBlackLevel=11570, DateTime=Tue Jun 16 10:35:17 EDT 2015}


Waterfall {streamId=0x42000000, bandwidth=0.016, centerFreq=18.0780006, size=1500,800}

WaterfallTile{FirstPixelFreq=18068150, BinBandwidth=5.0, LineDurationMS=80, _width=3360, _height=1, Timecode=7, AutoBlackLevel=12070, DateTime=Tue Jun 16 10:35:17 EDT 2015}


Waterfall {streamId=0x42000000, bandwidth=0.016, centerFreq=18.0780006, size=1500,800}

WaterfallTile{FirstPixelFreq=18068150, BinBandwidth=5.0, LineDurationMS=80, _width=3360, _height=1, Timecode=8, AutoBlackLevel=12070, DateTime=Tue Jun 16 10:35:18 EDT 2015}

etcetera

Those are the headers, representative data is:
[ 0]    short    11918  
[ 1]    short    13398  
[ 2]    short    13198  
[ 3]    short    12942  
[ 4]    short    12830  
[ 5]    short    13334  
[ 6]    short    13838  
[ 7]    short    14238  
[ 8]    short    14262  

Again, comparing this to William's approach in a window of 600x200 there are 600 points to plot and the strength goes from 0 (strongest) to 200 (noise floor) dividing that by 7 colors is about 28 strengths per color {0-27 violet, 28-55 blue, 56-83? yellow etc

Question 1) I have no idea what to do with 11918. Unlike the height of the window there is no way, it seems, to know, apriori, where 11918 lies. It is possible that those are bogus numbers but they all seem to be roughly similar if they were intermixed with 0's and numbers in the 32,000 range I'd be concerned.

I did try looking at the numbers being added to the short[]. The complete first packet was empty. Is that reasonable?

Thanks! BTW, this is 1 of 4

Stu Phillips - K6TU

A'boy...  after this answer, please let me know I can improve my post on this topic so that all the information goes in one place.


The data in the pan adaptor and the data in the Waterfall tile are clearly related in so far as they are both derived from a FFT that is yielding signal power in a bin bandwidth.  I don't know the size of the FFT that's run in the radio (and other than for intellectual curiosity, I don't NEED that answer. 

However, the data in the pan is scaled to a particular dBm range (the pan adaptor range of min to max values in dBm) and then allocated a dynamic range so that everything fits in the Y pixel height.

Any value < dbm_min is forced to dbm_min, any value greater than dbm_max is forced to dbm_max.

Moreover, the effective bandwidth of a "pixel" on the X scale is simply the bandwidth of the pan adaptor / number of x pixels.  So if you have a pan with a bandwidth of 0.2 MHz and x pixel size of 1024, then each pixel represents approximately 195 Hz. 

Think of the vertical scaling in terms of pixels/dbm as the dynamic range.

Think of the horizontal scaling in terms MHz/pixel as the resolution.

So the data sent to render the pan display is pre-calculated and you simply draw the result - done.

The Waterfall tile sends you a larger number of pixels than you requested in the pan adaptor.  In fact, you can't request the width tile the radio sends you, you get what you are given as a function of the panafall x_pixels value.

As I said in my post, the WaterfallTile has a wider frequency bandwidth than that you select on the pan adaptor via the bandwidth setting.  This is done so you can scroll the display and (depending on how far/fast you scroll the frequency) likely already have the data needed to seamlessly regenerate the waterfall without smearing or delay - these were design goals for what FRS implemented.

BUT the number of "pixels" (bins), the bandwidth of a "bin" and the dynamic range of the data in the WaterfallTile are completely different than the pan adaptor data.

ASIDE COMMENT: The values you show in your example above have the raw values of the firstPixelFreq and binBandwidth - they need to be scaled - see my post on this topic. 

https://community.flexradio.com/flexradio/topics/generating-a-pan-adaptor-and-waterfall-display

The dynamic range of the data in the WaterfallTile is a 16 bit unsigned integer spread across some dBm range.  Again, I don't know the range but I would hazard a guess that its the dynamic range of the analog/digital converter in the SCU from it minimum discern-able level to the point of overload. FlexRadio doesn't spec this range in the data sheet but it does say that the native resolution of the ADC is 16 bits so I think I have a good guess ;-)

The numbers you get in the WaterfallTile are from 0-65535 and represent a larger dynamic range than the pixels/dBm on the pan adaptor.  You have to map this range to a color - remembering that the color space is 24 bits (R G which is larger again.

The AutoBlackLevel included in the tile tells you what the radio thinks is the value of the black threshold in a color gradient.  So any value less than the AutoBlackLevel should be mapped to the lowest color in the color gradient.

Rendering the waterfall is laid out in steps in the above linked post.

Please take another look at it...
Stu K6TU

PS: If this is helpful, shameless plug... please show "like" love...

Walt - KZ1F

That referenced post is a good one Stu. As you, may, recall, my core library is 100% +/- (it certainly is expected to be 100%) compatible with Flexlib even though it uses a different model, much as you pointed out Objective-C does (which I did not know until you pointed that out). There is the VitaFrequency class which I use a facsimile of that can return the frequency in MHz or just the long getFreqHz. What is printing in the log message is just the way I displayed it. I did recently (since I posted that) changed the format as I figured someone would point that out.  Here is what it looks like now.

WaterfallTile{FirstPixelFreq=18.0682, BinBandwidth=5.0, LineDurationMS=80, _width=3360, _height=1, Timecode=29, AutoBlackLevel=12164, DateTime=Tue Jun 16 16:19:48 EDT 2015}

The pan data comes back via a different event sink but, yes, it does have high and low dbm as well as all the other FlexLib data relevant to the panadapter. Steve had mentioned it would be wider and in the code when Eric was testing he took the width of the panadapter and multiplied it by 1.2 for the width of the waterfall. As of a few weeks ago I understood why Eric did that (Thank you Steve).

The stmt in the aforementioned thread starting with "In other words, the Waterfall "bins" start before " is misleading I think as it isn't clear that the waterfall's first pixel is to the left of the panadapter's first pixel and continues to the right of the end of the panadapter. The continuing beyond the right side of the pan is clear, starting before the beginning is less so. You could maybe change it, oh, can't now as James added a comment so you can't edit it. Anyway, just say the waterfall overlaps on both sides the panadapter range.

The other thing is, and maybe this is a bug as I don't know anyone yet that has done the waterfall. You told me yesterday the iPad app doesn't do them and, as I recall, it is a spinner for freq in addition to hand typing? That was, likely, shortly after you finished it a year or so ago. Ah, my point. I get tiles with 0 as the autoblack followed by the next one with a value. The first tile, except for header info is all zeros and they start life at tc=4 or some such.

So the data is from 0 to  65536, sometimes any value below autoblack can be considered not eligible for painting (easiest way to make it black) but sometimes not as the autoblack is not always populated. In my examples above, they are, in others, they aren't. Are you sure about the autoblack?  Going row to row I would think where black is should be the same. Further, I would also think, where black is would be identical, across rows, not the data, the autoblack.

But, in general, that was a good explanation Stu, thanks!

Stu Phillips - K6TU

The current iPad (K6TU Control) does not render a waterfall but I am working on a second app that does.  K6TU Control allows a finger drag on the frequency display to change the frequency.

I don't know that the first tile being all zeros is a bug.  0 is simply another value to render and regardless of the state of autoBlackLevel should be rendered as the first color in a gradient.

There are indeed changes in the autoBlackLevel between tiles.  If you log the values you will see that they change independent of any parameter changes to the panafall. The percentage change is small - too small to be noticed with most color gradients.

Well, as I'm unable to change the original post, I will have rely on the comment that describes what firstPixelFrequency means.  I think a reasonably close (or at least second) read makes it pretty obvious.

Still waiting for the "like" - it takes time to write these response as you know...

Hopefully others will find the post helpful as they work to understand panafall.

Good luck with your library.  The client itself is a lot more work - regardless of the effort I've spent on the Objective C interface code, the application logic that sits on top is many (MANY) times the effort to write and maintain.

Stu K6TU

Walt - KZ1F

But Stu, I am waiting for the incentive? Like stuff you would know from your day job.
Thank you for the good luck, I was telling William I have a couple of tcp/ip related things to add and it will be done. It depends on what you mean by client, the control surface or the core library. The core library was infinitely 'harder'. As you told someone else a couple of weeks ago (and I will paraphrase) it wasn't rocket science it was just....gruelling. Basically what you did, a from scratch, facsimile of Flexlib, except mine is 100% (best I can determine) compatible, which was a design goal. It will run on anything...any GA OS except iPad..actually it'll run there too after running through a language converter. As I said, I punted on the waterfall as it represented the highest DoD. The UI, this is the fun stuff. Last count, I have something like 120 classes just for the core, itself. When it hits Android, there will be some very exciting things that make less than good value on Linux, Mac, Windows, Raspberry Pi (maybe this would make sense too). Next year comes the AI version.