API Primer - a work in progress

Doug - K3TZR

As John (G3WGV) mentions in his API Primer, once you get started working on any sort of Flex Radio Client, you'll need to be able to explore the FlexAPI. A good way to do that is, as John also suggests, to build a small program to experiment with API comments. I've taken his advice and built such a program for use on a Mac. If anyone is interested, I can put the source code on Dropbox and/or GitHub. With a little work, I can also create a DMG.

Here's a snapshot of the program taken just after I pressed the connect button showing the replies from the Radio as a result of establishing a TCP connection to it.



and here after sending it an "info" command (sent commands in green, replies in red):

John G3WGV

Hi Doug,

That's an excellent piece of work, well done!

I have been working on a very similar program for Windows and am also willing to make both the source code and the executable available if people are interested. The code is written in Delphi XE7, using the INDY 10.6 internet component suite.

Underlying the test harness application is my Flex API helper code, which is now nearly fully developed. This code provides broadly the same "middleware" capabilities as the FlexLib API but in a Delphi XE environment. This code is also available to anyone who might want it.

Here's a screen shot just after connection & slice subscription.

Doug - K3TZR

Hi John,

Thanks for the reply. I've put my source code on GitHub

(https://github.com/DougPA/FlexAPITester/tree/master/FlexAPITester)

I can supply an executable as well. It's written in Swift 3 using the GCDAsyncSocket (Objective-C) library. 

Now we need to put theses things to use and enhance your already very good API Primer. I've been tinkering with this for some time now and the lack of information about commands, their usage and their timing has made things difficult.

Like you, I've written code to perform the same functionality as FlexAPI. In my case because I couldn't run FlexAPI on a Mac. I'm in the process of testing a Mac client (i.e. a Mac version of SmartSDR). It's far from complete but so far I'm only trying to confirm that my FlexAPI implementation is complete and capable of supporting a client. I'll put the API code on GitHub as soon as I've tested it adequately.

Some of the code for the API Tester was "borrowed" from my FlexAPI implementation so its part of testing my work. I've implemented all of the Discovery, TCP and UDP aspects of the FlexAPI but didn't choose to put Discovery and UDP into the Tester, maybe the next version.

As I learn things I'll send you information that you may want to incorporate into your API Primer.

Thanks for starting this.

73's Doug

Doug - K3TZR

Kevin & John,

I've taken Kevin's list and my own very similar list and tried to understand the sequence and response from the Radio for these commands. I'm continuing to work on it (there are a lot of commands!). I've used my API Tester app (see posting below) to investigate all of this.

Here's a link to dropbox folder containing a spreadsheet (Excel). It has three sheets, the first shows the commands and the responses they generate. I've tried to "name" the responses and you can see the "named" responses listed on the second sheet. The third sheet is a list of all of the Error Codes I've seen so far and an attempt at an explanation for each one.

There are a lot more commands to be added but I'd appreciate any comments. I'm working on a full-fledged GUI client and understanding the sequence/response of the commands is critical to make it work.

There seems to be a difference in the responses to a GUI vs a non-GUI client. I'm not sure I fully understand that at this point. This list, so far, is from the perspective of my non-GUI client (running on my Mac) while the GUI client (SmartSDR) is running on my Windows computer.

https://www.dropbox.com/sh/vyl91c7f85potnt/AACWSiEy_M4XDuvVFaG705Yza?dl=0

Kevin

Very nice work. I'm still just learning C# on Visual Studio so have not gotten as far as creating my own API tester. Is calling the API from different languages that much different? Trying to catch up!

Will this information be posted to the wiki?

73,
Kev K4VD

Doug - K3TZR

Kevin,

I only have a reading knowledge of C#, I know enough to look through the FlexAPI code and understand roughly what's happening. Learning any language / development environment is a lengthy process. I've been focused on the Mac (Xcode and Swift).

I think the concepts are identical but of course the devil is in the details. In my case (the source is on GitHub) I used a well know open source library (on the Mac) to do the heavy lifting for TCP and UDP handling. If you look at my code there are only  a few dozens of lines of code that I had to write.

Everything I'll publish is certainly free to be used in any way. I'm not exactly sure how something gets included in the Wiki.

Doug - K3TZR

Here's an update to my Flex API Commands spreadsheet. I've learned a lot in the last few days and the spreadsheet has changed considerably. If you have the previous one, throw it away and use this one. I'll update it as I learn more. The name will include the date of last update so it should be easy to see if there's anything new.

Be sure to look at the first sheet in the spreadsheet. It's a short explanation of all the other sheets.

https://www.dropbox.com/sh/vyl91c7f85potnt/AACWSiEy_M4XDuvVFaG705Yza?dl=0

I've also updated/corrected the FlexAPITester and put the updated source on GitHub.

MikeBurns

Hi Doug,

Nice work on the API. I'm developing an interface in Mathematica, which is not ideal for real-time ethernet processing (but is wonderful for processing data). Your spreadsheet has been invaluable in this process (as is John's Primer). I've been exploring the API using Terminal on my mac but that is a tedious process.

It would be great if you could put up the executable of your swift program. Its been a long time since I've used Xcode and I've only been lightly following swift (too many languages in the world!) so it would take me a while to get it going from your GitHub source. If its not too much trouble to put up an executable that would be much appreciated.

Thanks,
Mike - AB1LD

Doug - K3TZR

Mike, I just put an executable into the Dropbox folder. I'm not very familiar with the process so please let me know if it works (it should). 73's Doug

Kevin

John:

In the API document you show the following table for scaling meter readings:


I captured SWR, forward and reflected power via the API. It looks like SWR has no scaling. For forward and reflected power I had to use this formula found in the Gary's fqapsac code that seems to return the correct result:

double myData = Math.Pow(10.0, (data / 10)) / 1000;

With RF Power set to 5 I got the following readings:

forward power: 36.45313 translates to 4.4 watts
reflected power: 18.70313 translates to 0.074 watts
SWR: 1.210938 calculated from above it's 1.3:1

Could the reason for the difference in scaling be where we are getting our data from? I'm using flexlib which might be doing some scaling already. I need to check that.

I finally figured out how to discover, connect and collect some data with the help of Mark WS7M and his videos. The conversion above came from Gary Robinson's fqapsac code along with a whole buch of other neat stuff. After a night's work I have a little hello world app that presents froward and reflected power and swr. Radio was set to 36 on the RF Power slider.



There's definitely a whole lot more to this stuff than I expected. I think I learned 12 new terms today and only had to unlearn 5 I thought I knew. Not a bad day.

73,
Kev K4VD

Jay Nation

Kev
Ok, you've leapfrogged right past me. Thanks for the hints about where to look next. You can expect email. I need a good day, too!

  SDRgadgets

#FlexRadio IRC chat

  73, Jay - NO5J

John G3WGV

Hi Kev,

Yes, FlexLib does the scaling for you, see Meter.cs around line 152:

        public void UpdateValue(short new_raw_value)
        {
            _raw_value = new_raw_value;

            switch (_units)
            {
                case MeterUnits.Dbm:
                case MeterUnits.Dbfs:
                case MeterUnits.SWR:
                    _value = _raw_value / 128.0f;
                    break;
                case MeterUnits.Volts:
                case MeterUnits.Amps:
                    _value = _raw_value / 1024.0f;
                    break;
                case MeterUnits.Degrees:
                    _value = _raw_value / 64.0f;
                    break;
            }

            OnDataReady(this, _value);
        }

If the raw data had no scaling then you could only represent integer numbers. I guess the scaling was set to maximise the accuracy that could be obtained with the 16 bit signed integer.

There is also a reference to this in the Wiki. I notice that the page has been changed recently to show the volts/amps scaling factor as 256 but as far as I can see that is wrong. I am investigating. As a general principle, FlexLib is the definitive documentation for the API.

73, John, G3WGV

Kevin

Excellent John... I was planning to look into the library today to try and find this. Thanks for saving me the time.

So, just to put things into perspective for me, a lot of your document discusses the inner workings of flexlib and maybe even how to do things directly while the approach I'm taking actually uses flexlib to do all the heavy lifting. Does this sound right?

For example, in your document you provide some details on discovery, connection, streaming data, subscriptions, etc.

But, if using flexlib, this all boils down to setting up some event handlers and doing an API.Init(). From there, the radio is basically discovered when it becomes available on the network. Then a Connect() seems to take care of all the details by doing the work of connection, subscriptions and capturing streamed data.

Full disclosure... this is my first time doing anything in a programming language beyond "hello world" so my understanding may still be way off.

Kev

John G3WGV

Hi Kev,

Not quite. My primer is on the radio API itself. FlexLib is a structured interface between the API and an application. It handles many of the details of the API, on behalf of the application, making the application much simpler to code. Doing the scaling to correctly represent meter values as a floating point value is but one example. The API.Init() and Connect() functions you mention are similar examples - there is quite a lot involved at an API level in that Connect() function!

For no other reason than that I wanted to, I have, in effect, written my own FlexLib. That meant I had to grapple with the raw API and it was from that epic struggle that my API primer emerged :-)

Good luck with your programming. It can be extremely rewarding and outrageously frustrating in equal measure but there is something very satisfying in using code that you wrote to make QSOs. It's another sort of construction project really.

73, John.

Kevin

Gotcha. I'm starting to see how the pieces fall together.

And I agree, it seems programming is a new aspect of homebrewing for me. I realize I'm having a good time after something finally works. Until then, I'm pulling my hair out trying to figure out if I'm misunderstanding the API, the library, the code, the language or the development environment! I've gotten a lot of help from the community though so I'm in real good shape I think.

Kev

John G3WGV

Earlier I wrote:

I notice that the page has been changed recently to show the volts/amps scaling factor as 256 but as far as I can see that is wrong. I am investigating.

Word from the coalface is that the meter scaling will be changing in a future release. At the moment it is still as per my API primer.

This must be one of those rare occasions where the documentation gets ahead of the code!

MikeBurns

Doug,

Thank you. I got it up and running with no problems (after updating Xcode and getting the correct IP address.)

This will make things easier.

Mike - AB1LD

Doug - K3TZR

Mike,

There is a new FlexAPITester on GitHub and a new executable in the DropBox folder. This version fixes some bugs and adds the ability to listen to Flex Discovery broadcasts (See the ReadMe on GitHub).

There's also a new app at both location called FlexBroadcaster, it generates Discovery broadcasts to simulate having multiple radios.

73's Doug

MikeBurns

Thanks Doug, I'll check it out. I don't know if the new version fixes it or not, but the old FlexAPITester doesn't like DaxIQ packets - says they are invalid. I'm struggling now trying to decode those, after grabbing them with some python.

Mike

Doug - K3TZR

That's interesting. The code doesn't really do anything with UDP packets. It receives them and restarts a timer. If the timer expires, the little green button on the lower right should turn red. 

It must be that it doesn't properly decode the the Vita packet. Here's the applicable code:

        if let vitaPacket = _vita.decode(packet: data) {

            // restart the timer

            _streamTimer.scheduleRepeating(deadline: DispatchTime.now(), interval: .seconds(1), leeway: .milliseconds(100))      // Every second +/- 10%

            // set the event handler

            _streamTimer.setEventHandler { [ unowned self] in                

                // timer fired, UDP stream timed out

                self._delegate.udpStream(active: false)

            }            

            // TODO: Packet statistics - received, dropped            

            switch vitaPacket.type {                

            case .ifData, .ifDataWithStream, .extData, .ifContext, .extContext:

                _delegate.udpError("Unexpected packetType - (vitaPacket.type.rawValue)")

                case .extDataWithStream:

                

                // Stream of data - figure out what type and call the dispatcher

                switch (vitaPacket.classCode) {                    

                case .daxAudio:

                    // FIXME: Class Code not in use

                    break                    

                case .daxIq24:

                    // FIXME: Class Code not in use

                    break                    

                case .daxIq48:

                    // FIXME: Class Code not in use

                    break                    

                case .daxIq96:

                    // FIXME: Class Code not in use

                    break                    

                case .daxIq192:

                    // FIXME: Class Code not in use

                    break                    

                case .discovery:

                    // FIXME: Class Code not in use

                    break                    

                case .meter:

                    // pass the data to the Radio's meter stream handler (there is only one)

                    //                    _radio?.meterStreamHandler(vitaPacket)

                    break                    

                case .opus:

                    // pass the data to the Opus stream handler (there is only one)

                    //                    _radio?.opus.streamHandler(vitaPacket)

                    break                    

                case .panadapter:

                    // pass the data to the specified (by Id) Panafall's stream handler

                    //                    _radio?.panafalls[vitaPacket.streamId]?.streamHandler(vitaPacket)

                    break                    

                case .waterfall:

                    // pass the data to the specified (by Id) Waterfall's stream handler

                    //                    _radio?.waterfalls[vitaPacket.streamId]?.streamHandler(vitaPacket)

                    break

                }

            }            

        } else {

            _delegate.udpError("Invalid packet received")

        }

I think you're seeing the message at the bottom (the last else) which means the decoder doesn't think the packets are valid (the decoder failed to decode them). I'm not using DaxIQ so may have missed something. Send me a sample if you can (douglas.adams@me.com), I'll try to fix it.

If you look at the Vita.swift file on GitHub, that's where I decode the packets. it might help you with your Python or you might see my mistake.

Doug - K3TZR

Mike,

Do you have a layout for the Vita packets containing DaxIq? I haven't been able to find one.

Mark_WS7M

The API Primer is wonderful.  It covers much of what I had to learn by experimentation.  I will try if you are willing to add to it where I can.  I'll send you snippets of stuff I've learned.

To all here who are programming I'm willing to help in anyway I can.  I have helped a few on here solve issues with programs they were writing.  It helps me to help you.

I'm a device programmer by trade.  Been making PC's talk to things and control them for over 30 years.  Been a ham since roughly 1972.  This radio and the ability to program it brings two things together for me so if I can help spread the love I will.

Mark - WS7M

Kevin

To all that are interested, Mark has given me enough offline guidance where I've become much more comfortable both with the radio's API and with the programming environment I'm using. I basically started with no clue and learning about for loops and variables from a book to making a connection to the radio, reading meters and presenting them in a window after viewing Mark's two videos and getting some help by email. I still have a lot of questions but I'm slowly working through them.

Thanks Mark.

73,
Kev K4VD

MikeBurns

I'll take a look at it, though it may be this weekend before I get to it.

For the IQ format, I look at the flexlib routines (though I'm not very familiar with C# so its a laborious process). My python program does the header decoding and separates out the payload. The IQ packets go through this process ok - its common to all the vita streams. Its the interpretation of the 32 bit data words that I'm working on. Currently I dump everything in the UDP streams to a .json file that I then use for analysis. I'll send you one of the json files (or maybe just a sample considering email size limits) if that helps any. I can also send you wireshark captures if you'd like, but I presume you can get those yourself.

I can grab the IQ packets OK, and decode the header data, etc. The payload data is specified to be in 32 bit IEEE float format but, reading from wireshark captures, the float data doesn't really make sense. I swapped the endian-ness in the python float conversion and the data then makes more sense - like having both positive and negative values and with a physical range of values (though more restricted than I expect), but still doesn't look like I expect a waveform to look. There's also a DC offset to one of the two channels. Not sure what the issue is - since I'm not totally sure I even have the daxIQ stream set up properly.

 I'll continue to stumble around. Let me know if there is anything more I can do to help you out. The python UDP decoding was actually written by a colleague and he has a GitHub entry for it. I’ll check with him and if its OK post the link for others to use. I’ve made some of my own modifications to it and if I figure out GitHub I could post those also. His code actually follows the organization of the flexlib code pretty closely, and I suspect the same would be true in swift, so it might not be very hard to get it working if you want to.

I just learned that Microsoft now has a Visual Studio for the Mac that does .NET and C#. I downloaded that last night and can actually get the flexlib code into it, though haven't got to the actual running of it yet. If I can get enough of it going I can compare my decoding to the gold standard. It turns out Mathematica (the main language I program in) has a very good .NET interface, so maybe I can go that route instead of the python capture. Too little time!

Doug - K3TZR

Mike,

I'm assuming that the DaxIQ data is in a Vita packet with a packet.type of:

        extDataWithStream = 0x03

 and a class.code of:

        daxIq24 = 0x00e3  or

        daxIq48 = 0x00e4  or

        daxIq96 = 0x00e5  or

        daxIq192 = 0x00e6

I that isn't true, that could explain why I can't decode them.

Here's a screen capture from a copy of the Vita standard:

Again, if this isn't correct, that explains it. There is also a lot of manipulation of the fields regarding "endian-ness". I think it's pretty clear in my Vita.swift file where the endianness is being adjusted. One or more of those may be incorrect.

Here's an example of the manipulation of the data in the "payload" of a Panadapter stream:

        // get a pointer to the data in the payload

        if let binsPtr = vitaPacket.payload?.advanced(by: kByteOffsetToBins).bindMemory(to: UInt16.self, capacity: dataFrame.numberOfBins) {            

            // Swap the byte ordering of the data & place it in the dataFrame bins

            for i in 0..<dataFrame.numberOfBins {

                dataFrame.bins[i] = CFSwapInt16BigToHost( binsPtr.advanced(by: i).pointee )

            }

        }

It's in Swift but I think the intent is fairly clear.The data is Unsigned 16 bit integers in Big Endian form. I would expect the DaxIQ frames to look similar(?). Since the IQ data may be +/- I'm guessing that it needs to be divided by some constant (the constant being the zero point).

I also looked at Xamarin (now part of Visual Studio). I asked Flex why they didn't consider using it. At the time, a year or two ago, there was a very large licensing fee for businesses using it but that may not apply to people like us (i.e. non-commercial usage) or maybe it has changed now that Microsoft bought the company.

Doug - K3TZR

Mike,

I just tried to setup DaxIq to the FlexAPITester. For some reason I'm not seeing the packets (and the error you describe). Can you tell me exactly the steps you go through to get to the point where you see the error? If you can and if I can recreate I may learn something and be able to fix the error.

Thanks,

Doug

MikeBurns

Hi Doug,

The exact commands I send to start the IQ stream in FlexAPITester after connecting are:

sub pan all
display panafall create x=1024 y=700
display panafall set 0x40000000 center=1.00 autocenter=0 bandwidth=0.1
sub daxiq all
stream create daxiq=1
dax iq set 1 pan=0x40000000 rate=24000

and immediately start getting the error:
---- UDP Error, Invalid packet received ---

I don't have a deep understanding of the details of the commands above, and they might not all be necessary or even correct, but it does start an IQ stream that I can capture with the python program and at least superficially has the right information in it.

I'll take a look at the swift code, my python code and the flexapi code and see if I can reconcile them, but that will take a little while.

Mike

MikeBurns

Doug,

The class IDs that I check for are 0x02e3, 0x02e4, 0x02e5 and 0x02e6.
Its always possible that you may have picked off the "2" in how you access the data. The python program I use does some fancy bit selection when its checking packet information.

Mike

Doug - K3TZR

You're absolutely correct, I'm looking at it now and it seems that I'm expecting 0x00e3 but getting 0x02e3 for the packet class code for daxiq24. Do you know which is correct. Sometime (in the murky past) I put together the enum for packet class code and have the value 0x00e3. Here's a list of the enum:

        case meter = 0x8002

        case panadapter = 0x8003

        case waterfall = 0x8004

        case opus = 0x8005

        case daxIq24 = 0x00e3

        case daxIq48 = 0x00e4

        case daxIq96 = 0x00e5

        case daxIq192 = 0x00e6

        case daxAudio = 0x03e3

        case discovery = 0xffff

I know that meter, panadapter, waterfall and opus are correct because I have them all working.

Doug - K3TZR

OK, I changed the packet class codes to have the "2".

Are the daxIq packets class type IFDataWithStream?, I had assumed (bad idea) that they were ExtDataWithSTream.