Tiny Python Panadapter

Mark Erbaugh

Martin Ewing, AA6E, published an article in the April 2014 QST detailing his Tiny Python Panadapter (TPP). This is Python code for a panadapter on a microcomputer, such as the BeagleBone or Raspberry PI.

I'm interested it writing Python code to talk to my Flex 6300, so I thought I'd see if I could get TPP working running on my Windows 7 computer with my Flex 6300. It was pretty easy. This could make a good starting point for more sophisticated code.

Download the TPP project from SourceForge (https://sourceforge.net/projects/tinypythonpanadapter.

Install Python 2.7 (I used the latest - 2.7.9).

Install three python libraries: numpy, PyAudio and pygame. I found these in .whl format (which installs the needed binary files) on the Unofficial WIndows Binaries for Python Extension Packages at http://www.lfd.uci.edu/~gohlke/pythonlibs/

Run the program pa.py(python pa.py)  from the TPP source. This lists all the audio devices on your system. Look for the DAX Audio IQ RX 1 device. On my system it was device 51.

Enable DAXIQ channel 1 in SmartSDR and DAX control panel. Set it to 48000 samples

Run the program iq.py passing the index of the audio device (python iq.py --index=51) and you should see a basic panadadapter display). You can play with the other parameter settings.

lyndy brannen

Thanks Mark.
I will  try this out over the weekend.

Mark Erbaugh

I've gotten Martin's Tiny Python Panadapter working with FlexLib API. The program now uses FlexLib API to get the IQ data from an IQ data stream and DAX is not used. It also uses FlexLib API to get and set the frequency.

The program still requires that SSDR be running and it really doesn't do anything that SSDR doesn't do better, but it was a good way to learn how to talk to FlexLib.

James Whiteway

Good work Mark!! Sounds like you are making good progress.
james
WD5GWY

Walt - KZ1F

Hi Mark,
   That sounds pretty cool. I vaguely remember that article but only having had a passing flirtation with Python it feel into the technically interesting category. What does your program actually do with Flexlib? I thought it just worked from connecting to the com port which is the DAX IQ stream?

Walt

Mark Erbaugh

The FlexLib API lets you do most of the things that you can do with SSDR. It does not use. virtual COM port (aka SmartCAT), but talks to the Flex directly over the Ethernet connection. The DAX IQ is a data stream which represents the I and Q signals and is used for things like panadapters and CW Skimmer. The DAX program converts these into virtual audio ports since most programs that work with this kind of data expect it from a sound card, but with the FlexLib API, you can get the data directly. The original TPP was designed to use a sound card input and my first experiment was to get it working using DAX. My next step was to get it working w/o DAX. Right now it doesn't do anything that can't be done with SSDR, but it was a learning experiment.. One possible project that I am considering is to generate some of the displays that are in PowerSDR, but currently not in SSDR. Another thing I would like to do is to interface my TMate 2 box to the 6300.

James Whiteway

Very cool ideas! Is the panadapter in TPP able to handle very wide bandwidths?
james
WD5GWY

Mark Erbaugh

The IQ data appears to be limited to 96 kHz, but there is also. Panadapter data stream that I'm guessing provides the data for the Panadapter in SSDR. That's on my list to experiment with. One of the reasons for getting TPP workin was so I would have the tools to view other data streams form the Flex.

Walt - KZ1F

The reason I ask Mark is prior to anything ssdr related one has to set up those virtual audio streams and configure them for 48000 vs 40000 to make the math faster etc per Greg's setup YouTube. So ssdr does all the configuring after you pick the radio you want to connect to, it creates a radio object and that radio object gets populated with all sorts of things, one of which is the  IQ streams. Then you'd have to mark one in use by selecting the DAX IQ stream you were interested in, if in ssdr it would be on the left side context menus under Display. I was curious how you did that in Python and not use ssdr to configure it? I am not sure how much processing occurs re: IQ before one selects it vs. after one selects the DAX context menu and then selects the channel. I suspect the later.  Where you did that all natively by Python calls directly to flexlib I would love to see that code.

Walt

Mark Erbaugh

I don't have it with me, but I'll post it when I get home this evening. The code to attach to the stream is pretty simple. Basically, call the radio object's CreateIQStream passing it the DAX IQ channel number. You then call that IQ stream object's RequestIQStreamFromRadio method.  You also need to attach a callback routine to receive the data. On the SSDR side, you just set up a Panadapter to use the DAX IQ channel.  I found that I could change that at will (at least from 1 to None and back), and the Python code got data only when it was set to the channel requested.

I would think it would be possible to do the Panadapter setup through FlexAPI calls, but while playing with that I found it somewhat problematic, so I guess I don't fully understand how that all works.

The IQ stream has a SampleRate parameter which can be changed by the Python code. I've only tried 48000 and 96000. I don't know if other values would work.

N7BCP

Mark, I can add you as a contributor to the GitHub if you want a place to share the complete code. You'll have to create an account.

Martin AA6E

@Mark -  This is great work and an unexpected turn for the TPP project.  You are also welcome to put code into the TPP Sourceforge project.  Your call.

I am a little puzzled about how you would use an IQ panadapter working off DAXIQ channels in the real world.  Potentially, you could have 4 TPP displays running alongside 8 SSDR panadapters on your '6700.  Overwhelming! 

It does make sense as a learning and experimental tool.  Fiddling with signals in the Python environment is a lot easier than in C#, at least for me.

73 Martin AA6E

Walt - KZ1F

I think doing anything not in a C# environment is better than in a C# environment.
BTW, what does the I and Q stand for? I guess I'll have to google.

Mark Erbaugh

Martin, I agree, the only benefit of adapting TPP like I've done is the learning experience. I used your code as an introduction to numpy, pyGame and signal processing theory.

My goal of using the IQ data would be to decode signals. I would like to learn how to decode some of the digital modes with Python code. Are you aware of any tutorials on how to decode CW, RTTY, PSK31, etc?

To help understand the code and where to add in my code, I did a fair amount of refactoring and I probably broke some of the features for the other radios, so I don't think the code is worth putting into the Sourceforge project, but thanks for the offer.

I agree in Python over C#. Of course, I've done a lot a Python programming, though I've never used numpy or pyGame.

Mark Erbaugh

Walt,  Looks like you posted at the same time I was replying to Martin, so our comments got intermingled.

(non-technical follow)
I stands for in-phase and Q stands for Quadrature, or 90 degrees out of phase.  They are the basis of SDR radios.

Gerald Youngblood (Mr. Flex) wrote a 4 part article about it in QEX a decade ago.  Here's one link I found to those articles: https://sites.google.com/site/thesdrinstitute/A-Software-Defined-Radio-for-the-Masses.

Mark Erbaugh

As promised, here is some of the Python / FlexLib API code I wrote:

import clr
clr.AddReference("FlexLib")

from Flex.Smoothlake.FlexLib import API

import Queue

from iq_opt import opt
import numpy as np


class Data(object):

    def __init__(self):
        self.api = API()
        self.radio = None
        self.iq = None
        self.ProgramName = "Python"
        self.api.RadioAdded += self.cbRadioAdded
        self.api.Init()
        self.queue = Queue.Queue(32)
        self.buffers = []
        self.data_len = 2 * opt.buffers * opt.size
        self.cur_len = 0


    def cbRadioAdded(self, radio):
        self.api.RadioAdded -= self.cbRadioAdded
        self.radio = radio
        radio.Connect()
        self.iq = self.radio.CreateIQStream(1)
        self.iq.RequestIQStreamFromRadio()
        self.iq.SampleRate = opt.sample_rate
        self.iq.DataReady += self.cbIQDataReady

    def cbIQDataReady(self, stream, data):
        l_data = tuple(data)
        l = len(l_data)
        new_len = self.cur_len + l
        if new_len < self.data_len:
            self.buffers.append(l_data)
            self.cur_len = new_len
        else:
            excess = new_len - self.data_len
            needed = l - excess
            self.buffers.append(l_data[:needed])
            data1 = np.concatenate(tuple(self.buffers))
            self.buffers = []
            if excess:
                self.buffers.append(l_data[needed:])
            self.cur_len = excess
            re = np.array(data1[0::2])
            im = np.array(data1[1::2])
            try:
                if opt.rev_iq:
                    self.queue.put(np.array(im + re * 1j), False)
                else:
                    self.queue.put(np.array(re + im * 1j), False)
            except Queue.Full:
                self.iq.DataReady -= self.cbIQDataReady
                self.iq.Close()
                self.radio.Disconnect()
                print 'Queue Full'

    def Frequency(self):
        if self.iq is not None and self.iq.Pan is not None:
            return self.iq.Pan.CenterFreq
        else:
            return 0.0

    def AdjustFrequency(self, adj):
        if self.iq is not None and self.iq.Pan is not None:
            self.iq.Pan.CenterFreq += adj


    def Terminate(self):
        self.iq.DataReady -= self.cbIQDataReady

This class uses FlexLib API to connect to an IQ data stream from the Flex and put the results as a complex (real and imaginary) numpy array into a Python Queue object. The main TPP panadapter code which is running in a different thread, gets this data from the Queue. The Queue is use to synchronize between the threads.

The thing that may be a little confusing is the manipulation of with new_len and data_len. The TPP panadapter code expects the data to be in chunks of a certain length determined by opt.buffers and opt.size, but the data comes in from FlexLib in different sizes. There's probably a more elegant way to handle that in the FFT processing code, but I don't really understand that, so I just collect data from multiple callbacks to build an array of the length that TPP is expecting.

Walt - KZ1F

Something where the history is better discussed offline. I've had a series of conversations with FRS about a portable version of SDR. Of course, that starts with a portable version of flexlib. The net of those was I was going to prove the feasibility of this by replicating ssdr to a Linux platform and that somewhat morphed into an Android. I recently retired and have been designing software systems my entire career. So there is a measure of straight port plus a measure of redesign.  I thought about a year ago I was pushed into retirement and had that worked out I'd be past all this by now but I just recently started in earnest. So far I I've tried to pretty much punt on the the waterfall and bandscope so when you said you took a Python app and using the C# dll. Ya know what's humorous, I flirted with Python early on while at Monster.com, that was 8 years ago. I believe I can accurately read your code while much of the C# is ****?? What's the best way to converse offline Mark?
I believe you are referring to what Eric calls Tiles? I used the same queueing mech to do the same thing at Monster.

Walt

N7BCP

Rather than taking the conversation off line, use a code collaboration site like github and others can benefit and contribute.

Walt - KZ1F

Larry, not trying to take Mark's conversation offline...trying to clone a slice of it offline, No Pun Intended. Someday I'll open the portable version, just not today.