Help required for SDC Skimmer for Flex

KD0RC

Excellent! I recently got that document. I need to remember to refer to it before looking in the Wiki...

UT4LW

Thank you very much for the valuable information!

UT4LW

I found some issues with IQ streaming over UDP connection.

Command to create a stream: stream create daxiq channel port=udpPort;

I receive packets of 4128 bytes each. 32 bytes - header. 4096 bytes remain, or

512 complex float samples. The first value is I, the second is Q, and so on.

This is right?

FFT conversion does not work correctly, there is an opinion that I am not getting an IQ stream.

K1FR

There are 4128 32 bit samples. Believe you may not starting a correct place in the udp stream. More when I get home.

K1FR

Yuri:

Sorry for last post which is pretty garbled. Tried to send from iPhone while multitasking. Bad idea!! Disregard the 4128 samples. Will try to explain below.

Please see the attached .jpg which is a screenshot of WireShark looking at a VITA49 packet from my radio. And, please refer to the .xlsx I sent earlier with the outline of the VITA49 format. (Please note: an error in that spreadsheet - the ClassID high and low fields are indeed filled by the radio. Do not think they are essential for what I am doing though). Also - as before - I am no expert. Below is what I think is correct but still working on it.

Note that the VITA49 header block begins at the first of the blue hex dump and is 0x1C560408. The last two bytes (0408) represent the packet size in number of 32 bit words and is decimal 1032. WireShark is showing a length of 4128 which is in bytes = 4 x 1032. VITA49's packet size of 0x0408 is the total number of 32 bit words which includes all headers and all data and a trailer if present. (The header bit is set to yes for a trailer from what I have seen so far, and all packets seem to have four bytes of zeroes at the end which I assume is the trailer??).

So, now in the WireShark .jpg scan down past the header words and you will see what I believe is the first data word which is 0x00004040. The next is 0xoooo5o41 and so forth until reaching the all zero trailer. If I have done the arithmetic correctly, that says there are 1024 32 bit numbers of data samples in each udp packet from the radio. Apparently, 512 I alternating with 512 Q samples.

Definitely think any program using these streams should verify a couple of things. Is the streamID matched to what the client thinks it owns? Look at the cyclic counter in the header and maybe the timestamps to see if a packet got lost. Finally, check the packet size in the header to be sure it is always 4128.

I have not yet tried to FFT any samples. So far I have concluded the 32 bit words are little-endian. A post from Flex a few years ago says they are signed fixed point decimal normalized to be between -1 and +1. I have so far recorded streams from the radio at 24 ksps, and the I and Q values are falling in the range.

Hope this is not confusing. Please let me know what you find. I have a feeling you are a far more advanced programmer than I! 😀

73, Tom K1FR

UT4LW

Tom, thanks!

I found my mistake! Due to inattention, for some reason I decided that the data block begins with the 32nd byte. That's right - 4096 bytes, starting from the 28th byte.

UT4LW

New problem. ))

For some users, the transmission of the IQ stream through the UDP does not work.

There are no errors in the log text.

The situation is also interesting in that for some users the transmission of IQ works only on part of the receivers. For example: two stickkimmers work, two do not work.

Attached is the text of the initialization:

-----

< C14|display pan set 0x40000000 daxiq_channel=1

> R14|0|

< C15|stream create daxiq 1 port=4995

> S785CE200|radio daxiq_capacity=16 daxiq_available=16

S785CE200|display pan 0x40000000 daxiq_channel=1 

S785CE200|display waterfall 0x42000000 daxiq_channel=1 

S785CE200|stream 0x20000002 type=dax_iq daxiq_channel=1 pan=0x40000000 daxiq_rate=48000 client_handle=0x785CE200 active=1 ip=192.168.68.101

S785CE200|radio daxiq_capacity=16 daxiq_available=14

>R15|0|20000002

K1FR

Have they started IQ channels for each receiver via SmartSDR? I always have a stream up and running before my client connects. Not sure if that is essential or not.

73, Tom K1FR

UT4LW

You don't need to run DAX to send an IQ stream over UDP. If DAX is loaded and IQ channels are active, then they are disabled automatically as soon as I create an IQ stream with a command:

stream create daxiq 1 port=4995

However, nothing arrives at the port.

As soon as I close the IQ stream via UDP, the IQ channel in DAX is activated automatically.

Everything seems to be working correctly, there are no errors, but nothing comes to the port.

K1FR

Yuri: not near computer but will check when get home in couple hours. Will send what is working for me Tom

K1FR

Yuri:

I believe the syntax you have for stream create is not the one for V3.3. Here is what is working for me:

 msg = "c158|client udpport 4996\n"

msg = "c159|stream create type=dax_iq daxiq_channel=1\n"

msg = "c160|stream set 0x20000001 daxiq_rate=24\n"

73, Tom K1FR

K1FR

Yuri:

One more note - I just verified that a slice needs to be active and needs to be set to a channel in SmardSDR to have an active DAX IQ channel. If I start my client without having set up a DAX IQ channel in the left hand panel on SmartSDR (not the DAX control panel), the client gets the following back from the radio:

radio response = b'R158|0|\n'

radio response = b'R159|0|20000001\n'

radio response = b'S5A477DFA|radio daxiq_capacity=16 daxiq_available=16\nR160|0|\n'

That is as normal and what I expect. However, my udp port never sees any data and my client hangs until it times out. If before it times out I click on the SmartSDR button and set it to Channel 1, packets immediately start streaming to my port.

So far, I have not found a way via the TCP/API client to tell a slice it has a DAX IQ channel. Will do some more sleuthing on that.

73, Tom K1FR

UT4LW

Tom, before creating the IQ stream, I send a command that binds the IQ channel to the panorama:

C14|display pan set 0x40000000 daxiq_channel=1

K1FR

Thanks, Yuri!!!

That works like a champ for sure. A big help.

Also, for anyone else reading this thread - my earlier post erroneously implied a slice has to have its DAX IQ channel set. IQ channels are not associated with slices, but with panadapters as in Yuri's command above.

Tom

UT4LW

So, the skimmers and the receive mode in the DIGI section work!

Now we need to organize the transmission through the TCP-IP API.

Does anyone have experience in developing such programs? What commands? How is the request for audio blocks and where to send them?

K1FR

I have not tried tapping the DAX audio via the API. However, I did have some luck with DAX_IQ and DAX audio via the Windows sound system before changing over to the API for the IQ recorder. In Python it was as easy as the snippet shown below.

# Set up capture parameters

devs = sd.default.device

sd.default.device = (44, 17)  # 44 is DAX I/Q 1 and 48 is DAX 1

devs = sd.default.device

# Do the recording

myrecording = sd.rec(int(duration*fs), samplerate=fs, channels=2)

sd.wait()

np.save("Flex.npy", myrecording)

73, Tom K1FR

K1FR

Well, I have now spent some time trying to use the DAX IQ stream via the TCP/IP UDP Vita-49 stream. Find myself really confused and hoping someone can get me on track.

QUICK SUMMARY: Some old posts on the Community from the gurus at Flex cause me to believe the data is fixed point, 32 bit, left justified, with values between +/- 1. I need a steer on how to interpret that format. Think I am close, but.....

I am getting IQ data and can decode it, plot it, and run it through FFT. However, the results are not very good. Mainly, the levels are too high regardless of signal levels on the radio. As a comparison, I also record the IQ data coming from the Windows sound system. Those data result in reasonable signal levels and I can plot a power spectrum that almost overlays what I see on the SSDR display.

Attached are three .xlsx grabs from Wire Shark that will show the kind of data I am getting. One is from a panadapter with a strong local BC station, one has a weak 7.85 mhz CHU signal, and one has an empty 28 mhz panadapter.

As you can see, even for what should be low levels, almost all of the samples have the second most significant bits set to "1". For example: 0xC1xxxxxx or 0x40xxxxxx. (VITA and Wire Shark are reversing the order of the bytes) My understanding of IEEE fixed point is that the MSB is the sign bit and the second is the integer bit, the remaining 30 bits being the fractional part. Having the integer bit always set to "1" is what has me confused. Decoding those values using the Python module fixedpoint results in samples with "DC" components at +0.5 and -0.5 with what appear to be reasonable signal values riding on them. I tried masking the integer bit, and that gives me more reasonable FFT results. However, my gut tells me that is a kludge of a fix that lacks understanding on my part! :)

I do notice that for cases with a panadapter full of strong signals, the IQ samples start filling in the least significant bits. Guess this is what they meant by "left justified". But, that leads to more confusion on my part...... I am unsure how to process that. Right shift by 8 or 16 bits depending on content?? Again, my level of understanding the big picture is coming up short.

Could someone set me straight on this please? And, if you have time please let me know if a Wire Shark grab of your radio looks similar to mine. Sorry for this post being so long!

73, Tom K1FR

shark dump 1.22 mhz local BC at -40 dbm.xlsx

shark dump 7.85 mhz CHU at -70dbm.xlsx

shark dump 28.1 mhz no sigs -110dbm floor.xlsx

K1FR

Please disregard my last post - I had an input from Flex that makes me think I am way off base with my assumption about format! Will post again once I know more.

Sorry to those who slogged through my long post. 😀

BTW - I continue to be amazingly impressed with the support from Flex. From Mike to the help desk to the 100 pound brain guys behind the design.

73, Tom K1FR

UT4LW

Hi, Tom!

This is very interesting information!

I ran into a similar problem. It turned out that the sample values are significantly larger than 1.0f. However, all calculations in skimmers and DIGI modules are carried out in relative values, so this did not affect the result.

I don't currently have a Flex-6x00 transceiver on my desk, so I've decided to defer this issue.

Perhaps you have information about the nuances of converting a sequence of bytes in an IQ stream to float values, please share it)).

K1FR

Yuri:

Many thanks! That really helps to know that my radio is working like others. Frankly, I do not think many of us have even messed with the UDP VITA stream or this would have been found earlier. I am going to make an input to the Flex engineers to see if this behavior is what they intended in their design.

Well, I have had good success converting the data. Please know that I am far from an experienced software writer, so there are probably better ways of doing what I do.

Using Python here. I have the IQ byte stream for each VITA load and separate the bytes into int32 by using a python method called struct.unpack (there is probably a C++ equivalent somewhere). I then mask the bit next to the sign bit (i.e. bit 30) to always be zero. So, now my I/Q values are more reasonable - not sure if they are proper, but I can deal with them and get OK results. Masking that bit obviously drops the values by 2, but since almost all the data should still be much lower, and there are plenty of bits to the right of bit 30 which are not hung up at '1', it makes sense to me. I then scale the corrected int32 by: decimal_Q = (intQ / 2 ** 31). With Python, that gives me a float32. I then set its sign according to the value of the sign bit. I am uncertain about the 2 to the 31st part. Perhaps it should be 2 ** 30. But 31works pretty well.

When I take the I/Q data from the Windows sound system DAXIQ channel, I get numbers that give me PSDs that almost overlay what SSDR is showing. I am still working on "tuning" my code to get the UDP VITA stream to do the same.

I hope this makes sense. Please let me know if you have questions and thank again for your help!!

73, Tom K1FR

Ricardo_PY2PT

Hi guys,

Please, anyone using SDC with Flex in a remote Environment?

Regards

Ricardo

Mike-VA3MW

I use my remotely all the time. I have been running remotely since about 2005. This is on 10M from about an hour ago.

Long ago, it became very obvious that having a PC at the remote end is a must have and it makes so many things much easier. I could type it all in, but I did it in a video.

Hopefully, I answered most questions in the video.

73

https://www.flexradio.com/videos/hf-remote-why-a-pc-at-the-remote-end/

Piero Ferrua

Excuse my ignorance, I understand that Yuri's (UT4LW) SDCx64 program works using the connection protocol with Flex6600 SmartSDR API .

How should it be configured to operate using a portable PC remotely i.e. away from the base station while using the MAESTRO? I've already made several attempts but nothing good.

Differently in fixed station (QTH) with PC and Flex6600 on the same network, the SDCx64 works perfectly

Mike-VA3MW

In order to use the SDC Skimmer (or CW Skimmer), you must run it on a PC local to the radio.

The reason is there is too much DAX IQ data to reliably send over the internet and not have a data loss or data corruption issue.

You would manage the SDC Software by using a Remote Desktop connection to the remote computer. The Spot data will show up on your Maestro even if you are using SmartLink.

73

UT4LW

Hi!

I want to write a telegraph program using the commands "cw ptt.." and "cw key.."

Turn on PTT and transmit CW is working. But I can't enable sidetone. Is there such a possibility?

This is a TCP port communication log. < - send command, > - transceiver response:

< C33|cw sidetone 1

> R33|0|

< C34|cw ptt 1 time=0x0000 index=1 client_handle=0x5BF1A6C9

> R34|0|

< C35|cw key 1 time=0x000A index=2 client_handle=0x5BF1A6C9

> R35|0|

< C36|cw key 0 time=0x00A0 index=3 client_handle=0x5BF1A6C9

> R36|0|

...

< C65|cw ptt 0 time=0x0B09 index=32 client_handle=0x5BF1A6C9

> R65|0|

KD0RC

Hi Yuri, you cannot get the Flex to provide sidetone unless the keying source is the key line or the ACC pin 4/5 lines. The issue is that even the small latency of a LAN is too much to use when keying this way. Over the WAN, it is even worse.

In my application (TeensyMaestro), I have a speaker in the box and send the sidetone myself when using the Ethernet keying option.

UT4LW

Hello Len! Thanks for the answer!

I think the cw ptt and cw key commands have a "timestamp" parameter that is meant to reduce the impact of network delays.

Checked: the telegraph transmit works, the signal is emitted into the air, on another receiver it sounds smoothly and without distortion of the length of the signals.

Why is it not possible to have telegraph self-control in headphones that are connected to the transceiver?

Maybe for this it is necessary to bind the client that forms the transmit with the GUI client?

KD0RC

Hi Yuri, I am sure that is technically possible, but Flex doesn't support it. If they did support it, it would work for sending code from text (keyboard, N1MM, etc), but it would not work using paddles or a straight key due to latency. I assume that this is why they do not support it.

Binding to the GUI client won't change this aspect. I found that binding to the GUI client does allow other things to work properly, so I always do a bind if there is a GUI client available.

UT4LW

In this case, there is a question: why were these commands (cw ptt and cw key) developed?

I can generate and transmit signals on the air, but I cannot control what is transmitted on the air. Agree, this is not logical. The monitoring of a transmitted signal should be independent of how the signal is formed.

Ok, I'll have to go back to WinKey 😏

KD0RC

Hi Yuri, I don't work for Flex, but my assumption is so that remote CW would be possible. My homebrew Flex control device (TeensyMaestro) has a keyer in it and can key the rig directly, in which case the rig's sidetone can be used, or keyed via Ethernet (LAN or WAN) in which case the keyer sidetone is used.