IQ stream from 6700 radio

Paul RN3A

Working on my new Project, which works with IQ audio stream from 6700 radio. I am getting some weird results when applying LP FIRs to the stream. Filter coefficients were obtained from MatLab Filter Design and Analisys Toolbox, and Simulink execution worked pretty well. But something is not coming out when I attempt to build the "live" model. I suspect that I misinterprete IQ stream data. Please confirm if the following assumptions are correct. If not, please provide correct interpretation.

1. IQ stream data is arriving in 256 size float blocks.
2. "Even" elements (0, 2,4...254) are I packets, "Odd" elements (1,3,5...255) are Q packets, and represent single sample at given sample rate. So incoming blocks should be interpreted as 128 size I and Q packets. So, if we are running IQ stream with 192000 Sample Rate, there should be  192000 / 128 = 1500 blocks per second. 
3. The [0] and [1] elements arrive first, [254] and [255] elements arrive later on the time scale.
4. Each sample (I or Q) have depth, defined by float space in C# (i.e. 4 bytes or 32 bits)
5. Samples are normalized by module |1|, i.e. their valuses will never exceed 1 or -1.
6. When applying fequency conversion before filtering, I and Q NCOs do not have  to be set to SIn/Cos, since I and Q packets from the radio already contain phase information. In other words, single NCO can be used for both streams.

Please advise.

73! Paul RN3A

Steve-N5AC

1. IQ stream data is arriving in 256 size float blocks.

The data is packetized and I believe it is 256 samples per block.  But really this shouldn't matter much in your code.  Your code should decode the VITA-49 headers and then locate the actual samples, add them to a circular buffer and then you can check for the block size you need on your end and process whenever you have the correct number of samples.

2. "Even" elements (0, 2,4...254) are I packets, "Odd" elements (1,3,5...255) are Q packets, and represent single sample at given sample rate. So incoming blocks should be interpreted as 128 size I and Q packets. So, if we are running IQ stream with 192000 Sample Rate, there should be  192000 / 128 = 1500 blocks per second. 

Yes we would say even samples are I, odd are Q.  At one point we were sending only Ethernet datagrams with < 1500 bytes, but I believe we may now be sending larger datagrams and letting the protocol stack split them apart for on-the-wire transmission... again, you should look at the packet headers and read how many samples are in the buffer you get and process as appropriate.  I would not build my code around a particular implementation if possible.

3. The [0] and [1] elements arrive first, [254] and [255] elements arrive later on the time scale.

Yes, everything sends in time BUT they can scramble on the wire -- this rarely happens to us, but in a multi processor system can happen more frequently.  I would again recommend looking at the headers to be sure you have things in the right order and that you have not missed packets.  We look at the packet count (running 4-bit counter) and if we skip a packet we wait a short time to see if it will show up before proceeding to assume it is gone.

4. Each sample (I or Q) have depth, defined by float space in C# (i.e. 4 bytes or 32 bits)

No, DAX Audio channels are typical IEEE-754 single-precision floats with a 23-bit mantissa.  This represents 138dB of instantaneous dynamic range in the container.  8-bits are reserved for the exponent. But DAX IQ channels are 32-bit fixed point I/Q.  Do you have the VITA-49 specification to look at how they are packaged?

5. Samples are normalized by module |1|, i.e. their valuses will never exceed 1 or -1.

Correct for DAX Audio, but DAXIQ are fixed point and as I recall we attempted to left justify them.  There are certain advantages to doing this in case you want to change the precision later and rescale.

6. When applying fequency conversion before filtering, I and Q NCOs do not have  to be set to SIn/Cos, since I and Q packets from the radio already contain phase information. In other words, single NCO can be used for both streams.

Generally, most people want to have DAX I/Q so that they can use a complex mixer with 100% image rejection to isolate a specific frequency(s) in the I/Q channel.  Once the mixing is done, typically filtering and downsampling are also done.  How you do all this depends on your demodulator and if you really want to mix before processing ... The I and Q channels presented have been mixed with a complex LO, 90-degrees out of phase so you get both a I and Q channel which are 90-degrees out of phase.

Paul RN3A

Steve, thank you. Things are still not coming out, so I had to dive deep into Vita and FlexLib. 

1. There is an event trigger for IQStreamDataReady. This event is triggered when FlexLib internal AddData routine returns a block of data in float[256] size format, and also returns iq_stream varaible, which carries information on capacity, IP address, Panadapter, port and so on. The only information, which seems to be relevant for further DSP is Sample Rate.  AddData routine also checks for received packets order, by checking VitaIFDataPacket packet header, which is returned from Vita class routines. The received packets order is checked in the range of +/- 1 packet out of range, and sorts them in the right order. 
So this is how I see it in the code. 
The float [256] data is filled by packet payload function inside VitaIFDataPacket class in Vita DLL. And sure looks like that all "formalities" out there are  already taken care of. 
Also, there are no on event trigger call points inside Vita class. 
Having said all that, I still have more questions than answers:
- If the above is correct, why do I need to re-check Vita IF packets again?
- How do I suppose to synchronize IQStreamDataReady event trigger with received Vita data packet? Do I supposed to do it at all?

However, if I need to recreate the algorithm which has apparently been already created, what is the purpose for that? The data payload in IF packet from radio simply carries data (bytes), which are formed in the radio. Each data "element" is 4 bytes long, which matches the float space. All bits in every bytes are swapped in the right order, as per Vita-49 protocol. All looks good... 

So what do I miss?

I have decided to record 60 seconds IQ stream samples of live audio with a couple of artificially created marker points for the reference (high level carriers on the known frequencies) and try to process them with MatLab model. I am using Hartley type Complex Mixers and LP FIRs with decimation to the required rate and BW. The simulated data works very well, but I am still struggling to get it working with "live" data.

Paul RN3A

Paul RN3A

One more thing about fixed point data format. VITA does not specify user-defined format of any of the allowed type :
1. Fixed-point unsigned
2. Fixed-point signed
3. VRT unsigned floating-point*
4. VRT signed floating-point
5. IEEE 754 single-precision (32-bit) floating-point
6. IEEE 754 double-precision (64-bit) floating-point

I assume you are sending #2 - FIxed-point signed. Is it correct?

In which order are they placed in 4 byte (32 bit) structure? 

How many bits from MSB to binary point?

73, Paul RN3A

Eric-KE5DTO

Paul/Steve: I think there is confusion here.  Correct me if I'm wrong, but Steve's response here addresses using the over-the-wire API while Paul is using the FlexLib API.  I think you guys are just missing each other here.  Let me take a shot at clarifying in another reply.

Eric-KE5DTO

Paul,

Your questions made me go back to the code to review.  FlexLib has been treating DAX IQ streams as if they were coming in as floats.  However, the radio was sending the stream as fixed point integers.  This is likely the root of the problem here.  I will work on correcting this and getting you an updated version of FlexLib that fixes this.

Paul RN3A

Thank you, Eric.

If so, I believe there is a quick fix here. You place payload data right into float space variable, byte by byte. Is this true? If so, you just need to specify fixed point params, the rest can be handled by BitConverter.GetBytes( float )., and then converted to true float, double or integer. 

Eric-KE5DTO

It probably makes more sense to skip the float and go straight from the bytes in the packet to the fixed point integers.  I'll keep you posted.

Eric-KE5DTO

Upon further investigation, we are already doing the conversion in the VitaIFDataPacket constructor.  I was having a hard time understanding how the DAX IQ would work at all without this.  So back to the original questions from a FlexLib perspective:

1. IQ stream data is arriving in 256 size float blocks.

This is correct.

2. "Even" elements (0, 2,4...254) are I packets, "Odd" elements (1,3,5...255) are Q packets, and represent single sample at given sample rate. So incoming blocks should be interpreted as 128 size I and Q packets. So, if we are running IQ stream with 192000 Sample Rate, there should be  192000 / 128 = 1500 blocks per second.

This is all accurate.  However, I couldn't tell you whether the I or Q (odd or even) samples were leading or trailing by 90 degrees.

3. The [0] and [1] elements arrive first, [254] and [255] elements arrive later on the time scale.

This is correct.

4. Each sample (I or Q) have depth, defined by float space in C# (i.e. 4 bytes or 32 bits)

Yes, each I/Q sample pair is represented by 8 bytes (2 floats x 4 bytes/float).

5. Samples are normalized by module |1|, i.e. their valuses will never exceed 1 or -1.

Correct.

6. When applying fequency conversion before filtering, I and Q NCOs do not have  to be set to SIn/Cos, since I and Q packets from the radio already contain phase information. In other words, single NCO can be used for both streams.

Usually you would use a complex oscillator where the two channels are 90 degrees out of phase and multiply this with the signal.  Sin/cos is one of the easier methods to do this.

Paul RN3A

Erich, sorry, but once again re: #4,

Samples are of float type, and are seen in C# environment as float. The reason I want to confirm is, because all values are very small, like  0.0XXX .. 0.000XXX , i.e e-2 to e-4. Is this normal? 


I was taking samples of 60 sec length block and looking at max and min values, all values were in this range. This does not look right - the band was full of very strong stations, and there should be at least some readings with the values close to 0.2…0.7.  I was playing with this sample, processing through FFT, trying to find harmonics on the known frequencies, and there were none… 


The example I was using is as follows:


Radio tuned at 20m with 14,100 as center Panadapter frequency, at 192000 sample rate. There were 2 high level carriers on 14,070 and 14,150 (+20db according to S-meter). I have inserted these carriers with signal generators, so they mixed with live air signal at the antenna.  So after FFT I was expecting to see strong harmonics  on +50 kHz and  - 30 kHz, but there were none. At the same time, MatLab simulations  were acting as expected.  The only difference - input data was simulated by me,  not live data. 


Thanks and regards - Paul.

Eric-KE5DTO

Paul,

Do you mean signals like S9+20 = -53dBm?  If so, consider the following: The radio will accept signals over 0dBm.  This means there is another 50dB+ of dynamic range above what you were testing.  The float space should handle this.  I'm actually surprised the numbers you gave weren't getting smaller numbers (i.e. -50dBFS).

If you meant +20dBm, then something must be wrong as the radio is supposed to set a protective circuit if it gets signals above some threshold (somewhere around +13dBm).  At those levels, I would expect the fixed point to be full or overflowing.

Paul RN3A

+20dB over S9, which is not +20dBm   +20dBm is a killer.

OK, I see. Float space accepts ranges between -3.4 E 38 to 3.4 E 38, which significantly exceeds 100dB range.  

Thank you, Eric.

Steve-N5AC

If you're working on things such as this, I think its important to recognize some important points about number spaces and dynamic range.  Since what you are looking at is voltage, a doubling of the voltage in the power realm (holding impedance constant) yields a quadruple in the power output.  This means that adding a bit (doubling the voltage) yields 4x the power (6dB).  In other words, each bit is worth 6dB of dynamic range.  

For fixed point numbers, you consider two things: how much dynamic range can the number space handle, and then how much of that number space are you using.  In the case of IQ output from the radio, we use 32-bit samples, but the entire 32-bits are not used.  A 32-bit number can handle 6 x 32 = 192dB of dynamic range.  I don't have the actual number we use in front of me, but it is a lot less than this.

For floating point, there are two kinds of dynamic range: total dynamic range and instantaneous dynamic range.   Think about it this way, if you were measuring a single value with a very wide dynamic range and reporting it as a floating point number, you would want to show a very small number in one sample and a very large number in the next, potentially.  Used this way, the total dynamic range is something like 413dB (I just ran this calculation and I could be off ... but the point is that it is very large).  

For radio, though, this is not useful.  In radio we want to be able to hear a very small signal in the presence of a very large signal.  We need for the number to represent both faithfully at the same time.  This is called instantaneous dynamic range.  To find the instantaneous dynamic range, we are again concerned with the number of bits that represent the actual number.  For an IEEE-754 floating point number, there are 23-bits in use.  So we use 6 x 23 = 138dB.  So you notice that this is actually less than the fixed point number -- so why would we convert to floating point instead of staying with fixed point?

Floating point relieves the programmer (that's both you and us if you are using these numbers) from the burden of determining how to scale every time we multiply.  If you multiply in a mixer, and you're using fixed point, the numbers get very large.  You must hold them in a larger accumulator and then scale appropriately.  If you mishandle the numbers at all, you truncate and lose bits.  Floating point is much easier to work with.

Finally, you could also go to double-precision floating point which has a 52-bit mantissa for 312dB of instantaneous dynamic range, but the CPU load generally quadruples or worse for handling doubles everywhere so it is rarely used.

Paul RN3A

Thank you, Steve.

This is very helpful. So the bottom line is:
- The data which I receive from radio is indeed of float (4 bytes) signed type.
- There is no need for additional measures to scale data - it is already within the dynamic range of the radio.

This matches very well with the results I finally managed to get from "live" signal on DAX channels (to begin with). After simple FFT I can see complex spectrum harmonics at exactly where I expect to see them, and it also perfectly matches MatLab simulation. 
Having said this, I just want to add something, which was not documented or discussed before, for those who might be interested. 
DAX audio data is ALWAYS sent with 48000 sample rate and appear in 256 size float type blocks. There are TWO channels sent simultaneously, so even memebrs of blocks are "left" channel, and odd elements are "right" channel (or vice versa, but this is irrelevant). So with every block you receive 128 "stereo" samples at 48000 sample rate. 

Now I need to get correct "live" results with IQ streams.