Questions about AGC-T algorithm

IW7DMH, Enzo

Questions about AGC-T algorithm

Hello, I read the "DIRECT CONVERSION RECEIVERS" White Paper of Stephen Hicks and I have some questions about the AGC-T function and its related control.

1- is the AGC-T funcion implemented in the same way in the new Direct Sampling Receivers?
2- if so, what is the real effect of the sliding control when AGC dropdown control is set to OFF?

Probably I am missing something, but when AGC is OFF the algorithm should not modify gain in any mode. On the contrary, expecially for dx little signals, I find it very effective (also when AGC is OFF) and can make the difference between copy or not copy a small signal.

73' Enzo
iw7dmh

Tim - W4TME

AGC Off disables the automatic gain adjustment algorithm and allows you to set (fix) the gain manually.  It will not change in response to the received signal strength.

Steve-N5AC

To expand on this a little further and to be sure a broad audience understands, I want to explain the purpose of AGC and AGC-T in our radios.  The dynamic range of the radios we sell is vast and covers in the vicinity of 100dB.  This means that the strength of the weakest to strongest signals that could be heard are 10,000,000,000:1.  Our ears have a dynamic range that is similar or larger (most would say something like 140dB), but this is not the level where we are comfortable processing information.  Very loud signals are painful and very weak ones require straining to hear. So the purpose of AGC is to apply gain to weak signals and remove gain with strong signals to get them all within a comfortable listening range.  When you listen to signals in your radio, they are generally mapped to a range of hearing that is comfortable and pleasant. 

To accomplish this, we could simply have gain that amplifies weak signals and attenuates loud ones and so provides a roughly constant level in your ears.  But we want to be smarter about this -- no one really wants to hear noise so we want to avoid applying gain to noise if we can.  This is where the AGC-T comes in.  The AGC-T sets the maximum gain we would apply under any circumstances.  Since the noise floor is relatively constant on a given band at a given time, we can set the AGC-T so that it never applies gain to noise, but it will apply gain to signals just out of the noise.  In doing so, we can reduce the level of noise you hear and help signals pop out of the noise.

The speed of the AGC (FAST, MED, SLOW) sets how quickly or slowly the AGC recovers after a strong signal.  If you are listening to a loud voice signal, AGC slow will resist increasing the gain between syllables and therefore reject most of the noise which is at a level far below the signal. FAST and MED, then, provide faster levels of recovery for when you want the system to more closely follow the the dominant signal in the passband.  Any time you have a very strong signal that causes the gain in the AGC to be reduced, you could experience a loss of gain to a weak signal you are listening to.  The filter passband edges which are continuously adjustable and TNFs can be used to eliminate signals that might interfere with operation.

As Tim mentioned, with AGC off, the control just becomes an absolute gain slider.  The more you slide it to the right, the more gain is applied and the louder the signal and noise will be.  The benefits of increasing SNR with AGC will be lost in this mode.  The main gotcha with running phone or CW with AGC off is that you could have the gain turned up and encounter a very loud signal. There would not be an immediate gain reduction and so your ears might get a wallop from the signal in question.  The only time I run with AGC off is for digital modes that do not want any amplitude changes made to the signal.  I often run wide bandwidths (5kHz) for some digital modes like WSJT (JT65/JT9) since we are flat all across that width.  If I used AGC, the occasional RTTY station at 14.080ish would run down the gain and push my WSJT signals further into the noise and make decoding harder (remember AGC calculations are made on all signals within the filter passband).  For this reason, I run AGC OFF and then I make sure that the peak signal never runs the signal meter in DAX past the right side.  I generally shoot for about 80%ish on a loud signal. Joe's software will work fine on signals in the noise, but we can't pass it a distorted mess from exceeding the dynamic range of the number system ;-)

A good starter setting for phone/CW would be to set AGC at MED and AGC-T at 40ish and see how you like that. Everyone has different preferences so your mileage may vary based on what you are trying to copy and what the bands are like.

Kevin WB4AIO

I monitor the levels coming out of my system with an average-reading VU meter. (Either James Chapman's VUMeter program or the VU section of the free Orban Loudness Meter.) I adjust the VUMeter scale so strong signals read about 0 VU on frequent peaks.

Then I tune off to a point in the band where I hear no signals, just normal band background noise. I then adjust the AGC-T for a reading of -15 dB on the VU meter.

For me, that is the magic spot where signals "pop out" of the noise, without the noise being fatiguing to listen to -- yet weak signals are still strong enough to understand. This would apply to PowerSDR or SmartSDR.

I also almost always run Custom AGC settings (this relates to PowerSDR; not sure about SmartSDR settings). I keep the normal 2 ms attack time; that sounds best. But I slow the recovery time down to its maximum setting: 5000 ms. (I actually wish I could make it a bit longer.) This makes listening a real pleasure. Only in severe thunder static conditions do I use one of the default AGC settings to get a faster recovery time between crashes.

Jay Nation

Kevin
Funny. I thought I was the only one doing that. Including the Oban meter, but not the James Chapman VU meter, hadn't seen that one. Have you looked at FlexMeter-1.0.0.5?
https://community.flexradio.com/flexradio/topics/a-small-utility-to-read-smartsdr-radio-metering-points-with-ethernet-api. It reads out the levels that the radio sends to SmartSDR. Even if SmartSDR doesn't show them to you, the radio knows what it's doing. We juat have to figure out how to interpret the numbers its putting out. 
73, Jay - NO5J

Jay Nation

Hadn't paid real attention to the numbers from FlexMeter, until now. but tonight on 20m monitoring WSJTX with a 50Hz filter width of the noise floor during the quiet intervals.

the SLC LEVEL 0 reading is -121 dBm.
the SLC AGC+  0 reading of that 50Hz is -39dBFS.
the SLC 24kHz 0 reading  of that 50Hz is -103dBFS.

widening the filter back out to 3500 Hz dring the quiet intervals
the SLC LEVEL 0 reading is -99dBm
the SLC AGC+  0 reading is -16dBFS + or - 1dBFS
the SLC 24kHz 0 reading is -103dBFS + or  -1dBFS

It's a guess but SLC LEVEL 0 might just be the current SmartSDR Smeter level in dBm they seem to track together.

Another guess is the SLC AGC+  0 level might be how much AGC gain is being applied. 15dBFS seems to be the reading of what I'm trying to adjust the AGC-T to acheive, the way I normally set things up.

Another guess the SLC 24kHz 0  -103dBFS not changing much might be explained by a 100dBm birdie in the passband. its usually the same reading unless a stronger signal gets dropped across it.
putting a narrow TNF on it makes it drop a little.

Maybe someone at Flexradio can define the meaning of those 3 levels they send to SmartSDR?

But I do like the -16 to -15dBFS reading for the SLC AGC+  0 measurement that probably has nothing to do with the -15dBm reading i got from the Oban meter when
 I was searching for the sweetspot for AGC-T setup.  

All of this was done with Preamp set to 0, AGC speed set to fast, AGC-T set to 50, DAX RX gain set to 20, and WSJTX set with the slider centered on the 4th index from the bottom and the WSJTX level meter reading 10 dB during the quiet Intervals. 

My usual method is switch to dummy load, set DAX to 20, set AGC-T to 50 and read a 0dB level on the WSJTX level meter. then switch to the antenna, the level will jump up to 9 to 10 dB during a quiet interval, and rises to almost 20dB when it hits 20 I lower AGC-T 1 notch, when I see the quiet intervals drop below 7dB I raise AGC-T one notch.  total adjustment range of AGC-T is between 47 to 51, but it ends up most of the time set at 50. On 40m I use the -10dB attenuator and adjust the same way.

73, Jay - NO5J

Steve-N5AC

Let me see if I can explain the metering (hopefully I can since I wrote the code).  For each mode there is a receiver signal chain.  The chain consist of a series of blocks that perform each of the functions in the receiver including metering, demodulation, noise mitigation, level adjustments, etc.  After each block we have an option to add another meter to measure what the levels look like after the block.  My recollection is that we generally have about 5 meters in each receiver chain, but we may have changed this over time.  

In all cases the meter performs an "average square root sum of the squares" measurement or what is known as an average magnitude as well as a peak magnitude.  Mathematically, the magnitude of any given IQ point is sqrt(I^2+Q^2).  I took a few liberties with this algorithm for performance reasons, but that's essentially it.

Here's what the ones you've identified mean:

24kHz: our slice receivers run at 24ksps because this allows us up to about 20kHz bandwidth without too much trouble.  The receiver is formed by filtering and decimating the input from the antenna from either 245.76Msps or 122Msps down to 24ksps.  At that point, we take a measurement of the level.  So this is a magnitude of the entire 24kHz of bandwidth -- anything you see +/- 12 kHz from the carrier will be included in this measurement.  The level has also not been calibrated and it originates from fixed-point numbers so it is in dBFS (decibels away from full scale) and will always be negative.  This meter is useful for us, but probably has limited utility for an operator.

LEVEL: This meter occurs after the passband filter and after calibration is applied.  There is a complex series of calibration tables that use the band, which filters are in use, which preamp(s) are in use, etc. that calculate the absolute signal level in dBm given a specific level in dBFS.  LEVEL is the meter that we show in the slice receiver and it is calibrated.

AGC+: This meter comes from the output of the AGC at the end of the signal chain and shows the effects of any AGC, APF, NR, etc. on the signal.  The signal the AGC measures is what is going to your speaker/headphones.  It is back in dBFS as there is no need for a calibration on this level -- it's all about a level that will be sent to headphones/speakers.  You could think of this as a "loud meter" if you like, but it measures what is in your ears.

So AGC adjustments will affect the AGC+ meter, but will not affect the LEVEL meter.  Adjustments in the passband filter will affect the LEVEL meter, but not the 24kHz meter.  You can see the effect of bandpass filtering on SNR by watching the meter in an empty band and reducing the bandpass filter. For every reduction of 50% in the filter, you will see a lowering of the noise by 3dB.  You can learn a lot about how noise works just by playing with the filter and watching the meter outputs ;-)

Another meter that is interesting to watch is the panadapter.  Each pixel in the panadapter is a meter showing the noise level in a single bin of the FFT.  You can tell the bandwidth of each pixel by comparing the noise floor of the slice receiver to that of the panadapter.  For example, you can zoom out until the average pixel level reads the same as the slice meter and at that point you are about 500Hz per pixel assuming you have the slice receiver passband set to 500Hz.  This should roughly jive if you take the visible panadapter bandwidth in Hz and divide by about 2000 (this isn't precise for a number of reasons).  Now as you zoom in with the panadapter, you are "looking into the noise floor."  On a 6700 you can see 25dB into the noise floor (if you use WSJT, you know JT65 can hear signals something lie 24-27 dB below the noise floor) so you can see weak signals like WSJT that you can't hear nor see on other radios.  On the 6500/6300 you can see about 19dB into the noise floor with the panadapter.

DrTeeth

Absolutely fascinating. A lot of it goes over my head but I still like reading your technical posts. Many thanks and keep posting.

Jay Nation

Steve
Wow, thanks for this explanation. These are the types of things I'm curious about. I'd like to be able to understand it "all". Each new detail helps explains a bit more of the mystery. Give's me more pieces of puzzle that I hope will move me closer to solving the puzzle and assembling the "big picture". The more in focus that picture becomes, the more ideas I get about using the SmartSDR API, and SmartDSP API. When I know more about what the radio is doing, I can wonder more about what it might be able to do. Then I can move on to the what can "I" make it do. It's already capable of so much, I'm usually to busy enjoying it, to worry about, "but what else?". 

Thank You

73, Jay - NO5J

Jay Nation

6 months ago I could only spell API, Initially the wikipedia explanation, explained that while I could spell it, it was going to take me a while to learn to API, and a bit more learning to avoid creating "cold API joints" and what all these funny little tubes with wires and colored stripes were for. 

The more I explore, I'm more certain I'm following an the right path through this wilderness. Who knows?, Maybe the world really is round after all. I've seen whats behind, Let's see whats ahead! I'll know more tommorow, follow at your own risk. I wonder what that is, I wonder what it will feel like when I know. See what happens when you look?

It's easy, just keep us in sight, and you might not get lost, on the journey to API.

It's probably that a way!

Let Us Explore!

73, Jay - NO5J

IW7DMH, Enzo

Thank a lot for your explanations, Steve.
I hope your last two post will be reported in official Flex manuals. I didn't aware of the feature mentioned below until you talked about it.
... you can zoom out until the average pixel level reads the same as the slice meter and at that point you are about 500Hz per pixel assuming you have the slice receiver passband set to 500Hz.  This should roughly jive if you take the visible panadapter bandwidth in Hz and divide by about 2000 (this isn't precise for a number of reasons).  Now as you zoom in with the panadapter, you are "looking into the noise floor."
In this aspect Flex really shines.
I'll take more time to really figure out what you have written but I have other questions:

1- as SSDR can access measurement data by itself, could be implemented an adaptive AGC-T function? User could set its preferred audio level (listening in a "only noise" frequency) and SSDR could adapt the threshold to match that setting;

2- as the listening window of each slice is about +/- 12KHz, could be implemented an additional "slice receiver" only in that window, also in the 6300? I am thinking about a way to use the second slice for skimming and the other one to work, if needed, a split. I know very well I can do it with a 6500 or a 6700, but I am considering the software aspect: as the window data is already sampled for the main slice, is it very hard adding a second audio conversion for the additional slice? (hope I have explained well in my poor english);

3- last but not least: where can I buy a polo like the one you dressed in this event? https://www.youtube.com/watch?v=LuWaStl0GgI

Thank a lot for your help

73' Enzo
iw7dmh

Steve-N5AC

1- as SSDR can access measurement data by itself, could be implemented an adaptive AGC-T function? User could set its preferred audio level (listening in a "only noise" frequency) and SSDR could adapt the threshold to match that setting;

We've discussed this in the past.  It's something we are likely to look into in the future when we have some time.

2- as the listening window of each slice is about +/- 12KHz, could be implemented an additional "slice receiver" only in that window, also in the 6300? I am thinking about a way to use the second slice for skimming and the other one to work, if needed, a split. I know very well I can do it with a 6500 or a 6700, but I am considering the software aspect: as the window data is already sampled for the main slice, is it very hard adding a second audio conversion for the additional slice? (hope I have explained well in my poor english);

Actually early in SmartSDR slices were implemented like this -- and two could occupy the same 24kHz bandwidth without anyone really understanding that this was happening.  The issue was that as the slice was moves outside of the window, it then required it's own 24kHz slice of the spectrum.  If you had already used all of them up, we'd be in a quandary: would we stop the slice from moving further?  Would we close it?  Would we generate an error?  Our initial foray into this world was to stop it from moving.  So early in the design you would not be able to move slices further.  This simply confused everyone "my slice is stuck -- it's hit an invisible wall -- why is that wall there?"  In the end, we decided that our customers were likely to not like a feature that says "two, but maybe up to more slices" combined with a slice hitting a wall.  So we pulled this from the system and required each slice to have it's own slice of the spectrum because it seemed that predictability more important.

3- last but not least: where can I buy a polo like the one you dressed in this event? https://www.youtube.com/watch?v=LuWaStl0GgI

Not the exact shirt, but we have all black FlexRadio shirts for sale online now here: http://cart.flexradio.com/FlexRadio-Polo-Shirt--Black_p_934.html

Al_NN4ZZ

Hi Enzo,
Here is an idea to automate the AGC-T setting per your Q&A with Steve below on this topic.

https://community.flexradio.com/flexradio/topics/option-to-automate-the-agc-t-setting

The idea has 33 votes so far.  Please add you vote and comments if this is what you are looking for.   It would be nice to see it in V1.5 but suspect is probably is more likely to be considered for V2.x or later.  

Regards, Al / NN4ZZ  
al (at) nn4zz (dot) com




Steve - N5AC (VP Engineering) replied to this question: Questions about AGC-T algorithm

1- as SSDR can access measurement data by itself, could be implemented an adaptive AGC-T function? User could set its preferred audio level (listening in a "only noise" frequency) and SSDR could adapt the threshold to match that setting;

We've discussed this in the past.  It's something we are likely to look into in the future when we have some time.

IW7DMH, Enzo

Actually early in SmartSDR slices were implemented like this ...

I don't know if it was before or after the 6300 and actually using a 6500 or 6700 it doesn't make sense.
But using a 6300 it could make. I would stop moving it further but I would try to draw it in a different way so one could be aware of the difference kind of slice. I would also keep it simpler and more ****. Just two controls: volume and bandwidth (I try to picture it below).
Probably users of 6300 rig would be glad for this.
 

Alan, K2WS

Steve, your insight on the inner workings of the DSP really help make sense out of Smartsdr! I hope you guys include a "theory of operation section" in the eManual along with block diagrams. Your discussion on metering has me thinking. Why not allow the user to switch to the AGC+ meter when adjusting a TNF, NR or whatever? How about having the S-meter indicate where the AGC-T is set, in addition to its normal function? How about an indicator in the S-meter that shows when the slice receiver passband width is less than, equal to or greater than the waterfall pixel bin bandwidth? I bet that would clear up confusion about the noise floor of the waterfall being different then the passband noise level in the slice receiver. Any thoughts on this?

73, Alan K2WS

Steve-N5AC

There's a spectrum of desired quantity of information in the display.  With PowerSDR we tended to throw up anything that might be of use.  With SmartSDR we are working to try to only provide things we believe are necessary and holding back some of the extras.  This is largely because we've been told that too much information can be intimidating.  Also, when we provide information that is not well understood in the past, it has generated large numbers of support calls with folks trying to understand what they are looking at.  For example, we used to allow customers to calibrate the FLEX-5000/3000 series radios using software.  But minor deviations from radio to radio, which are completely normal with analog components, would generate phone calls.

The discussions surrounding the voltage reading on the FLEX-6000 is a perfect example.  We use this reading internally to determine the state of the master fuse and also to understand if the voltage is below a level tolerable by the internal supplies.  We opted not to show this to customers because of the "man with two voltmeters" problem (which is right?).  DDUtil is reading this voltage out of the radio and showing it to customers.  We have explained where this comes from and that there is, necessarily, some drop in the voltage.  There are regularly discussions about why this voltage is different, why our measurements might have the wrong calibration (they don't, it's just drop internally), etc.

For myself, I would prefer that the dashboard of my Suburban look like an F-16.  I'm an instrumentation ****.  We have metering that measures about 10 voltages and something under that in current measurements.  We use these internally for diagnostics and I'd probably show them all in a display ;-) The more information, the better.  I have learned by talking to others that everyone does not share this view of the world and that having a lot of data that is not immediately relevant is not useful for most folks.  So this is the overall approach that we have taken.

Now that I've rambled about how we got here, we could come up with displays/indicators that show the things you are talking about.  I especially like the dB differential in the slice and panadapter metering.  I would recommend adding it to the Idea section and encouraging folks to upvote it.  No doubt that we will return to the list periodically and grab things from there to go implement that have wide interest.

DrTeeth

So you mean you'd throw a fit if I asked if 13.7V supply is okay, hi hi?

Rather than *more* instrumentation, I would like the readability/usability issues addressed first. Al (NN4ZZ) has produced some excellent mockups that would transform SSDR's power meter for example.

Are there any plans to modify SSDR's interface or to allow more customisation? For example, I'd replace the graphic equaliser with the DAX control panel in a blink of an eye. I use the latter far more and it is not as accessible as the former.