Flex 6700, K3, KX3 IC-7800 CQ WWDX SSB Shoot Out

KY6LA_Howard

FLEX 6700
- K3 - KX3 - IC-7800 Shoot – Out  Report  - CQ WW DX SSB 2014

Nov 24-25, 2014

 

Background:

Dennis N6KI and I have been friends for years.  We fight over everything and anything.  I am usually on the Bleeding Edge of Technology and Dennis usually takes the most conservative possible positon. I am very technology oriented and would rather try something new each time even if it may sometimes disappoint while Dennis prefers the old standard tried and true.  Dennis is the ultimate world class contestor placing near or at the top in every contest.. while I occasionally do well in a contest (and win a couple), I would rather do it a different NEW way each time.   My approach to testing tends to be a rather detailed scientific approach to try to produce reproducible results while Dennis prefers a random unstructured approach to see if it feels right for contesting.    Do not take our ranting at each other below seriously,…. We are still friends…

 

CQ WW DX SSB SHOOT OUT:

Dennis conceived of the idea that we should test modern radios against legacy radios during the 2014 CQ WWDX SSB contest because it was a world class splatterfest of the worst possible conditions.

We decided to use the NX6T Contest Station Site in Fallbrook CA –  I was part of Dennis team up there when we came first in NA in the JIDX Contest and second in the world … it’s a great site on a hill 900’ above the surrounding terrain…with no obstructions in all directions about 20 miles east of the Pacific Ocean.

Quadcopter shots by Paul NN6X

 

https://drive.google.com/folderview?id=0B0fUPinob3F4bG5ubGMyREJtQzg&usp=sharing

 

 

            The site is rural and pretty quiet from an RF point of view

 

            The intent was to test the following rigs in contest conditions

 

1.      Flex 6700

2.      K3

3.      KX3

4.      IC-7800

 

 

FIRST SHOOT OUT REPORT FROM KY6LA

Albert Einstein said that the definition of Insanity is to repeat the same experiment and expect different results

I have participated in a number of Shootouts with Dennis N6KI including SteppIR, Honda vs Yamaha Generators and a number of different Legacy vs Modern Radio Shootouts

All these shootouts invariably start the same way.    Those of us with Scientific backgrounds try to set up the experiments with detailed documentation processes, normalization of as many variables as possible so we are comparing apples to apples and a detailed WRITTEN Plan of Experimentation so that we can have Reproducible Results. 

Enter Dennis, N6KI, the King of Irreproducible Results.... who demands total control of the observations

He immediately throws out all the carefully and exacting setups that had normalized the variables to a single consistent set.   He then continues to fiddle with parameters throughout the experimental time so that there is absolutely no documentation and no way to produce objective reproducible results.  We invariably end up with a mish mash of subjective observations colored by the personal biases of the various different observers and absolutely no way to reproduce anything.

The last two days were no different..  than every other previous N6KI Shootout

No written plan of Action, No experimental procedure established.   No coordination of Normalization of Parameters No objective observers and no way to produce reproducible results.  Everyone frustrated with Dennis.

John W6JBR the audio engineer and I spent a few hours on Friday normalizing audio between receivers.  

On Saturday Dave N6EEG and I tried to do the same.  

Invariable N6KI started to do the comparative testing by immediately grabbing the tuning knob on his K3 to continually change frequencies and adjust parameters throughout the experiment.  After trying and failing to get N6KI to follow any reproducible protocol, I gave up in frustration....

I spent most of my time trying to manually track N6KI's random frequency gyrations so that we could all be on the same frequency for tests... ultimately I suggested that we connect the CI-V ports on the radios so that they would all track together.. BUT N6KI would not accept that - rather he said we should just manually tune so that they all sound "ABOUT" the same... which of course was impossible because N6KI continued to fiddle with the tuning knob...

My FIRST CONCLUSION

I will NEVER participate in another "Shootout" UNLESS N6KI is either totally banned from the proceedings or N6KI has his hands tied behind his back so that he can’t continuously and randomly change parameters all the time..

Results:

1.        Surprisingly the Flex with its visual interface was by far the easiest/quickest radio to set up and the least likely to **** up during a contest

 

 (Albeit I had accidently turned on the NR on the 6700 for much of the experiment and did not notice that until the end because NR was one of the few parameter on a hidden drop down menu on the 6700)

 

2.        The IC-7800 was so outperformed by the K3, KX3, and 6700 that we quickly dropped it from the testing rotation

 

3.        Likewise we did not spend too much time with the KX3.. it performed well (better than the 7800) but lacked sophisticated filtering

 

4.        All radios hear the very weakest signals - as you would expect since the atmospheric noise floor on 20M is about -115dBm and the radios    can hear down to -130 dBm or better

 

5.        The Notch filter on the IC-7800 and the 6700 were better than the K3/KX3 at removing a CW Signal in the SSB Bandpass

 

6.        ONLY the 6700 could produce more than one notch filter at a time so it was the only radio to be able to remove more than 1 CW signal from a SSB Bandpass-  the 6700 had the most powerful and flexible filters and filtering

 

7.        The 6700 had the lowest Phase Noise (the background static you hear jumping around because of the splatter)  followed by the KX3, K3 and finally the IC-7800

 

This is consistent with the expectations of the different technologies 

- lower phase noise = lower operator fatigue

 

8.        We ultimately came down to comparing received audio.. but that is highly subjective as different people hear very differently. 

a.       For example I had my 6700's equalizer peaked for my hearing... so while I could hear well on it, I had trouble hearing on the K3 as it was not peaked for my hearing. 

b.      Dennis had his K3 peaked for his hearing so I could not hear well on his K3 but he said my 6700 sound lacked    "Brilliance" - albeit that might have been due to the fact that the NR was on...

c.       With these subjective hearing tests.. the K3, KX3 and 6700 could all hear the very weakest ones and could find them inside a splatter zone.

d.      the background noise was easily the lowest on the 6700 and for my hearing was the easiest to discern the content of the weakest signals

 

9.        We really did not do a consistent IMD3 test with a strong signal beside a very weak signal as that was impossible to do with N6KI continually changing frequencies.

Dennis did record audio albeit with a hand held recorder so you can see hear the audio and judge for yourselves

OTHER SHOOT OUTS

BY CONTRAST  We did a shoot out at WA3IHV between has IC-7700 (which has a Better receiver than the IC-7800) and his Flex 6700

We normalized for all parameters and just tested a single parameter - Reception of JT-65

Results    -    Weakest Signal Received

       IC -7000        -19 dB S/N

       FLEX 6700    -26 dB S/N

Clearly the 6700 is a better digital receiver… Why because with DAX – everything stays in the digital domain for digital communications.. While the 7700 needed an external audio to digital to audio conversion device (Rigblaster) which introduces about 6 dB of losses and distortion into the decoding process

 

 

 

REBUTTAL FROM DENNIS N6KI

 

At 11:21 PM 10/26/2014, Dennis Vernacchia wrote:

Howard's FLEX 6700 didn't live up to his expectations/exagerations,  so, well, in his normal Hot Air spewing manor, Howard attempts to invalidate the test results and discredit me for the PRACTICAL, NEVER INTENDED TO BE
SCIENTIFIC, TEST that was done during the SPLATTERFEST of all contests that occurs annually, The CQWW SSB WORLDWIDE CONTEST !

NO "SCIENTIFIC" TYPE TEST WAS EVER PROMISED TO BE DONE at the onset of my clearly explained proposition to Howard,.....but, ...Howard,  sporting early Dimentia, was repeatedly told this fact and that the test would just be a very practical listening test using reasonanly good external Berringer Audio equipment and mostly the defacto standard Heil headsets that the majority of hams all over the world utilize for DXing and Contesting
( We all know that most rag chewers, like Howard, use Speakers )
 
Howard arrived late as usual and with the help of **** K6KAL and John W6JBR, spent just over an hour reassembling Howard's FLEX station.  So when Howard was happy with setup and was out of my way, I then came in and got the K3, KX3 and ICOM IC-7800 set up. Did I have some technical difficulty for a few moments, well yes, some of my contest club Ops had mucked up some of the dozens of Configuration Menus of my K3 and IC-7800 unlike Howard's ready to go configured FLEX, which he alone diddles at home for hours on end
trying to make the vapor-ware driven radio work like some of our top notch, BUT, ACCORDING TO
Dr. White, "OBSOLETE LEGACY RADIO S"

Howard or I then tuned in the weakest SSB station we could find on whatever frequency band was most active at the moment, insuring the maximum amount of QRM from adjacent stations.

I told Howard to tune that station in and tune any interference out with all the resources available to him with his FLEX 6700 -

 After Howard was satisfied that he had done everything in his FLEX's system to make that weak station understandable by anyone who was on site and desired to listen in,
( We had four sets of headsets available, 3 Heils and whatever brand Howard brought for his challenged hearing condition.),.... I then tuned in the same station on a K3, KX3 and an ICOM IC-7800 and quickly determined that the K3 consistantly won out over the KX3 and IC-7800
In it's ability to hear a weak station with QRM and QRN,

Receive audio to all the radiod under test was controlled with the use of a Berringer audio control panel/mixer and a Berringer 1 to 4 output variable amplitude Headset Splitter to account for impedance difference in any of the headsets and a very well isolated  antenna splitter so the Flex and K3 always received simultaneously with the same antenna.


Well, the FLEX, at best, could equal the K3's signal strength and but the signal clarity or brilliance , just wasn't to be attained by the FLEX, meaning that the FLEX had a slightly hollow muffled sounding audio, most likely due to "over-processing" when Howard would make adjustments to the FLEX.

It turned out the K3 had the best clarity 90 percent of the time for test stations signals that Howard had "tuned" in, so,.... feeling sorry knowing that Howard was not really someone one could call a "proficient" radio operator,..... I and sometimes John W6JBR (EX WA2OOB who is a retired Hollywood Movie industry Audio Engineer with 75 IMDB credits and 30+ years as an excellent amateur operator) one of my best contest SSB trainees and Dave N6EEG, one of best SSB contest Ops with over 5 years SSB contest operating experience at the Nash-Ville test site, either directed Howard to make adjustments to the signal on the FLEX which could not be clearly tuned in and understood by the above mentioned experienced Ops , at times actually put our hands on a external optional knob so we could adjust the FLEX to at least, audio-wise, have sime brilliance or clarity for the audio heard in our head sets.

Now, was this a "SCIENTIFIC TEST"???, **** NO!
IT WAS NEVER MEANT TO BE A SCIENTIFIC TEST !

WE CAME, WE LISTENED, WE CONQUERED,
......ERRRR, I meant,.....The K3 CONQUERED.

Anyone who would like audio clips of what we all heard is invited to email me and decide for yourself
what was heard by the radios.

N6KI

 

 

MY REPLY TO DENNIS N6KI

 

At 10:33 AM 10/27/2014, Dr Howard S White wrote:

 

I would love to repeat the test WITHOUT DENNIS but using Neutral Observers under strictly controlled Scientific conditions so that we can have reproducible results.   

 

On Friday I was up at Nashville several HOURS before the test.  The setup time on the Flex involved carrying the boxes in, plugging them in and it was working.  Total elapsed time -  about. 10 minutes. Setup was instantaneous as the Flex visual interface is very quick and easy to use.

 

Dennis admits that his K3 took hours of fiddling to get right. 

 

The phase noise, (background hiss) was significantly lower on the 6700 which makes for much less operator fatigue and causes signals to jump out with more clarity but we made no measurements of relative phase noise.   Perhaps Dennis did not want to measure this as the Flex easily won, but more likely he is so used to having background static that he was unaware that Modern Radios are quieter 

 

The. 6700 clearly had more flexible, more powerful Brick wall filters but we made no measurements of relative effectiveness of the filters.  Perhaps Dennis did not want to measure this as the Flex easily won, but more likely he is so used to having to buy multiple expensive analog filters that barely work that he was unaware that Modern Radios have infinite numbers of FREE BRICK WALL filters. 

 

 

Basically the N6KI Shoot Out came down to a subjective observation of audio "Briliance" as the only difference in radio performance.   This is an incredibly subjective measure as it is totally dependent on the hearing range characteristics of the observer,

 

As I said, I could barely make out signals on the K3 that were perfectly clear on my 6700 and the same was true. Vice versa with Dennis. Each of us  had optimized our radios to our own hearing characteristics.  So to me the K3 lacked "Brilliance" while to Dennis the 6700 lacked brilliance.  Of course, with Dennis constantly fiddling and changing frequencies there was no way to collect reproducible results. 

 

A more objective test would be to (remove Dennis) have neutral observers adjust the K3 and the 6700 themselves to optimize each radio for his particular hearing characteristics and to the same bandwidths and filters and to the EXACT same frequency,   This would normalize the results so that we were actually comparing a single variable. Then have each of at least 3 different neutral observers rate the comparative audio.  

 

This would go a long way towards producing reproducible results

 

We should have put Audio Spectrum Analyzers on the outputs so we can see what really is happening with the audio

 

That way we could eliminate the subjective nature of the audio reports.

 

BUT then Dennis did not want to do anything “Scientific”

I might note that last year Ernest W4EG subjected himself to a N6KI CQ WW SSB Shootout with his 6700.    The conclusion this year by Dennis N6KI is that the 6700 was now a much better  receiver and the only difference seemed to be subjective "Brilliance" compared to the K3

 

The Debate will likely continue for a while

George KF2T

Laughing hard enough to cry! Now, THAT'S Ham Radio.

KY6LA_Howard

if you think this one was funny you would be in stitches if you read the thread fight between Dennis and I when we tested Yamaha Propane powered Generators vs Honda Gas Powered Generators.

Ken - NM9P

LOL! LOL! ROTFL!
You guys sound like my in-laws.  They will argue about almost anything just to get a rise out of the other!
But it is all in fun!  most of the time....

You guys must be a hoot to watch in action.

Ken - NM9P

KY6LA_Howard

I posted some videos on Facebook...

Yes we love to fight... been doing that for at least 20 years..

Gerald-K5SDR

Fun read!  

If you actually had NR on the whole time, that would explain the audio.  NR is use a LMS adaptive filter that will always "color" and reduce the "brightness" of the signal.  I always prefer a properly set AGC-T.  

By the way, I ran the contest with my filter set with the low cut set at 250 Hz.  That makes a huge improvement in intelligibility for my ears.  I had the high cut set to 2800 Hz.  I could even run narrower without a problem if needed.

Ernest - W4EG

As you noted Howard,

I had to relied on Tim and you to setup the radio. I barely knew how to turn the radio on/off using SmartSDR v1.0 original release.

Dennis and I agree that the KX3 with its filter was the winner; and a tie between the K3 and the 6700.  

The 6700 was handicapped, due to the AGC, NB, NR, and other receiver adjustment still not implemented in v1.0. And no memory retention. 

For those of you that don't know Howard and Dennis; I like to invite you to the San Diego DX Club meetings to see these two in actions!

Is mutual love and admiration after they agree to disagree.  

Sergey R5AU

Wow, fantastic post Howard, thank you for precise work

DrTeeth

I just cannot wait until the next episode, hi hi.

Get a webcam in the San Diego radio club STAT!

Barry N1EU

Just about any two radios will perform as well in the splatter fest that is ssb contesting.  Ergonomics are the main issue.

CW contest would be a much more discerning battle field.

Reggie

I disagree.  Test a Flex 6300 radio against a Kenwood TS-2000 in a SSB contest, and you will see and hear the amazing performance of the Flex.

KY6LA_Howard

John NA6L Wrote  10/28/14  12:20PM

All,

Every one of you knows 10x more than me about the deeper aspects of HAM radio but I’d still like to share my thoughts.

 

Quantification will be very difficult because:

 

1 .We all hear differently.

2. The tested radios will each have unique audio properties from the Mfg.

3. What may sound best to one operator/listener may not work for another operator using the same settings

4. The champion of one brand of radio may be “slightly” biased toward his favorite mark.

5. Each set of headphones has its own audio characteristics.

6. Many top end radios are going to be very close in performance (especially the Flex 6000s and K3) and once adjusted for optimum performance can any impartial observer actually “hear” a 3%, 5% or even a 10% difference?  I guess that you could add instrumentation but we are actually interested in what we hear.  

7. There is a considerable amount of operator skill involved to adjust the radios to find the sweet spot to allow hearing the desired station by reducing the splatter from the nearby stations – RX filter choice and adjustment, shift adjustment, RX audio settings, noise reduction  etc.

 

The challenge with #7. is having “judges” quantify what the control operator is hearing at the control operator’s best settings and possibly  not the best audio settings for the “judges”.   For example – my least expensive radio (Yaesu FT-950 – about the cost of the filters in my K3) has a wonderful control (knob) called “contour” that allows “perturbation” of the I.F. filter passband that will quickly peak or null specific audio frequencies, very useful for emphasizing the desired audio and/or diminishing the splatter audio (several of the other Yaesu models also have this control).  Note that the best “Contour” settings for my ears and my headphones may sound like **** to another listener.

 

Just to toss this on the wall and see what sticks, my initial thoughts/suggestions for the test would be:

 

DOUBLE BLIND TEST

Use a bank of judges using a double blind environment, fully separated from the radios and operators (listen from another room).

 

When the judges see a wall mounted  green light come on they score the “green” radio and when they see the red light come on they score the “red” radio – this would be done at least a few dozen times during a variety of conditions.  Additional “blindness” could be added by swapping the red and green radio light color assignment at intervals (could be random) that would be documented and be correlated later -- IOW, for one 30 min period the K3 would be green and the Flex red and then during another 30 minute interval the light assignments could be swapped (K3 red and Flex green) – once again this would be switched and only known and documented by a neutral observer -- the control officer.

 Other colors could be added for additional radios.      

 The person who sets up and operates the red/green system could not be part of the judging although the switching gear could also be blinded.   

 

If the test procedure could be blind enough and the judges given enough instruction to operate the tested radios during splatter conditions, everyone could be an operator and also be a judge. 

Think of the potential here, Howard KY6LA and Dennis N6KI blindly judging the K3 and Flex?

Because there would be a “salad” of operators and judges there should be a very good sample of data that could be crunched into some useful metrics. 

 

Regards, John NA6L

KY6LA_Howard

Great point John

 

I really love your idea of the double blind test..

It would mean that we could actually remove Dennis' random fiddling from the test

And Likely have reproducible results which you could take to the bank

 

Overall the tests this week proved that the K3 and 6700 are pretty close and easily the top of the Ham Radio Food chain.

I suspect that you are 100% correct that operational difference were really more due to operator settings than due to specific rig characteristics

 

I would amend your test by having the same operator optimize the K3 and 6700 to his particular hearing

Then run a blind test

Then have a different Operator set them both up and run the tests again

This would normalize for the different hearing characteristics of the different setup operators

And give the judges different setups to listen to

Ken - NM9P

I Agree.  
But I don't quite understand the report of "No Brilliance" on the 6700?  
I can understand that statement if the NR was ON and the RX EQ was not active.  But on my 6500 I can add so much "brilliance" with the RX EQ that it makes my ears hurt!  
The RX EQ, when adjusted properly, along with judicious shifting of the RX filter bandwidth and adjustment of the AGC-T, often made the difference between a copyable signal and NO COPY on a crowded or noisy band.

Ken - NM9P

KY6LA_Howard

I suspect that I had the NR on for part of the test - I finally noticed it by looking at the K6TU iPad App.. as NR is hidden in a drop down menu

Frankly I could hear much better on the 6700 than on the K3 but that was because I had optimized the 6700 for my hearing characteristics and the K3 was optimized for N6KI

Which is why i suggest that the same operator set up both radios for a test then the next operator set up the radios for a test

Ken - NM9P

I just re-read this thread and had another good laugh.  If I remember correctly, this test was made with SSDR V.1.0?  Three years later, I wonder how things would sort out in a refined test with the latest firmware in each radio?  

Perhaps you should wait and have a shoot-out with a Flex-6600M, KX3S, IC-7851 or IC-7610, and Kenwood TS-990 or something else from Yaesu or Kenwood.  That would be an interesting match-up....

Ken - NM9P

Michael Coslo

Howard - if you ever set up another test, I would be willing to assist in any way that I could. I am a first class pain in the backside, but I understand how to conduct testing using defined protocols. And sticking to them. Oustide of Amateur radio, I deal a lot with relational databases.

KY6LA_Howard

Maybe again for 2017 CQ WW SSB But Michael I would need someone with a pair of handcuffs to tie down N6KI so he can't constantly change parameters during the shootout or we will not have reproducable results. In the interim the IC7800 is gone, the K3's have been upgraded to K3S, a Maestro has been added to the 6700 to improve Contest Ergonomics. We tested and discarded an IC7851, The IC-7300 was too easily overloaded to be used in Contests, acquired an Elad FM_Duo for future testing, and we all gave up on Kenwood and Yaesu years ago.

Mark WS7M

Howard... You guys are buddies to the end I think!