Flex 6700, K3, KX3 IC-7800 CQ WWDX SSB Shoot Out

FLEX 6700
- K3 - KX3 - IC-7800 Shoot – Out  Report  - CQ WW DX SSB 2014

Nov 24-25, 2014

Background:

Dennis N6KI and I have been friends for years.  We fight over everything and anything.  I am usually on the Bleeding Edge of Technology and Dennis usually takes the most conservative possible positon. I am very technology oriented and would rather try something new each time even if it may sometimes disappoint while Dennis prefers the old standard tried and true.  Dennis is the ultimate world class contestor placing near or at the top in every contest.. while I occasionally do well in a contest (and win a couple), I would rather do it a different NEW way each time.   My approach to testing tends to be a rather detailed scientific approach to try to produce reproducible results while Dennis prefers a random unstructured approach to see if it feels right for contesting.    Do not take our ranting at each other below seriously,.... We are still friends...

CQ WW DX SSB SHOOT OUT:

Dennis conceived of the idea that we should test modern radios against legacy radios during the 2014 CQ WWDX SSB contest because it was a world class splatterfest of the worst possible conditions.

We decided to use the NX6T Contest Station Site in Fallbrook CA –  I was part of Dennis team up there when we came first in NA in the JIDX Contest and second in the world ... it’s a great site on a hill 900’ above the surrounding terrain...with no obstructions in all directions about 20 miles east of the Pacific Ocean.

Quadcopter shots by Paul NN6X

https://drive.google.com/folderview?id=0B0fUPinob3F4bG5ubGMyREJtQzg&usp=sharing

            The site is rural and pretty quiet from an RF point of view

            The intent was to test the following rigs in contest conditions

1.      Flex 6700

2.      K3

3.      KX3

4.      IC-7800

FIRST SHOOT OUT REPORT FROM KY6LA

Albert Einstein said that the definition of Insanity is to repeat the same experiment and expect different results

I have participated in a number of Shootouts with Dennis N6KI including SteppIR, Honda vs Yamaha Generators and a number of different Legacy vs Modern Radio Shootouts

All these shootouts invariably start the same way.    Those of us with Scientific backgrounds try to set up the experiments with detailed documentation processes, normalization of as many variables as possible so we are comparing apples to apples and a detailed WRITTEN Plan of Experimentation so that we can have Reproducible Results. 

Enter Dennis, N6KI, the King of Irreproducible Results.... who demands total control of the observations

He immediately throws out all the carefully and exacting setups that had normalized the variables to a single consistent set.   He then continues to fiddle with parameters throughout the experimental time so that there is absolutely no documentation and no way to produce objective reproducible results.  We invariably end up with a mish mash of subjective observations colored by the personal biases of the various different observers and absolutely no way to reproduce anything.

The last two days were no different..  than every other previous N6KI Shootout

No written plan of Action, No experimental procedure established.   No coordination of Normalization of Parameters No objective observers and no way to produce reproducible results.  Everyone frustrated with Dennis.

John W6JBR the audio engineer and I spent a few hours on Friday normalizing audio between receivers.  

On Saturday Dave N6EEG and I tried to do the same.  

Invariable N6KI started to do the comparative testing by immediately grabbing the tuning knob on his K3 to continually change frequencies and adjust parameters throughout the experiment.  After trying and failing to get N6KI to follow any reproducible protocol, I gave up in frustration....

I spent most of my time trying to manually track N6KI's random frequency gyrations so that we could all be on the same frequency for tests... ultimately I suggested that we connect the CI-V ports on the radios so that they would all track together.. BUT N6KI would not accept that - rather he said we should just manually tune so that they all sound "ABOUT" the same... which of course was impossible because N6KI continued to fiddle with the tuning knob...

My FIRST CONCLUSION

I will NEVER participate in another "Shootout" UNLESS N6KI is either totally banned from the proceedings or N6KI has his hands tied behind his back so that he can’t continuously and randomly change parameters all the time..

Results:

1.        Surprisingly the Flex with its visual interface was by far the easiest/quickest radio to set up and the least likely to screw up during a contest

 (Albeit I had accidently turned on the NR on the 6700 for much of the experiment and did not notice that until the end because NR was one of the few parameter on a hidden drop down menu on the 6700)

2.        The IC-7800 was so outperformed by the K3, KX3, and 6700 that we quickly dropped it from the testing rotation

3.        Likewise we did not spend too much time with the KX3.. it performed well (better than the 7800) but lacked sophisticated filtering

4.        All radios hear the very weakest signals - as you would expect since the atmospheric noise floor on 20M is about -115dBm and the radios    can hear down to -130 dBm or better

5.        The Notch filter on the IC-7800 and the 6700 were better than the K3/KX3 at removing a CW Signal in the SSB Bandpass

6.        ONLY the 6700 could produce more than one notch filter at a time so it was the only radio to be able to remove more than 1 CW signal from a SSB Bandpass-  the 6700 had the most powerful and flexible filters and filtering

7.        The 6700 had the lowest Phase Noise (the background static you hear jumping around because of the splatter)  followed by the KX3, K3 and finally the IC-7800

This is consistent with the expectations of the different technologies 

- lower phase noise = lower operator fatigue

8.        We ultimately came down to comparing received audio.. but that is highly subjective as different people hear very differently. 

a.       For example I had my 6700's equalizer peaked for my hearing... so while I could hear well on it, I had trouble hearing on the K3 as it was not peaked for my hearing. 

b.      Dennis had his K3 peaked for his hearing so I could not hear well on his K3 but he said my 6700 sound lacked    "Brilliance" - albeit that might have been due to the fact that the NR was on...

c.       With these subjective hearing tests.. the K3, KX3 and 6700 could all hear the very weakest ones and could find them inside a splatter zone.

d.      the background noise was easily the lowest on the 6700 and for my hearing was the easiest to discern the content of the weakest signals

9.        We really did not do a consistent IMD3 test with a strong signal beside a very weak signal as that was impossible to do with N6KI continually changing frequencies.

Dennis did record audio albeit with a hand held recorder so you can see hear the audio and judge for yourselves

OTHER SHOOT OUTS

BY CONTRAST  We did a shoot out at WA3IHV between has IC-7700 (which has a Better receiver than the IC-7800) and his Flex 6700

We normalized for all parameters and just tested a single parameter - Reception of JT-65

Results    -    Weakest Signal Received

       IC -7000        -19 dB S/N

       FLEX 6700    -26 dB S/N

Clearly the 6700 is a better digital receiver... Why because with DAX – everything stays in the digital domain for digital communications.. While the 7700 needed an external audio to digital to audio conversion device (Rigblaster) which introduces about 6 dB of losses and distortion into the decoding process

REBUTTAL FROM DENNIS N6KI

At 11:21 PM 10/26/2014, Dennis Vernacchia wrote:

Howard's FLEX 6700 didn't live up to his expectations/exagerations,  so, well, in his normal Hot Air spewing manor, Howard attempts to invalidate the test results and discredit me for the PRACTICAL, NEVER INTENDED TO BE
SCIENTIFIC, TEST that was done during the SPLATTERFEST of all contests that occurs annually, The CQWW SSB WORLDWIDE CONTEST !

NO "SCIENTIFIC" TYPE TEST WAS EVER PROMISED TO BE DONE at the onset of my clearly explained proposition to Howard,.....but, ...Howard,  sporting early Dimentia, was repeatedly told this fact and that the test would just be a very practical listening test using reasonanly good external Berringer Audio equipment and mostly the defacto standard Heil headsets that the majority of hams all over the world utilize for DXing and Contesting
( We all know that most rag chewers, like Howard, use Speakers )
 
Howard arrived late as usual and with the help of Dick K6KAL and John W6JBR, spent just over an hour reassembling Howard's FLEX station.  So when Howard was happy with setup and was out of my way, I then came in and got the K3, KX3 and ICOM IC-7800 set up. Did I have some technical difficulty for a few moments, well yes, some of my contest club Ops had mucked up some of the dozens of Configuration Menus of my K3 and IC-7800 unlike Howard's ready to go configured FLEX, which he alone diddles at home for hours on end
trying to make the vapor-ware driven radio work like some of our top notch, BUT, ACCORDING TO
Dr. White, "OBSOLETE LEGACY RADIO S"

Howard or I then tuned in the weakest SSB station we could find on whatever frequency band was most active at the moment, insuring the maximum amount of QRM from adjacent stations.

I told Howard to tune that station in and tune any interference out with all the resources available to him with his FLEX 6700 -

 After Howard was satisfied that he had done everything in his FLEX's system to make that weak station understandable by anyone who was on site and desired to listen in,
( We had four sets of headsets available, 3 Heils and whatever brand Howard brought for his challenged hearing condition.),.... I then tuned in the same station on a K3, KX3 and an ICOM IC-7800 and quickly determined that the K3 consistantly won out over the KX3 and IC-7800
In it's ability to hear a weak station with QRM and QRN,

Receive audio to all the radiod under test was controlled with the use of a Berringer audio control panel/mixer and a Berringer 1 to 4 output variable amplitude Headset Splitter to account for impedance difference in any of the headsets and a very well isolated  antenna splitter so the Flex and K3 always received simultaneously with the same antenna.


Well, the FLEX, at best, could equal the K3's signal strength and but the signal clarity or brilliance , just wasn't to be attained by the FLEX, meaning that the FLEX had a slightly hollow muffled sounding audio, most likely due to "over-processing" when Howard would make adjustments to the FLEX.

It turned out the K3 had the best clarity 90 percent of the time for test stations signals that Howard had "tuned" in, so,.... feeling sorry knowing that Howard was not really someone one could call a "proficient" radio operator,..... I and sometimes John W6JBR (EX WA2OOB who is a retired Hollywood Movie industry Audio Engineer with 75 IMDB credits and 30+ years as an excellent amateur operator) one of my best contest SSB trainees and Dave N6EEG, one of best SSB contest Ops with over 5 years SSB contest operating experience at the Nash-Ville test site, either directed Howard to make adjustments to the signal on the FLEX which could not be clearly tuned in and understood by the above mentioned experienced Ops , at times actually put our hands on a external optional knob so we could adjust the FLEX to at least, audio-wise, have sime brilliance or clarity for the audio heard in our head sets.

Now, was this a "SCIENTIFIC TEST"???, HELL NO!
IT WAS NEVER MEANT TO BE A SCIENTIFIC TEST !

WE CAME, WE LISTENED, WE CONQUERED,
......ERRRR, I meant,.....The K3 CONQUERED.

Anyone who would like audio clips of what we all heard is invited to email me and decide for yourself
what was heard by the radios.

N6KI

MY REPLY TO DENNIS N6KI

At 10:33 AM 10/27/2014, Dr Howard S White wrote:

I would love to repeat the test WITHOUT DENNIS but using Neutral Observers under strictly controlled Scientific conditions so that we can have reproducible results.   

On Friday I was up at Nashville several HOURS before the test.  The setup time on the Flex involved carrying the boxes in, plugging them in and it was working.  Total elapsed time -  about. 10 minutes. Setup was instantaneous as the Flex visual interface is very quick and easy to use.

Dennis admits that his K3 took hours of fiddling to get right. 

The phase noise, (background hiss) was significantly lower on the 6700 which makes for much less operator fatigue and causes signals to jump out with more clarity but we made no measurements of relative phase noise.   Perhaps Dennis did not want to measure this as the Flex easily won, but more likely he is so used to having background static that he was unaware that Modern Radios are quieter 

The. 6700 clearly had more flexible, more powerful Brick wall filters but we made no measurements of relative effectiveness of the filters.  Perhaps Dennis did not want to measure this as the Flex easily won, but more likely he is so used to having to buy multiple expensive analog filters that barely work that he was unaware that Modern Radios have infinite numbers of FREE BRICK WALL filters. 

Basically the N6KI Shoot Out came down to a subjective observation of audio "Briliance" as the only difference in radio performance.   This is an incredibly subjective measure as it is totally dependent on the hearing range characteristics of the observer,

As I said, I could barely make out signals on the K3 that were perfectly clear on my 6700 and the same was true. Vice versa with Dennis. Each of us  had optimized our radios to our own hearing characteristics.  So to me the K3 lacked "Brilliance" while to Dennis the 6700 lacked brilliance.  Of course, with Dennis constantly fiddling and changing frequencies there was no way to collect reproducible results. 

A more objective test would be to (remove Dennis) have neutral observers adjust the K3 and the 6700 themselves to optimize each radio for his particular hearing characteristics and to the same bandwidths and filters and to the EXACT same frequency,   This would normalize the results so that we were actually comparing a single variable. Then have each of at least 3 different neutral observers rate the comparative audio.  

This would go a long way towards producing reproducible results

We should have put Audio Spectrum Analyzers on the outputs so we can see what really is happening with the audio

That way we could eliminate the subjective nature of the audio reports.

BUT then Dennis did not want to do anything “Scientific”

I might note that last year Ernest W4EG subjected himself to a N6KI CQ WW SSB Shootout with his 6700.    The conclusion this year by Dennis N6KI is that the 6700 was now a much better  receiver and the only difference seemed to be subjective "Brilliance" compared to the K3

The Debate will likely continue for a while