Pure CW - its all about that tone, bout that tone, no noise

Al_NN4ZZ

What is the viability of a CW mode that analyzes the signal which includes noise, hiss, etc and detects the desired CW signal, then triggers a side tone output?    In other words, the software would look for the CW signal but not send it and the other noise to the audio output.  Instead of trying to minimize the noise with audio peaking, RX equalization, and noise reduction, it would eliminate it completely.  In this "pure CW" mode, the operator would only hear a clean CW side tone and zero noise.   
 
If the "pure CW" mode could detect weak CW signals where noise is more of an issue and humans have a harder time hearing the signal,  that would be huge.   Armchair copy on even the weakest of signals. 

There must be some significant challenge I'm overlooking since no one has done this as far as I know.    (on the other hand, sometimes when you don't know that something can't be done, you just do it anyway).

Regards, Al / NN4ZZ  
al (at) nn4zz (dot) com

p.s.  Apologies to Meghan Trainor  and 'All About That Bass' for ripping off the tittle. 

Sergey R5AU

Al, nice idea, but you know too many or/if in case realization, I think may be better to have extremely linear receiver with Hi IMD in passband. I know you doubts, ADC in 6700 has ENOB below 13. Let's see how the 1.5 will looks like with noise mitigation

Al_NN4ZZ

Hi Sergey,
I often see weak CW signals down at the noise floor and figured if I can see them and they are rendered in the display then maybe with the processing capability of the FPGA they could also be detected and drive a side tone.  Traditional radios and even many other SDRs don't have the processing power to consider this.  

At this point my first interest is to see whether FRS and others think this is even viable.  If we could get solid copy on those very weak signals, it would be a game changer for DXers.

Regards, Al / NN4ZZ  
al (at) nn4zz (dot) com

  

Sergey R5AU

i am with you Al, the issue is only - pileup 
For clear air no doubts, CW sign detectable easily , necessary just define the passband for detection  (10z or 50hz or...)
Nevertheless i VOTE for this of course , such functionality should be switchable in any way.

Ken - NM9P

Intriguing idea.  My "Like" in this case is not necessarily a vote to do it, but certainly a vote to explore the possibility!  BTW, I have seen a software program, (perhaps CW Get?) that attempted to do this, but its decode algorithm couldn't cut it.

W7NGA

I like noise ...

Doug Hall

Al,
I've always been intrigued by this idea. I built such a regenerator circuit in the late 70s using an LM567 as a tone decoder driving a 555 for the regenerated CW. It was interesting, and it worked, but it required a strong, relatively noise-free CW signal, so for me it was little more than a novelty. I resurrected the project in the 90s when I was making DSP noise filters, using one of the boxes as a hardware platform for a DSP-based CW regenerator. It was better than my old hardware-based circuit, but I still never managed to get anything I thought was marketable. It started falling apart at the higher speeds, and while it made for nice copy with signals like W1AW it didn't improve my ability to copy CW signals. It's like having an AM or FM signal on the ragged edge of the squelch - ultimately I'd rather have the noisy signal and not miss anything.

The DSP in the Flex is an order of magnitude faster (well, really nearly two orders) than the DSP I was using, so perhaps this is feasible now. Personally (unless it's just dead simple) there are other things I'd rather see the MIPS and memory and programmer resources used for.

73,
Doug K4DSP

Al_NN4ZZ

Hi Doug,
Agree, the FPGA has so much more power, it seems like it could work for even very weak signals.  I doubt this idea would get any significant priority given all of the other work there is to be done.   So I'm mainly looking to see if the FRS team agrees it is feasible and if have they ever considered it before.

I suppose that if it was truly a capability that set them apart from the rest of the "old technology" radios, it would have some real user and marketing benefit.   So a vote for the idea doesn't mean you want it soon, just that you would like to know if it's possible and then for FRS to prioritize it appropriately.

Regards, Al / NN4ZZ  
al (at) nn4zz (dot) com

Tom Warren

Al,

   I think it's a great idea and added my vote to the list.  I would think this, or something similar, might qualify as part of the  'science project' mentioned for part of v1.5

Tom W4TMW

Joe, KQ1Q

Re a "pure CW" mode which could detect weak CW signals where noise is more of an issue and humans have a harder time hearing the signal, this would essentially be a CW decoder which then re-encodes the audio signal instead of printing it out. Programs like this exist and of course Elecraft can decode CW. Some programs like MRP40 supposedly do well in high noise environments, but I haven't used it.

Conceptually the request is something like MRP40 but re-written for state-of-the-art computational hardware and which re-encodes the signal for human audio interpretation, incorporated as a feature of the Flex 6000 series.

The core problem is software decoding of CW in a weak signal, high noise environment. It's true the Flex 6000 DSP is powerful but that's just hardware -- gigaflops and GMACS. The latest Intel CPUs rival or exceed this, although they burn a lot more power doing it. The i7-4770K is benchmarked at 182 Linpack gigaflops if using AVX2 vector instructions, and the E5-2687W v3 (although a server CPU) does nearly a teraflop. 

If CPU horsepower alone could produce revolutionary results in CW decoding, someone could write a program for this and run it on a high-end PC. The fact this hasn't been done implies there's a diminishing law of returns and additional CPU/DSP resources may not benefit greatly. Unlike PSK31, JT65, etc, CW decoding cannot mandate timing  and envelope characteristics of the transmitted signal -- they are sent by humans. So these attributes cannot be used to aid decoding.

But the real problem is the software development required and evaluating the cost/benefit ratio of that. I think given limited development resources, so far FRS is prioritizing other areas.

There are radios that have CW-specific, adjustable noise reduction algorithms, e.g, TS-590S. Maybe the NR work FRS is doing on SSDR 1.5 could achieve some of the goal via NR instead of CW decoding/re-encoding.

Al_NN4ZZ

Hi Joe,
My idea wasn't to actually decode the CW.    Rather it was to just use the CW signal (regardless of how well or poorly timed the elements and spacing are) to trigger a side tone.  The idea is that by triggering a side tone instead of presenting the original audio, the CW we hear would be free of any noise.  And even a very weak signal would be presented as a strong clear CW tone.  The operator would still do the decoding.  

This is not to say that decoding is a bad idea, but it does have all the additional challenges you mentioned.

The FPGA provides a tremendous increase in processing power as compared to a PC so that is what I was hoping to get some feedback on from FRS.    Given I can see a weak CW signal on the display, it seems like it could be feasible. 

Regards, Al / NN4ZZ  
al (at) nn4zz (dot) com 

rfoust

I like the title - made me giggle. :-)

Joe, KQ1Q

Al, although you described it differently, I think achieving this in a broadly useful way would equate to either decoding the CW or more advanced CW-specific NR.

IOW the rising and falling edge of each element must be detected and somehow separated from the noise -- without injecting error or artifacts. If that isn't done properly, noise characteristics would simply be impressed on the side tone.

The side tone method is conceptually like using tracing paper to re-draw an underlying diagram.  If the source is noisy the final result will also be. It's true that unique characteristics might distinguish the signal from the background noise (like a line drawing on dirty paper). But either sophisticated human or software judgment is required as that signal is traced, moment by moment to produce the output.

The magic which makes the noise really go away is some algorithmic process such as NR (auto-correlation, spectral subtraction, etc.), or decoding the source which also involves noise processing to separate out the signal. If making a perfectly clear CW signal in a weak signal, high noise environment was simple as triggering a sidetone, I don't think the radio manufacturers would be spending so much effort on NR.

Re FPGA processing, as I mentioned the Virtex parts used in the 6000 aren't any faster than a state-of-the-art CPU. However they are much lower cost and consume far less power, so that's why high-end general purpose CPUs are not typically used in embedded applications. I'm just not sure any amount of computation (whether FPGA or CPU) would make a big difference.

Anyway that's just my impression, maybe FRS will comment.

Al_NN4ZZ

Joe, Thanks for your input.  Makes sense, will be interested in the FRS feedback.   If anyone can do it, they are a good bet. Regards, Al / NN4ZZ

Carl K5HK

Al,  This idea was used years ago in marine autoalarms and other receivers to some good effect depending on the level & type of the noise and how it keyed the audio tone.  I saw it years ago using tube technology very limited "digital processing" of sorts.   Suspect the right flexible algorhythums usings modern digital technics and processors could do a lot better job.  34 year merchant mariner here as Radio Electronics Officer so listened in to plenty of good static over all those years not to mention my ham activities before, during and after.

73, Carl / K5HK

Mark Erbaugh

I remember when I saw my first demo of the sdr1000. The filter was narrow and

Ed - W2RF

CW Skimmer attempts to read the CW out of the noise and actually decode it. That's one step past a sidetone!

But in my experience CWS begins to fail at about the same point as my ability to copy by ear.

73 Ed W2RF

Sergey R5AU

Ed, this is exactly what i mean, you know Algorithms VS True Detection VS Latency will be big issue, however basic algorithm is very close to the very narrow peak filter with further Freq detector. In any way it will be intersting "to talk to know" here different opinions regarding True CW and of course FRS feedback is appreciated

Ed - W2RF

On the bright side Skimmer can identify and decode many closely spaced CW signals simultaneously. Adding this type of functionality to the Flex 6000 would form the core of built in skimming.

Sergey R5AU

Well, not a fantasy but close to reality is signal reconstruct for RTTY too, it will be True RTTY also.P.S. in case realization of the Forward algorithms (for RX )simular algorithms can be used for transmission - in this case will be like direct CW keying or FSK RTTY. For me still open question: "How to maximize copy of the desired signal
 and eliminate unwanted signals ?" You know , how this is algorithms will run in the hash of the pile-up, looks intersting. 

Ed - W2RF

I'm not sure many Flexers are aware that both CW and RTTY skimmer support for the Flex are available today.

SDR-Bridge includes a driver that enables both CW Skimmer Server and RTTY Skimmer Server to connect with DAX IQ channels. Up to four bands can be skimmed simultaneously.

It's installed automatically with Bridge, and is fairly simple to get working. In case of problems I'm available for help.

73 Ed W2RF

Chris WX7V

I use CWSkimmer and SDR Bridge myself - I like the ability to use multiple decodes but wish there was an squelch like adjustment similar to FLDIGI that can be used to reduce some of the decode clutter and random characters that otherwise show up when using the raw text mode.

Any tips on this?

Thanks

Mark_WS7M

In our company we have one of the worlds leading FPGA developers. He is so good that Cyclone and others come to him for advice.

Our instrument uses multiple FPGAs to examine high speed laser reflection data off of cells. I showed him Smart SDR and the Flex and he thought it was cool but felt the FPGA is being under utilized in what it is doing.

Not being an FGPA developer myself I cannot say but knowing what he does with our firehose of data it is pretty amazing. So I would venture to say there is a lot of room to do more in the FPGA but it takes real skill to realize it properly.

Pete K6TJ

[{"insert":"It would be useful to send a CW sidetone to the remote flex radio, have such a decoder that would key the transmitter. Since there's little noise with the incoming signal, it could be quite fast perhaps. Has anyone tried this?\nIt's reverse from the rest of this thread. This tread works on receive, this is a way to send CW remotely for transmit\n"}]