VPN degredation on 2.3.7

Jim Gilliam

VPN degradation on 2.3.7 When I run 2.3.7 remotely on a VPN I  get at  best two yellow bars. I down graded back to 2.2.8, I get 5 green bars. It appears the newer version might be hogging more bandwidth than the previous version.

Jim, K6QE

Mike-VA3MW

Hi Jim

The bars don't really tell the entire story, so take those with a grain of salt.  Do you notice any change in your operating?

Also, do you notice a change in your network latency numbers?  

Mike

Jim Gilliam

The latency numbers are a function of the number of bars showing, so yes. However, I noticed no degradation in service. I thought this might be worth pointing out if there is a latency problem.

Tim - W4TME

The new version (2.3.7) no longer fragments the large datagrams used for spectrum display, audio and metering (VITA-49 data) to resolve the issue where ISPs or the endpoint routers violate RFC 791 and do not allow for packet fragments to pass (no spectrum display while using SmartLink). The radio now breaks up the large datagram into individual 1500 byte frames and the client has to reassemble them rather than allowing the TCP stack to do it.  This takes more computational resource overhead on the radio and the client (the TCP/IP stack did this very efficiently).

If you are running a VPN, then there is a possibility that the packets exceed your network path MTU once the encapsulation (wrapper) is done on the VITA-49 data.  I would look at your VPN endpoints and see what their performance metric tell you between using 2.2.8 (the old method) and 2.3.7 the new one.

The Network Health indicator shows VITA-49 packets the client expects to receive from the radio and if one is missing, it shows it as a drop, but as Michael indicates, that is not necessarily a bad thing operationally if you are not hearing gaps on the audio or seeing display frames not refreshing (blank display). 

Jim Gilliam

Thank you, Tim.

Strangely, when I get one red I am getting a latency of 34 ms on the VPN. This is about what I was getting before when I had 4 to five green bars on the previous version. When I switch to Smarlink I get an indication or three yellow bars but my latency reads 116 ms. There doesn't seem to be a correspondence to the number of bars showing and the latency.

Jim, K6QE

Marc Lalonde

Same  thing here i just try for the third time to run 2.3.7  but again fail 
start whit Triangle on display , then got panadapter but run in red whit > 25% packet loss

this time i do a cold reboot before and after update  and not better

VPN  is SoftEther  runing on Small Atom Win10 PC on remote site

Timo - OH5KW

After installing 2.3.7 my Win10 PC via Softether VPN has had SSDR same  kind of problems.  Waterfall works ok, but the Panadapter is updating once per second or so :-(
I would like have this fixed.
 
Any way, I can use my radio on digital modes via VPN on the remote PC with this hybrid setup:
-First I start SmartSDR without VPN, using SmartLink.
-When SSDR is running. I open VPN tunnel to the radio. Immediately DAX and other clients connect to the radio normally. SSDR keeps running via SmartLink!

No need to downgrade, I think.

73, Timo OH5KW

Burch - K4QXX

I run DogparkSDR via VPN and since the upgrade, the Panadater and waterfall are jerky and not smooth anymore.  They both used to be as smooth as when I was using the computer local.  Doesn't seem to affect the audio, just the display.

Burch

K4QXX

Mario_DL3LSM

Hi Burch,

I think it's normal behaviour that a GUI client released before 2.3.7 will not work with 2.3.7 out of the box because of the new way fragmentation is handled for the panadapter and waterfall streams. There is a beta version on the Dogpark website which is tested with 2.3.7. Have you tried this version?

73, Mario DL3LSM

Burch - K4QXX

Yes, I am using the latest beta.  Version 1.13 beta 15.  

Marc Lalonde

Hi Tim  it not clear to me  , first you said 2.3.7 no longer  fragments the large datagrams   , then you said it  Radio now breaks up the large datagram into individual 1500 byte frames ??????   this look like fragmetation to me  ?

need to have clear direction for find and solve the issue or setting to do on VPN or else?  i quite nood in VPN ,for now i just downgrade  ,but likely that this new method remain for future 
and honestly the lack of persistence  of 2.2.8 really **** me off  since i run near exclusively on transverter ,so it very bug me and will for sure not stay forever whit 2.2.8

also >27% packet loos  IS a bad thing if you operate remote , operate legal limit and said no problem it safe ,i just have bit less that 75% of the control of my radio ;-)
i have multiple level of runaway condition safety but prefer to not use it ..
for be more serious @ 27% this start to be choppy and cause issue 

and not see yet solution for that even if i not the only one to report issue, and remember the remote operation is key selling point of your radio  ,this must work .

so may great to have some how-to or else for fix that ;-)

W1IMD

Tim, I too am running Softether and seeing over 30% dropped packets. When I do not run the VPN my packet loss is 0%. Could you give us network challenged folks an idea as to what we should be looking at to correct this? Thanks!

Tim - W4TME

This is detailed networking.  A single datagram that will render one frame of the panadapter can and is most likely more than 1500 bytes.  The client software needs the entire datagram before it can render one frame.

We used to send the entire datagram through the UDP socket and because it was larger than 1500 bytes (actually the payload is a little smaller) the TCP/IP stack broke the datagram into fragments as per RFC 791 and the receiving host's TCP/IP stack put them together into the large datagram and passed them up the stack to the application for rending.

Since a lot of Internet providers now violate RFC 791 and do not allow for fragmented packets as per RFC 791, we now split up the datagram into to individual chunks of data that will fit in a single Ethernet frame and send those to the client, where it has to do the reassembly in user mode rather than in the more efficient kernel mode where the TCP/IP stack runs.

Marc Lalonde

ok good an clear now ;-) , still need to figure reason it fail on VPN ?
if worked before this mean that VPN / Router and else is able to pass 1500 and more datagram , so normally it it now 1500 or less it must work ??

did VPN  may **** the packet and make it hard to application to reassemble back  together ?    , i knot that fragmented packet is now mostly see as suspicious since many use that for exploit  and DOS Attack 

as usual thank for the QSO ;-)

Tim - W4TME

we are investigating why the performance degradation when using a VPN.  We do not have any answers yet.

Matt NQ6N

After installing 2.3.7 I noticed that the connection quality indicator in SmartSDR would typically be orange or red, rather than green as it had been in the previous version. 

I do not notice any difference in the actual performance of SmartSDR however, so I assumed that perhaps the bars were "re-normed" so that only a direct LAN connection or similar would earn a green "very good" status. 

I am seeing 0.26% packet loss at present over my SoftEther VPN with 25ms RTT average and worst case 165ms.

Two quick product suggestions that I think are relevant to this issue which would be very helpful:

Suggestion 1: Improved network latency "histogram":  Instead of just showing real time latency and Max Latency, show the max latency for several different rolling time windows. I suggest real time, 1 minute, 5 minute, and 30 minute.  If it shows 120ms as the max latency in the 30 minute window but say 40ms in the 1 and 5 minute windows, that is not cause for concern, but if it routinely shows 120ms max in the 1 minute window that indicates a much more unstable network. 

Suggestion 2: Create a "latency health check" view that measures all aspects of the system latency including DSP filter skirt settings, PC audio hardware, network latency, VPN latency, etc., that are relevant to SmartSDR performing optimally.  This would help those setting up remote stations to figure out where to focus attention.  This could even be a command line script that prints information to the console. I suggest this since I suspect Flex has already created tooling to measure many of these one way or another. In digging into this in my own station, it seems that consumer grade internet is the major culprit, but given the amount of time I've invested so far (not much) it has been difficult to get tot the bottom of which components of the internet link need attention. 

One question about packet fragmentation that I've been wondering about in reading this:  Is the approach to packet fragmentation adopted by IPV6 impactful for this kind of issue? Is SmartSDR likely to perform better or worse (all else being equal) on a mediocre connection over IPV4 or IPV6?  Why? 

One other question since others in this thread use SoftEther:  Is anfyone using traffic shaping on their VPN interface? I recently set up SQM on the tap_soft interface (bridged to the LAN interface at the remote site) and now the buffer bloat results are much improved, but I can't tell if the setting ought to improve SmartSDR performance.  

73,
Matt NQ6N