The life of a remote HF Operator

Mike-VA3MW

This is the life of remote operating. 

About 5:45 today, I go to turn on the remote HF rig.  No Go - not sure why.  

 I know it is online, etc. 

I do a remote reboot of the radio via my control panel (local to the radio).  This often clears up some port issues on the pfSense router.

Seconds later, the entire remote site disappears.  No Web cams.. nothing. 

****... what was wired incorrectly? 

Did the modem power end cord up over on the radio reboot relay?   No, that is a 100% separate control system.  Designed so that you have to actually log into a RPI to reboot the modem with a script you have to hand type in.  Nope, that isn't it.

Maybe the pfSense firewall crashed.  No simple recovery remotely there.  **** again.

Every 20 minutes, the RPI checks for Internet and reboots the Modem.  I'll wait until 6pm.  6:02pm. The router should reboot about 6:05pm if no  internet.   Nothing.  !@#$%$%^^&

I see a 90 minute drive in my future.  Each way.  In the dark dark (not a big deal for me as I can listen to the hockey game).  

Walk away from the keyboard.  Go have dinner.  Pray to the remote HF radio gods that this will recover.  

6:11pm.  Email arrives from Hydro One (power company).  There is a Hydro outage in your area.  Estimated uptime 6:30pm.  

6:34pm  Everything back online.

Moral of the story.  Sometimes the radio outage is just the symptom.  Not the fix.

Moral 2 - If it can go wrong, it will.

Mike va3mw

Alan

I see a UPS for the Raspberry and the modem/router in your future.

Alan

Mike-VA3MW

It is all UPSd. 

When the power goes out like this, so do the upstream switches on the cable network.  

The emails showed up after the power came on.  

My stuff never powered down as my power was never off.  It was a cable company outage. 

Joe - KC2TN

Auto start generator with transfer switch?

Neil D Friedman N3DF

I would open a help desk ticket and let Tim fix it.

Mike-VA3MW

That wouldn't help in this case.  The root cause is that the Power Failure in the area took out my ISP, but not my QTH.  So, if I really needed to fix this, I would have to have a 2nd ISP with a hot switch over.  At this time, it isn't worth the expense.

Mike

Steve K9ZW

Just a good reminder that if even if your remote station stays up and your are up at your user/client end, that if the infrastructure goes down between you're off the air until it is restored.

While loss of "Hydro" (Canadian speak for "Electric Power") was the root, the actual causal-agent was that Mike's ISP dropped offline when the ISP also lost power to relay sites. 

Not every part of the infrastructure that serves any of us is fully backed up.  The wrong site goes down whether through power loss or damaged cables or a fire, and the hope is that the out of service equipment can be switched around.  

In more remote areas the equipment that services the internet is not well backed up due to economics and logistics.

Good old Frontier lost the internet to my Island QTH's entire Island for four days last month (October 2019) due to heavy wind damage to infrastructure.  It happens.  Frontier's reliability is shaky to the point that I do not leave my station running unless in residence on the Island.  

Running truly remote sites in wilderness has lots of extra adventures that come your way.  Tonno ES5TV recently relayed that mice and moose have eaten outdoor cables at his Estonia QTH.  George W9EVT put in a big generator as the Island loses power so often and sometimes for days (we are near neighbors on the Island).  Folks have reported intruders, including thieves, messing with their unattended stations.  All extra adventures. 

73

Steve
K9ZW

Blog:  http://k9zw.wordpress.com  

 

Keith WA0BEU

Thanks for the info Mike, and great comments!

BL7IB

Mike,

You said rebooting "often clears up some port issues on the pfSense route". I found things are running more smoothly if I setup fixed IP and fixed port mapping.

I am using a device from msnswitch.com to auto-reboot modem and router automatically when no internet detected.

The device is almost 10 years old. Thanks for inspiring idea. I will try to use Raspberry pi and relay board to replace it.

Did you code yourself or you found some free nice program to do the job?

73

Rocky

BL7iB

Mike-VA3MW

Hi Rocky

Yes, that is an ok option. We usually don't want to recommend that to customers who do not have any networking experience since it rapidly becomes an issue with attempting to solve an issue. :)

Here is what we see that impacts SmartLink and Wifi operations:

Network Mesh installed and the Mesh is not in a bridge mode, which results in 2 unique networks

WiFi extenders that also create unique subnets (aka, they need to be in a bridge mode). We are not a fan of WiFi extenders as they make it look like you have full signal, but not any improved bandwidth.

Hardcoded port forwards that aren't updated when DHCP address on the radio changes (DHCP reservations are preferred over hard coded IP addresses for the radios)

WiFi AP's that block client to client communications.

I hope that helps and thanks for your comments! 73