The DREADED SINGLE POINT OF FAILURE HAS STRUCK.
I have backups and UPS on everything as well as multiple servers etc. BUT my single point of failure is a single Internet connection with a single modem and single router on the critical path. In this case, I am pretty sure it is the one UPS that powers the modem and router that failed. Likely the battery had gone which is an easy fix.
My most obvious solution is to add a second Internet line, modem and router and run a second subnet. Possible but I am time limited before we head back to France next week so I don't think I can get that working before I leave
Anyone have any bright ideas as to how I might eliminate the single point of failure?
Last year I had a similar problem where my Internet went down while waiting a the airport to go on a two week cruise. Blah!! When I finally got home there was no apparent failure except I could not connect to the Internet. I had to power-off/power-on my modem before the ISP would recognize my router. So, apparently someone was screwing around at the Central office. Since then, I have put my modem on my remotable powerline strip should that happen again. Also, when I went through the power cycle of my modem, my I/P address changed and had to had to adjust my Dyn.com DNS service to accommodate the change.
This is also a possible Single point of failure that would render a rig totally unusable without human or electro-mechanical intervention.
As far as your question - If you want true internet fail-safe diversity, you might wish to obtain a second internet connection from a different provider/means. i.e. if your normal connection is by Cable Modem, then have a DSL line installed for backup, so that it would require two different companies to lose service at the same time.
If I were in a communications-critical situation, I would pursue that further. But it is much too expensive for my small-time operation. I don't even want to pay for 3Mb upload speed yet!
Ken - NM9P
I would use UPS on the router only if I am at the location. It could be convenient to have internet access in case of a power outage. Some routers will allow you to have a 4g dongle connected and revert to a 4g connection if the main WAN fails.
I would however, only use a surge protector and not UPS if I am not there. That doesn't cover the eventuality of needing an internet/router reset without having internet access. That is a very tricky one.
I have access to neighbors WiFi as back up. I use Fiber and they use ADSL so we are on different connections and providers. I use these tiny https://www.itead.cc/sonoff-wifi-wireless-switch.html wifi switches that will allow me to restart devices. I have the router connected to 2 WiFi switches in series one on my neighbors WiFi and one with my own WiFi.
My first thought though is that these tiny wifi switches have the option of a Default to ON and also a timer ON-OFF capability. This means that you could setup your internet router switch to shutdown and restart once a week. That means it will reset itself every so often regardless of status. This is just a thought, not thrilled about rebooting the router every week... but if you are going to be gone without anyone with physical access that might be the way to go.
Advanced remote switches/routers have a feature where if they don't respond to a ping request several times it will reboot. Not cheap though.
Some other things I have setup on my remote operation:
- All the PCs will power back on in case of a power failure. This is a BIOS setting and not the default one. I also do not Halt on errors.
- Make sure your Ethernet and wifi adapters do not have energy saving feature on. It will turn them off.
- I have 3 PCs in the same LAN which I can access remotely. I keep them on but with no monitors and on lowest power consumption setting. Each one is on a different segments of the network. One directly to one of the internet router ports, one on wifi and one on the main gigabit switch that distributes internet in the home.
- I have 2 systems to log into the devices, Teamviewer (with 2 step verification) and Chrome Remote.
- I can also SSH into a linux box (a Raspberry Pi).
- I can access the internet router from the outside.
- I have cameras looking into radioshack room and server room.
- I have smoke sensors and water sensors on the floor.
- I rely heavily on direct wire connections, do not trust WiFi, so I have ON OFF switches that are ethernet connected to the LAN.
- In some cases, if I am not going to be in the location for a while I will run and ethernet cable on the ground from one location to another instead of relying on WiFi.
This could be a great thread, I am certainly interested in hearing other ideas.
As I have been running a remote base for over 10 years (2006), my modem was my first SPoF. I called the ISP and they said: "Your modem is in a state - just reboot it". Yeah, right. A 3 hr winter drive away.
The perfect solution is another ISP (cable or DSL). However, in my case, I immediately added a Christmas tree timer to power cycle the router/modem every morning at 3am. It is down for about 2 minutes. Since then, I've been pretty good.
Lately, my new modem has a network port that disappears. I proved this and called the cable company. They said "No one has reported this, so that isn't it". I said "I am not surprised as the first thing you tell everyone is to reboot everything and therefor the problem goes away." The rep said: "oh" :)
I used the KISS principle.
When you come home next, have the tower and beams installed and you will have a 2nd station for emergency use. Will work good for the next S. Cal earthquake. And might be a real low-noise location !
Always fun installing a new station from scratch.
One thing you might wish to try is a second ISP, and using DD-WRT on your primary router. DD-WRT will allow you to connect multiple WAN sources, designate one as a primary, and use the secondary to fail over to. This feature is typically only found on much more expensive enterprise-grade routers for business. Just a thought...
I would love to have a software button to turn off the radio remotely. As my only option now is to turn the power supply off which is probably not a good practice to turn off the radio.
What? You don't have an OC-192 SONET ring installed at your home QTH with parallel backed inverters fed from a Telco +48V distribution system? Shame on you. :-)
Boy you're right on the remote complexities. N4CC and I operate a remote station on the edge of the Okefenokee Swap. Great QTH into Asia from FL since the site looks down into a north-south river valley.
I purchased a used telecom shelter from Sprint Cellular. It has redundant HVAC, genset input, TVSS surge suppression, and security/environmental monitoring all left over from the Sprint folks.
A rack-mounted Digital Loggers webswitch automatically pings various devices on the network in search of life. If none, the webswitch initiates a hard boot of the crashed device. BIOS is also set on shack PC to re-start after a power failure and at 5am as a Hail Mary if necessary. Very few 1 hour long trips to the site. Between VNC and TeamViwer, just about every problem can be reset remotely. A telco-operated AC power switch is at the site but we haven't had the telco line installed. That's a great way to restart a DSL/Cable modem when it can't be pinged from WAN. Next, we're looking to add high power Wi-Fi transceivers for connection to another site about 7 miles away for a redundant ISP. The main tower is 140 ft AGL and at that height, there's good line of sight into Jacksonville.
Curiouser and Curiouser
Flew back to L J tonite...
It was not the UPS.. or the Modem - IT was the ROUTER.
I did the usual disconnect the router wait 1 minute and repowered it again but it did not work.
Swapped in one of my spare older slower Routers... that worked OK but no longer running 1GB and none of my fixed IP or static routes were there...
Fortunately I have lots of backups.....as well as up to date..screen shot documentation ... but it took only a few seconds of slower operations to convince me that I needed a top of the line router..
So I raced to Costco before they closed and picked up a NetGear R7900 (AC3000) Nighthawk, Why that one.. it was the only one they had.. and Fry's was add an extra hour total. And Yes I could have had a R8000 from Amazon by t0AM tomorrow.. But I need thinks to run 100% tonite...
As a lark I put back the original failed router into the network..
Lo and Behold after being off for an hour or so, it powered up and everything is working 100%
So what the heck caused the failure???????
as the power reset did nothing a couple of hours ago
Either way there is no way I can afford to leave a potential single point of failure in my network especially since we will be out of the country until Oct. so the R7900 will become the primary router.
Long experience has taught me that every disaster has a positive result.
First: I now have a Spare Router (the original one) that is already programmed with my network configurations. By just simply changing it's IP and primary gateway address I can easily connect it into my system and have access via a second ISP through it...
I have done that before in commercial systems so I know that it easy to do and works..
Now if I can just get a second vendor's ISP connection installed before I leave I will have a totally redundant system...
Second: The R7900 Supports OPENVPN so I now will have a second VPN Network connection available...
AND SOME PEOPLE THINK RELIABLE REMOTE OPERATIONS IS EASY!!
METHINKS I SPOKE TOO SOON!!!
It turns out that neither the Netgear R7900 or R8000 support the OpenVPN Client for iOS (R7000 does)
So for all intents and purposes both those Netgear Routers are USELESS for SmartSDR for IOS.
Not being one to spend money on obsolete equipment like the R7000 its back to the drawing board for me...
It means a lot more work to load tables but IIRC the ASUS Routers doe support OpenVPN
OTOH my SoftEther VPN has been most reliable but once I got on the redundancy kick it became apparent to that I need a second VPN as well..
Something like this one:
I usually buy from Ali Express from commercial sellers. Some, but not all of these products make it to eBay.
I'm currently remote from my 6500 at home. I have an R7000 router at home in which I have loaded VPN settings for both IOS and Windows. It is OpenVPn that runs as a server on the Router., as you know I can connect from this remote location using VPN to the 6500 at home using either my IPad with Ssdr iOS or my Windows laptop with Ssdr. Both can used at my choosing from this location. The VPN server in the router takes care of the interfacing. The R7000 is a great Router and hasn't fallen down too far on the obsolescence curve. The VPN settings are provided by Netgear and the router setup takes 5 minutes. They also have settings for Android devices. Nice.I am not proficient in IT, but the setup was as easy as it could possibly be. The nice added feature is the security to get into my NAS at home for files , securely. I kow you would like some performance data,; all I can say is that I have 50MMPS down at 10 MMPS up at home and at my remote location. SSDR connections are no problem and at high quality . I run fast Network on the IOS app.
- 5028 Conversations
- 1525 Followers
- 2964 Conversations
- 616 Followers
- 3487 Conversations
- 897 Followers
- 936 Conversations
- 392 Followers
- 838 Conversations
- 147 Followers
- 2874 Conversations
- 825 Followers
- 908 Conversations
- 247 Followers
- 1153 Conversations
- 286 Followers
- 406 Conversations
- 125 Followers