Westin maintenance @ 2:30PM PST
Hello, I'll be doing a reboot of the Westin server @ 2:30PM PST today. This means all traffic and services there will stop for a few minutes. I'll send a second email when it's all completed. --Bart
What? No automatic failover and redundancy!!! :) On Fri, May 22, 2015 at 2:22 PM, Bart Kus <me@bartk.us> wrote:
Hello,
I'll be doing a reboot of the Westin server @ 2:30PM PST today. This means all traffic and services there will stop for a few minutes. I'll send a second email when it's all completed.
--Bart
_______________________________________________ PSDR mailing list PSDR@hamwan.org http://mail.hamwan.net/mailman/listinfo/psdr
-- ------------------------------ John D. Hays K7VE PO Box 1223, Edmonds, WA 98020-1223 <http://k7ve.org/blog> <http://twitter.com/#!/john_hays> <http://www.facebook.com/john.d.hays>
The network itself has automatic failover and redundancy. The website hamwan.org (non-redundantly hosted on the Westin server) does not. IIRC, that's the only service at Westin that is not duplicated elsewhere. Tom On Fri, May 22, 2015 at 2:34 PM, John D. Hays <john@hays.org> wrote:
What? No automatic failover and redundancy!!!
:)
On Fri, May 22, 2015 at 2:22 PM, Bart Kus <me@bartk.us> wrote:
Hello,
I'll be doing a reboot of the Westin server @ 2:30PM PST today. This means all traffic and services there will stop for a few minutes. I'll send a second email when it's all completed.
--Bart
_______________________________________________ PSDR mailing list PSDR@hamwan.org http://mail.hamwan.net/mailman/listinfo/psdr
--
------------------------------ John D. Hays K7VE
PO Box 1223, Edmonds, WA 98020-1223 <http://k7ve.org/blog> <http://twitter.com/#!/john_hays> <http://www.facebook.com/john.d.hays>
_______________________________________________ PSDR mailing list PSDR@hamwan.org http://mail.hamwan.net/mailman/listinfo/psdr
And PSRG website would be affected too. And possibly our email system? (this list) Anyways, the server's back now. Outage started 14:41 PST and ended 14:51 PST. 10 minutes, not bad. --Bart On 5/22/2015 2:36 PM, Tom Hayward wrote:
The network itself has automatic failover and redundancy. The website hamwan.org <http://hamwan.org> (non-redundantly hosted on the Westin server) does not. IIRC, that's the only service at Westin that is not duplicated elsewhere.
Tom
On Fri, May 22, 2015 at 2:34 PM, John D. Hays <john@hays.org <mailto:john@hays.org>> wrote:
What? No automatic failover and redundancy!!!
:)
On Fri, May 22, 2015 at 2:22 PM, Bart Kus <me@bartk.us <mailto:me@bartk.us>> wrote:
Hello,
I'll be doing a reboot of the Westin server @ 2:30PM PST today. This means all traffic and services there will stop for a few minutes. I'll send a second email when it's all completed.
--Bart
_______________________________________________ PSDR mailing list PSDR@hamwan.org <mailto:PSDR@hamwan.org> http://mail.hamwan.net/mailman/listinfo/psdr
--
------------------------------------------------------------------------ John D. Hays K7VE
PO Box 1223, Edmonds, WA 98020-1223 <http://k7ve.org/blog> <http://twitter.com/#%21/john_hays> <http://www.facebook.com/john.d.hays>
_______________________________________________ PSDR mailing list PSDR@hamwan.org <mailto:PSDR@hamwan.org> http://mail.hamwan.net/mailman/listinfo/psdr
_______________________________________________ PSDR mailing list PSDR@hamwan.org http://mail.hamwan.net/mailman/listinfo/psdr
You did note the smiley face? BTW, two MikroTik routers in parallel can failover nicely. On Fri, May 22, 2015 at 2:36 PM, Tom Hayward <tom@tomh.us> wrote:
The network itself has automatic failover and redundancy. The website hamwan.org (non-redundantly hosted on the Westin server) does not. IIRC, that's the only service at Westin that is not duplicated elsewhere.
Tom
On Fri, May 22, 2015 at 2:34 PM, John D. Hays <john@hays.org> wrote:
What? No automatic failover and redundancy!!!
:)
On Fri, May 22, 2015 at 2:22 PM, Bart Kus <me@bartk.us> wrote:
Hello,
I'll be doing a reboot of the Westin server @ 2:30PM PST today. This means all traffic and services there will stop for a few minutes. I'll send a second email when it's all completed.
--Bart
_______________________________________________ PSDR mailing list PSDR@hamwan.org http://mail.hamwan.net/mailman/listinfo/psdr
--
------------------------------ John D. Hays K7VE
PO Box 1223, Edmonds, WA 98020-1223 <http://k7ve.org/blog> <http://twitter.com/#!/john_hays> <http://www.facebook.com/john.d.hays>
_______________________________________________ PSDR mailing list PSDR@hamwan.org http://mail.hamwan.net/mailman/listinfo/psdr
_______________________________________________ PSDR mailing list PSDR@hamwan.org http://mail.hamwan.net/mailman/listinfo/psdr
-- ------------------------------ John D. Hays K7VE PO Box 1223, Edmonds, WA 98020-1223 <http://k7ve.org/blog> <http://twitter.com/#!/john_hays> <http://www.facebook.com/john.d.hays>
I'm just trying to share details about the network. I figure that's what people want to see here. We recently migrated Seattle-ER1 from an RB2011 to a VM on an ESXi host. This was done for a few reasons, but most importantly, crypto is much faster in the new x86 CPU than in the RB2011. This will allow us to support more VPN terminations. (I believe the RB2011 maxed out around 18 Mbps of VPN traffic.) Seattle-SRV1 is another VM on the same host. Today's outage was for a reboot of the ESXi host. Bart is more familiar with the details, but I understand the reboot was required to get the VMs to show up properly in vCenter server. This momentarily took out the two VMs, Seattle-ER1 and Seattle-SRV1. During the outage, the network re-routes around Seattle-ER1, but services only provided on Seattle-SRV1 remains inaccessible. After a few minutes, everything boots back up and service is restored. The topology at the Westin looks something like this: Seattle-ESXi Seattle-ER1 Seattle-SRV1 Seattle-QueenAnne PtP Each of these systems participates in OSPF, so if we have a failure of Seattle-ER1, Seattle-SRV1 will begin routing all traffic through Queen Anne. My guess is this will eventually make its way back to the Internet via our Tukwila datacenter. (The other possibility is exit through Corvallis via a VPN tunnel.) For VM failover, we'd need a second ESXi host and configuration for automatic failover. My preference is that we make services redundant rather than systems, so if a VM or ESXi itself fail, another system at another site continues to provide the service. This is how we have DNS configured, for instance. I bet no one noticed any downtime for DNS during this maintenance event. Unfortunately, DNS is a lot simpler to make redundant than a SQL-backed website like hawan.org. Tom On Fri, May 22, 2015 at 3:31 PM, John D. Hays <john@hays.org> wrote:
You did note the smiley face?
BTW, two MikroTik routers in parallel can failover nicely.
On Fri, May 22, 2015 at 2:36 PM, Tom Hayward <tom@tomh.us> wrote:
The network itself has automatic failover and redundancy. The website hamwan.org (non-redundantly hosted on the Westin server) does not. IIRC, that's the only service at Westin that is not duplicated elsewhere.
Tom
On Fri, May 22, 2015 at 2:34 PM, John D. Hays <john@hays.org> wrote:
What? No automatic failover and redundancy!!!
:)
On Fri, May 22, 2015 at 2:22 PM, Bart Kus <me@bartk.us> wrote:
Hello,
I'll be doing a reboot of the Westin server @ 2:30PM PST today. This means all traffic and services there will stop for a few minutes. I'll send a second email when it's all completed.
--Bart
_______________________________________________ PSDR mailing list PSDR@hamwan.org http://mail.hamwan.net/mailman/listinfo/psdr
--
------------------------------ John D. Hays K7VE
PO Box 1223, Edmonds, WA 98020-1223 <http://k7ve.org/blog> <http://twitter.com/#!/john_hays> <http://www.facebook.com/john.d.hays>
_______________________________________________ PSDR mailing list PSDR@hamwan.org http://mail.hamwan.net/mailman/listinfo/psdr
_______________________________________________ PSDR mailing list PSDR@hamwan.org http://mail.hamwan.net/mailman/listinfo/psdr
--
------------------------------ John D. Hays K7VE
PO Box 1223, Edmonds, WA 98020-1223 <http://k7ve.org/blog> <http://twitter.com/#!/john_hays> <http://www.facebook.com/john.d.hays>
_______________________________________________ PSDR mailing list PSDR@hamwan.org http://mail.hamwan.net/mailman/listinfo/psdr
participants (3)
-
Bart Kus -
John D. Hays -
Tom Hayward