From 11:25 to 11:45pm Pacific Time, a small portion of our network experienced packet loss due to a traffic flood generated by a customer’s server. This server has been terminated and all network activity is operating normally at this time. We will continue to monitor the situation closely and our technology and networking teams will be reviewing the event in full detail tomorrow.
With a winter storm affecting travel and power throughout the Northwest, we have limited support coverage at this time. If you have an issue that needs to be addressed immediately, please escalate it to urgent. Our team will respond accordingly.
Our data center is unaffected by the storm and has full Internet connectivity and power.
Today, Friday, January 13th, 2012 at 14:22 PST the primary load balancer in one of our main load balancing clusters crashed during routine maintenance. This crash resulted in the load balancer ceasing to respond to network traffic on all interfaces. Within seconds of this happening the backup load balancer for this cluster came online and began to assume its fail-over role.
This fail-over resulted in a few minutes of downtime for some sites that are balanced by this cluster and as little as a few seconds for others. Our support and technical engineers were alerted of the problem shortly after by our monitoring system and began to diagnose and document the crash. At 14:31 the primary load balancer came back online and our technicians had determined that the machine was stable. At approximately 14:36 the primary load balancer reassumed primary control over the load balancing cluster and traffic was again shifted from the backup load balancer to the primary. This resulted in a brief period of downtime as the services spun up again on the primary. At this time the load balancers are online and stable and as always we will continue to monitor them closely for any problems should they occur.
If you have any questions or notice any anomalies that cause concern please don’t hesitate to contact our Support team.
(Web Only Post)
Our monitoring system has informed us that vsh156 is experiencing
problems. This is affecting around 8 Blocks or Virtual Private
Servers on our network. We’re presently engaged in getting the
machine back up, at which point we will be migrating customers off of
it. We expect this issue to be resolved within the next 20-30
minutes.
(Web Only Post)
The link is now back up and running.
Please let us know if you are still seeing any issues.
We’re currently experiencing problems on the link between our primary site
and our nearby backup site. We are in touch with our vendor and we are
currently investigating this issue.
The only services affected by this are backups, which we expect to resume
when the link comes back up.
Please contact support via phone or through Box Panel and escalate a ticket
if you believe you have been affected by this.
Our offices will be closed Sunday and Monday, 1st and 2nd January, in
observance of the new years holiday. The office will remain closed on
Monday for general business and we will have reduced staff for support
issues and limited phone support. As always, our on-call staff will be
available throughout the holiday and weekend.
If you have an urgent issue that needs to be addressed while our offices
are closed, please make sure to open an urgent ticket or call our phone
system and leave a voicemail marking your message as urgent.
To mark your phone call as urgent, please do the following:
Dial 1-800-613-4305.
At the voice prompt, press 1.
Then press #.
Finally, press 2 and leave a voicemail.
To open an urgent support ticket, ensure you are logged into Box Panel and
visit Under the ticket body, please check the box “Open this Ticket as
Urgent”. This will escalate your ticket and bring
it to attention of our team.
Happy New Year from all of us at the Blue Box Group!
Note: None of our data center properties were affected by this issue. This only affected our office suite.
The fire alarm in our office building has triggered. Our offices will be temporarily closing until building management has resolved this issue. Inbound phone calls will not presently be answered. In the interim, our offsite teams will be responding to all alerts and escalated tickets.
To mark your phone call as urgent, please do the following:
Dial 1-800-613-4305.
At the voice prompt, press 1.
Then press #.
Finally, press 2 and leave a voicemail.
To open an urgent support ticket, ensure you are logged into Box Panel and visit http://support.bluebox.net/. Under the ticket body, please check the box “Open this Ticket as Urgent”. This will escalate your ticket and bring it to attention of our team.
We will update this page when we are back online in the office.
Update - 11:10AM Pacific - We’ve gotten the all-clear from the Fire Department and our team can now re-enter our office suite. An issue with the HVAC fan belt in our office’s HVAC unit caused the system to billow smoke and odors from vents. The system has been disabled while the HVAC company comes to determine the root cause. There may be a continued lag in our teams general response as there is still a lingering odor, so we’re working on ventilating the space. A portion of our team has relocated to alternative space to support our off site staff. We will update when all staffing is back to normal.
Update - 9:00AM Pacific - As of 2pm yesterday, all staff were back working from the office and the HVAC system was working as expected.
(Web Only Post)
