Full nyc16 power outage

We have just been informed that due to an auxiliary electrical failure, power systems have failed at our nyc16 facility at 121 Varick St in New York City.  All New York servers and services are now offline and we have no ETA for service restoration.  We will provide updates as we have them.

Most systems should be configured to power on when power service resumes and we can assist via ticket as needed when power becomes available. We will, of course, honor full SLA credits for the period(s) in which power has been unavailable in the facility. We apologize for any inconvenience this has caused you or your business.

The following information was provided by our facilities vendor at 121 Varick St, known upstream as LGA4:

"The auxiliary power for our LGA4 Network Operations Center has gone out. LGA4 data center is will be going down. NOC responses will be delayed as other elements need to be contained to ensure the safety of our crew as we endure the effects of Hurricane Sandy. We are in the process of relocating our team to our near by LGA1 data center.

At this point it is encouraged that customers who have the ability to, remotely shut down any critical equipment or services to mitigate the risk of any interruptions to service. Our staff is evacuating the site and remote hands services will be suspended until the threat passes. Customer access is also restricted until further notice.

We will report back as new information becomes available about the condition of our Network Operations Center and Internet Data Center at 121 Varick St."

If you need assistance or have any questions, please contact support@steadfast.net or visit our helpdesk.

Update: October 30 2012, 2:13 PM CDT

The nyc16 datacenter currently has no electricity service, including supply to the elevator and interior lighting.  The Fire Department of New York is currently working to clear the area of water and debris, at which point work will begin to restore utility electricity service and generator service. 

We have had a few customers ask about physical access to the nyc16 facility. We are unable to grant access at this time, and access will be delayed at least until lighting and power is restored. 

Update: October 30 2012, 7:54 PM CDT

Damage evaluation at the nyc16 datacenter has located an Automatic Transfer Switch (ATS) in need of repair. Crews are scheduled to be dispatched within the next 24 hours to begin work to repair the switch, and begin procedures to restore power to the facility.

Update: October 31 2012, 4:07 PM CDT

Evaluation of the generator at the facility has concluded that the generator is not repairable.  A temporary generator has been secured, and expected to be delivered tomorrow (November 1 2012).

Updates will continue to be published as they become available.

Update: November 1 2012, 9:38 AM CDT

A temporary rollup generator has been secured and is in transit to the site. ETA 4 hours. Once the rollup generator arrives electrical contractors will have to work to tie in the required power bus. ETA for complete restoration is currently unknown. 

Update: November 1, 2012 4:42 PM CDT

The temporary generator has been installed, and temporary cooling is in place.  Critical systems are beginning to come back online. Please open a ticket with support@steadfast.net, or by submitting a ticket on our website, if your system has not yet started to come back online.

Updates will continue to be published as they become available.