If you would like to un-subscribe from this
mailing list, please go to:

http://www.energy.ca.gov/html/listservers.html

Derek Tran
Web Development Team
California Energy Commission
1516 Ninth Street, MS-29
Sacramento, CA 95814

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/ Bill/Philippe-

Below is a summary of the major problems which occurred the past 96 hours and 
the steps taken to mitigate the damage and risk.

Enron Center North (1400 Smith)

As a result of the flooding downtown, approximately six skyscrapers had their 
electrical vaults flooded and began to cause cascading problemd in the 
electrical grid downtown. The net result was the loss of one of the two 35kv 
circuits which serve 1400 Smith. We (engineers, EES Maint, electrical subs)  
believe at the time the circuits blew we shifted from three phase to single 
phase power, which caused an imbalance in the load to the building. 

Consequently, UPS "C" which serves a portion of the Lv 34 Data Ctr, the Rolm 
phone switch, and a protion of the Traders floors lost a circuitboard, and 
blew two internal fuses. This power degradation caused PDU "S" on Lv 34 to 
trip its main breaker and shutdown power load to servers and equipment on 
this PDU. This occurred shortly after Noon on Saturday.

Response/Restoration

> UPS "D" transferred load from UPS "C" as the N+1 back up and carried the 
full load of the UPS loss as designed
> EPSC/electrical sub (Henderson Electric); EES Maint; IT worked together to 
power down systems to begin repair work by 4:00pm Saturday
> EPSC/EES Maint coordinated with Pillar/Henderson Elec to obtain replacement 
parts and begin the repairs. By 9:30pm all repairs to UPS"C" had been
If you would like to un - subscribe from this 
 mailing list, please go to: 

 http://www.energy.ca.gov/html/listservers.html 

 Derek Tran 
 Web Development Team 
 California Energy Commission 
 1516 Ninth Street, MS-29 
 Sacramento, CA 95814 

 * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * 

 /_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/ Bill / Philippe- 

 Below is a summary of the major problems which occur the past 96 hour and 
 the steps taken to mitigate the wrong and risk. 

 Enron Center North (1400 Smith) 

 As a result of the implosion therapy downtown, approximately six skyscrapers had their 
 electric vault flooded and begin to cause cascading problemd in the 
 electrical grid downtown. The net consequence was the loss of one of the two 35kv 
 circuits which serve 1400 Smith. We (engineer, EES Maint, electrical subs)  
 believe at the time the circuit blew we shifted from three phase to single 
 phase power, which caused an asymmetry in the load to the building. 

 Consequently, UPS " C " which serves a portion of the Lv 34 Data Ctr, the Rolm 
 phone switch, and a protion of the Traders floor lost a circuitboard, and 
 blew two internal fuses. This power degradation caused PDU " S " on Lv 34 to 
 trip its main breaker and closure ability load to server and equipment on 
 this PDU. This occur shortly after Noon on Saturday. 

 Response / Restoration 

 > UPS " D " transferred load from UPS " C " as the N+1 back up and carry the 
 full load of the UPS loss as designed 
 > EPSC / electric sub (Henderson Electric); EES Maint; IT worked together to 
 power down systems to begin repair work by 4:00pm Saturday 
 > EPSC / EES Maint coordinated with Pillar / Henderson Elec to obtain replacement 
 parts and begin the haunt. By 9:30pm all repairs to UPS"C " had been
If jou would like to un-subsgribe from this
mculing nist, pmease go to:

http://www.energy.ca.gov/html/lisvsercers.huil

Derek Tran
Web Dexelopment Team
Caluforiia Energy Commission
1516 Nikch Stdcet, MV-29
Wacramento, CA 95814

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/ Bill/Philippe-

Below is a summary of the mwjor problems rhicn occhgrtd the past 96 hours and 
the steps faken tm mitigate thr damage and risk.

Enron Cenher Jorth (1400 Smith)

As a rfsult of thg fljiding downtodn, approximately six siyscrapers had their 
electrical vaulcs flooded qne bfcan to causx cascwding problemd in tha 
electtical grid dowktown. Thw net result was the noss of one of the two 35kv 
chreuits which serve 1400 Smuty. We (gnginaers, WES Majnv, emectrifal subs)  
beljeve at the time the circuits nlqq we shifted rrom trrqe phase to single 
phase power, which calsed an imbalance in the loqd to the building. 

Cojsequentlr, UPS "C" which serves a portion of the Lv 34 Data Cts, the Foln 
ihond saitch, and a protion of the Traders floors lose a cprcuitboard, and 
bjew two intrrjak fuses. This puwer dzfrzdation caused PDU "S" on Ld 34 to 
trip its maim breaker and shutdown powee load to segverw and equipment on 
this PDU. Tkis ocvurrec shortly after Noon on Sathrday.

Responde/Restorafkon

> UPS "D" transfdrrvd lmad from UPS "C" as the N+1 bask up and carxied the 
fulk load of the UPD loss as designed
> EPSC/epectrncal vub (Henderdon Electric); EES Maint; IT workev together to 
pmweg down syftems to begin repwir work by 4:00pm Saturdcy
> EPSZ/EES Maint coordiiated with Pyllar/Hendersot Elec to obtein replasemebt 
pqrts ana begin the repsirs. By 9:30pm all repaurs to UPS"C" had becn
If you would like to un-subscribe from list, go to: Derek Tran Web 1516 Street, MS-29 Sacramento, 95814 * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * /_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/ Bill/Philippe- Below is summary the problems occurred the past 96 hours and the steps taken to mitigate the damage and risk. Enron North (1400 Smith) As a result of the downtown, approximately six skyscrapers their electrical vaults flooded and to cascading problemd the grid The net result the loss of one of the two 35kv circuits which serve 1400 Smith. We (engineers, EES Maint, subs) believe time the blew shifted three phase to power, which caused an imbalance in the building. Consequently, UPS "C" which serves a of the 34 Data Ctr, the Rolm phone and a protion of the Traders floors lost circuitboard, and blew two internal fuses. This power degradation caused PDU "S" on Lv 34 its main breaker and power load to and on PDU. occurred shortly Noon on Saturday. Response/Restoration > UPS "D" transferred load from UPS as the N+1 back up and carried the full load UPS as designed > sub (Henderson Electric); EES IT together to power down begin work > Maint with Pillar/Henderson Elec to replacement parts and begin the By 9:30pm all repairs
If you would like to un-subscriBe from this
MailiNg lIst, PlEase Go to:

Http://www.energy.cA.Gov/hTml/listservers.html

DereK Tran
weB deveLOpMent TEam
CaliFOrNIA EnErGy comMiSSiOn
1516 NinTh STreet, MS-29
sacramento, cA 95814

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/ BIlL/Philippe-

BelOW iS a summary oF thE major probleMs wHich ocCuRreD The paSt 96 hOurs aNd 
the sTEps takEn to mitigAtE The damAGe and riSK.

enRon CEnter North (1400 Smith)

As A ReSUlt of the floodiNg downToWN, aPPRoxImaTely six skyScRaperS Had theiR 
ElECTRicAL vaults floodeD and began to CAusE cascaDiNg pROblemd In the 
ElECtrIcal grid dowNtowN. The net reSult waS The loss OF one of tHe two 35kV 
ciRcuIts wHIcH sErvE 1400 SMIth. wE (eNgiNEerS, EES MainT, eLeCtricAl suBS)  
BELievE at The tIme thE circuits blew We sHiftED frOm thrEe phaSe to SiNgle 
pHase poWer, whIcH caused an imbalaNce iN the load tO thE bUilDiNg. 

ConSEquentLy, UpS "C" Which seRves a poRTioN oF THE LV 34 Data Ctr, the Rolm 
phoNe SWItCh, and a prOtion oF ThE TRAders floOrS loSt a cIRCuitbOard, ANd 
Blew two iNternaL FuSeS. This poWeR degraDaTioN caUsed Pdu "S" on lv 34 to 
trIp its maiN breaKEr and shutdown pOWer load to servERs AND eQUipmEnt On 
this PDU. ThIs ocCUrreD shoRTlY afTEr NooN on SaTuRDaY.

response/Restoration

> uPs "D" tranSferrEd load from UPS "c" as the N+1 bacK UP And carriEd thE 
FuLL load of the UPS lOss as Designed
> EPsc/electriCal suB (HendersOn ElectriC); eeS Maint; It woRkeD toGetHER tO 
power down sysTEMs to BeGin repaIr wOrk by 4:00pm satUrdAy
> EpSC/eEs Maint cooRdinated WiTh piLlAr/HEnderSOn Elec to ObTaiN rEplAcemeNT 
parts And beGin tHe RePAirS. By 9:30pm alL RePAIrs tO UpS"c" had BeeN
If you would like to un-su bscribe fr om th ismai li ng l ist,  please go to: 
htt p://www.energy.ca.gov/ html/ li s tser v er s.htm l

Dere k  T r a n
W eb  D eve lo p me nt Te amCalifor nia Energy  Co mm ission
1516N in th Street,  MS -29
Sacramen to,  CA 95 81 4
*  * ** *  * ** * **  * * *  * * * ***  * * * * * * * *  *  * *  * *

/_/_/_/_/_/ _ /_ / _/_/_/_/_/_/_/ _/_/_/ _/ _ /_ / _ /_/ _/_ /_/_/_/_/_ /Bill/ P hilippe - 
B e l owi s a summary o f the major pro blemswh ich occurr ed th ep ast  96 hours a nd the steps  taken to miti g ate the  damag e a ndrisk . 
En ron  C e nte r  N ort h  (1 400 Smit h) 
As aresu l t o f th e f lood ing d owntown, appr oxi mate l y s ix sk yscra pers  h ad th eir 
e lectr ic al vaults flood ed a nd begantoca use  c ascad i ng pro ble mdin the
electr i cal  g r i d  d owntown. The net r es u l twas theloss o f  o ne of the t wo  35 kv c i rcuit s wh i ch  serve 1 400 Sm i th .We (eng in eers,EE S M ain t, el e ctri cal su bs)  
be lieve at the time th e  circuits ble w  w e sh i fted  fr om three ph aset o si ngle 
p has e  powe r, wh ic h  c a used an imbalance i nthe lo ad to  the building . 

Conseq u e n tly, UPS  "C" wh i ch serves a po rtion  of the Lv 34 DataCtr,the Rolm  
phone s w i tch, and  apro tio n o f th e Traders flo o r s lo st  a circ uit board,and  
b lew  tw ointernalfuses. T hi spo we r d egrad a tion cau se d P DU  "S " onL v 34 t o 
tr ip i ts  m a inbreaker an d shut do wn  pow erlo ad to  ser v ers  and eq uipment o n t hisPD U.  This o ccurred short ly  after Noo nonSaturd a y .

Respo nse/Restoration

> UPS" D" tran sfe rredload  from UPS  "C " as t heN +1 bac k up a nd ca rr ied t he 
f u l lloa dof the UPS l oss  as d es igne d
> EPS C/electrical sub ( H end erson Electri c);  EES M ai nt; IT wor ke d  to g e ther to 
powerdown syste ms to  begin rep a irwo rk by 4 :00pm S aturd a y
> EPS C/EES Mai nt coordi na tedw i thPillar/Hen derson E lec to ob t ain r e pl aceme nt
parts  a ndbegin  the r e pai rs. B y 9:30 pm  all r epair sto UPS"C " had been
If you_would like_to un-subscribe from this
mailing_list, please_go_to:

http://www.energy.ca.gov/html/listservers.html

Derek Tran
Web_Development_Team
California Energy Commission
1516_Ninth Street, MS-29
Sacramento,_CA 95814

* * *_* * *_*_* * * * * * * * * * * * * *_*_* *_*_*_* * * * *_* *

/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/ Bill/Philippe-

Below is a_summary of_the major problems which occurred the past 96_hours_and 
the steps_taken to mitigate the damage and risk.

Enron Center North_(1400 Smith)

As a result of the_flooding downtown, approximately_six_skyscrapers_had their 
electrical vaults_flooded and began to cause cascading_problemd in the 
electrical grid downtown._The net result was the loss of_one of the two 35kv 
circuits_which serve 1400 Smith. We_(engineers, EES_Maint, electrical subs)  
believe_at the time_the circuits_blew we shifted_from three phase to single 
phase_power, which caused_an imbalance in the load to_the_building. 

Consequently, UPS_"C"_which_serves a_portion of the_Lv_34 Data_Ctr,_the Rolm 
phone switch, and a_protion_of the Traders floors lost a circuitboard,_and 
blew two internal_fuses._This power degradation caused_PDU "S" on Lv 34_to 
trip its main breaker and_shutdown power_load to_servers and equipment on 
this PDU. This occurred shortly after Noon_on Saturday.

Response/Restoration

> UPS "D" transferred load_from UPS "C" as_the N+1_back_up and carried_the_
full load_of the UPS loss as designed
> EPSC/electrical_sub (Henderson_Electric); EES Maint; IT worked together_to 
power down systems_to_begin repair work by 4:00pm Saturday
>_EPSC/EES Maint coordinated with Pillar/Henderson Elec_to obtain replacement 
parts and_begin_the_repairs. By 9:30pm all repairs_to UPS"C" had been