The URL can be used to link to this page

Home My WebLink AboutOES 28-08 REPORT TO EXECUTIVE COMMITTEE Report Number: OES 28-08 Date: October 14, 2008 '1 ~-_.__.~-~_.~----"-~_.._---,._----_._._-_.._-~----~~-.------+----- From: Everett Buntsma Director, Operations & Emergency Services Subject: Information Technology (IT) Server Room Ventilation and Air Conditioning, Civic Complex File: A-1440 Recommendation: 1. That Report OES 28-08 regarding the Ventilation and Air Conditioning (HVAC) requirements for the Information Technology (IT) Server Room in the Civic Complex be received; 2. That staff be directed to proceed with a formal tender process for the provision of ventilation and air conditioning in the Information Technology Server Room. Executive Summary: The IT Server Room on the second floor of the Civic Complex is experiencing serious climate control problems. Three factors were identified that necessitated the HVAC be upgraded within the computer room during 2007: the first being additional hardware being added during the past several years to accommodate network growth; the second to compensate for the warming of the Civic Complex's air conditioning system that the computer room partially depends upon; and the third to replace the roof-mounted unit that was installed approximately five years ago that is now insufficient to meet demand. The 2008 Capital Budget included a $20,000.00 allowance for the IT computer room air conditioning replacement; however, the total cost based on informal quotations received on the Consultant's design is estimated to be approximately $175,000.00, therefore, it was decided to re-submit this project in the 2009 Capital Budget and implement a temporary solution until then. However, during mid-July, a temporary unit was installed within the computer room to assist with rising temperatures, but during the night of September 4, 2008, the unit failed resulting in a complete shutdown of the computer room servers as a result of temperatures reaching 108 degrees Fahrenheit. The shutdown of the computer network resulted in the lost productivity of one-half day for most employees, and a full day for others, depending on the application system accessed. The ability of the City to provide the expected level of customer service to its residents also was impacted. As this failure could occur again at any time, staff are seeking authorization to proceed through the formal tendering process now. Report OES 28-08 October 14, 2008 Subject: IT Server Room 2 Ventilation and Air Conditioning Page 2 Financial Implications: The 2008 Capital Budget included a $20,000 allowance for IT Server Room air conditioning, but the total cost based on quotations received is estimated to be approximately $ 175,000.00. This is the realistic cost for a long-term solution. Funding for the expenditure, if approved by Council after the results of the Tender Call are known, will be provided from the 2008 Current Budget Contingency Account and/or Federal Gas Tax Funds and therefore, will not increase the total approved Budgets for 2008. Sustainability Implications: At present, the internal remedial measures being used to cool the Server Room are not energy or cost effective. If a properly designed upgrade is constructed, the climate control will once again fall within the City's Sustainability Guidelines. Adequate ventilation and ducting will reduce the energy requirement for cooling the area thus reducing the carbon footprint. In addition, the potential loss of network time is in itself not sustainable due to loss of productivity and customer service. Background: The need for upgraded ventilation and climate control in the IT Server Room, on the second floor of the Civic Complex was recognized in 2007 and budgeted for as a capital item in the 2008 Capital Budget for a total amount of $20,000.00. This budget estimate was originally based on an assumption that additional compressor units could simply be added to the roof. Engineering consultants (TSH) were commissioned to provide working sketches and specifications for the required upgrades in spring of 2008. During this process, it became evident that the required modifications would exceed the original budget estimate, as the ducting and ventilation was considered to be inadequate for a simple fix. Therefore, it was decided to defer the project until 2009, and during 2008, implement a temporary solution using a portion of the $20,000.00. As stated, the temporary solution failed causing a complete shutdown in the City's computer network. Informal quotations from a list of qualified contractors were requested in July 2008 and site meetings were arranged by staff with each of the contractors. CORP0227-07/01 revised Report OES 28-08 October 14, 2008 Subject: IT Server Room Ventilation and Air Conditioning Page 3 :3 Quotations were received from three qualified contractors and ranged from $140,000.00 to $175,000.00. These quotations were based on detailed working sketches and a site visit. Details of the actual bids are not included due to confidentiality issues. As the prices greatly exceeded budget, it was decided to defer this project to the 2009 Capital Budget. In consultation with the Manager of Information Technology, an interim solution involving portable air conditioners was agreed upon in July 2008. However, because of the ducting limitations only one air conditioner could be operating and it was the only solution available within the budget restraints. On September 4, 2008, the temporary system failed. While it is unknown how long the computer room was without air-conditioning, it is known that at 8: 1 0 am, the temperature within the computer room was 108 degrees Fahrenheit and metal components were hot to the touch. The following identifies both the known impact of this equipment failure, as well as the potential impact. Known 1. The City as a whole suffered a minimum of four and one-half hours of lost productivity, as the network was known to be inoperative from 7:00 am to 11 :30 am. 2. Most servers went into thermal mode and shut down to protect against the extreme heat. The newer servers did not shut down, but powered down the processing capacity to reduce the heat output and power requirements. 3. The server hosting the financial, budget, customer care and property systems failed to start at 11 :30 am, most likely as a result of heat damage. HP was called and according to the maintenance contract, the server was operational by 5:00 pm. Potential 1. It is quite possible the City will experience computer equipment failure(s) within the next three to five weeks. The result will be lost productivity. 2. It is possible that components of the telephone system could fail within the coming weeks due to the extreme heat experienced. If that were to occur, the City would be unable to respond to incoming telephone calls, or to place outgoing calls. A system identified as the Corporation's most important system would be unavailable for an unknown time period. CORP0227-07/01 revised Report OES 28-08 October 14, 2008 Subject: 14 IT Server Room Ventilation and Air Conditioning Page 4 3. It is possible that components of the email system could fail within the coming weeks due to the extreme heat experienced. If that were to occur, the City would be unable to send emails, and incoming emails could be lost. Staff have set up two portable air conditioning units to maintain an appropriate temperature in the Server Room. However, this solution is not effective in the long term. At present, hot air discharging from the Server Room is increasing the cooling load demand and creating a static air pressure problem in the entire Civic Complex. It is therefore recommended that staff be authorized to proceed with a formal tender process. Once completed, the results, together with financing will be presented to Council for approval in 2008. Attachments: Not applicable Prepared By: Approved/Endorsed By: /::- ) 1'/ /' / Ton(t~1 Division Head, Facilities Operations Everett unts Director, Operations & Emergency Services Prepared By: Approved/Endorsed By: /1 / j i' i' i ...---'" il 1._ ..' 1-1-- ;; . '-.....- '......- ~ <C:-f;~[?:=~ c_' Gillis A. Paterson Director, Corporate Services & Treasurer JOI},$torms Manager, Information Technology TP:mld Copy: Chief Administrative Officer Recommended for the consideration of Pickering City~ouncil /' // CORP0227 -07 /01 revised