Site History & Original Submission
Anglicare WA is a non-profit social services agency that depends on funding from state and federal governments, as well as donations. When I first started at Anglicare WA, the standard PC was a Pentium 133 with 16-megabytes of RAM. The standard operating system was Windows 95 and Office 97 was the office suite. Over time, it became clear that the technology needs of the agency would not be met by the hardware and software that was in place.
The IT manager at the time wrote a submission for a grant from Lotteries West. The original submission asked for approximately $650,000. The final grant amount awarded was $550,000. The original submission outlined the following issues that needed to be resolved:
- A lack of a common hardware platform. This made deployment, inventory and support more difficult.
- A lack of a common software platform. IT was required to support several versions of Windows and Office. This made user documentation and training very difficult and resulted in a different user experience between PCs.
- Aging hardware in critical areas, such as servers, required replacement.
- Capacity issues with servers, such as available drive space for user files.
- A single point of failure existed as most server functions (email, file/print, Active Directory, VPN, DNS, DHCP) existed on one server
Review
The original submission contained a number of items that were revised over time. The original submission called for all the work to be outsourced. Quotes recieved for the outsourcing ranged from $100,000 to $200,000 – a substancial amount of the grant money. Management decided to insource the project, with as much work as possible being undertaken in-house. This method would allow money to be saved and the better decisions to be made in the context of the agency’s needs.
I began this process with reviewing the original submission as some time had passed between the approval of the grant and the go-ahead to start the rollout. This meant that hardware had improved, new versions of software had been released and the needs of the agency had changed. I drafted a number of documents in response to the original submission. These documents included a revised specifications document for desktop, laptops and routers. I also drafted a document that outlined a list of IT-centric objectives the rollout should achieve. These objectives were:
- A Common Hardware Platform (CHP) based on high quality hardware. The CHP would facilitate a common user experience, reduce support requirements and allow for longer system life cycles.
- A Common Software Platform (CSP) to provide a common user experience for all staff, thus reducing training requirements, support costs and issues due to incompatibilities.
- Create a Support Infrastructure to support the investment in technology. The cornerstone of this was my insistance that all hardware have a minimum of a 3 year warranty, as well as extended and extra support options on key points of the infrastructure. An example of this was the 8×5 4 hour response time support contracts on all servers.
- Create a Robust WAN Infrastructure based on high quality hardware. In this case, Cisco routers were used.
- Build a Server Infrastructure that would last 3-5 years and allow plenty of expansion in capacity.
- Increase Agency Service Delivery through effective use of the new technology. An example was to bring email up to “carrier grade” reliaibility, so that it “just works”, like the telephone. Collaboration features in Office 2003 was another part of this goal.
In the period between the original submission and the go-head, I had been testing out various options for hardware and software, to find the best products before fully committing to them. A positive outcome of this was the changing of the original submission spec of white box desktops to HP corporate-grade desktop systems. Other key changes in hardware specifications I made included:
- In the original submission, branch office servers were “souped up” desktops (a bit more RAM, a second hard drive for RAID). I changed these to be HP Proliant ML350 servers, with 3 year warranties and support contracts.
- The original storage solution was an IDE RAID system made by Promise. The specifications of the unit revealed that throughput was a serious issue with it. After research and demonstrations by vendors, the Promise unit was replaced with a Hitachi SAN.
- The backup solution specified in the submission was a tape autoloader. This was replaced with a Quantum M1500 tape library, which had larger capacity and faster backup speeds.
- Software licenses were changed. Because of the minimal price difference, all servers were upgraded to Windows 2003 Enterprise Edition.
- Exchange 2003 Standard Edition was the original specification. Research showed that Standard Edition limits mail stores to 16-gigabytes. At the time, Anglicare WA was using 9-gigabytes. Thus, the Exchange specification was changed to Enterprise Edition, which supports 75-gigabytes for mail stores.
- The final number of servers at head office was reduced from the original 6 to 4 due to increased hardware specifications in the final purchased units.
This review period involved a lot of research and consultation with vendors. A number of documents were written, justifying the changes that were made and outlining the reasons for the changes.
Implementation
In late 2004, there was a rapid acceleration in the rate of hardware failure in the servers at head office. The main server had both drives in its boot mirror fail in a 24 hour period. A third drive in the data mirror also failed. The Terminal Server also suffered a hardware failure and had to be reinstalled from scratch. With these issues, it became clear the rollout needed to happen sooner rather than later.
In December 2004, the purchase order was written out for the first phase. This phase covered the following items:
- 4 new servers
- The Hitachi SAN
- The Quantum tape library
- 1 new HP 42RU rack
- HP switches
- 50 desktop systems and monitors
- Associated software licensing
Immediate priority was given to resolving the server issues. I assembled the servers and installed Windows 2003 on them. The physical installation of the servers, SAN and tape library was handled by the vendor for warranty reasons. Two of the servers were setup first. ATHENA, the new file/print server, and HERMES, the new Exchange server. Migration of the Exchange data from the old Exchange server was the first major task of the upgrade. A test move with one mailbox was performed with no ill effects, followed by a followup test with half a dozen boxes. The remainder were migrated over a weekend. The next task was the migration of user data. A new login script, based on VBScript, was deployed to reset the network drive mappings to the new server. The new backup software, Veritas Backup Exec, was installed and configured to backup the necessary data. With the failing core of the network stabilised with the new hardware, a parellel rollout followed, with the rollout of desktop systems at head office and the rollout of desktops, servers and other hardware at remote sites.
In order to maximise the budget, a number of thin clients were purchased for “light” users, connecting to the new Terminal server. The thin clients were plug and play – cost effective and easily deployed.
To speed up desktop deployment, the Standard Operating Environment (SOE) was deployed using Group Policies. The Group Policy was developed and tested during the research phase of the rollout and the result was almost all applications in the SOE being deployed using this method. Group Policy was also used to enforce the “classic” theme in XP, facilitating the common user experience goal of the rollout.
The upgrading of the network involved two tasks. Firstly, the cabling at head office was such that all cables ran to the server room. In some cases, the cables exceeded the 100 meter limit for ethernet and also made cable management very difficult. To resolve this, the building was recabled:
- Cabling on ground floor and second floor were altered, to terminate in a comms cabinet with a switch.
- From the switch on ground and second floors, a fibre optic cable ran to the first floor, into the server room where the core switches resided.
- Rewiring of the phone system.
- Relabeling of all outlets and ports.
The second task was the upgrading of the network hardware. Previously, 10-megabit hubs and switches had formed the “core” network. The new core network consisted of two HP Procurve 2824 gigabit switches. The secondary switches were HP Procurve 2650s.
Other network upgrade tasks included migrating all remote sites over to a common ISP and deploying Cisco 837 ADSL routers. The routers were configured to create a VPN tunnel back to head office, allowing transparent networking and email access for users. It also made support and maintenance much easier.
Final Analysis
The rollout finished around August 2005. The original timeline was for the rollout to finish in July. The project managed to remain under budget. In terms of the goals I laid out, I think they were achieved:
- Common Hardware Platform – Everyone ended up with either a HP DC7100 or a thin client. All monitors were 17 inch and everyone had the same keyboard and mouse.
- Common Software Platform – All users had the same interface. Office 2003 was the standard office suite for all users.
- Support Infrastructure – Every piece of hardware had the 3 year warranty. Some exceeded this (lifetime on the HP switches). All servers included a 4 hour response time support contract. All Cisco routers had a support contract too.
- Robust WAN – Head office was migrated over to a 2-megabit fibre link. This could be upgraded to 10-megabits. Where possible, remote sites were given ADSL links with a Cisco 837 router.
- Long Lasting Server Infrastructure – A number of IT professionals have positively commented on the final product, saying how well it was put together and noting the high quality of the hardware used.
- Increased Agency Service Delivery – Email “just works”. Server downtime in business hours was greatly reduced. Desktop issues relating to crashes and blue screens were reduced.
Lastly, staff feedback during and after the rollout was positive.