CCAM Computational Facility Hardware Resources
Our facility is a reflection of the rapid growth of computational biology at CCAM. We started in 1999 with an 8-node compute cluster and a small fileserver that supported the Virtual Cell project and was located in an office extension. We have now grown to two dedicated environmentally-controlled server rooms housing a variety of hardware resources, currently with >5 TFlops compute power, >50 TB storage, and Terascale switching bandwidth, and undergoing continuous expansion.
Compute Clusters
A mix of older and newer architectures:
Alpha Cluster
- Originally a 16 CPU BeoWulf cluster of Compaq DS-20 servers running True64 Unix
- The oldest component of our resource, and at present partially (6 servers) decommissioned
Xeon Blades
- A total of 388 CPUs (Q2 in 2008) (Xeon 2.8 - 3.2 GHz) with 2 - 4 GB RAM partitioned as follows
- A 60 CPU cluster of RLX 2800i blades running Rocks Cluster 4.3
- A 200 CPU cluster (expanding to 328 CPU in Q2 of 2008) of Dell PowerEdge 1855, 1955, and M600 blades running Rocks Cluster 4.3
Large Memory SMP Supercomputer
- A 16 CPU and 64 GB RAM IBM x450 system
- A 16 CPU and 64 GB RAM SGI Altix 350 system
- A 32 CPU and 96 GB RAM SGI Altix 3700 system
Pentium Blades
- 18 CPUs (PIII 1.2 - 1.33 GHz) with 0.5 - 1 GB RAM
- HS318 blades running Linux
- Dedicated (Charles Wolgemuth)
X-Servers
- 70 CPUs
- 1U servers running OS-X 10.3.1
- Dedicated (Greg Huber and Brenton Graveley)
Shared Storage
Apart from the storage resources of compute clusters and specialized servers, we currently have several high-capacity scalable shared storage systems supporting CIFS, NFS, and Appletalk:
- An Isilon clustered storage system of 8 IQ 200 nodes for 16 TB of storage (expanding to 28 TB in Q2 of 2008))
- An iSCSI SAN with >20 TB of RAID enclosures, 2 switches, 3 front-end servers (one dedicated - Bill Mohler)
- Two stand-alone file servers with 4.2 TB of direct connect RAID storage
Managed Switches
- 16 Class 2 and Class 3 managed switches
- >396 Gb ports
- 3 HiGig links (10 Gbs) interconnecting switches dedicated to the Xeon blade cluster
- 2 Force10 C300 modular switches expected H1 2008 for cluster and core switching infrastructure upgrade
Dedicated Servers
Several specialized servers that make up the Virtual Cell back-end Infrastructure:
- 3 Database servers running Oracle and JDBC services, including one redundant server and a Quad AMD Opteron
- 1 Messaging server running SonicMQ
- Distributed services for Platform LSF, RMI, Data, Compile/link, Management across 14 nodes
- Compute servers (80 nodes)
Several specialized servers for Enterprise Computing support, among which:
- 5 Active Directory Domain Controllers for domains vcell.uchc.edu and ccam.uchc.edu with DNS
- 2 VPN servers and 1 DHCP server
- 2 Backup servers
- 2 SNMPc network monitoring servers
- 1 server running 5 FlexLM license services and 2 SCM services (Envy and Clearcase)
- 2 management servers (private LANs, Ghostcast imaging/deployment, iSNS, StorageManager)
- Several file servers (see also Shared Storage above)
Environmental Control
- 15 tons of cooling monitored 24/7 by two environmental temperature/humidity probes
- 10 ton of additional cooling expected H1 2008
- 8 managed power strips controllable via HTTP and SNMP
- A total of 80kW of UPS via two 40kW subsystems with reduntant power modules
- A Cat5E KVM over IP system expected in H1 2008 for remote server managemnt
