There are several types of storage available on the cluster. Their usage is described below.


The cluster's main network storage device is a dual-head NetApp FAS6210. Most users have their home directories on one of the NetApp device's heads. These devices provide high-performance and highly reliable storage. However, there are limitations on their usage:

As the NetApp is not a permanent data storage device, it is not backed up. Should anything physically happen to the unit, there will be no way to retrieve the data.

There is a facility, however, for immediate retrieval of mistakenly deleted data:

Node Scratch Disks

Each node contains a local disk which can be used by an SGE job for temporary data storage. Data on these disks is automatically deleted after 2 weeks (or sooner if circumstances dictate). While these directories are generally only accessible for the lifetime of the job that creates them, it is possible to access their contents should a job crash or be terminated. To do so:

Scratch NetApp

In August 2011, we upgraded our primary NetApp from a daul 3070 system to the dual 6210 system discussed above. Given an exceedingly low trade-in cost, we kept the old dual 3070 system. It is now setup as a cluster wide scratch space.

One of the main reasons we bought a new NetApp is that it is trivial, given the number of nodes we have, to completely overwhelm the old system. So you still have to be judicious in your use of the old NetApp in its new role. While hammering /scrapp and/or /scrapp2 will no longer slow down *every* cluster user, it will significantly slow down the jobs which are using those volumes. So, please, continue to take care in your data access patterns.

QB3cluster: Storage Usage (last edited 2012-01-17 21:51:04 by Joshua_Baker-LePain)