DCC Storage Groups
Within DCC we offer multiple levels/types of storage of varying capacity/usage.
Home
Current capacity 220Tb
This storage is, by default, available to all staff and students as your Linux “home” directory, i.e. the one that you see when you connect to Linux systems within the HPC environment (e.g. login.gbar.dtu.dk or login.hpc.dtu.dk)
It is also accessible to non Linux users via SAMBA:
Windows \\home.cc.dtu.dk\username MAC smb://home.cc.dtu.dk/username
This is backed up to tape, and space is limited thus quotas are enforced on all home directories. Space here should be used for important data, documents, source code, and output results etc.
Default quota is 30Gb
Command to check your current quota usage.
Check current usage of all directories:
cd ~ && du -h --max-depth=1 .
Scratch filesystems
Current capacity 98TB + 142Tb
This is allocated on a request basis. We have three shared high performing (I/O) parallel file system between all HPC nodes and it offers Low-Latency and High Speed connectivity via an Infiniband network. This should be treated as temporary storage for the duration of HPC jobs. It is accessible from HPC nodes as /work1/<username> or /work3/<username>
**** Please remember that our scratch-filesystems are not backed up ****
Capacity and performance of the scratch file system increases by adding more storage “nodes”. The concept is that departments with several heavy users are expected to contribute to the storage pool.
Users can easily check their scratch usage with a dedicated command.
Technical Information
For those interested in the underlying technical information…
We use Dell EMC for home storage, this offers certain benefits including hardware redundancy in the disk/server setup, hourly/daily snapshots of data (quicker data recovery), snapshot replication to secondary server. Tape backup from secondary server to remote DTU site.
Scratch is a Distributed file systems utilising BeeGFS connected via an infiniband network to the HPC cluster nodes.