System login and general usage

The Data Archive system is Unix-based and requires you to login to the node with your username and password. The system allows key-based authentication. A more in-depth introduction and explanation can be found on the SSH usage page.

You can login using a terminal or tool supporting SFTP or SSH connections. For data transfers GridFTP, HPN-SSH and rsync are also supported.

Guidelines for storing data

The Data Archive is designed to store large amounts of data for long-term archiving. This means that there are a few guidelines we ask our users to follow to make optimal use of the system and keep performance to a maximum:

• Try to store files of significant size (> 1 GB) as much as possible. Smaller files will always be accepted, but will lower the performance of restoring your files from tape.
• If you have many small files, make sure to pack them using a file archiving tool like tar or dmftar.
• Try to pack your files before uploading them to the archive, possibly by using dmftar which allows remote tarring.
• Organise your files in such a way that in case the files are needed again only parts of the data set need to be restored from tape.
• Avoid storing unpacked software packages, these usually contain a lot of small files. Instead pack these as well, or refer to a specific software repository.

Optimal archiving of data files

To facilitate users in archiving their data files while following the above guidelines, the internal tool dmftar has been developed. A detailed explanation and usage instructions can be found here.

High-performance data transfers

The Data Archive supports high-performance data transfer protocols such as HPN-SSH and GridFTP. Please refer to the documentation and instructions here.

Using the archive file system

More information about the usage of the archive file system can be found here