What is the Lisa system

The Lisa system is a cluster computer consisting of several hundreds of multi-core nodes running the Linux operating system. The system is installed and maintained by SURFsara.

Research Capacity Computing Services (RCCS)

The Lisa system is used for the SURFsara service Research Capacity Computing Services (RCCS). RCCS is a SURFsara compute service for researchers coping with many large computational tasks. RCCS covers computing tasks which are typically characterized by a large amount of independent, moderately parallel, computing tasks. The tasks itself can be run in parallel on the computing system. RCCS is typically for those compute tasks which in practice cannot be run on either departmental or university computing systems, due to their size.

Participants

The following participants are involved in the Lisa system:

• The University of Amsterdam (UvA)
• The VU University Amsterdam (VU)
• The SURF organization (SURF, which has taken over from NWO)

System configuration

The Lisa system is constantly evolving and growing to satisfy the needs of the participants. At this moment the configuration is as follows:

Summary

CPU nodes

Total number of cores    7484
Total amount of memory   26 TB
Total peak performance   149 TFlop/sec
Disk space               400 TB for the home file systems
Operating System         Debian Linux
Mellanox                 InfiniBand network
                         Bandwidth FDR: 56 Gbit/sec
                         Latency   FDR: 1.3 µsec 

GPU nodes

Total number of cores        276
Total amount of memory       5.9 TB
Total peak performance (SP)  1.026 TFlop/sec
Total peak performance (DP)  32 TFlop/sec
Disk space                   400 TB for the home file systems
Operating System             Debian Linux
Interconnect                 40 Gbit/s ethernet

Managing Lisa

Most of the software that is used to manage the Lisa system is Open Source. We are using the following software to manage the system:

• CFEngine 3, A configuration engine
• As batch software we use Torque and Maui. For Torque we have developed a pbs Python interface. So it is now possible to develop batch utilities in Python instead of C.
• torque_2_deb, SURFsara has developed software to make a Debian package from the Torque source
• Ganglia. For ganglia SURFsara has developed an jobmonarch add-on to monitor OpenPBS/Torque queues. https://ganglia.surfsara.nl
• SALI, SALI (Sara Automatic Linux Installer) is a tool that allows you to install Linux on multiple machines at once. It support several protocols for downloading by way of aria2 to install a machine. For example, BitTorrent and rsync are supported. SALI originates from SystemImager and still uses the same philosophy. It is a scalable method for performing unattended installation. SALI is mostly used in cluster setups.