New CCIAM cluster: access and scratch usage policy

By /

1. Computing resources

The new cluster provides four Slurm partitions: genoa (CPU computing), hopper (GPU computing), skylake (CPU computing, legacy nodes), kepler, pascal and volta (GPU computing, legacy nodes) and a visualization partition.

Partition Usage Nodes Processors Cores per node Memory per node GPUs per node
genoa CPU computing 16 AMD EPYC 9655 192 768 GB —
hopper GPU computing and large memory needs 1 AMD EPYC 9555 128 2,304 GB 8 × NVIDIA H200
visu02 Visualization 1 AMD EPYC 9555 128 [to be completed] [to be completed]
skylake CPU computing (legacy Rheticus nodes) Up to 158, as migration proceeds Intel Xeon Gold 6142 32 192 GB —
kepler GPU computing (legacy Rheticus nodes) 7 Intel Xeon E5-2680 v3 24 320 GB 2 × NVIDIA Tesla K80 (dual-GPU cards, i.e. 4 GPUs)
pascal GPU computing (legacy Rheticus nodes) 2 Intel Xeon E5-2680 v4 28 320 GB 2 × NVIDIA Tesla P100
volta GPU computing (legacy Rheticus nodes) 5 Intel Xeon Gold 5218 32 380 GB 4 × NVIDIA Tesla V100

The skylake partition, the associated visu partition and the kepler, pascal and volta GPU partitions group the legacy Rheticus nodes: these nodes are added progressively, as they are migrated.

Note that you will also see the gpu partition, which is reserved for MesoNET projects.

The partition is selected at submission time with the Slurm --partition (or -p) option, for example sbatch -p genoa job.sh.

2. Connection

Access is via SSH to the login node hedy.mesocentre.univ-amu.fr. The access port is specified in your copernicus space (https://mesocentre.univ-amu.fr/copernicus/login.html).

ssh -p <portssh> <identifiant>@hedy.mesocentre.univ-amu.fr

To avoid specifying the port on every connection, add the following to ~/.ssh/config on your workstation:

Host hedy
    HostName hedy.mesocentre.univ-amu.fr
    Port <portssh>
    User <identifiant>

You can then connect with ssh hedy. File transfers use the same port: scp -P <portssh> … (capital P) or rsync -e "ssh -p <portssh>" ….

NB: the ssh-copy-id command does not work: using the copernicus interface is mandatory to add SSH public keys.

3. Storage spaces

The scratch is a temporary working space; the former Rheticus spaces are available read-only for data recovery.

Space Type Capacity Access Usage
/scratch BeeGFS on NVMe, shared by all partitions 450 TB (quota 2 TB/user) Read and write Data for running jobs only; subject to the policy in section 4
/home-reth Former Rheticus home — Read-only Recovery of data from the former cluster
/save-reth Former Rheticus save — Read-only Recovery of data from the former cluster
/home New home quota:
500 GB/user
500 GB/project		 Read and write User data

No writing is possible in /home-reth and /save-reth: copy the files you need for your jobs. The scratch must not be used for archiving; results to be kept must be transferred to your own storage spaces.

4. Scratch usage policy if overall scratch occupancy exceeds 60%

The scratch is reserved for data from running jobs.
The following policy will only be applied:

  • if overall scratch occupancy exceeds 60%
  • until we set up a storage offering (late 2026, early 2027)

To prevent the scratch from being used as a storage space, the five largest consumers are identified every day. When overall scratch occupancy exceeds 60%, they must halve their usage within 14 days; otherwise, all their data on the scratch will be deleted.

This policy is expected to evolve according to how the scratch is used.

4.1 Daily measurement

  1. Every day, the volume occupied on the scratch by each user is measured (total volume of the files they own), as well as the overall scratch occupancy rate.
  2. Users are ranked by decreasing volume. The first five make up the “top 5” of the day.

4.2 Notification

When the measured overall occupancy rate is strictly greater than 60%, any user in the top 5 who is not already subject to an ongoing procedure receives a notice by email, at the address associated with their account. No notice is issued if occupancy is less than or equal to 60%. The day of the notice is denoted D0. The notice states:

  • the reference volume V0, equal to the volume measured on D0;
  • the threshold to reach, equal to V0 / 2;
  • the deadline, equal to D0 + 14 days.

4.3 Release condition

The procedure is closed without deletion if the volume occupied by the user remains less than or equal to V0 / 2 for at least 72 consecutive hours, this period lying entirely between D0 and the deadline.

  • Any exceedance of the threshold during these 72 hours resets the count to zero.
  • The reduction must therefore be effective by D0 + 11 days at the latest.
  • Compliance with the condition is checked by measurements taken [frequency to be completed, e.g. every hour].

4.4 Outcome of the procedure

  • Condition met: the procedure is closed and the user is informed by email.
  • Condition not met by the deadline: all of the user’s data on the scratch is deleted at D0 + 14 days. As the scratch is not backed up, this data cannot be recovered.

4.5 Special cases

  • Leaving the top 5 during the grace period does not end the procedure, nor does a drop in overall scratch occupancy below 60%: only the condition in section 4.3 closes it.
  • A user with an ongoing procedure does not receive a new notice; V0 and the deadline remain those set on D0.
  • After closure, any new entry into the top 5 triggers a new full procedure, with a new V0 and a new 14-day period.

4.6 Example

A user occupying 2 TB is notified on Monday, November 2 (D0), a day on which the scratch is 65% full. Their threshold is 1 TB and their deadline is Monday, November 16. If they go down to 0.9 TB on Friday, November 6 and stay there until Monday, November 9 at the same time, the procedure is closed. If they only go below 1 TB on Saturday, November 14, the 72 hours cannot be completed before the deadline: their data is deleted on November 16.

For any questions: mesocentre-techn@univ-amu.fr

Scroll to Top