User Tools

Site Tools


supercomputer:liger:gpfs

homeusing the cluster

Appropriate File System Usage

The following table gives an overview of the available file system resources on Liger Clusters.

Recommendation: ICI-SC has defined an environment variable which should be used as a base path for reading/writing large scratch files. Since the target of may change over time, it is recommended to use this variable instead of hard-coded paths.

Today NO BACKUP is performed neither on nor
Please do a copy by your own or ask your IT services.
A backup solution is under study for a futur deployment. Sorry for this inconvenience.

GPFS Filesystem

File System Usable Space Limit Backup Description / Usage
814 TB no limit NO Full path to the user's standard scratch directory inside GPFS

temporary storage location for applications with large size and I/O demands;
data are automatically deleted (files after 90 days by modification
and access date, empty directories after 3 days)
30 TB 5GB per user NO1) Full path to the user's home directory inside GPFS

for source code, scripts, small static data, binaries, libraries, and applications
/[group] 45 TB 100GB per project
1 millions files
NO2) Full path to limited available user's project data directory inside GPFS

storage for projects in collaboration with ICI; needed space must
be applied for explicitly
/softs 2 TB denied for users NO3) Software repository

modules, flexlm, XF libraries, etc.

Visibility

Variable Mount point Access Type Protocol
/scratch/[login] Login
Compute+Visu
GPFS device
/home/[login] Login
Compute+Visu
GPFS NFS
/data/ Login
Compute+Visu
GPFS NFS
not defined /softs Login
Compute+Visu
GPFS NFS

More details about

Good practice notes

(use tar for lot of small files, don't rename directories)

  • Avoid a lot of small files
    Numerousness small files should be reorganized within tar-archives to avoid long access times due to deficiencies in file processing of the underlaying operating system.
  • Avoid renaming of directories
    a rename of directories within the data path should be done carefully because all data beyond the changed directory must be backed up once again. If a large amount of data is affected, it prevents backup of really new data in the entire file system and wastes system resources like CPU time.
1)
will soon be available
2) , 3)
in study
supercomputer/liger/gpfs.txt · Last modified: 2016/04/21 23:10 by Richard.Randriatoamanana@ec-nantes.fr