Storage and compression¶
Storage¶
How does automatic backup of project areas work at UPPMAX?
What is this 'glob' folder in my home folder?
- The glob directory found in your home has been deprecated since early 2017.
- It is now a normal directory and shared your default 32GByte sized home.
-
The glob directory remains to not interfere with scripts who might reference ~/glob in the source code.
-
Historically, the glob directory was the main storage area for storage of user data.
- It was shared by all nodes.
- The directory was used for files needed by all job instances and could house files exceeding the quota of the home directory.
- Job input and output files was (and can still be) stored here.
-
You might also be interested in our disk storage guide.
Compression¶
File compression guide
How can I compress my files as quickly and efficiently as possible?
-
You can use this SBATCH script [1] to run the compression in parallel as a node job, with a parallel version of the highly efficient bzip2 compression software.
-
Remember to modify the appropriate #SBATCH parameters at the top of the file, according to your project, and the estimated time to compress your files.
-
[1] Thanks Roman Valls Guimera, for contributing this script.