Submitit

Submitit

Submitit is a lightweight tool for submitting Python functions for computation within a Slurm cluster. It basically wraps submission and provide access to results, logs and more. Slurm is an open source, fault-tolerant, and highly scalable cluster management and job scheduling system for large and small Linux clusters. Submitit allows to switch seamlessly between executing on Slurm or locally. Development is in progress for an OAR plugin, to facilitate the passage between OAR and Slurm based resource managers. Source code, issues and pull requests can be found here.

Basic usage

Submitit installation

pip can be used to install the stable release of submitit:

flille:

pip install submitit

otherwise, conda can be used to install submitit from the conda-forge:

flille:

conda install -c conda-forge submitit

an installation from Source can also be used to get the latest version on the main branch:

flille:

pip install git+https://github.com/facebookincubator/submitit@main#egg=submitit