Funk: Difference between revisions
Lpouilloux (talk | contribs) |
|||
(91 intermediate revisions by 3 users not shown) | |||
Line 1: | Line 1: | ||
{{Portal|User}} | |||
{| border="0" cellpadding="5" | |||
|- | |||
|width=150px valign="top"| __TOC__ | |||
|valign="top"| | |||
'''(F)ind yo(U)r (N)odes on g5(K)''' is a tool that help you to plan your experiments on Grid'5000. It is a Python script based on [https://gitlab.inria.fr/mimbert/execo Execo] that: | |||
* give you the number of nodes available at a date and for walltime | |||
* find the slots for a combination of resources and a walltime | |||
* find the slot with the maximum number of nodes for a period and a walltime. | |||
Some basic examples can be found [[#examples|here]]. | |||
|valign="top"| [[File:Funk_Slots.png|300px]] | |||
|} | |||
Projet source, issue and feature requests can be found [https://github.com/lpouillo/Funk here]. | |||
==Running the tool== | |||
===Inside Grid'5000=== | |||
{{Term|location=frontend|cmd=$ <code class="command">funk</code> -h}} | |||
See below for options descriptions. | |||
== | ===Outside Grid'5000=== | ||
You need to install [https://mimbert.gitlabpages.inria.fr/execo/userguide.html#installation execo] and [https://mimbert.gitlabpages.inria.fr/execo/userguide.html#more-advanced-usages configure it for Grid'5000]. | |||
You also need [https://pypi.python.org/pypi/httplib2 python-httplib2] to make use of the request on the Grid'5000 [[API]]. | |||
You need | If you have no admin rights, install it with : | ||
easy_install --user httplib2 | |||
or | |||
pip install --user httplib2 | |||
Once it is done, you can clone the repository with: | Once it is done, you can clone the repository with: | ||
git clone https://github.com/lpouillo/Funk.git | |||
and run the script. | |||
==<span id="examples">Usage</span>== | |||
I want to find how many nodes are currently available on sites lyon,nancy,rennes for a walltime of 3 hours | |||
{{Term|location=frontend|cmd=<code class="command">funk</code> -m <code class="replace">date</code> -r <code class="replace">lyon,nancy,rennes</code> -w <code class="replace">3:00:00</code>}} | |||
I have a deadline in one week and I absolutely need to run a large | |||
scale experiment on the whole platform (avoiding periods under usage policy | |||
restrictions) | |||
{{Term|location=frontend|cmd=<code class="command">funk</code> -m <code class="replace">max</code> -r <code class="replace">grid5000</code> -w <code class="replace">4:00:00</code> -e <code class="replace">"2014-02-17 16:00:00"</code> -c}} | |||
I need specific resources combination for my experiments and I want to know when a slot will be available, with deploy job type, and reserve it automatically (-y option) : | |||
{{Term|location=frontend|cmd=<code class="command">funk</code> -m <code class="replace">free</code> -r <code class="replace">taurus:2,nancy:100</code> -w <code class="replace">10:00:00</code> -o <code class="replace">"-t deploy"</code> -y}} | |||
= | Please be aware of the [[Grid5000:UsagePolicy|Grid'5000 Usage Policy]] or use <code class="replace">-c</code> to avoid usage policy periods. | ||
A walltime for a night reservation is 11:58:00 or is for a weekend 62:58:00. | |||
==Options== | ==<span id="options">Options</span>== | ||
=== | ===General options=== | ||
Define mode and controls I/O. | |||
-h, --help show this help message and exit | |||
--version show program's version number and exit | |||
-m MODE, --mode MODE Setup the mode: date, free or max | |||
default = date | |||
-y, --yes Perform the reservation automatically | |||
--quiet Run without printing anything | |||
-v, --verbose Run in verbose mode | |||
-p PROGRAM, --program PROGRAM | |||
The program to be run when the reservation start | |||
--plots Draw a Gantt plot and the slots | |||
--json Output the computed data to standard output | |||
--json-file JSON_FILE | |||
Output the computed data to a json file | |||
=== | ===Reservation=== | ||
{{Warning|text=[[OARGrid2]] has been discontinuited and is not maintained anymore.<br> | |||
If you use an old version of Funk, please adapt your usage by using "--no-oargrid" option.}} | |||
Customize your Grid'5000 reservation. | |||
-r RESOURCES, --resources RESOURCES | |||
Comma separated list of Grid'5000 elements (grid5000, site or cluster) | |||
-r element1,element2 for date and max modes | |||
-r element1:n_nodes1,element2:n_nodes2 for free mode | |||
default = grid5000 | |||
-b BLACKLIST, --blacklist BLACKLIST | |||
Remove clusters from planning computation | |||
-R RATIO, --ratio RATIO | |||
Apply a given ratio to the host resources | |||
-o SUBMISSION_OPTS, --submission_opts SUBMISSION_OPTS | |||
Extra options to pass to the oarsub/oargridsub command line | |||
-k, --kavlan Ask for a KaVLAN | |||
-n SUBNET, --subnet SUBNET | |||
Ask for subnets. slash_22=1 will retrieve a /22 subnet on every site of your requests, | |||
but you can specify site1:slash_22=2,site2:slash_19=1 | |||
-j JOB_NAME, --job_name JOB_NAME | |||
The job name passed to the OAR subjobs | |||
default = FUNK | |||
-q QUEUE, --queue QUEUE | |||
The OAR queue to use | |||
default = default | |||
--virtual Use only clusters with hardware virtualization | |||
--infiniband Use only clusters with infiniband or myrinet interfaces | |||
--green Use only clusters that have energetical power measurements | |||
--no-oargrid Don't use oargridsub and perform parallel oar reservations | |||
--force-oargrid Force the usage of oargrisub, even if only one site is involved | |||
== | ===Time=== | ||
Define options related to date and time. | |||
-w WALLTIME, --walltime WALLTIME | |||
Reservation walltime in OAR format | |||
default = 1:00:00 | |||
-s STARTDATE, --startdate STARTDATE | |||
Starting date in OAR format | |||
default = 1443470554 | |||
-e ENDDATE, --enddate ENDDATE | |||
End date in OAR format | |||
default = 1445284954 | |||
-c, --charter Avoid charter (usage policy) periods |
Latest revision as of 11:15, 12 June 2024
(F)ind yo(U)r (N)odes on g5(K) is a tool that help you to plan your experiments on Grid'5000. It is a Python script based on Execo that:
Some basic examples can be found here. |
![]() |
Projet source, issue and feature requests can be found here.
Running the tool
Inside Grid'5000
See below for options descriptions.
Outside Grid'5000
You need to install execo and configure it for Grid'5000.
You also need python-httplib2 to make use of the request on the Grid'5000 API. If you have no admin rights, install it with :
easy_install --user httplib2
or
pip install --user httplib2
Once it is done, you can clone the repository with:
git clone https://github.com/lpouillo/Funk.git
and run the script.
Usage
I want to find how many nodes are currently available on sites lyon,nancy,rennes for a walltime of 3 hours
I have a deadline in one week and I absolutely need to run a large scale experiment on the whole platform (avoiding periods under usage policy restrictions)
I need specific resources combination for my experiments and I want to know when a slot will be available, with deploy job type, and reserve it automatically (-y option) :
Please be aware of the Grid'5000 Usage Policy or use -c
to avoid usage policy periods.
A walltime for a night reservation is 11:58:00 or is for a weekend 62:58:00.
Options
General options
Define mode and controls I/O.
-h, --help show this help message and exit --version show program's version number and exit -m MODE, --mode MODE Setup the mode: date, free or max default = date -y, --yes Perform the reservation automatically --quiet Run without printing anything -v, --verbose Run in verbose mode -p PROGRAM, --program PROGRAM The program to be run when the reservation start --plots Draw a Gantt plot and the slots --json Output the computed data to standard output --json-file JSON_FILE Output the computed data to a json file
Reservation
![]() |
Warning |
---|---|
OARGrid2 has been discontinuited and is not maintained anymore. |
Customize your Grid'5000 reservation.
-r RESOURCES, --resources RESOURCES Comma separated list of Grid'5000 elements (grid5000, site or cluster) -r element1,element2 for date and max modes -r element1:n_nodes1,element2:n_nodes2 for free mode default = grid5000 -b BLACKLIST, --blacklist BLACKLIST Remove clusters from planning computation -R RATIO, --ratio RATIO Apply a given ratio to the host resources -o SUBMISSION_OPTS, --submission_opts SUBMISSION_OPTS Extra options to pass to the oarsub/oargridsub command line -k, --kavlan Ask for a KaVLAN -n SUBNET, --subnet SUBNET Ask for subnets. slash_22=1 will retrieve a /22 subnet on every site of your requests, but you can specify site1:slash_22=2,site2:slash_19=1 -j JOB_NAME, --job_name JOB_NAME The job name passed to the OAR subjobs default = FUNK -q QUEUE, --queue QUEUE The OAR queue to use default = default --virtual Use only clusters with hardware virtualization --infiniband Use only clusters with infiniband or myrinet interfaces --green Use only clusters that have energetical power measurements --no-oargrid Don't use oargridsub and perform parallel oar reservations --force-oargrid Force the usage of oargrisub, even if only one site is involved
Time
Define options related to date and time.
-w WALLTIME, --walltime WALLTIME Reservation walltime in OAR format default = 1:00:00 -s STARTDATE, --startdate STARTDATE Starting date in OAR format default = 1443470554 -e ENDDATE, --enddate ENDDATE End date in OAR format default = 1445284954 -c, --charter Avoid charter (usage policy) periods