Storage Manager: Difference between revisions
No edit summary |
No edit summary |
||
Line 2: | Line 2: | ||
== How it works == | == How it works == | ||
The storage API works with '''access''' object. An access specify a list of | The storage API works with '''access''' object. An access specify a list of addresses and a termination condition for a specific location on a NFS Server. In this page, we will focus on sharing the user home. Each user home directory is located on <code>/export/home/</code><code class="replace">username</code> on the NFS server of each site : <code>nfs.</code><code class="replace">site</code><code>.grid5000.fr</code> | ||
Each time you start a job, an access to your home | Each time you start a job, an access to your home of the local NFS server is automatically created for all the assigned nodes. You may want to use this page if you want to allow additional addresses to access your home or if you want to access your home over-site. | ||
You will need to use the [[API|Grid'5000 API]], the entry-point for the home of a '''user''' on a specific '''site''' is : | You will need to use the [[API|Grid'5000 API]], the entry-point for the home of a '''user''' on a specific '''site''' is : | ||
https://api.grid5000.fr/stable/sites/<code class="replace">site</code>/storage/home/<code class="replace">username</code>/access | <nowiki>https://api.grid5000.fr/stable/sites/</nowiki><code class="replace">site</code>/storage/home/<code class="replace">username</code>/access | ||
{{Note|text=Only the home owner can interact with his or her accesses, so you need to be authenticated. It's done automatically on the frontend. With curl, you can also use the <code>-u </code><code class="replace">username</code> option and you will be prompt for your password before the request is send.}} | |||
{{Note|text=Only the home owner can interact with his or her accesses, so you need to be authenticated. It's done automatically on the frontend.}} | |||
== Usage == | == Usage == | ||
We will assume we want to work with the home of user <code class="replace">jpicard</code> on <code class="replace">nancy</code>, so the storage API entry-point will be: | We will assume we want to work with the home of user <code class="replace">jpicard</code> on <code class="replace">nancy</code>, so the storage API entry-point will be: | ||
<nowiki> https://api.grid5000.fr/stable/sites/</nowiki><code class="replace">nancy</code>/storage/home/<code class="replace">jpicard</code>/access | |||
=== Create access === | === Create access === | ||
Accesses creation is done by sending POST to the API. | Accesses creation is done by sending POST to the API with a JSON payload. | ||
Let's assume we want to give access to the nodes <code class="replace">dahu-1.grenoble.grid5000.fr</code> and <code class="replace">dahu-2.grenoble.grid5000.fr</code>. We also need a termination condition, it can be a specific date or a job termination. | |||
==== Until termination ==== | |||
If we want to use a specific time (for example <code class="replace">2018-12-25 19:38</code>), we send the following request to the Storage API: | |||
curl -X POST <nowiki>'https://api.grid5000.fr/stable/sites/</nowiki><code class="replace">nancy</code>/storage/home/<code class="replace">jpicard</code>/access' | |||
-H "Content-Type: application/json" | |||
-d '{"hosts": [<code class="replace">"dahu-1.grenoble.grid5000.fr", "dahu-2.grenoble.grid5000.fr"</code>], | |||
"termination" : {"until":"<code class="replace">2018-12-25 19:38"</code>}}' | |||
If we want to | ==== Job termination ==== | ||
curl -X POST 'https://api.grid5000.fr/stable/sites/nancy/storage/home/jpicard/access' -H "Content-Type: application/json" -d '{" | If we want our access to be valid until the end of a Grid'5000 job, for example the job <code class="replace">4548</code> of <code class="replace">grenoble</code>, we do: | ||
curl -X POST <nowiki>'https://api.grid5000.fr/stable/sites/</nowiki><code class="replace">nancy</code>/storage/home/<code class="replace">jpicard</code>/access' | |||
-H "Content-Type: application/json" | |||
-d '{"hosts": [<code class="replace">"dahu-1.grenoble.grid5000.fr", "dahu-2.grenoble.grid5000.fr"</code>], | |||
"termination" : {"job": <code class="replace">4548</code>, "site": <code class="replace">"grenoble"</code>}} | |||
If we want to give access to all the assigned nodes of a job, we don't need to give any hosts, the API will complete theme for us: | |||
curl -X POST <nowiki>'https://api.grid5000.fr/stable/sites/</nowiki><code class="replace">nancy</code>/storage/home/<code class="replace">jpicard</code>/access' | |||
-H "Content-Type: application/json" | |||
-d '{"termination" : {"job": <code class="replace">4548</code>, "site": <code class="replace">"grenoble"</code>}} | |||
{{Warning|text= The access is create immediately and until the termination of the job, even if the job has not yet started.}} | {{Warning|text= The access is create immediately and until the termination of the job, even if the job has not yet started.}} | ||
Line 29: | Line 39: | ||
{{Note|text=<code class="env">debian9-x64-std</code>,<code class="env">debian9-x64-big</code> and <code class="env">debian9-x64-nfs</code> use autofs for the | {{Note|text=<code class="env">debian9-x64-std</code>,<code class="env">debian9-x64-big</code> and <code class="env">debian9-x64-nfs</code> use autofs for the home directories of the local NFS server. So <code>cd /home/jpicard/</code> will automount the home of jpicard if you have the right}} | ||
=== List access === | === List access === | ||
To list all access, you should do: | To list all access, you should do: | ||
curl https://api.grid5000.fr/stable/sites/<code class="replace">nancy</code>/storage/home/<code class="replace">jpicard</code>/access | curl <nowiki>https://api.grid5000.fr/stable/sites/</nowiki><code class="replace">nancy</code>/storage/home/<code class="replace">jpicard</code>/access | ||
You get a list of accesses: | You get a list of accesses: | ||
{ | { | ||
Line 55: | Line 65: | ||
} | } | ||
You can also ask for a specific access by providing its identifier, for instance : | You can also ask for a specific access by providing its identifier, for instance : | ||
curl https://api.grid5000.fr/stable/sites/<code class="replace">nancy</code>/storage/home/<code class="replace">jpicard</code>/access/<code class="replace">G5k-home_jpicard_u_1535456240_1</code> | curl <nowiki>https://api.grid5000.fr/stable/sites/</nowiki><code class="replace">nancy</code>/storage/home/<code class="replace">jpicard</code>/access/<code class="replace">G5k-home_jpicard_u_1535456240_1</code> | ||
=== Delete an access === | === Delete an access === | ||
Accesses deletion is done by sending DELETE to the API. | Accesses deletion is done by sending DELETE to the API. | ||
To delete an access you specify it's identifier, for instance: | To delete an access you specify it's identifier, for instance: | ||
curl -X DELETE https://api.grid5000.fr/stable/sites/<code class="replace">nancy</code>/storage/home/<code class="replace">jpicard</code>/access/<code class="replace">G5k-home_jpicard_u_1535456240_1</code> | curl -X DELETE <nowiki>https://api.grid5000.fr/stable/sites/</nowiki><code class="replace">nancy</code>/storage/home/<code class="replace">jpicard</code>/access/<code class="replace">G5k-home_jpicard_u_1535456240_1</code> | ||
== API Specification: == | == API Specification: == | ||
The Storage API uses the same status codes as the rest of [[API#Status_Codes|the Grid'5000 API]]. | |||
=== GET /stable/sites/<code class="replace">site</code>/storage/home/<code class="replace">username</code>/access === | === GET /stable/sites/<code class="replace">site</code>/storage/home/<code class="replace">username</code>/access === | ||
fetch a list of access. | fetch a list of access. | ||
Line 74: | Line 84: | ||
=== DELETE /stable/sites/<code class="replace">site</code>/storage/home/<code class="replace">username</code>/access/?job=<code class="replace">job_id</code>&site=<code class="replace">job_site</code> === | === DELETE /stable/sites/<code class="replace">site</code>/storage/home/<code class="replace">username</code>/access/?job=<code class="replace">job_id</code>&site=<code class="replace">job_site</code> === | ||
delete all access | delete all access with a job termination '''job_id''' of site '''job_site''' | ||
=== POST /stable/sites/<code class="replace">site</code>/storage/home/<code class="replace">username</code>/access === | === POST /stable/sites/<code class="replace">site</code>/storage/home/<code class="replace">username</code>/access === | ||
Create new access. The payload must be in JSON format, and contains the following | Create new access. The payload must be in JSON format, and contains the following properties: | ||
{| class="wikitable" | {| class="wikitable" | ||
|- | |- | ||
! Parameter !! Type !! Description & Examples | ! Parameter !! Type !! Description & Examples | ||
|- | |- | ||
| ipv4 || | | ipv4 || Optional || Array of ipv4 allowed to access NFS. eg: | ||
["172.16.72.1", | ["172.16.72.1", | ||
"172.16.72.25"] | "172.16.72.25"] | ||
|- | |||
| hosts || Optional || Array of hosts allowed to access NFS. eg: | |||
["grisou-1.nancy.grid5000.fr", | |||
"grisou-2.nancy.grid5000.fr"] | |||
|- | |- | ||
| termination || Mandatory || Termination condition can be of two types "Until" or "Job". The form is a structure/hash. | | termination || Mandatory || Termination condition can be of two types "Until" or "Job". The form is a structure/hash. |
Revision as of 11:16, 3 September 2018
The Storage API is used to allow NFS access across machines on Grid'5000. This page focus on accessing user home directories.
How it works
The storage API works with access object. An access specify a list of addresses and a termination condition for a specific location on a NFS Server. In this page, we will focus on sharing the user home. Each user home directory is located on /export/home/
username
on the NFS server of each site : nfs.
site
.grid5000.fr
Each time you start a job, an access to your home of the local NFS server is automatically created for all the assigned nodes. You may want to use this page if you want to allow additional addresses to access your home or if you want to access your home over-site. You will need to use the Grid'5000 API, the entry-point for the home of a user on a specific site is :
https://api.grid5000.fr/stable/sites/site
/storage/home/username
/access
Usage
We will assume we want to work with the home of user jpicard
on nancy
, so the storage API entry-point will be:
https://api.grid5000.fr/stable/sites/nancy
/storage/home/jpicard
/access
Create access
Accesses creation is done by sending POST to the API with a JSON payload.
Let's assume we want to give access to the nodes dahu-1.grenoble.grid5000.fr
and dahu-2.grenoble.grid5000.fr
. We also need a termination condition, it can be a specific date or a job termination.
Until termination
If we want to use a specific time (for example 2018-12-25 19:38
), we send the following request to the Storage API:
curl -X POST 'https://api.grid5000.fr/stable/sites/nancy
/storage/home/jpicard
/access' -H "Content-Type: application/json" -d '{"hosts": ["dahu-1.grenoble.grid5000.fr", "dahu-2.grenoble.grid5000.fr"
], "termination" : {"until":"2018-12-25 19:38"
}}'
Job termination
If we want our access to be valid until the end of a Grid'5000 job, for example the job 4548
of grenoble
, we do:
curl -X POST 'https://api.grid5000.fr/stable/sites/nancy
/storage/home/jpicard
/access' -H "Content-Type: application/json" -d '{"hosts": ["dahu-1.grenoble.grid5000.fr", "dahu-2.grenoble.grid5000.fr"
], "termination" : {"job":4548
, "site":"grenoble"
}}
If we want to give access to all the assigned nodes of a job, we don't need to give any hosts, the API will complete theme for us:
curl -X POST 'https://api.grid5000.fr/stable/sites/nancy
/storage/home/jpicard
/access' -H "Content-Type: application/json" -d '{"termination" : {"job":4548
, "site":"grenoble"
}}
Warning | |
---|---|
The access is create immediately and until the termination of the job, even if the job has not yet started. |
After that, you should be able to mount the home where you want:
Note | |
---|---|
|
List access
To list all access, you should do:
curl https://api.grid5000.fr/stable/sites/nancy
/storage/home/jpicard
/access
You get a list of accesses:
{ "G5k-home_jpicard_j_1666466-nancy_1": { "ipv4": [ "172.16.64.97" ], "termination": { "job": 1666466, "site": "nancy" } }, "G5k-home_jpicard_u_1535456240_1": { "ipv4": [ "172.16.64.16" ], "termination": { "until": 1535456240, } } }
You can also ask for a specific access by providing its identifier, for instance :
curl https://api.grid5000.fr/stable/sites/nancy
/storage/home/jpicard
/access/G5k-home_jpicard_u_1535456240_1
Delete an access
Accesses deletion is done by sending DELETE to the API. To delete an access you specify it's identifier, for instance:
curl -X DELETE https://api.grid5000.fr/stable/sites/nancy
/storage/home/jpicard
/access/G5k-home_jpicard_u_1535456240_1
API Specification:
The Storage API uses the same status codes as the rest of the Grid'5000 API.
GET /stable/sites/site
/storage/home/username
/access
fetch a list of access.
GET /stable/sites/site
/storage/home/username
/access/access_id
fetch information of a specific access
DELETE /stable/sites/site
/storage/home/username
/access/access_id
delete a specific access
DELETE /stable/sites/site
/storage/home/username
/access/?job=job_id
&site=job_site
delete all access with a job termination job_id of site job_site
POST /stable/sites/site
/storage/home/username
/access
Create new access. The payload must be in JSON format, and contains the following properties:
Parameter | Type | Description & Examples |
---|---|---|
ipv4 | Optional | Array of ipv4 allowed to access NFS. eg:
["172.16.72.1", "172.16.72.25"] |
hosts | Optional | Array of hosts allowed to access NFS. eg:
["grisou-1.nancy.grid5000.fr", "grisou-2.nancy.grid5000.fr"] |
termination | Mandatory | Termination condition can be of two types "Until" or "Job". The form is a structure/hash.
Until termination takes a date as an until parameter. The access will be deleted after the specify date. eg: {"until" : "2018-12-25 19:38"} Job terminations takes two parameters, job and site, which are respectively the id and the site of a Grid'5000 job. When the job end, the export is deleted. eg: {"job": "4548, "site":"nancy"} |