Storage Manager: Difference between revisions

From Grid5000
Jump to navigation Jump to search
No edit summary
 
(4 intermediate revisions by 3 users not shown)
Line 4: Line 4:
{{TutorialHeader}}
{{TutorialHeader}}


The Storage Manager manages access to NFS shares (in particular, users' '''home''' directories) on Grid'5000.
The Storage Manager manages access to NFS shares on Grid'5000. It is used for user's home and [[Group Storage]].  


This page describes its API, and focuses on accessing user's '''home''' directories.
== How it works ==
The storage API works with '''access''' object. An access is defined by a list of addresses and a termination condition for a specific location on an NFS Server.


== How it works ==
On the frontends and on <code class="env">nfs</code> environments (such as <code class="env">centosstream9-nfs</code>), <code class="env">debianXX-big</code> environments (such as <code class="env">debian11-big</code>)  autofs is used for the local home and group storage directories.  '''The accesses are created automatically in this case'''. You may want to use this page if you want to allow additional addresses to access your home (like when you are using VLAN), if you want to access your home over-site or to access group storage on other environments.
The storage API works with '''access''' object. An access is defined by a list of addresses and a termination condition for a specific location on an NFS Server. Each user's home directory is located on <code>/export/home/</code><code class="replace">username</code> on the NFS server of each site: <code>nfs.</code><code class="replace">site</code><code>.grid5000.fr</code>


'''Each time you start a job, an access to your local home directory is automatically created for all the assigned nodes. You may want to use this page if you want to allow additional addresses to access your home (like when you are using VLAN) or if you want to access your home over-site.'''
You will need to use the [[API|Grid'5000 API]], the entry-point for the home of a '''user''' on a specific '''site''' is:
You will need to use the [[API|Grid'5000 API]], the entry-point for the home of a '''user''' on a specific '''site''' is:
  <nowiki>https://api.grid5000.fr/3.0/sites/</nowiki><code class="replace">site</code>/storage/home/<code class="replace">username</code>/access
  <nowiki>https://api.grid5000.fr/stable/sites/</nowiki><code class="replace">site</code>/storage/home/<code class="replace">username</code>/access
 
For [[Group Storage]], the entry-point of a '''storage''' on a server '''server''' of a '''site''' is:
For [[Group Storage]], the entry-point of a '''storage''' on a server '''server''' of a '''site''' is:
   <nowiki>https://api.grid5000.fr/3.0/sites/</nowiki><code class="replace">site</code>/storage/<code class="replace">server</code>/<code class="replace">storage</code>/access
   <nowiki>https://api.grid5000.fr/stable/sites/</nowiki><code class="replace">site</code>/storage/<code class="replace">server</code>/<code class="replace">storage</code>/access
For example, <code><nowiki>https://api.grid5000.fr/3.0/sites/lille/storage/storage1/my-team-storage/access</nowiki></code>
For example, <code><nowiki>https://api.grid5000.fr/stable/sites/lille/storage/storage1/my-team-storage/access</nowiki></code>
{{Note|text=Only the home owner can interact with his or her accesses, so you need to be authenticated. It's done automatically on the frontend. With curl, you can also use the <code>-u </code><code class="replace">username</code> option and you will be prompted for your password before the request is sent.}}
{{Note|text=Only the home owner or a group member can interact with his or her accesses, so you need to be authenticated. It's done automatically on the frontend. With curl, you can also use the <code>-u </code><code class="replace">username</code> option and you will be prompted for your password before the request is sent.}}


== Usage ==
== Usage ==
We assume that we want to manage the home of user <code class="replace">jpicard</code> on <code class="replace">nancy</code>, so the storage API entry-point will be:
We assume that we want to manage the storage group <code class="replace">delta</code> on the <code class="replace">storage1.nancy</code>.grid5000.fr, so the storage API entry-point will be:
   <nowiki> https://api.grid5000.fr/3.0/sites/</nowiki><code class="replace">nancy</code>/storage/home/<code class="replace">jpicard</code>/access
   <nowiki> https://api.grid5000.fr/stable/sites/</nowiki><code class="replace">nancy</code>/storage/<code class="replace">storage1</code>/<code class="replace">delta</code>/access
=== Create access ===
=== Create access ===
Accesses creation is done by sending a POST request to the API with a JSON payload.
Accesses creation is done by sending a POST request to the API with a JSON payload.
We must provide a list of '''hosts''' to whom we want to give access to, for instance <code class="replace">dahu-1.grenoble.grid5000.fr</code> and <code class="replace">dahu-2.grenoble.grid5000.fr</code>. We also need a termination condition, it can be a specific date or a job termination.
We must provide a list of '''hosts''' to whom we want to give access to, for instance <code class="replace">dahu-1.grenoble.grid5000.fr</code> and <code class="replace">dahu-2.grenoble.grid5000.fr</code>. We also need a termination condition, it can be a specific date or a job termination.
{{Note|text=To give access to a home from a specific VLAN, you can use its domain name, for example for dahu-1 on VLAN 2, you can add <code class="replace">dahu-1-kavlan-2.grenoble.grid5000.fr</code>.}}
{{Note|text=To give access to a storage from a specific VLAN, you can use its domain name, for example for dahu-1 on VLAN 2, you can add <code class="replace">dahu-1-kavlan-2.grenoble.grid5000.fr</code>.}}
==== Until termination ====
==== Until termination ====
If we want to use a specific time (for example <code class="replace">2018-12-25 19:38</code>), we send the following request to the Storage API:
If we want to use a specific time (for example <code class="replace">2018-12-25 19:38</code>), we send the following request to the Storage API:
   curl -X POST <nowiki>'https://api.grid5000.fr/3.0/sites/</nowiki><code class="replace">nancy</code>/storage/home/<code class="replace">jpicard</code>/access'  
   curl -X POST <nowiki>'https://api.grid5000.fr/stable/sites/</nowiki><code class="replace">nancy</code>/storage/<code class="replace">storage1</code>/<code class="replace">delta</code>/access'  
   -H "Content-Type: application/json"  
   -H "Content-Type: application/json"  
   -d '{"hosts": [<code class="replace">"dahu-1.grenoble.grid5000.fr", "dahu-2.grenoble.grid5000.fr"</code>],  
   -d '{"hosts": [<code class="replace">"dahu-1.grenoble.grid5000.fr", "dahu-2.grenoble.grid5000.fr"</code>],  
Line 35: Line 35:
==== Job termination ====
==== Job termination ====
If we want our access to be valid until the end of a Grid'5000 job, for example the job <code class="replace">4548</code> of  <code class="replace">grenoble</code>, we do:
If we want our access to be valid until the end of a Grid'5000 job, for example the job <code class="replace">4548</code> of  <code class="replace">grenoble</code>, we do:
   curl -X POST <nowiki>'https://api.grid5000.fr/3.0/sites/</nowiki><code class="replace">nancy</code>/storage/home/<code class="replace">jpicard</code>/access'
   curl -X POST <nowiki>'https://api.grid5000.fr/stable/sites/</nowiki><code class="replace">nancy</code>/storage/<code class="replace">storage1</code>/<code class="replace">delta</code>/access'
   -H "Content-Type: application/json"  
   -H "Content-Type: application/json"  
   -d '{"hosts": [<code class="replace">"dahu-1.grenoble.grid5000.fr", "dahu-2.grenoble.grid5000.fr"</code>],  
   -d '{"hosts": [<code class="replace">"dahu-1.grenoble.grid5000.fr", "dahu-2.grenoble.grid5000.fr"</code>],  
       "termination" : {"job": <code class="replace">4548</code>, "site": <code class="replace">"grenoble"</code>}}'
       "termination" : {"job": <code class="replace">4548</code>, "site": <code class="replace">"grenoble"</code>}}'
If we want to give access to all the assigned nodes of a job, we don't need to give any hosts, the API will fill them for us:
If we want to give access to all the assigned nodes of a job, we don't need to give any hosts, the API will fill them for us:
   curl -X POST <nowiki>'https://api.grid5000.fr/3.0/sites/</nowiki><code class="replace">nancy</code>/storage/home/<code class="replace">jpicard</code>/access'
   curl -X POST <nowiki>'https://api.grid5000.fr/stable/sites/</nowiki><code class="replace">nancy</code>/storage/<code class="replace">storage1</code>/<code class="replace">delta</code>/access'
   -H "Content-Type: application/json"
   -H "Content-Type: application/json"
   -d '{"termination" : {"job": <code class="replace">4548</code>, "site": <code class="replace">"grenoble"</code>}}'
   -d '{"termination" : {"job": <code class="replace">4548</code>, "site": <code class="replace">"grenoble"</code>}}'
Line 46: Line 46:
{{Warning|text= The access is created immediately and is valid until the termination of the job, even if the job has not started yet.}}
{{Warning|text= The access is created immediately and is valid until the termination of the job, even if the job has not started yet.}}


The '''nfs_address''' field will contain the address used to manually mount the home:
The '''nfs_address''' field will contain the address used to manually mount the storage:
{{Term|location=node|cmd=mount srv-data.nancy.grid5000.fr:/export/home/<code class="replace">jpicard</code> <code class="replace">/mnt/</code>}}
{{Term|location=node|cmd=mount <code class="replace">storage1.nancy</code>.grid5000.fr:/export/group/<code class="replace">delta</code> <code class="replace">/mnt/</code>}}
 


{{Note|text=Debian <code class="env">nfs</code>, <code class="env">big</code> and <code class="env">std</code> environments (such as <code class="env">debian11-x64-nfs</code>) use autofs for the home directories of the local NFS server. So doing <code>cd /home/jpicard/</code> will automount the home of jpicard if you have the rights.}}


{{Warning|text=When access is explicitly given to a group storage (e.g. this is not done automatically by the job management system), this group storage can be mounted using the ''mount'' command as super-user (via ''sudo-g5k'' or in a ''deploy'' job). However, trying to access to the group storage in the autofs mount point in <code>/srv/storage/</code> is not supported.}}
{{Warning|text=When access is explicitly given to a group storage (e.g. this is not done automatically by the job management system), this group storage can be mounted using the ''mount'' command as super-user (via ''sudo-g5k'' or in a ''deploy'' job). However, trying to access to the group storage in the autofs mount point in <code>/srv/storage/</code> is not supported.}}
Line 56: Line 54:
=== List access ===
=== List access ===
To list all accesses, you should do:
To list all accesses, you should do:
   curl <nowiki>https://api.grid5000.fr/3.0/sites/</nowiki><code class="replace">nancy</code>/storage/home/<code class="replace">jpicard</code>/access
   curl <nowiki>https://api.grid5000.fr/stable/sites/</nowiki><code class="replace">nancy</code>/storage/<code class="replace">storage1</code>/<code class="replace">delta</code>/access'
You get a list of accesses:
You get a list of accesses:
   {
   {
     "G5k-home_jpicard_j_1666466-nancy_1": {
     "G5k-group_delta_j_1666466-nancy_1": {
       "ipv4": [
       "ipv4": [
         "172.16.64.97"
         "172.16.64.97"
Line 67: Line 65:
         "site": "nancy"
         "site": "nancy"
       },
       },
       "nfs_address": "srv-data.nancy.grid5000.fr:/export/home/jpicard"
       "nfs_address": "storage1.nancy.grid5000.fr:/export/group/delta"
     },
     },
     "G5k-home_jpicard_u_1535456240_1": {
     "G5k-group_delta_u_1535456240_1": {
       "ipv4": [
       "ipv4": [
         "172.16.64.16"
         "172.16.64.16"
Line 76: Line 74:
         "until": 1535456240,
         "until": 1535456240,
       },
       },
       "nfs_address": "srv-data.nancy.grid5000.fr:/export/home/jpicard"
       "nfs_address": "storage1.nancy.grid5000.fr:/export/group/delta"
     }
     }
   }
   }
You can also ask for a specific access by providing its identifier, for instance:
You can also ask for a specific access by providing its identifier, for instance:
   curl <nowiki>https://api.grid5000.fr/3.0/sites/</nowiki><code class="replace">nancy</code>/storage/home/<code class="replace">jpicard</code>/access/<code class="replace">G5k-home_jpicard_u_1535456240_1</code>
   curl <nowiki>https://api.grid5000.fr/stable/sites/</nowiki><code class="replace">nancy</code>/storage/<code class="replace">storage1</code>/<code class="replace">delta</code>/access/<code class="replace">G5k-group_delta_u_1535456240_1</code>


=== Delete an access ===
=== Delete an access ===
Access deletion is done by sending a DELETE request to the API with the access identifier.
Access deletion is done by sending a DELETE request to the API with the access identifier.
  curl -X DELETE <nowiki>https://api.grid5000.fr/3.0/sites/</nowiki><code class="replace">nancy</code>/storage/home/<code class="replace">jpicard</code>/access/<code class="replace">G5k-home_jpicard_u_1535456240_1</code>
  curl -X DELETE <nowiki>https://api.grid5000.fr/stable/sites/</nowiki><code class="replace">nancy</code>/storage/<code class="replace">storage1</code>/<code class="replace">delta</code>/access/<code class="replace">G5k-group_delta_u_1535456240_1</code>


== API Specification ==
== API Specification ==


[https://api.grid5000.fr/doc/3.0/#tag/storage See here] for the storage manager API specification.
[https://api.grid5000.fr/doc/stable/#tag/storage See here] for the storage manager API specification.

Latest revision as of 14:27, 26 July 2023

See also: Storage overview | Storage Manager | Group Storage | Ceph | Local (on nodes) disks reservation
Note.png Note

This page is actively maintained by the Grid'5000 team. If you encounter problems, please report them (see the Support page). Additionally, as it is a wiki page, you are free to make minor corrections yourself if needed. If you would like to suggest a more fundamental change, please contact the Grid'5000 team.

The Storage Manager manages access to NFS shares on Grid'5000. It is used for user's home and Group Storage.

How it works

The storage API works with access object. An access is defined by a list of addresses and a termination condition for a specific location on an NFS Server.

On the frontends and on nfs environments (such as centosstream9-nfs), debianXX-big environments (such as debian11-big) autofs is used for the local home and group storage directories. The accesses are created automatically in this case. You may want to use this page if you want to allow additional addresses to access your home (like when you are using VLAN), if you want to access your home over-site or to access group storage on other environments.

You will need to use the Grid'5000 API, the entry-point for the home of a user on a specific site is: 

https://api.grid5000.fr/stable/sites/site/storage/home/username/access

For Group Storage, the entry-point of a storage on a server server of a site is: 

 https://api.grid5000.fr/stable/sites/site/storage/server/storage/access

For example, https://api.grid5000.fr/stable/sites/lille/storage/storage1/my-team-storage/access

Note.png Note

Only the home owner or a group member can interact with his or her accesses, so you need to be authenticated. It's done automatically on the frontend. With curl, you can also use the -u username option and you will be prompted for your password before the request is sent.

Usage

We assume that we want to manage the storage group delta on the storage1.nancy.grid5000.fr, so the storage API entry-point will be: 

  https://api.grid5000.fr/stable/sites/nancy/storage/storage1/delta/access

Create access

Accesses creation is done by sending a POST request to the API with a JSON payload. We must provide a list of hosts to whom we want to give access to, for instance dahu-1.grenoble.grid5000.fr and dahu-2.grenoble.grid5000.fr. We also need a termination condition, it can be a specific date or a job termination.

Note.png Note

To give access to a storage from a specific VLAN, you can use its domain name, for example for dahu-1 on VLAN 2, you can add dahu-1-kavlan-2.grenoble.grid5000.fr.

Until termination

If we want to use a specific time (for example 2018-12-25 19:38), we send the following request to the Storage API: 

 curl -X POST 'https://api.grid5000.fr/stable/sites/nancy/storage/storage1/delta/access' 
 -H "Content-Type: application/json" 
 -d '{"hosts": ["dahu-1.grenoble.grid5000.fr", "dahu-2.grenoble.grid5000.fr"], 
      "termination" : {"until":"2018-12-25 19:38"}}'

Job termination

If we want our access to be valid until the end of a Grid'5000 job, for example the job 4548 of grenoble, we do: 

 curl -X POST 'https://api.grid5000.fr/stable/sites/nancy/storage/storage1/delta/access'
 -H "Content-Type: application/json" 
 -d '{"hosts": ["dahu-1.grenoble.grid5000.fr", "dahu-2.grenoble.grid5000.fr"], 
      "termination" : {"job": 4548, "site": "grenoble"}}'

If we want to give access to all the assigned nodes of a job, we don't need to give any hosts, the API will fill them for us: 

 curl -X POST 'https://api.grid5000.fr/stable/sites/nancy/storage/storage1/delta/access'
 -H "Content-Type: application/json"
 -d '{"termination" : {"job": 4548, "site": "grenoble"}}'
Warning.png Warning

The access is created immediately and is valid until the termination of the job, even if the job has not started yet.

The nfs_address field will contain the address used to manually mount the storage:

Terminal.png node:
mount storage1.nancy.grid5000.fr:/export/group/delta /mnt/


Warning.png Warning

When access is explicitly given to a group storage (e.g. this is not done automatically by the job management system), this group storage can be mounted using the mount command as super-user (via sudo-g5k or in a deploy job). However, trying to access to the group storage in the autofs mount point in /srv/storage/ is not supported.

List access

To list all accesses, you should do: 

 curl https://api.grid5000.fr/stable/sites/nancy/storage/storage1/delta/access'

You get a list of accesses: 

 {
   "G5k-group_delta_j_1666466-nancy_1": {
     "ipv4": [
       "172.16.64.97"
     ],
     "termination": {
       "job": 1666466,
       "site": "nancy"
     },
     "nfs_address": "storage1.nancy.grid5000.fr:/export/group/delta"
   },
   "G5k-group_delta_u_1535456240_1": {
     "ipv4": [
       "172.16.64.16"
     ],
     "termination": {
       "until": 1535456240,
     },
     "nfs_address": "storage1.nancy.grid5000.fr:/export/group/delta"
   }
 }

You can also ask for a specific access by providing its identifier, for instance: 

 curl https://api.grid5000.fr/stable/sites/nancy/storage/storage1/delta/access/G5k-group_delta_u_1535456240_1

Delete an access

Access deletion is done by sending a DELETE request to the API with the access identifier. 

curl -X DELETE https://api.grid5000.fr/stable/sites/nancy/storage/storage1/delta/access/G5k-group_delta_u_1535456240_1

API Specification

See here for the storage manager API specification.

Retrieved from "https://www.grid5000.fr/mediawiki/index.php?title=Storage_Manager&oldid=99909"