Storage5k: Difference between revisions

From Grid5000
Jump to navigation Jump to search
(→‎Overview: cosmetic fix)
(Redirected page to Unmaintained:Storage5k)
 
(89 intermediate revisions by 14 users not shown)
Line 1: Line 1:
{{Maintainer|Nicolas Niclausse}}
#REDIRECT [[Unmaintained:Storage5k]]
{{Portal|Admin}}
{{Status|In production}}
{{See also|[[Storage5k]] | [[Storage5k_Admin|Storage5k Admin]]}}
 
= Grid'5000 storage resources =
 
{| class="checks" style="width: auto;"
! class="left" |Sites
! VM Name
! Disk partition
! Size
! status
|-
| class="left" |[[Sophia:Home|Sophia]]
| stock.sophia.grid5000.fr
| /dev/sda
| 2000 GB
| [[Image:Check.png]]
|-
| class="left" |[[Grenoble:Home|Grenoble]]
|
|
|
| [[Image:Fail.png]]
|-
| class="left" |[[Lille:Home|Lille]]
|
|
|
| [[Image:Fail.png]]
|-
| class="left" |[[Lyon:Home|Lyon]]
|
|
|
| [[Image:Fail.png]]
|-
| class="left" |[[Nancy:Home|Nancy]]
|
|
|
| [[Image:Fail.png]]
|-
| class="left" |[[Orsay:Home|Orsay]]
|
|
|
| [[Image:Fail.png]]
|-
| class="left" |[[Rennes:Home|Rennes]]
|
|
|
| [[Image:Fail.png]]
|-
| class="left" |[[Sophia:Home|Bordeaux]]
|
|
|
| [[Image:Fail.png]]
|-
| class="left" |[[Toulouse:Home|Toulouse]]
|
|
|
| [[Image:Fail.png]]
|}
 
= Purpose =
This page presents ways to use storage5k for the storage reservation in Grid'5000.
 
= Overview =
 
Each user has a default storage space (/home) exported via NFS. For larger storage needs, a user can book another storage space for a determined period.
storage5k allows users to reserve "storage" resources with OAR.
 
= Usage =
=== First check the chunk disk size of the oar storage resources  ===
 
malachheb@fsophia:~$ storage5k -a chunk_size
Chunk size: 10737418240 (10 GB)
 
In this example the chunk size is 10GB, so for a reservation of 100 GB, you should ask for 10 chunks.
 
=== Storage reservation  ===
A storage reservation is an oar job with the resources type is "storage". For each storage reservation you should specifie the chunk number and the walltime (1h by default)<br>
you can submit a storage reservation with two way:
*with storage5k command:
{{Term|location=frontend|cmd=<code class="command">storage5k -a add -l chunks=2,walltime=2</code>}}
  malachheb@fsophia:~$ storage5k -a add -l chunks=2,walltime=2
  [ADMISSION RULE] Modify resource description with type constraints
  [ADMISSION RULE] Modify resource description as storage
  Generate a job key...
  OAR_JOB_ID=648
  Reservation mode : waiting validation...
  Reservation valid --> OK
 
*with oarsub command:
{{Term|location=frontend|cmd=<code class="command">oarsub -t deploy -l chunks=2,walltime=2 -r "2011-12-09 11:57:00"</code>}}
  malachheb@fsophia:~$ oarsub -t deploy -l chunks=2,walltime=2 -r "2011-12-09 11:57:00"
  [ADMISSION RULE] Modify resource description with type constraints
  [ADMISSION RULE] Modify resource description as storage
  Generate a job key...
  OAR_JOB_ID=649
  Reservation mode : waiting validation...
  Reservation valid --> OK
 
{{Note|text= A reservation can take a few minutes if the space is too large, due to the initialization of the reserved partition in the storage server.
}}
 
{{Warning|text= you can not submit a storage reservation simultaneously with compute nodes or other oar resources (vlan, subnet) }}
 
=== check your Storage reservation ===
 
Each storage reservation is identified by the job cpuset_name (e.g, malachheb_649), to displya the details of your reservations, you can use "storage5k -a info".
 
{{Term|location=frontend|cmd=<code class="command">storage5k -a info</code>}}
 
  malachheb@fsophia:~$ storage5k -a info
  storage5k 1.0.0
  Job_Id: 648
    State: Running
    Reservation Id: malachheb_648
    Chunks: 2
    Chunks_size: 10737418240
    Space reserved: 21474836480
    Data Directory: /data/malachheb_648
    Free space: 20018364416
    Used space: 1119879168
    Source nfs: stock.sophia.grid5000.fr:/data/malachheb_648
Also you can use "oarstat" to display your job related to the storage reservation
 
Each reservation is mounted by default in the directory "/data/" (e.g, /data/malachheb_648). You can make a soft link for this directory in your home.
 
=== Use your reserved space ===
 
The reserved  space is used after to store input/output of your experiences, so you need to mount this space in your reserved computing nodes. It exist two use cases:
 
* Nodes reserved by a Interactive job: All your reserved spaces will be mounted/umounted automatically in these nodes.
* Nodes reserved by a deploy job: The user should mount/umount manually her reserved space in these nodes as follows:
 
{{Warning|text= Before any action to mount/umout you reserved space, you should deploy your system image with kadeploy3 in your reserved nodes (verify that you can access to these nodes as root user).<br> The deployed image should contain the "nfs-common" package, if not, you should install this package manually in reserved nodes
<br> List of images that contain "nfs-common": lenny-x64-nfs, lenny-x64-big, squeeze-x64-nfs, squeeze-x64-big, debian-x64-5-prod, squeeze-x64-prod
}}
 
** Mount/Umount All reserved space in ALL reserved nodes of the job (the deploy job with jobid 651 contain two nodes suno-6 and suno-7):
 
{{Term|location=frontend|cmd=<code class="command">storage5k -a mount -j 651</code>}} 
 
  malachheb@fsophia:~$ storage5k -a mount -j 651
  mount the storage reservation malachheb_651 in node suno-6.sophia.grid5000.fr [ Success]
  mount the storage reservation malachheb_651 in node suno-7.sophia.grid5000.fr [ Success]
 
{{Term|location=frontend|cmd=<code class="command">storage5k -a umount -j 651</code>}}
 
  malachheb@fsophia:~$ storage5k -a umount -j 651                                                                                                                                                           
  umount the storage reservation malachheb_651 in node suno-6.sophia.grid5000.fr [ Success]
  umount the storage reservation malachheb_651 in node suno-7.sophia.grid5000.fr [ Success]
 
** Mount All reserved space in one reserved node of the job:
 
{{Term|location=frontend|cmd=<code class="command">storage5k -a mount -j 651 -m suno-6.sophia.grid5000.fr</code>}} 
 
  malachheb@fsophia:~$ storage5k -a mount -j 651 -m suno-6.sophia.grid5000.fr
  mount the storage reservation malachheb_651 in node suno-6.sophia.grid5000.fr [ Success]
 
{{Term|location=frontend|cmd=<code class="command">storage5k -a umount -j 651 -m suno-6.sophia.grid5000.fr</code>}}
 
  malachheb@fsophia:~$ storage5k -a umount -j 651 -m suno-6.sophia.grid5000.fr                                                                                                                                                           
  umount the storage reservation malachheb_651 in node suno-6.sophia.grid5000.fr [ Success]
 
=== Delete your storage reservation ===
* storage5k -a del -j $jobid
 
{{Term|location=frontend|cmd=<code class="command">storage5k -a del -j 648</code>}}
 
  malachheb@fsophia:~$ storage5k -a del -j 488787
  Deleting the job = 488787 ...REGISTERED.
  The job(s) [ 488787 ] will be deleted in a near future.
 
{{Warning|text= If you remove your storage reservation, you will lose all your data in the reserved space.<br>
Remember to save your data before the deletion of your storage reservation.
}}

Latest revision as of 09:30, 22 October 2019

Redirect to:

Retrieved from "https://www.grid5000.fr/mediawiki/index.php?title=Storage5k&oldid=81759"