Subnet reservation

Problem statement

Users deploying VMs on Grid'5000 need to attribute IP addresses to them. Usually, what is done is that users take IPs from "free to use" IPs as described in Virtual Network Interlink. When several users use this technique, they may both provide the same IPs to their machines, thus creating a conflict.

To overcome this issue, OAR can reserve subnets, enabling scheduling and sharing among users.

Available subnets

Each site of Grid'5000 is allocated a /14 ip block giving thus a total of 262142 different IPs. Therefore, the maximum range of IPs has to be in this /14 subnet.

The /14 address block available on each site is divided in four /16 blocks as described here. The first three /16 are used for IP reservation through OAR. Because the first two are contiguous, we can see them as a single /15 block. Therefore, the reservable range of IPs is composed of one /15 and one /16 subnet, for a total of 131070 + 65534 = 196604 IPs.

Concerning the smallest subnet size, it is a /22, which is equivalent to 1022 hosts per subnet. We do not allow smaller subnets to avoid a scheduling overhead in OAR. Therefore, there are 192 reservable /22 subnets.

Requesting subnets

Using the Grid'5000 API

For an understandable use of the Grid'5000 API, see the API page and the practicals

Authentication

This example using the Grid'5000 API uses the Restfully, a wrapper of REST requests.

If you operate from outside Grid'5000, you need to use your login and password to be able to connect to the API.

  echo "grid5000:
    username: your-grid5000-login
    password: your-grid5000-password" >> ~/.restclient && chmod 600 ~/.restclient

If you operate from within Grid'5000, the authentication is currently automatically handled for you (since you already had to connect to the frontend via SSH).

Code example

The following code will reserve one /22 subnet and one node.

#!/usr/bin/env ruby
require 'rubygems'
require 'restfully' # gem install restfully
require 'yaml'

config = YAML.load_file(File.expand_path("~/.restclient"))['grid5000']
config[:base_uri] = "https://api.grid5000.fr/sid/grid5000/sites/rennes/jobs.json"

Restfully::Session.new(config) do |jobs, session|
  jobs.submit({:resources => "{type='subnet'}/slash_22=1+nodes=1,walltime=00:30:00", :command => "sleep 500"})
end

From inside G5K

Subnet reservation through OAR is similar to normal resource reservation.

To reserve 4 /22 subnets and 2 nodes, just type:

frontend:

oarsub -l {"type='subnet'"}/slash_22=4+/nodes=2 -I

You can of-course have more complex request. To obtain 4 /22 on different /19 subnets, you can type:

frontend:

oarsub -l {"type='subnet'"}/slash_19=4/slash_22=1+/nodes=2/core=1 -I

Getting your networks

Using the Grid'5000 API

	Warning
	The 2.0 API does not take subnets into account (as as of 03/02/2011 sid API does not either). The following code is just here to be ready when the new API server implementation will be deployed.

Authentication

See Authentication

Code example

See the tutorial to learn to use the API. The following code will get the subnets of a job. Using the Subnet class, we compress the subnets returned by the API (only /21 are returned in the API). You can see different printing options in the cfg definition.

#!/usr/bin/env ruby

require 'rubygems'
require 'restfully' # gem install restfully
require 'yaml'
require 'subnets'

config = YAML.load_file(File.expand_path("~/.restclient"))['grid5000']
config[:base_uri] = "https://api.grid5000.fr/sid/grid5000/sites/rennes/jobs/123456789.json"

subnets = nil
Restfully::Session.new(config) do |job, session|
  subnets = job['resources_by_type']['subnets']
end

s = Subnets.new(subnets, {'broadcast' => false, 'netmask' => false, 'prefix' => true, 'ips' => false, 'compress' => true})
s.print

If the job requested 3 /21 subnets, running this sample code may print:

#get_subnets_api.rb
10.8.0.0/20
10.8.32.0/21

From inside Grid'5000

The simplest way to get the list of your allocated subnets is to use the get_subnets script provided on the head node of the submission. OAR provides a property file ($OAR_RESOURCE_PROPERTIES_FILE) where the resources are described. By default, the script uses this file if no other is specified with the -f option.

# get_subnets
10.8.0.0
10.8.8.0

The -p options prints the CIDR format

# get_subnets -p
10.8.0.0/21
10.8.8.0/21

Different other printing options are available (-b to display broadcast address, -n to see the netmask, and -a is equivalent to -bnp):

# get_subnets -a
10.8.0.0/21	10.8.7.255	255.255.248.0
10.8.8.0/21	10.8.15.255	255.255.248.0

You can also compress the subnets into a larger one if they are contiguous:

# get_subnets -cp
10.8.0.0/20

However, when not contiguous, the networks are not merged.

# get_subnets -p
10.8.0.0/21
10.8.16.0/21
# get_subnets -pc
10.8.0.0/21
10.8.16.0/21

Another option available is to get the subnets of any job (running or terminated) using the -j option. This can only be used from a computer where oarstat is usable. Because it uses oarstat that will query the OAR database, it is slower than just reading a property file

# get_subnets -j 123456 -p
10.8.0.0/21
10.8.16.0/21