API: Difference between revisions

From Grid5000
Jump to navigation Jump to search
(New API: reference API)
Line 1: Line 1:
{{Status|Draft}}
== ReferenceAPI ==
{{Maintainer|Cyril Rohr}}
=== Synopsis ===
The reference API provide the reference data of Grid5000. Information such as the list of sites, clusters, nodes, environments, etc. can be queried using this API.
You can also obtain a specific version of any data, list all the versions of a given information, and get an archive of all or part of the data.


== Current State ==
=== REST API ===
[[Image:g5k-api-deployment.png|Current Deployment Schema]]
* API entry-point: http://localhost:8080
* note on formats: it is not required to explicitly indicate the format in the URL, e.g.:
  GET /resource.json
You may prefer to set the Accept HTTP header to the correct mime type, e.g.:
  GET /resource
  Accept: application/json
If you put both, the Accept HTTP header will be ignored.


An early prototype of the upcoming API is available at: https://helpdesk.grid5000.fr/api/rennes. Not stable, only a subset of sites are included (grenoble, bordeaux, rennes, sophia).
==== Get a specific version of a resource ====
 
  GET /<code class="replace">%resource%</code>.<code class="replace">%format%</code>?version=<code class="replace">%version%</code>
Enhancements are planned on a regular basis. Backwards-compatibility is not assured. Servers might be down at any time. Please refer to this page to get the main entry-point of the API as it may change.
* Accepted formats: json, zip
 
* Version parameter: can be omitted (latest version is returned), or it can be: a commit id (40 characters length) or a Unix UTC time (number of seconds since the Unix epoch: 1970-01-01 00:00:00 UTC)
The API is totally distributed. That is, the data is pulled from local datasets (OAR2) without the need for a centralized database.
* Comments: the zip format will return a zip archive containing the set of directories and files corresponding to the required data.
 
* Examples:
== REST API ==
  GET /.json
 
will return the description of Grid5000:
API entry-point: https://helpdesk.grid5000.fr/api/rennes
  HTTP/1.1 200 OK
 
  Etag: "d69bfd1891582824f5a192fa41a3f444a3d52854"
The availability of an HTML format means that you can use your browser as a native API client (e.g. click on https://helpdesk.grid5000.fr/api/rennes/sites).
  Last-Modified: Wed, 28 Jan 2009 13:46:19 GMT
=== GET /sites [html | json] ===
  Content-Type: application/json;charset=utf-8
list of sites (cached for 1 hour). JSON response has the following format:
  Content-Length: 345
  Connection: close
 
   {
   {
     "items": [], # array of site hashes ({"uid": "...", "location": "...", "uri": "...", ...})
     "environments": [
     "errors": [], # array of error messages
      "\/environments\/sid-x64-base-1.0"
     "generated_at": 123456789 # number of seconds since epoch (UTC time)
    ],
    "sites": [
      "\/sites\/bordeaux",
      "\/sites\/grenoble",
      "\/sites\/lille",
      "\/sites\/lyon",
      "\/sites\/nancy",
      "\/sites\/orsay",
      "\/sites\/rennes",
      "\/sites\/sophia",
      "\/sites\/toulouse"
    ],
    "uid": "grid5000",
     "type": "grid",
     "uri": "\/"
   }
   }
=== GET /sites/<code class="replace">{site_uid}</code> [html | json] ===
 
site's details
You can then follow the URIs to get more information on the sub-resources. For example, you can get more details about Bordeaux by doing:
=== GET /sites/<code class="replace">{site_uid}</code>/hosts [html | json] ===
  GET /sites/bordeaux.json
list of site's hosts (cached for 90 seconds). JSON response has the following format:
this will return something like:
  HTTP/1.1 200 OK
  Etag: "45f78b07665ed58f843a741d6927d60a4db35ba3"
  Last-Modified: Wed, 28 Jan 2009 13:46:19 GMT
  Content-Type: application/json;charset=utf-8
  Content-Length: 604
  Connection: close
 
   {
   {
     "items": [], # array of host hashes ({"uid": "...", "cluster_uid": "...", "site_uid": "...", ...})
     "environments": [
     "errors": [], # array of error messages
      "\/sites\/bordeaux\/environments\/sid-x64-base-1.0"
     "generated_at": 123456789 # number of seconds since epoch (UTC time)
    ],
  }
    "name": "Bordeaux",
=== GET /sites/<code class="replace">{site_uid}</code>/hosts/<code class="replace">{host_uid}</code> [html | json] ===
    "location": "Bordeaux, France",
host's details
    "latitude": null,
=== GET /sites/<code class="replace">{site_uid}</code>/clusters [html | json] ===
     "security_contact": null,
list of site's clusters (cached for 1 hour). JSON response has the following format:
     "clusters": [
  {
      "\/sites\/bordeaux\/clusters\/bordemer",
     "items": [], # array of cluster hashes ({"uid": "...", "site_uid": "...", ...})
      "\/sites\/bordeaux\/clusters\/bordeplage",
     "errors": [], # array of error messages
      "\/sites\/bordeaux\/clusters\/bordereau",
     "generated_at": 123456789 # number of seconds since epoch (UTC time)
      "\/sites\/bordeaux\/clusters\/borderline"
    ],
    "uid": "bordeaux",
    "type": "site",
     "user_support_contact": null,
    "description": "",
    "longitude": null,
    "email_contact": null,
    "web": null,
     "uri": "\/sites\/bordeaux",
     "sys_admin_contact": null
   }
   }
and so on.
==== Get the list of all versions of a resource ====
  GET /<code class="replace">%resource%</code>/versions.<code class="replace">%format%</code>
* Accepted formats: json
* Response: the list of all changes that were made to the resource.


== XML-RPC API ==
The API closely follows the principles of the REST architecture. A 404 status code is returned when a resource does not exist. A 500 status code is returned when the server encountered an error. A 200 status code is returned when the request is successful.


Experimental feature. This is mainly an auto-generated wrapper on top of the REST API. This requires an XML-RPC library that supports HTTP Basic Authentication.
=== Getting Started ===
==== Creating an SSH tunnel ====
The API is only available from the Grid5000 frontends. As a consequence, the first thing you have to do when trying to query an API is to create an SSH tunnel to the API server, via your site's access machine:
  ssh -N -L 8080:131.254.202.98:8080 <code class="replace">login</code>@access.<code class="replace">site</code>.grid5000.fr


API entry-point: https://helpdesk.grid5000.fr/api/rennes/xmlrpc
Note: when you are done with your queries, you can hit CTRL-C to destroy the tunnel.


=== sites.index() ===
==== cURL example ====
list of sites (cached for 1 hour). Same response format as for JSON response.
  curl -i http://localhost:8080/sites.json
=== sites.show(site_uid:string) ===
site's details
=== hosts.index(site_uid:string) ===
list of site's hosts (cached for 90 seconds) with their states. Same response format as for JSON response.
=== hosts.show(site_uid:string, host_uid:string) ===
host's detail
=== clusters.index(site_uid:string) ===
list of site's clusters. Same response format as for JSON response.


== Getting Started ==
==== Ruby example ====
=== Ruby client (REST API) ===
First, make sure you've got *Ruby* and *Rubygems* installed. Then install the required gems:
Put this code in a g5k-rest-client.rb file. It will output the current list of sites and, for each one, the list of its clusters and dead nodes:
  sudo gem install rest-client json --no-ri --no-rdoc
Put this code in a g5k-reference-api-client.rb file. It will output the current list of sites and, for each one, the list of its clusters:
  require 'pp'
   require 'rubygems'
   require 'rubygems'
   require 'rest_client' # sudo gem install rest-client
   require 'rest_client' # sudo gem install rest-client
   require 'json' # sudo gem install json
   require 'json'       # sudo gem install json
 
 
   api = RestClient::Resource.new('https://helpdesk.grid5000.fr/api/rennes', 'G5Klogin', 'G5Kpassword')
   api = RestClient::Resource.new('http://localhost:8080')
   begin
   begin
     # get the list of sites
    puts "---- Getting Grid5000"
     sites_response = JSON.parse api['/sites'].get(:accept => 'application/json') # or api['/sites.json'].get
     # start at the root of the reference data (= grid5000)
     sites_response['items'].each do |site|
     grid5000 = JSON.parse api['/'].get(:accept => 'application/json')
      puts "------------------ #{site['uid']} ------------------"
     pp grid5000
      puts "Clusters = [#{site['clusters'].map{|cluster| cluster['uid']}.join(", ")}]"
    puts "\n---- Getting sites"
 
    grid5000['sites'].each do |site_uri|
       # get the site's hosts. we use the URI contained in the response to locate and get new information.
       site = JSON.parse api[site_uri].get(:accept => 'application/json')
      hosts_response = JSON.parse api["#{site['uri']}/hosts"].get(:accept => 'application/json')
      pp site
       puts "Dead hosts = [#{hosts_response['items'].select{|host| host['state'] == 'dead'}.collect{|host| host['uid']}.join(", ")}]"
       puts "\n---- Getting #{site['uid']} clusters"
    end
      site['clusters'].each do |cluster_uri|
   
        cluster = JSON.parse api[cluster_uri].get(:accept => 'application/json')
    # display errors if any
        pp cluster
    sites_response['errors'].each do |error|
       end
       puts error
     end
     end
   rescue RestClient::ResourceNotFound
   rescue RestClient::ResourceNotFound
Line 99: Line 143:


Run with:
Run with:
   ruby g5k-rest-client.rb
   ruby g5k-reference-api-client.rb
 
=== Ruby client (XML-RPC API) ===
Put this code in a g5k-xmlrpc-client.rb file. It will output the current list of sites and, for each one, the list of its clusters and dead nodes:
  require 'xmlrpc/client'
  require 'pp'
  server = XMLRPC::Client.new3( :host => "helpdesk.grid5000.fr",
                                :path => "/api/rennes/xmlrpc",
                                :user => "G5Klogin",
                                :password => "G5Kpassword",
                                :use_ssl => true,
                                :timeout => 5 )
  begin
    sites_response = server.call("sites.index") # equivalent to GET /sites
    sites_response['items'].each do |site|
      puts "------------------ #{site['uid']} ------------------"
      puts "Clusters = [#{site['clusters'].map{|cluster| cluster['uid']}.join(", ")}]"
 
      # get the site's nodes. we need to know the RPC call before runtime to get new information
      hosts_response = server.call("hosts.index", site['uid']) # equivalent to GET /sites/{site_uid}/hosts
      puts "Dead hosts = [#{hosts_response['items'].select{|host| host['state'] == 'dead'}.collect{|host| host['uid']}.join(", ")}]"
    end
    # display errors if any
    sites_response['errors'].each do |error|
      puts error
    end
  rescue XMLRPC::FaultException => e
    puts "Error:"
    puts e.faultCode
    puts e.faultString
  end
 
Run with:
  ruby g5k-xmlrpc-client.rb
 
== Planned ==
* details responses
* make available the node's attributes (cpu, memory, etc.)
* choice: do we settle on using a REST architectural style ?
* experiment submission

Revision as of 11:22, 29 January 2009

ReferenceAPI

Synopsis

The reference API provide the reference data of Grid5000. Information such as the list of sites, clusters, nodes, environments, etc. can be queried using this API. You can also obtain a specific version of any data, list all the versions of a given information, and get an archive of all or part of the data.

REST API

  • API entry-point: http://localhost:8080
  • note on formats: it is not required to explicitly indicate the format in the URL, e.g.:
 GET /resource.json

You may prefer to set the Accept HTTP header to the correct mime type, e.g.:

 GET /resource
 Accept: application/json

If you put both, the Accept HTTP header will be ignored.

Get a specific version of a resource

 GET /%resource%.%format%?version=%version%
  • Accepted formats: json, zip
  • Version parameter: can be omitted (latest version is returned), or it can be: a commit id (40 characters length) or a Unix UTC time (number of seconds since the Unix epoch: 1970-01-01 00:00:00 UTC)
  • Comments: the zip format will return a zip archive containing the set of directories and files corresponding to the required data.
  • Examples:
 GET /.json 

will return the description of Grid5000:

 HTTP/1.1 200 OK
 Etag: "d69bfd1891582824f5a192fa41a3f444a3d52854"
 Last-Modified: Wed, 28 Jan 2009 13:46:19 GMT
 Content-Type: application/json;charset=utf-8
 Content-Length: 345
 Connection: close
 
 {
   "environments": [
     "\/environments\/sid-x64-base-1.0"
   ],
   "sites": [
     "\/sites\/bordeaux",
     "\/sites\/grenoble",
     "\/sites\/lille",
     "\/sites\/lyon",
     "\/sites\/nancy",
     "\/sites\/orsay",
     "\/sites\/rennes",
     "\/sites\/sophia",
     "\/sites\/toulouse"
   ],
   "uid": "grid5000",
   "type": "grid",
   "uri": "\/"
 }
 

You can then follow the URIs to get more information on the sub-resources. For example, you can get more details about Bordeaux by doing:

 GET /sites/bordeaux.json

this will return something like:

 HTTP/1.1 200 OK
 Etag: "45f78b07665ed58f843a741d6927d60a4db35ba3"
 Last-Modified: Wed, 28 Jan 2009 13:46:19 GMT
 Content-Type: application/json;charset=utf-8
 Content-Length: 604
 Connection: close
 
 {
   "environments": [
     "\/sites\/bordeaux\/environments\/sid-x64-base-1.0"
   ],
   "name": "Bordeaux",
   "location": "Bordeaux, France",
   "latitude": null,
   "security_contact": null,
   "clusters": [
     "\/sites\/bordeaux\/clusters\/bordemer",
     "\/sites\/bordeaux\/clusters\/bordeplage",
     "\/sites\/bordeaux\/clusters\/bordereau",
     "\/sites\/bordeaux\/clusters\/borderline"
   ],
   "uid": "bordeaux",
   "type": "site",
   "user_support_contact": null,
   "description": "",
   "longitude": null,
   "email_contact": null,
   "web": null,
   "uri": "\/sites\/bordeaux",
   "sys_admin_contact": null
 }

and so on.

Get the list of all versions of a resource

 GET /%resource%/versions.%format%
  • Accepted formats: json
  • Response: the list of all changes that were made to the resource.


The API closely follows the principles of the REST architecture. A 404 status code is returned when a resource does not exist. A 500 status code is returned when the server encountered an error. A 200 status code is returned when the request is successful.

Getting Started

Creating an SSH tunnel

The API is only available from the Grid5000 frontends. As a consequence, the first thing you have to do when trying to query an API is to create an SSH tunnel to the API server, via your site's access machine:

 ssh -N -L 8080:131.254.202.98:8080 login@access.site.grid5000.fr

Note: when you are done with your queries, you can hit CTRL-C to destroy the tunnel.

cURL example

 curl -i http://localhost:8080/sites.json

Ruby example

First, make sure you've got *Ruby* and *Rubygems* installed. Then install the required gems:

 sudo gem install rest-client json --no-ri --no-rdoc

Put this code in a g5k-reference-api-client.rb file. It will output the current list of sites and, for each one, the list of its clusters:

 require 'pp'
 require 'rubygems'
 require 'rest_client' # sudo gem install rest-client
 require 'json'        # sudo gem install json
 api = RestClient::Resource.new('http://localhost:8080')
 begin
   puts "---- Getting Grid5000"
   # start at the root of the reference data (= grid5000)
   grid5000 = JSON.parse api['/'].get(:accept => 'application/json')
   pp grid5000
   puts "\n---- Getting sites"
   grid5000['sites'].each do |site_uri|
     site = JSON.parse api[site_uri].get(:accept => 'application/json')
     pp site
     puts "\n---- Getting #{site['uid']} clusters"
     site['clusters'].each do |cluster_uri|
       cluster = JSON.parse api[cluster_uri].get(:accept => 'application/json')
       pp cluster
     end
   end
 rescue RestClient::ResourceNotFound
   puts 'Resource not found.'
 rescue RestClient::RequestTimeout
   puts 'Timeout.'
 rescue RestClient::Unauthorized
   puts 'Unauthorized.'
 rescue RestClient::RequestFailed
   puts 'Request failed.'
 rescue RestClient::ServerBrokeConnection 
   puts 'Connection broken.' 
 rescue Exception => e
   puts e.message
 end


Run with:

 ruby g5k-reference-api-client.rb