Reconfigurable Firewall

Quick overview

Using IPv6 and Grid'5000 reconfigurable firewall, it is possible to connect directly from the Internet to Grid'5000 nodes. You need to reserve some node(s), activate IPv6 on these node(s), then issue some request to the firewall API to open the firewall for these node(s). You can jump to the #Example usage section if you want to skip the detailed explanation and directly see how to do that.

Background

In IPv4, all communications from inside Grid'5000 to the outside Internet are NATed to the single public IP address of Grid'5000. It is possible to initiate a network connection from inside Grid'5000 to the outside Internet and the connection tracking of the Grid'5000 firewalls will allow communications in most situations (i.e. for protocols that are supported by standard connection tracking mechanisms).

But two things are not possible, in IPv4:

• Initiating a direct network connection from the outside Internet to the inside of Grid'5000 (by direct connection we mean not through VPN or SSH tunnel). It is impossible to allow such connections since Grid'5000 nodes do not have public IPv4 addresses.
• having connections initiated inside Grid'5000 to the outside Internet, on protocols which are not supported by standard connection tracking mechanisms ("standard connection tracking mechanisms" is deliberately vague, it depends on the specific firewall version which may change).

In IPv6, all Grid'5000 nodes have a public, globally routable IPv6 addresses, and there is a Reconfigurable Firewall service which allows users to request firewall openings for specific IPv6 addresses, allowing connections initialed from the outside Internet to target specific IPv6 addresses inside Grid'5000. This page discusses the usage of this Reconfigurable Firewall service.

Features

The Reconfigurable Firewall (aka Dynamic Firewall) service is called g5kfw. It provides a REST API allowing:

• To request firewall openings associated with an OAR job. The request is checked to ensure that the requested openings only targets IPv6 addresses of nodes which belong exclusively to the OAR job of user performing the request. (It is not possible to request an opening to a shared resource, ie. you need to reserve a full host with all its cpu cores to be allowed to open the firewall).
• There can be several openings associated with an OAR job.
• At the end of the OAR job, all openings are removed.
• An opening is:
  • A list of destination hostnames (which resolve to IPv6 addresses) or IPv6 addresses. These are the IPv6 addresses of Grid'5000 nodes for which a connection from the Internet will be allowed,
  • The kind of protocols authorized. There are two kinds of protocols:
    • "tcp+udp": will open only for TCP / UDP flows. In this case, the list of TCP/UDP ports or port ranges that are to be allowed are also needed. "tcp+udp" is the default.
    • "all": means open any protocol, meaning any IPv6 packet are allowed. In this case, there is no concept of ports.
  • Optional source hostnames (which resolve to IPv6 addresses) or IPv6 addresses (/128) or networks, to restrict the opening to specific sources outside Grid'5000. If none given, there is no restriction.
• Additionally, for any destination for which an opening is requested, ICMPv6 is also automatically authorized.

Known limitations

Due to the way Grid'5000 Linux-based firewalls behave (interaction between nftables rules and connection tracking mechanisms), when firewall openings are removed (either because the user explicitly asks for removal or because the OAR job ends), existing established connections will not be closed.

API details

The reconfigurable Firewall API resource URLs are of the form https://api.grid5000.fr/stable/sites/<site>firewall/<jobid> where <site> <jobid> are the Grid'5000 site and the OAR job number for which one requests openings.

There are three operations than can be performed on these resource URLs:

• POST: adds openings to the current openings for the OAR job. The openings are given in the POST body as a JSON payload, with the format:

[
  {
    "addr": "space separated list of hostnames or ipv6 host (/128) addresses" or array of,
    "proto": (optional) protocols to open, can be "tcp+udp" (the default) or "all"
    "port": (mandatory if proto=="tcp+udp", forbidden if proto=="all") integer, or "space separated list of ports or port ranges" or array of. A port range is in the form "port1..port2",
    "src_addr": (optional) same as addr, for source addresses. Also possible to pass IPv6 network addresses (/x with x < 128)
  },
]

• DELETE: delete all openings for the OAR job.
• GET: returns all openings for an OAR job.

Following verification are performed:

• check that user making the request has permission on the job
• for an opening request, check that all requested destination addresses belong to the job
• for an opening request, check that the job has started

If the request is accepted, a success HTTP code is returned, which can be 200 if all the firewalls could be immediately configured, or 202 if one or more of the firewalls could not yet be configured. If one of the firewall could not be immediately configured (maybe there is a network issue, or the firewall is rebooting...) the service will retry to apply the opening every X seconds (X currently being set to 60 seconds)

If the request is refused or fails for any reason, an error code is issued (4xx or 5xx), and the response body contains a message with an explanation of the error.

Example usage

You can test this functionnality from anywhere outside Grid'5000, like a campus, your home, or any server / VM, provided that you have an IPv6 connectivity to the Internet.

Connect to a Grid'5000 site and submit a job. Here, we ask for job type allow_classic_ssh because later on we will connect to the node from outside Grid'5000 using SSH:

USER@fSITE:~$ oarsub --project g5k-staff -l nodes=1,walltime=1:00:00 -t allow_classic_ssh -I
USER@NODE:~$ cat $OAR_JOB_ID

From now on, take note of the SITE, OAR_JOB_ID and NODE.

Activate IPv6 on the node:

USER@NODE:~$ sudo-g5k dhclient -6 br0

The request for firewall opening can be issued from anywhere, but it's more convenient to issue it from a frontend as it will avoid having to authenticate.

The opening request must include the IPv6 address of the node, which is in the form CLUSTERNAME-NODENUMBER-ipv6.SITE.grid5000.fr, so you just need to add -ipv6 inside the node name NODE (for example, if NODE=sagittaire-12.lyon.grid5000.fr, then IPV6NODE=sagittaire-12-ipv6.lyon.grid5000.fr).

Then we send the request to the dynamic firewall service:

USER@flyon:~$ curl -i https://api.grid5000.fr/stable/sites/SITE/firewall/OAR_JOB_ID -d '[{"addr": "IPV6NODE", "port": 22}]'

If everything is ok, the answer code should be 200 or 202, and the response body should be similar to the request (except that all DNS names are converted to IPv6, and the response format is more "rigid").

For more complex requests, it possible the JSON inside a file:

USER@fSITE:~$ cat <<EOF > ~/example-firewall-opening.json
[
    {
        "addr": "IPV6NODE",
        "port": 22
    }
]
EOF

USER@flyon:~$ curl -i https://api.grid5000.fr/stable/sites/SITE/firewall/OAR_JOB_ID -d @/home/USER/example-firewall-opening.json

Then you can try sending pings to the node:

LOCALWORKSTATION:~$ ping -c 5 IPV6NODE

If you have an SSH key on your workstation which allows you to connect to Grid'5000 resources (it depends on your SSH setup), you should also be able to connect directly to the node:

LOCALWORKSTATION:~$ ssh USER@IPV6NODE
USER@NODE:~$ ...

Then, as soon as the OARJOB is finished, the firewall opening will be deleted.