Reconfigurable Firewall

See also: Hardware | KaVLAN | Network reconfiguration tutorial | Advanced KaVLAN | Advanced IPv6 | Reconfigurable firewall | Network emulation

	Note
	This page is actively maintained by the Grid'5000 team. If you encounter problems, please report them (see the Support page). Additionally, as it is a wiki page, you are free to make minor corrections yourself if needed. If you would like to suggest a more fundamental change, please contact the Grid'5000 team.

Quick overview

Using IPv6 and Grid'5000 reconfigurable firewall, it is possible to connect directly from the Internet to Grid'5000 nodes. You need to reserve some nodes (one or more), activate IPv6 on these nodes, then issue some requests to the firewall API to open the firewall for these nodes. You can jump to the #Example usage section if you want to skip the detailed explanation and directly see how to do that.

Background

In IPv4, all communications from inside Grid'5000 to the outside Internet are NATed to the single public IP address of Grid'5000. It is possible to initiate a network connection from inside Grid'5000 to the outside Internet and the connection tracking of the Grid'5000 firewalls will allow communications in most situations (i.e. for protocols that are supported by standard connection tracking mechanisms).

But two things are not possible, in IPv4:

• Initiating a direct network connection from the outside Internet to the inside of Grid'5000 (by direct connection we mean not through VPN or SSH tunnel). It is impossible to allow such connections since Grid'5000 nodes do not have public IPv4 addresses.
• having connections initiated inside Grid'5000 to the outside Internet, on protocols which are not supported by standard connection tracking mechanisms ("standard connection tracking mechanisms" is deliberately vague, it depends on the specific firewall version which may change).

In IPv6, all Grid'5000 nodes have a public, globally routable IPv6 addresses, and there is a Reconfigurable Firewall service which allows users to request firewall openings for specific IPv6 addresses, allowing connections initialed from the outside Internet to target specific IPv6 addresses inside Grid'5000. This page discusses the usage of this Reconfigurable Firewall service.

Features

The Reconfigurable Firewall (aka Dynamic Firewall) service is called g5kfw. It provides a REST API allowing:

• To request firewall openings associated with an OAR job. The request is checked to ensure that the requested openings only targets IPv6 addresses of nodes which belong exclusively to the OAR job of user performing the request. (It is not possible to request an opening to a shared resource, ie. you need to reserve a full host with all its cpu cores to be allowed to open the firewall).
• There can be several openings associated with an OAR job.
• At the end of the OAR job, all openings are removed.
• An opening is:
  • A list of destination hostnames (which resolve to IPv6 addresses) or IPv6 addresses. These are the IPv6 addresses of Grid'5000 nodes for which a connection from the Internet will be allowed,
  • The kind of protocols authorized. There are two kinds of protocols:
    • "tcp+udp": will open only for TCP / UDP flows. In this case, the list of TCP/UDP ports or port ranges that are to be allowed are also needed. "tcp+udp" is the default.
    • "all": means open any protocol, meaning any IPv6 packet are allowed. In this case, there is no concept of ports.
  • Optional source hostnames (which resolve to IPv6 addresses) or IPv6 addresses (/128) or networks, to restrict the opening to specific sources outside Grid'5000. If none given, there is no restriction.
• Additionally, for any destination for which an opening is requested, ICMPv6 is also automatically authorized.

Known limitations

Due to the way Grid'5000 Linux-based firewalls behave (interaction between nftables rules and connection tracking mechanisms), when firewall openings are removed (either because the user explicitly asks for removal or because the OAR job ends), existing established connections will not be closed.

API details

The reconfigurable Firewall API resource URLs are of the form https://api.grid5000.fr/stable/sites/<site>/firewall/<jobid> where <site> <jobid> are the Grid'5000 site and the OAR job number for which one requests openings.

There are three operations than can be performed on these resource URLs:

• POST: adds openings to the current openings for the OAR job. The openings are given in the POST body as a JSON payload, with the format:

[
  {
    "addr": "space separated list of hostnames or ipv6 host (/128) addresses" or array of,
    "proto": (optional) protocols to open, can be "tcp+udp" (the default) or "all"
    "port": (mandatory if proto=="tcp+udp", forbidden if proto=="all") integer, or "space separated list of ports or port ranges" or array of. A port range is in the form "port1..port2",
    "src_addr": (optional) same as addr, for source addresses. Also possible to pass IPv6 network addresses (/x with x < 128)
  },
]

• DELETE: delete all openings for the OAR job.
• GET: returns all openings for an OAR job.

Following verification are performed:

• check that user making the request has permission on the job
• for an opening request, check that all requested destination addresses belong to the job
• for an opening request, check that the job has started

If the request is accepted, a success HTTP code is returned, which can be 200 if all the firewalls could be immediately configured, or 202 if one or more of the firewalls could not yet be configured. If one of the firewall could not be immediately configured (maybe there is a network issue, or the firewall is rebooting...) the service will retry to apply the opening every X seconds (X currently being set to 60 seconds)

If the request is refused or fails for any reason, an error code is issued (4xx or 5xx), and the response body contains a message with an explanation of the error.

Using the reconfigurable firewall from EnOSlib

EnOSlib has support for the reconfigurable firewall API, which makes it easy to use.

Example code is provided here: EnOSlib tutorial to open ports with the reconfigurable firewall

Example usage tutorials

	Note
	You can test this functionality from anywhere outside Grid'5000, like a campus, your home, or any server / VM, provided that you have IPv6 connectivity to the Internet.

Opening inbound SSH from the Internet

Connect to a Grid'5000 site and submit a job:

USER@fSITE:~$ oarsub -l nodes=1,walltime=1:00:00 -I
USER@NODE:~$ echo $OAR_JOB_ID

From now on, take note of the SITE, OAR_JOB_ID and NODE.

Activate IPv6 on the node:

USER@NODE:~$ sudo-g5k dhclient -6 br0

The request for firewall opening can be issued from anywhere, but it's more convenient to issue it from a frontend as it will avoid having to authenticate.

The opening request must include the IPv6 address of the node, which is in the form CLUSTERNAME-NODENUMBER-ipv6.SITE.grid5000.fr, so you just need to add -ipv6 inside the node name NODE (for example, if NODE=sagittaire-12.lyon.grid5000.fr, then IPV6NODE=sagittaire-12-ipv6.lyon.grid5000.fr).

Then we send the request to the dynamic firewall service:

USER@flyon:~$ curl -i https://api.grid5000.fr/stable/sites/SITE/firewall/OAR_JOB_ID -d '[{"addr": "IPV6NODE", "port": 22}]'

If everything is ok, the answer code should be 200 or 202, and the response body should be similar to the request (except that all DNS names are converted to IPv6, and the response format is more "rigid").

For more complex requests, it's possible to have the JSON in a file:

USER@fSITE:~$ cat <<EOF > ~/example-firewall-opening.json
[
    {
        "addr": "IPV6NODE",
        "port": 22
    }
]
EOF

USER@flyon:~$ curl -i https://api.grid5000.fr/stable/sites/SITE/firewall/OAR_JOB_ID -d @/home/USER/example-firewall-opening.json

Then you can try sending pings to the node:

LOCALWORKSTATION:~$ ping6 -c 5 IPV6NODE

If you have an SSH key on your workstation which allows you to connect to Grid'5000 resources (it depends on your SSH setup), you should also be able to connect directly to the node:

LOCALWORKSTATION:~$ ssh USER@IPV6NODE
USER@NODE:~$ ...

Then, as soon as the OARJOB is finished, the firewall opening will be deleted.

Restricting access to specific source addresses

For security reasons, you might not want to allow the whole Internet to access your nodes. It's possible to restrict firewall rules to specific source IP addresses. Assuming the IPv6 network of your home or lab is 2001:db8:1842:1234::/64:

USER@flyon:~$ curl -i https://api.grid5000.fr/stable/sites/SITE/firewall/OAR_JOB_ID -d '[{"addr": "IPV6NODE", "src_addr": "2001:db8:1842:1234::/64", "port": 22}]'

Now SSH to the node should work from your home/lab, but not from anywhere else.

Allowing any network protocol

You may want to experiment with protocols other than UDP and TCP, such as GRE, SCTP, IPv6-in-IPv6 tunnels... In this case, you should specifically allow "all" protocols:

USER@flyon:~$ curl -i https://api.grid5000.fr/stable/sites/SITE/firewall/OAR_JOB_ID -d '[{"addr": "IPV6NODE", "proto": "all"}]'

Note that, as a side-effect, it also allows TCP and UDP traffic on all destination ports. Use with caution or restrict the source address.

Testing end to end IPv6 connectivity

To be able to use IPv6 from anywhere to a node in grid5000, it is mandatory to have end to end IPv6 between the two. If you need to check that you have IPv6 connectivity on the path from your external host to Grid5000, you can check by pinging ping6.grid5000.fr:

yourexternalhost$ ping -6 ping6.grid5000.fr

If this ping fails, the issue is most likely on your side (do you have IPv6 at your home or on your campus?).