KaVLAN: Difference between revisions
Line 10: | Line 10: | ||
= Overview = | = Overview = | ||
[[Image:Kavlan_admin.png|500px|right|thumb|KaVLAN architecture: see ''local VLANs'' in '''<font color="green">green</font>''', ''routed VLANs'' in '''<font color="blue">blue</font>''', ''global VLANs'' in '''<font color="purple">purple</font>''' and the default VLAN in '''<font color="red">red</font>''']] | [[Image:Kavlan_admin.png|500px|right|thumb|KaVLAN architecture: see ''local VLANs'' in '''<font color="green">green</font>''', ''routed VLANs'' in '''<font color="blue">blue</font>''', ''global VLANs'' in '''<font color="purple">purple</font>''' and the default VLAN in '''<font color="red">red</font>''']] | ||
[[KaVLAN]] provides network isolation capabilities for Grid'5000 users via a high-level, user-driven interface to 802.1q (VLAN). KaVLAN allows users to manage VLAN on their Grid'5000 nodes. The benefit is a complete level 2 isolation for users' experiments. It is however important to note that KaVLAN does not guarantee performance isolation: on sites with a hierarchical network (such as [[Nancy:Network|Nancy]]), inter-switch links may indeed be shared between various VLANs/experiments. | [[KaVLAN]] provides network isolation capabilities for Grid'5000 users via a high-level, user-driven interface to 802.1q (VLAN). KaVLAN allows users to manage VLAN on their Grid'5000 nodes. The benefit is a complete level 2 isolation for users' experiments. | ||
It is however important to note that KaVLAN does not guarantee performance isolation: on sites with a hierarchical network (such as [[Nancy:Network|Nancy]]), inter-switch links may indeed be shared between various VLANs/experiments. | |||
KaVLAN is to be used together with OAR and Kadeploy to do experimentations involving network reconfiguration. | KaVLAN is to be used together with OAR and Kadeploy to do experimentations involving network reconfiguration. |
Revision as of 00:42, 20 January 2016
Overview
KaVLAN provides network isolation capabilities for Grid'5000 users via a high-level, user-driven interface to 802.1q (VLAN). KaVLAN allows users to manage VLAN on their Grid'5000 nodes. The benefit is a complete level 2 isolation for users' experiments.
It is however important to note that KaVLAN does not guarantee performance isolation: on sites with a hierarchical network (such as Nancy), inter-switch links may indeed be shared between various VLANs/experiments.
KaVLAN is to be used together with OAR and Kadeploy to do experimentations involving network reconfiguration.
3 types of VLANs are available for users on Grid'5000:
KaVLAN name in OAR | type | first id | last id |
---|---|---|---|
kavlan-local | local | 1 | 3 |
kavlan | routed | 4 | 9 |
kavlan-global | global | 10 | 21 |
See the 2 schemas on the right which illustrate KaVLAN Architecture.
Please also note the installation status for all Grid'5000 sites:
Sites | Version | Status |
---|---|---|
Grenoble | 1.2.7-1 | ![]() |
Lille | 1.2.7-1 | ![]() |
Luxembourg | 1.2.7-1 | ![]() |
Lyon | 1.2.7-1 | ![]() |
Nancy | 1.2.7-1 | ![]() |
Nantes | 1.2.7-1 | ![]() |
Rennes | 1.2.7-1 | ![]() |
Sophia | 1.2.7-1 | ![]() |
Toulouse | 1.2.7-1 | ![]() |
Strasbourg | 1.2.7-1 | ![]() |
Louvain | 1.2.7-1 | ![]() |
1: Local VLAN
An local VLAN is completely isolated from the rest of Grid'5000: no routing is configured. Therefore, you have to hop by a special host to reach your nodes inside this kind of VLAN. A DHCP server is brought up for you, so once you have put your nodes in the VLAN, you can down-up the network interface, or restart the networking service (with kaconsole3, or an 'at' command for instance), or reboot the node (with kareboot3).
Then you will be able to reach any of your nodes within the VLAN using hostnames such as hostname-kavlan-VLAN_ID (adding the suffix -kavlan-VLAN_ID to the regular hostname), with a hop by the VLAN gateway machine, named: kavlan-VLAN-ID.
The figure below shows two jobs running with KaVLAN: each job has it's nodes isolated in a local VLAN (purple and green). The other nodes are all in the default VLAN (red). The only way to reach the isolated nodes is to hop by the VLAN's SSH gateway machine (kavlan-1 and kavlan-2 in the figure). The gateway has two Ethernet interfaces: one in the default VLAN and one in the dedicated VLAN. An other way to reach an isolated node is to use the kaconsole
command.
2: Routed VLAN
Unlike local VLANs which are isolated, routed VLANs are not isolated at the layer 3: IP packets are routed. Therefore you can reach the nodes inside a routed VLAN from the rest of Grid5000 (e.g. from the production network, or other routed VLAN). No need for a hop by a SSH gateway, as it is the case for local VLANs.
Nodes in the VLAN are reachable with the following hostname: hostname-kavlan-VLAN_ID (same naming scheme as for local VLANs), from the frontends for instance.
A DHCP service is provided in those VLANs, which will serve an IP to any cabled network interface of a node which is placed in the VLAN.
3: Global VLAN
global VLANs is are VLAN which is spread on all grid5000 sites (using IEEE 802.1ad encapsulation, also known as QinQ to provide a same layer 2 network for all sites). Therefore you can configure nodes of different sites on the same global VLAN.
There is exactly 1 and only 1 global VLAN available by site. If it is already reserved by another user, you can try to get one from another site (reservation must be made on the site of the global VLAN)
Since it is a same layer 2 network, no routing between the nodes which are placed in a global VLAN is required (even from site to site).
To reach nodes inside a global VLAN from outside, routing is configured on the router of the site where the global VLAN is reserved.
A DHCP service is provided in those VLANs, which will serve an IP to any cabled network interface of a node which is placed in the VLAN.
Reserving a VLAN
KaVLAN only works with deploy reservations; to obtain nodes and a VLAN, you must reserve kavlan resources (VLAN-IDs) with the oarsub
command. There are 3 kinds of resources defined in OAR: kavlan, kavlan-local, kavlan-global. For example, if you need 3 nodes and a local VLAN, you can run:
Then you can get the id of your VLAN using the kavlan
command
If you run this command from outside the shell which is started by OAR for your reservation, you must give the OAR JOBID.
This way, you should get an VLAN ID integer in the <1-3> range for local VLANs, <4-9> for routed VLANs, and greater than 10 for global VLANs (only one global VLAN ID is available per site, that should be that one).
See below the KaVLAN ID, and associated IP subnets (served by DHCP in the VLANs)
- Local VLANs (non-routed)
Site | KAVLAN-1 | KAVLAN-2 | KAVLAN-3 |
---|---|---|---|
All | 192.168.192.0/20 |
192.168.208.0/20 |
192.168.224.0/20
|
- Routed VLANs
Site | KAVLAN-4 | KAVLAN-5 | KAVLAN-6 | KAVLAN-7 | KAVLAN-8 | KAVLAN-9 |
---|---|---|---|---|---|---|
10.0.0.0/18 |
10.0.64.0/18 |
10.0.128.0/18 |
10.0.192.0/18 |
10.1.0.0/18 |
10.1.64.0/18
| |
Grenoble | 10.4.0.0/18 |
10.4.64.0/18 |
10.4.128.0/18 |
10.4.192.0/18 |
10.5.0.0/18 |
10.5.64.0/18
|
Lille | 10.8.0.0/18 |
10.8.64.0/18 |
10.8.128.0/18 |
10.8.192.0/18 |
10.9.0.0/18 |
10.9.64.0/18
|
Lyon | 10.12.0.0/18 |
10.12.64.0/18 |
10.12.128.0/18 |
10.12.192.0/18 |
10.13.0.0/18 |
10.13.64.0/18
|
Nancy | 10.16.0.0/18 |
10.16.64.0/18 |
10.16.128.0/18 |
10.16.192.0/18 |
10.17.0.0/18 |
10.17.64.0/18
|
10.20.0.0/18 |
10.20.64.0/18 |
10.20.128.0/18 |
10.20.192.0/18 |
10.21.0.0/18 |
10.21.64.0/18
| |
Rennes | 10.24.0.0/18 |
10.24.64.0/18 |
10.24.128.0/18 |
10.24.192.0/18 |
10.25.0.0/18 |
10.25.64.0/18
|
Toulouse | 10.28.0.0/18 |
10.28.64.0/18 |
10.28.128.0/18 |
10.28.192.0/18 |
10.29.0.0/18 |
10.29.64.0/18
|
Sophia | 10.32.0.0/18 |
10.32.64.0/18 |
10.32.128.0/18 |
10.32.192.0/18 |
10.33.0.0/18 |
10.33.64.0/18
|
Strasbourg |
10.36.0.0/18 |
10.36.64.0/18 |
10.36.128.0/18 |
10.36.192.0/18 |
10.37.0.0/18 |
10.37.64.0/18
|
Luxembourg | 10.40.0.0/18 |
10.40.64.0/18 |
10.40.128.0/18 |
10.40.192.0/18 |
10.41.0.0/18 |
10.41.64.0/18
|
Nantes | 10.44.0.0/18 |
10.44.64.0/18 |
10.44.128.0/18 |
10.44.192.0/18 |
10.45.0.0/18 |
10.45.64.0/18
|
Louvain | 10.48.0.0/18 |
10.48.64.0/18 |
10.48.128.0/18 |
10.48.192.0/18 |
10.49.0.0/18 |
10.49.64.0/18
|
- Global VLANs
Site | Global Vlan | Subnet | Router IP |
---|---|---|---|
KAVLAN-10 |
10.3.192.0/18 |
10.3.255.254
| |
Grenoble | KAVLAN-11 |
10.7.192.0/18 |
10.7.255.254
|
Lille | KAVLAN-12 |
10.11.192.0/18 |
10.11.255.254
|
Lyon | KAVLAN-13 |
10.15.192.0/18 |
10.15.255.254
|
Nancy | KAVLAN-14 |
10.19.192.0/18 |
10.19.255.254
|
KAVLAN-15 |
10.23.192.0/18 |
10.23.255.254
| |
Rennes | KAVLAN-16 |
10.27.192.0/18 |
10.27.255.254
|
Toulouse | KAVLAN-17 |
10.31.192.0/18 |
10.31.255.254
|
Sophia | KAVLAN-18 |
10.35.192.0/18 |
10.35.255.254
|
Strasbourg |
KAVLAN-19 |
10.39.192.0/18 |
10.39.255.254
|
Luxembourg | KAVLAN-20 |
10.43.192.0/18 |
10.43.255.254
|
Nantes | KAVLAN-21 |
10.47.192.0/18 |
10.47.255.254
|
Louvain | KAVLAN-22 |
10.51.192.0/18 |
10.51.255.254
|
- IP subnet assignments for the sites within a global VLANs
A global VLAN is a /18 subnet (16382 IP addresses). It is split so that every site gets one /23 (510 ip) in the global VLAN address space.
Example for the global VLAN of Lille, KAVLAN-12, whose address space is 10.11.192.0/18:
Bordeaux:10.11.192.1
→10.11.193.254
- Grenoble:
10.11.194.1
→10.11.195.254
- Lille:
10.11.196.1
→10.11.197.254
- Lyon:
10.11.198.1
→10.11.199.254
- Nancy:
10.11.200.1
→10.11.201.254
Orsay:10.11.202.1
→10.11.203.254
- Rennes:
10.11.204.1
→10.11.205.254
- Toulouse:
10.11.206.1
→10.11.207.254
- Sophia:
10.11.208.1
→10.11.209.254
- Strasbourg
Reims:10.11.210.1
→10.11.211.254
- Luxembourg:
10.11.212.1
→10.11.213.254
- Nantes:
10.11.214.1
→10.11.215.254
(More info in the Network page)
Setting up the VLAN
You can get all the options of the kavlan
command using --help:
# kavlan --help Usage: kavlan [options] Specific options: -i, --vlan-id N set VLAN ID (integer or DEFAULT) -C, --ca-cert CA CA certificate -c, --client-cert CERT client certificate -k, --client-key KEY client key -l, --get-nodelist Show nodenames in the given vlan -e, --enable-dhcp Start DHCP server -d, --disable-dhcp Stop DHCP server -V, --show-vlan-id Show vlan id of job (needs -j JOBID) -g, --get-vlan Show vlan of nodes -s, --set-vlan Set vlan of nodes -j, --oar-jobid JOBID OAR job id -m, --machine NODE set nodename (several -m are OK) -f, --filename NODEFILE read nodes from a file -u, --user USERNAME username -v, --[no-]verbose Run verbosely -q, --[no-]quiet Run quietly --[no-]debug Run with debug output -h, --help Show this message --version Show version
Once you have a kavlan reservation running, and know your vlan ID, you can put your nodes in your VLAN (and later, back into the default VLAN) at anytime during the lifetime of your job.
For nodes with multiple cabled network interfaces, you can specified any of those interfaces using:
- for the default interface:
nodename-X
-kavlan-ID
- for other interfaces:
nodename-X
-ethY
-kavlan-ID
For local VLANs, you are also allowed to ssh to the VLAN's SSH gateway, which is named kavlan-ID
.
To get more information on how to use a local VLAN, you can read the following tutorial: Network_isolation_on_Grid'5000#Change_the_VLAN_of_your_nodes_manually
If you want to learn how to use KaVLAN, you can try the tutorial on Network isolation on Grid'5000