Network reconfiguration tutorial

See also: Introduction tutorial | Advanced Tutorial | KaVLAN | KaVLAN VPN

	Note
	This page is actively maintained by the Grid'5000 team. If you encounter problems, please report them (see the Support page). Additionally, as it is a wiki page, you are free to make minor corrections yourself if needed. If you would like to suggest a more fundamental change, please contact the Grid'5000 team.

Introduction

This TP aims to discover a method to configure a network in Grid'5000 using KaVLAN.

KaVLAN is a tool on Grid'5000 which allow to the user to manage VLANs in the platform. It edits switch configuration to change the VLAN number of the port corresponding to the interface of a node. This method permits complete layer 2 isolation.

Three kinds of VLANs are available on Grid'5000, you can find more information on the page KaVLAN. In this TP, we will use only global Vlan and local Vlan.

In the first time, we will set up a simple topologie with 2 VLANs, a global and a local. In each VLAN there will be at least one node, and one interface of a node with 2 interfaces.

Set up topology

Reservations

A global VLAN is all over Grid'5000, so we have to reserve it on only one site. We will use Rennes and Nancy sites.

rennes:frontend:

oarsub -l {"type='kavlan-global'"}/vlan=1+{"type='kavlan-local'"}/vlan=1+{"cluster='paravance'"}/nodes=3,walltime=3 -I -t deploy

With this reservation we have, 1 kavlan-local, 1 kavlan-global and 3 nodes on paravance cluster. Paravance is a cluster of nodes with 2 interfaces.

Get your Vlans ID :

frontend:

kavlan -V

How to know which VLAN is global and which is local ? It's simple, it's write in KaVLAN (look at the first diagram ;)) :

• kavlan-local : [1-3]
• kavlan : [4-9]
• global : [10-20]

Deployment

Now we will deploy our nodes with debian wheezy minimal :

rennes:frontend:

kadeploy3 -e wheezy-x64-min -k -f $OAR_NODEFILE

During the deployment you can reserve and deploy a node on Nancy in other terminal :

nancy:frontend:

oarsub -I -l walltime=3 -t deploy

nancy:frontend:

kadeploy3 -e wheezy-x64-min -k -f $OAR_NODEFILE

We will install 'at' and 'tcpdump' on each node :

nancy:node:

taktuk -s -l root -f $OAR_FILE_NODES broadcast exec [ "apt-get update; apt-get --yes install at tcpdump" ]

	Warning
	A bug was spotted on this command refer to the bugzilla : 6442. If still not working when you are working on this TP, please install 'at' and 'tcpdump' manually now

Network configuration

Nancy

We will put static IP on our node on nancy and put it in Vlan. So set this configuration on the first interface (we will assume this is eth0) in /etc/network/interfaces

 
auto eth0
iface eth0 inet static
        address 192.168.1.1
        netmask 255.255.255.0
 

nancy:frontend:

ssh root@Hostname_node_nancy "apt-get --yes install at && echo 'service networking restart' | at now + 1 minute" && kavlan -s -i Global_Vlan_Id -m Hostname_Node_Nancy

Now we will reboot the node to take into account the modification (we can also install 'at' on the node, and restart the service networking after one minute)

Rennes

As we have 3 nodes on Rennes, we will call them node1, node2 and node3, of course during the TP node1=the hostname of your first node. For example node1=paravance-23.

We will put the node1 in the local vlan and restart the networking service to get a new IP address (there is DHCP server in kavlan-local).

rennes:frontend:

ssh root@node1 "apt-get --yes install at && echo 'service networking restart' | at now + 1 minute" && kavlan -s -i Local_Vlan_Id -m node1

Now we will put a node between the local and global vlan using node2. We have to set up the second interface. We know eth1 is linked with the API : https://api.grid5000.fr/sid/sites/rennes/clusters/paravance/nodes/paravance-1.json?pretty

So we have to add to /etc/network/interfaces on node2 :

 

auto eth1
iface eth1 inet static
        address 192.168.1.2
        netmask 255.255.255.0

rennes:frontend:

ssh root@node2 "apt-get --yes install at && echo 'service networking restart' | at now + 1 minute" && kavlan -s -i Local_Vlan_Id -m node1 && kavlan -s -i Global_Vlan_Id -m node2-eth1

Now we have :

[prod <-]--kavlan-ID--[-> local-vlan]
[local-vlan  <--node1-->]
[local-vlan <-]--node2--[-> global-vlan]
[global-vlan  <--node_nancy-->]

Now you should be able to ssh connect node2 from kavlan gateway, and ping 192.168.1.1 (node_nancy).

Routing

node1 and node_nancy are in two different VLAN, so currently it's impossible to send a packet between node1 and node_nancy. To do this, we will set route between the network in the local vlan and the network in the global VLAN (192.168.1.0/24)

	Note
	To know the network address in the local vlan, connect to node1, node2 or kavlan-ID and enter the command : ip route

	Warning
	Here we will assume this network is 192.168.192.0/20

Packets can't change of network without router between the vlans, so we will use our gateway (node2) to do this. Firstly we have to allow ip forwarding on node2.

rennes:node2:

sysctl -w net.ipv4.ip_forward=1

Now, if a IP packet with known network destination is sent to node2, it will be forwarded to the destination network.

But node1 don't know how to reach node_nancy network and vice versa. We will add a route to each node.

rennes:node1:

route add -net 192.168.1.0/24 gw IP_node2

	Warning
	IP_node2 here is the IP in the network 192.168.192.0/20

nancy:node:

route add -net 192.168.192.0/20 gw 192.168.1.2

Topology test

We will check if packet pass through our gateway :

Open two terminal :

rennes:node2:

tcpdump -i eth0 icmp

rennes:node1:

ping 192.168.1.1

On the first terminal you should see the ICMP packet are forwarded on the node2 :

IP 192.168.200.7 > 192.168.1.1: ICMP echo request, id 4270, seq 1, length 64
IP 192.168.1.1 > 192.168.200.7: ICMP echo reply, id 4270, seq 1, length 64
IP 192.168.200.7 > 192.168.1.1: ICMP echo request, id 4270, seq 2, length 64
IP 192.168.1.1 > 192.168.200.7: ICMP echo reply, id 4270, seq 2, length 64
IP 192.168.200.7 > 192.168.1.1: ICMP echo request, id 4270, seq 3, length 64
IP 192.168.1.1 > 192.168.200.7: ICMP echo reply, id 4270, seq 3, length 64