Sophia:Network

Network: Global | Grenoble | Lille | Luxembourg | Lyon | Nancy | Nantes | Rennes | Sophia | Strasbourg | Toulouse

IP networks in use

You have to use a public network range to run an experiment between several Grid5000 sites.

Public Networks

• computing : 138.96.20.0/22
• virtual : 10.164.0.0/14

Local Networks

• admin : 10.6.1.0/24
• ipmi : 10.96.20.0/24

Network

Physical view of the compute network

Gigabit Ethernet

Azur Cluster (IBM e325)

All the nodes are connected to 4 Cisco-3750 switches (gigabit ethernet). The switches are interconnected using a 32Gbps stack. The nodes are connected to the following switches:

• cisco1 (Master) : azur-1 to azur-15 (15 nodes), 10GB uplink

to the FastIron

• cisco2 : azur-16 to azur-36 (21 nodes)
• cisco3 : azur-37 to azur-55 (19 nodes)
• cisco4 : azur-56 to azur-72 (17 nodes)

The switches are dedicated to Grid5000 (since january 2007; before that, they were shared with the local cluster).

You can choose the switch for your reservation with OAR, using the switch name:

oarsub -l nodes=XX -p "switch='ciscoX'"

If you want XX nodes on a single switch, use

oarsub -l /switch=1/nodes=XX 

Network usage is available through cricket at : https://helpdesk.grid5000.fr/cricket/grapher.cgi?target=%2Fsophia%2Fcisco

OS used: Cisco IOS Software, C3750 Software (C3750-IPBASE-M), Version 12.2(25)SEB4

Helios Cluster (Sun X4100)

All the nodes are connected in gigabit ethernet to 2 Foudry EdgeIron switches (one Edgeiron 48GS and one EdgeIron24GS). The switches are interconnected using a 2x10Gbps stack (10GBps in each direction). The nodes are connected to the following switches:

• edgeiron48gs : helios01 to helios48 (48 nodes)
• edgeiron24gs : helios49 to helios56 (8 nodes)

You can choose the switch for your reservation with OAR, using the switch name: oarsub -l nodes=15 -p "switch='XXX'"

OS used: 2.2.6.35

Sol Cluster (Sun X2200 M2)

All the nodes are connected (in a non blocking way) to the main Foundry FastIron Super X switch.

47 nodes (from sol-1 to sol-47) have a second Ethernet interface connected to 2 Cisco 3750 stacked switches.

• IP addressing for eth1: sol-{1-47}-eth1, using the 172.16.128.0/24 network

The Cisco are interconnected to the main FastIron router with a single 1Gbps link. A separate VLAN is used.

Topology

The main switch is a Foundry FastIron Super X. It has 2 dual 10GB modules, 2 modules with 24 gigabit ports, and 12 gigabit ports on the management module (so 60 gigabit ports are available). 4 slots are currently free.

The Cisco 3750 stack (Azur) is connected to the FastIron switch with a 10GB link; The EdgeIron stack (helios) is connected to the FastIron with 2 aggregated 10GB links.

Each frontend is connected to the FastIron switch.

Network usage is available through cricket at: https://helpdesk.grid5000.fr/cricket/grapher.cgi?target=%2Fsophia%2Fgw-sophia.sophia.grid5000.fr

IP pool for virtualization

Ongoing tests with Kavlan are using half of the virtualization address pool (see Virtual_network_interlink ). The parameters of available IPs are:

 network 10.166.0.0/15
 netmask 255.254.0.0
 gateway 10.167.255.254

Myrinet

72 azur nodes and 56 helios nodes are connected to the 128 ports Myrinet 2000 switch.

You can submit jobs with OAR on Myrinet nodes using the 'myrinet' property: oarsub -l nodes=10 -p "myri2g='YES'".

MX driver is enabled by default on these nodes, IP emulation also.

• IP over MX addressing: helios-{1-56}-myri0 and azur-{1-72}-myri0 (192.168.7.{1-128})
• Myrinet network usage is available through cricket at: https://helpdesk.grid5000.fr/cricket/