Sophia:Network

Network: Global | Grenoble | Lille | Luxembourg | Lyon | Nancy | Nantes | Rennes | Sophia | Strasbourg | Toulouse

See also: Network Admins | Network survival guide

IP networks in use

You have to use a public network range to run an experiment between several Grid5000 sites.

Public Networks

• computing : 138.96.20.0/22
• virtual : 10.164.0.0/14

Local Networks

• admin : 10.6.1.0/24
• ipmi : 10.96.20.0/24

Network

Physical view of the compute network

Gigabit Ethernet

Azur Cluster (IBM e325)

All the nodes are connected to 4 Cisco-3750 switches (gigabit ethernet). The switches are interconnected using a 32Gbps stack. The nodes are connected to the following switches:

• cisco1 (Master) : azur-1 to azur-15 (15 nodes), 10GB uplink

to the FastIron

• cisco2 : azur-16 to azur-36 (21 nodes)
• cisco3 : azur-37 to azur-55 (19 nodes)
• cisco4 : azur-56 to azur-72 (17 nodes)

The switches are dedicated to Grid5000 (since january 2007; before that, they were shared with the local cluster).

You can choose the switch for your reservation with OAR, using the switch name:

oarsub -l nodes=XX -p "switch='ciscoX'"

If you want XX nodes on a single switch, use

oarsub -l /switch=1/nodes=XX 

Network usage is available through cricket at : https://helpdesk.grid5000.fr/cricket/grapher.cgi?target=%2Fsophia%2Fcisco

OS used: Cisco IOS Software, C3750 Software (C3750-IPBASE-M), Version 12.2(25)SEB4

Helios Cluster (Sun X4100)

All the nodes are connected in gigabit ethernet to 2 Foudry EdgeIron switches (one Edgeiron 48GS and one EdgeIron24GS). The switches are interconnected using a 2x10Gbps stack (10GBps in each direction). The nodes are connected to the following switches:

• edgeiron48gs : helios01 to helios48 (48 nodes)
• edgeiron24gs : helios49 to helios56 (8 nodes)

You can choose the switch for your reservation with OAR, using the switch name: oarsub -l nodes=15 -p "switch='XXX'"

OS used: 2.2.6.35

Sol Cluster (Sun X2200 M2)

All the nodes are connected (in a non blocking way) to the main Foundry FastIron Super X switch.

47 nodes (from sol-1 to sol-47) have a second Ethernet interface connected to 2 Cisco 3750 stacked switches.

• IP addressing for eth1: sol-{1-47}-eth1, using the 172.16.128.0/24 network ( 172.16.128.154 can be used as a gateway)

The Cisco are interconnected to the main FastIron router with a single 1Gbps link. A separate VLAN is used.

Suno Cluster (Dell R410)

All the nodes are connected (in a non blocking way) to the main Foundry FastIron Super X switch.

Topology

The main switch is a Foundry FastIron Super X. It has 2 dual 10GB modules, 2 modules with 24 gigabit ports, and 12 gigabit ports on the management module (so 60 gigabit ports are available). 4 slots are currently free.

The Cisco 3750 stack (Azur) is connected to the FastIron switch with a 10GB link; The EdgeIron stack (helios) is connected to the FastIron with 2 aggregated 10GB links.

Each frontend is connected to the FastIron switch.

Network usage is available through cricket at: https://helpdesk.grid5000.fr/cricket/grapher.cgi?target=%2Fsophia%2Fgw-sophia.sophia.grid5000.fr

Myrinet

72 azur nodes and 56 helios nodes are connected to the 128 ports Myrinet 2000 switch.

You can submit jobs with OAR on Myrinet nodes using the 'myrinet' property: oarsub -l nodes=10 -p "myri2g='YES'".

MX driver is enabled by default on these nodes, IP emulation also.

• IP over MX addressing: helios-{1-56}-myri0 and azur-{1-72}-myri0 (192.168.7.{1-128})
• Myrinet network usage is available through cricket at: https://helpdesk.grid5000.fr/cricket/