Rennes:Network
Jump to navigation
Jump to search
Network: Global | Grenoble | Lille | Luxembourg | Lyon | Nancy | Nantes | Rennes | Sophia | Strasbourg | Toulouse
Network topology
Experiment Network
- Our network connectivity is provided by a single switch: a Cisco 6509. It provides us with a gigabit ethernet connection to each cluster node, a 10G uplink to Renater and a Gigabit uplink to the IRISA Network (used for DNS/NTP access, and connection to satellite sites).
Management Network
- The service processors of HP Nodes are connected with HP ProCurve 2848 switches.
- The service processors of Sun Nodes are connected with a 10Mb hub.
- Cisco 3750 switches have a 64Gb switching capacity, and they are stacked with a 32Gb bus.
Node names
- Head nodes are assigned to compilation, debugging and running reservations through OAR.
- Compute nodes must be reserved before use with OAR on
frontend.rennes.grid5000.fr
Cluster |
Compute Nodes |
---|---|
Dell | paraquad{01..64}.rennes.grid5000.fr
|
HP | paravent{01..99}.rennes.grid5000.fr
|
Sun | parasol{01..64}.rennes.grid5000.fr
|
Addressing
Les plages d'addresses suivantes sont réservées et utilisées par Rennes :
- 131.254.202.0/23
- Used by the ethernet interface of nodes
- 192.168.202.0/23
- These addresses are not routed (ie they can only be accessed from head nodes at Rennes). They are used for the service processors (aka BMCs) or the
parasol
,paravent
andparaquad
nodes. - 192.168.200.0/23
- Used for IP over High Performance Networks.
The addresses assignement for routed addresses is :
131.254.202.0 -> 131.254.202.253
:parasol
, andparaquad
clusters.131.254.203.2 -> 131.254.203.101
:paravent
compute nodes (Ethernet)131.254.203.202 -> 131.254.203.204
:paravent
service nodes131.254.203.102 -> 131.254.203.201
:paravent
compute nodes (InfiniBand)
The addresses assignement for non routed addresses is :
192.168.202.100 -> 192.168.202.165
:parasol
service processors192.168.203.2 -> 192.168.203.102
: service processors forparavent
compute nodes192.168.203.202 -> 192.168.203.204
: service processors forparavent
service nodes
High Performance Networks
InfiniBand on HP nodes (paravent)
paravent01
to paravent66
have a 10GB Infiniband card (a Mellanox MT23108).
Infiniband nodes are connected through 7 Voltaire ISR-9024 switches.
Switchs topology :
Note:
- The nodes are connected to ports 1 to 13 (to port 14 for node parvent66).
- There are 2x4 uplinks per switch (ports 17-20 & 20-24).
- The first uplink (ports 17-20) seems to be not very stable.
- OAR property : infiniBand (Monika). Will be standardized with OAR2.
- IP over IB addressing : paravent[01..66].ib (192.168.201.3 à 192.168.201.68)
Myrinet 10G on HP nodes (paravent)
paravent67
to paravent99
have a 10GB Myrinet card.
- Drivers :
/opt/mx/
- OAR property : myri (Monika). Will be standardized with OAR2.
- IP over MX addressing : paravent[67..99].myri (192.168.201.69 à 192.168.201.101)