Rennes:Network: Difference between revisions
Line 67: | Line 67: | ||
== High-Speed Networks == | == High-Speed Networks == | ||
=== InfiniBand on HP nodes (paravent) === | === InfiniBand on HP nodes (paravent) === | ||
<code class="host">paravent01</code> to <code class="host"> | <code class="host">paravent01</code> to <code class="host">paravent66</code> have a 10GB Infiniband card (a Mellanox MT23108). | ||
Infiniband nodes are connected through 7 [http://www.voltaire.com/isr_9024.htm Voltaire ISR-9024] switches. | Infiniband nodes are connected through 7 [http://www.voltaire.com/isr_9024.htm Voltaire ISR-9024] switches. | ||
Revision as of 13:41, 4 July 2006
Network topology
- Our network connectivity is provided by a single switch: a Cisco 6509. It provides us with a gigabit ethernet connection to each cluster node, a 10G uplink to Renater and a Gigabit uplink to the IRISA Network (used for DNS/NTP access, and connection to satellite sites).
- The service processors of HP Nodes are connected with HP ProCurve 2848 switches.
- The service processors of Sun Nodes are connected with a 10Mb hub.
- Cisco 3750 switches have a 64Gb switching capacity, and they are stacked with a 32Gb bus. The uplink of each Cisco 3750 stack is a gigabit optical fiber (They are temporarily retired).
Node names
- Head nodes are assigned to compilation, debugging and running reservations through OAR.
- Compute nodes must be reserved before use with GridPrems or OAR.
Cluster |
Head Node from grid'5000 |
Head Node from IRISA |
Compute Nodes |
---|---|---|---|
Apple | tartopom-dev.rennes.grid5000.fr
|
tartopom-dev.irisa.fr
|
tartopom{01..32}.rennes.grid5000.fr
|
Dell | paraci-dev.rennes.grid5000.fr
|
paraci-dev.irisa.fr
|
paraci{01..64}.rennes.grid5000.fr
|
HP | paravent-dev.rennes.grid5000.fr
|
paravent-dev.irisa.fr
|
paravent{01..99}.rennes.grid5000.fr
|
Sun | parasol-dev.rennes.grid5000.fr
|
parasol-dev.irisa.fr
|
parasol{01..64}.rennes.grid5000.fr
|
Addressing
Les plages d'addresses suivantes sont réservées et utilisées par Rennes :
- 131.254.202.0/23
- Used by the ethernet and infiniband interface of nodes
- 10.254.202.0/23
- These addresses are not routed (ie they can only be accessed from nodes at Rennes). They are used for the service processors (aka BMCs) or the
parasoleil
andparavent
nodes.
The addresses assignement for routed addresses is :
131.254.202.0 -> 131.254.202.253
:parasol
,paraci
, andtartopom
clusters131.254.203.2 -> 131.254.203.101
:paravent
compute nodes (Ethernet)131.254.203.102 -> 131.254.203.201
:paravent
compute nodes (InfiniBand)131.254.203.202 -> 131.254.203.204
:paravent
service nodes
The addresses assignement for non routed addresses is :
10.254.202.100 -> 10.254.202.165
:parasol
service processors10.254.203.2 -> 10.254.203.102
: service processors forparavent
compute nodes10.254.203.202 -> 10.254.203.204
: service processors forparavent
service nodes
High-Speed Networks
InfiniBand on HP nodes (paravent)
paravent01
to paravent66
have a 10GB Infiniband card (a Mellanox MT23108).
Infiniband nodes are connected through 7 Voltaire ISR-9024 switches.
Switchs topology :
Myrinet on Dell nodes (paraci)
Nodes paraci17
to paraci32
have a 2 gigabit Myrinet network card, that has these characteristics:
- bus: PCI
- CPU: LANai9.0
These 16 cards are connected to a Myrinet switch M3-E32. This switch contains 3 M3-SW16-8S daughterboards, each with 8 ports. 8 ports of this switch are currently unused.
The driver installed is GM 2.1.19, it is installed in /stow-i386/gm-2.1.9
.
Myrinet-enabled nodes are tagged with the lanai9
property in OAR. This links display their current state in Monika: monika.