Rennes:Network: Difference between revisions

From Grid5000
Jump to navigation Jump to search
No edit summary
Line 2: Line 2:
{{Portal|Network}}
{{Portal|Network}}


== Network topology ==
= IP networks in use =


=== Experiment Network ===
You have to use a public network range to run an experiment between several Grid5000 sites.
 
=== Public Networks ===
 
* computing : '''131.254.202.0/23'''
* virtual : '''10.156.0.0/14'''
 
=== Local Networks ===
 
* ipmi : '''192.168.202.0/23'''
* 10G nets : '''192.168.200.0/24''' : see [[Rennes:Network#High_Performance_Networks|High Performance Networks]]
 
== Assignment policy ==
 
 
The addresses assignement for routed addresses is :
* <code>131.254.202.0&nbsp;&nbsp; -> 131.254.202.253&nbsp;</code>: <code class="host">paraquad</code>, and <code class="host">paramount</code> clusters.
* <code>131.254.203.2&nbsp;&nbsp; -> 131.254.203.101&nbsp;</code>: <code class="host">paravent</code> compute nodes (Ethernet)
* <code>131.254.203.202 -> 131.254.203.204&nbsp;</code>: <code class="host">paravent</code> service nodes
* <code>131.254.203.102 -> 131.254.203.201&nbsp;</code>: <code class="host">paravent</code> compute nodes (InfiniBand)
 
The addresses assignement for non routed addresses is :
* <code>192.168.202.100 -> 192.168.202.203&nbsp;</code>: <code class="host">paraquad</code>, and <code class="host">paramount</code> service processors
* <code>192.168.203.2&nbsp;&nbsp; -> 192.168.203.102&nbsp;</code>: service processors for <code class="host">paravent</code> compute nodes
* <code>192.168.203.202 -> 192.168.203.204&nbsp;</code>: service processors for <code class="host">paravent</code> service nodes
 
= Network =
 
== Experiment Network ==


[[Image:Rennes_experiment_network.png]]
[[Image:Rennes_experiment_network.png]]
Line 10: Line 38:
* Our network connectivity is provided by a single switch: a [http://www.cisco.com/en/US/products/hw/switches/ps708/ps711/index.html Cisco 6509]. It provides us with a gigabit ethernet connection to each cluster node, a 10G uplink to Renater and a Gigabit uplink to the IRISA Network (used for DNS/NTP access, and connection to satellite sites).
* Our network connectivity is provided by a single switch: a [http://www.cisco.com/en/US/products/hw/switches/ps708/ps711/index.html Cisco 6509]. It provides us with a gigabit ethernet connection to each cluster node, a 10G uplink to Renater and a Gigabit uplink to the IRISA Network (used for DNS/NTP access, and connection to satellite sites).


=== Management Network ===
== Management Network ==


[[Image:Rennes_management_network.png]]
[[Image:Rennes_management_network.png]]
Line 39: Line 67:
|}
|}


= Addressing =
==  Virtual (routable) ==
Can be used by users :
* 10.156.0.0/14 (Netmask: 255.252.0.0)


== High Performance Networks ==
== High Performance Networks ==
Line 100: Line 120:
* slot 15 10G-SW16LC-8C
* slot 15 10G-SW16LC-8C
* slot 16 10G-SW16LC-8C
* slot 16 10G-SW16LC-8C
==== 10G Ethernet uplink with Cisco ====
* Need to deploy and use [[Rennes:Network#Virtual_.28routable.29|virtual IPs]].

Revision as of 11:53, 18 June 2009

Network: Global | Grenoble | Lille | Luxembourg | Lyon | Nancy | Nantes | Rennes | Sophia | Toulouse

IP networks in use

You have to use a public network range to run an experiment between several Grid5000 sites.

Public Networks

  • computing : 131.254.202.0/23
  • virtual : 10.156.0.0/14

Local Networks

Assignment policy

The addresses assignement for routed addresses is :

  • 131.254.202.0   -> 131.254.202.253 : paraquad, and paramount clusters.
  • 131.254.203.2   -> 131.254.203.101 : paravent compute nodes (Ethernet)
  • 131.254.203.202 -> 131.254.203.204 : paravent service nodes
  • 131.254.203.102 -> 131.254.203.201 : paravent compute nodes (InfiniBand)

The addresses assignement for non routed addresses is :

  • 192.168.202.100 -> 192.168.202.203 : paraquad, and paramount service processors
  • 192.168.203.2   -> 192.168.203.102 : service processors for paravent compute nodes
  • 192.168.203.202 -> 192.168.203.204 : service processors for paravent service nodes

Network

Experiment Network

Rennes experiment network.png

  • Our network connectivity is provided by a single switch: a Cisco 6509. It provides us with a gigabit ethernet connection to each cluster node, a 10G uplink to Renater and a Gigabit uplink to the IRISA Network (used for DNS/NTP access, and connection to satellite sites).

Management Network

Rennes management network.png

  • The service processors of HP Nodes are connected with HP ProCurve 2848 switches.
  • The service processors of Sun Nodes are connected with a 10Mb hub.
  • Cisco 3750 switches have a 64Gb switching capacity, and they are stacked with a 32Gb bus.

Node names

  • Compute nodes must be reserved before use with OAR on frontend.rennes.grid5000.fr


Cluster

Compute Nodes

Dell paramount-{1..33}.rennes.grid5000.fr
Dell paraquad-{1..64}.rennes.grid5000.fr
HP paravent-{1..99}.rennes.grid5000.fr


High Performance Networks

InfiniBand on HP nodes (paravent)

paravent-1 to paravent-66 have a 10GB Infiniband card (a Mellanox MT23108). Infiniband nodes are connected through 7 Voltaire ISR-9024 switches.

Switchs topology :

Topologie infiniband.png

Note:

  • The nodes are connected to ports 1 to 13 (to port 14 for node parvent-66).
  • There are 2x4 uplinks per switch (ports 17-20 & 20-24).
  • The first uplink (ports 17-20) seems to be not very stable.
  • OAR property : ib10g=YES ([1]).
  • IP over IB addressing : paravent-[1..66]-ib0.rennes.grid5000.fr or paravent-[1..66].ib (192.168.201.3 à 192.168.201.68)

Myrinet 10G on DELL nodes

Nodes

  • paraquad-1 to paraquad-64 have a 10GB Myrinet card.
  • paramount-1 to paraquad-33 have a 10GB Myrinet card.
  • Drivers : /opt/mx/
  • OAR property : myri10g=YES and myri10gmodel=10g-pcie-8a-c ([2]).
  • IP over MX addressing : paraquad-[1..64]-myri0.rennes.grid5000.fr or paraquad-[1..64].myri (192.168.200.21 à 192.168.200.84)
  • IP over MX addressing : paramount-[1..33]-myri0.rennes.grid5000.fr or paramount-[1..33].myri (192.168.201.171 à 192.168.201.203)

Switch

  • slot m M3-MONITOR
    • uc microcode version: M3 Switch Firmware v3.0 Type: LCMONITOR Compiled on: Oct 29 2004 16:44:54
    • uc rom version: 9S08 Monitor v1.0 Compiled on: Oct 17 2004 23:15:08
    • firmware: vv0.9.9.35 Feb 5 2009 18:20:36
  • slot 0 10G-SW16LC-6C2ER
    • uc microcode version: M4 Switch Firmware v1.2 Type: 10G-SW16LC-6C2ER Compiled on: Oct 31 2008 14:57:55
    • uc rom version: 9S08 Monitor v1.0 Compiled on: Oct 17 2004 23:15:08
  • slot 1 10G-SW16LC-8C
    • uc microcode version: M4 Switch Firmware v0.9 Type: 10G-SW16LC-8C Compiled on: Nov 28 2005 16:49:14
    • uc rom version: 9S08 Monitor v1.0 Compiled on: Oct 17 2004 23:15:08
  • slot 2 10G-SW16LC-8C
  • slot 3 10G-SW16LC-8C
  • slot 4 10G-SW16LC-8C
  • slot 5 10G-SW16LC-8C
  • slot 6 10G-SW16LC-8C
  • slot 7 10G-SW16LC-8C
  • slot 12 10G-SW16LC-8C
  • slot 13 10G-SW16LC-8C
  • slot 14 10G-SW16LC-8C
  • slot 15 10G-SW16LC-8C
  • slot 16 10G-SW16LC-8C
Retrieved from "https://www.grid5000.fr/mediawiki/index.php?title=Rennes:Network&oldid=26983"