Grenoble:Network: Difference between revisions

From Grid5000
Jump to navigation Jump to search
No edit summary
No edit summary
Line 11: Line 11:
= Omni-Path Topology =
= Omni-Path Topology =


Nodes of the dahu, yeti and lama clusters are connected to Omni-Path switches (Intel Omni-Path 100Gbps), aside the Ethernet network.
Nodes of the dahu, yeti and lama clusters are connected to Omni-Path switches (Intel Omni-Path 100Gbps), beside the Ethernet network.


This Omni-Path network interconnects both the Grid'5000 Grenoble nodes and the nodes of the HPC Center of Université Grenoble Alpes (Gricad mésocentre).
This Omni-Path network interconnects both the Grid'5000 Grenoble nodes and the nodes of the HPC Center of Université Grenoble Alpes (Gricad mésocentre).

Revision as of 15:06, 2 December 2019

Overview of Ethernet network topology

GrenobleNetwork.png


Network devices models

  • gw: Dell S5296F-ON

More details (including address ranges) are available from the Grid5000:Network page.

Omni-Path Topology

Nodes of the dahu, yeti and lama clusters are connected to Omni-Path switches (Intel Omni-Path 100Gbps), beside the Ethernet network.

This Omni-Path network interconnects both the Grid'5000 Grenoble nodes and the nodes of the HPC Center of Université Grenoble Alpes (Gricad mésocentre).

The topology used is a fat tree with a 2:1 blocking factor:

  • 2 top switchs
  • 6 leaves: switchs with 32 downlinks to nodes and 8 uplinks to each top switch (48 ports total)

All 32 dahu nodes are connected to a same leaf. The 4 yeti nodes are connected to another leaf. TBC for lama.

                                                        +-----------------------------------------------------+               +-----------------------------------------------------+
                                                        |    OPA Top switch 1  (48 ports)                     |               |    OPA Top switch 2  (48 ports)                     |
                                                        +-----------------------------------------------------+               +-----------------------------------------------------+
                                                        |8 ports |8 p     |8 p     |8 p     |8 p     |8 p     |               |8 ports |8 p     |8 p     |8 p     |8 p     |8 p     |
                                                        +---+X+-------X---------------------------------------+               +---+X+-------X---------------------------------------+
                                                             X        X                                                            X        X
                                                             X        X                                                            X        X
                                                             X        X                                                            X        X
                                                    8 links  X        XXXXXXXXXXXXXXX                                              X        X
                                                             X                      X                                              X        XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
                                                             X                      X                                              X                                                                                 X
                                                             XXXXXXXXXXX            X                                              X                                                                                 X
                                                                       X            X                                              X                                                                                 X
                                                                       X            X 8 links                              8 links X                                                                                 X
                                                                       X            X                                              X                                                                        8 links  X
            +-----------------------------------------------------+    X            X                                              X                                                                                 X
            |    OPA switch 1  (48 ports)                         |    X            X                                              X                                                                                 X
            +-----------------------------------+--------+--------+    X            X                                              X                                                                                 X
            | 32 ports                          |        |        |    X            X                                              X                                                                                 X
            ++--+---------------+------------+--+---+X+--+--+X+---+    X            X                                              X                                                                                 X
             |  |               |            |       X       X         X            X                                              X                                                                                 X
             |  |               |            |       X       X         X            X                                              X                                                                                 X
             |  |               |            |       XXXXXXXXXXXXXXXXXXX            X                                              X                                                                                 X
      +------+  |               |            |               X                      X                                              X                                                                                 X
      |         |               |            |               X                      X                                              X                                                                                 X
      |         |               |            |               XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX                                                                                 X
      |         |            +--+---+        |                                      X                                                                                                                                X
      |         | +------+   |dahu N|  +-----+-+                                    X                                                                                                                                X
   +--+---+     +-+dahu 2|   +------+  |dahu 32|                                    X                                                                    +-----------------------------------------------------+     X
   |dahu 1|       +------+             +-------+                                    X                                                                    |    OPA switch 2  (48 ports)                         |     X
   +------+                                                                         X                                                                    +-----------------------------------+--------+--------+     X
                                                                                    X                                                                    | 32 ports                          |        |        |     X
                                                                                    X                                                                    ++--+---------------+------------+--+---+X+--+--+X+---+     X
                                                                                    X                                                                     |  |               |            |       X       X          X
                                                                                    X                                                                     |  |               |            |       X       X          X
                                                                                    X                                                                     |  |               |            |       X       X          X
                                                                                    X                                                              +------+  |               |            |       X       XXXXXXXXXXXXX
                                                                                    X                                                              |         |               |            |       X
                                                                                    X                                                              |         |               |            |       X
                                                                                    X                                                              |         |            +--+---+        |       X
                                                                                    X                                                              |         | +------+   |yeti 3|  +-----+-+     X
                                                                                    X                                                           +--+---+     +-+yeti 2|   +------+  |yeti 4 |     X
                                                                                    X                                                           |yeti 1|       +------+             +-------+     X
                                                                                    X                                                           +------+                                          X
                                                                                    X                                                                                                             X
                                                                                    X                                                                                                             X
                                                                                    XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

IP Range

  • Computing : 172.16.16.0/20
  • Omni-Path : 172.18.16.0/20
  • Virtual : 10.132.0.0/14