Grenoble:Hardware: Difference between revisions

From Grid5000
Jump to navigation Jump to search
No edit summary
No edit summary
Line 491: Line 491:
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Network:'''
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Network:'''
|  
|  
* eth0/enp1s0f0np0, Ethernet, configured rate: 25&nbsp;Gbps, model: Broadcom Inc. and subsidiaries BCM57414 NetXtreme-E 10Gb/25Gb RDMA Ethernet Controller, driver: bnxt_en - no KaVLAN<br />
* eth0/enp1s0f0np0, Ethernet, configured rate: 25&nbsp;Gbps, model: Broadcom Inc. and subsidiaries BCM57414 NetXtreme-E 10Gb/25Gb RDMA Ethernet Controller, driver: bnxt_en<br />
* eth1/ens15f1np1, Ethernet, configured rate: 25&nbsp;Gbps, model: Broadcom Inc. and subsidiaries BCM57414 NetXtreme-E 10Gb/25Gb RDMA Ethernet Controller, driver: bnxt_en - no KaVLAN<br />
* eth1/ens15f1np1, Ethernet, configured rate: 25&nbsp;Gbps, model: Broadcom Inc. and subsidiaries BCM57414 NetXtreme-E 10Gb/25Gb RDMA Ethernet Controller, driver: bnxt_en [[Advanced_KaVLAN#A_simple_multi_NICs_example|(multi NICs example)]]<br />
* eth2/ens4np0, Ethernet, configured rate: 100&nbsp;Gbps, model: Mellanox Technologies MT28908 Family [ConnectX-6], driver: mlx5_core - no KaVLAN<br/>
* eth2/ens4np0, Ethernet, configured rate: 100&nbsp;Gbps, model: Mellanox Technologies MT28908 Family [ConnectX-6], driver: mlx5_core [[Advanced_KaVLAN#A_simple_multi_NICs_example|(multi NICs example)]]<br/>
|-
|-
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''GPU:'''
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''GPU:'''
Line 538: Line 538:
* <span style="color:grey">eth1, Ethernet, model: Broadcom Inc. and subsidiaries BCM57414 NetXtreme-E 10Gb/25Gb RDMA Ethernet Controller, driver: bnxt_en - unavailable for experiment</span><br/>
* <span style="color:grey">eth1, Ethernet, model: Broadcom Inc. and subsidiaries BCM57414 NetXtreme-E 10Gb/25Gb RDMA Ethernet Controller, driver: bnxt_en - unavailable for experiment</span><br/>
|-
|-
|}''<small>Last generated from the Grid'5000 Reference API on 2025-02-18 ([https://gitlab.inria.fr/grid5000/reference-repository/commit/fa6d9442b0 commit fa6d9442b0])</small>''
|}''<small>Last generated from the Grid'5000 Reference API on 2025-02-26 ([https://gitlab.inria.fr/grid5000/reference-repository/commit/b63c99645d commit b63c99645d])</small>''

Revision as of 13:24, 26 February 2025

Hardware: Global | Grenoble | Lille | Louvain | Luxembourg | Lyon | Nancy | Nantes | Rennes | Sophia | Strasbourg | Toulouse

See also: Network topology for Grenoble

Summary

  • 7 clusters
  • 61 nodes
  • 2096 CPU cores
  • 60 GPUs
  • 390144 GPUs cores
  • 13.81 TiB RAM + 6.0 TiB PMEM
  • 96 SSDs and 68 HDDs on nodes (total: 242.45 TB)
  • 145.8 TFLOPS (excluding GPUs)

Clusters summary

Default queue resources

Cluster Access Condition Date of arrival Manufacturing date Nodes CPU Memory Storage Network Accelerators
# Name Cores Architecture
dahu 2018-03-22 2017-12-12 32 2 Intel Xeon Gold 6130 16 cores/CPU x86_64 192 GiB 240 GB SSD + 480 GB SSD + 4.0 TB HDD 10 Gbps + 100 Gbps Omni-Path
drac exotic job type 2020-10-05 2016-10-17 12 2 IBM POWER8NVL 1.0 10 cores/CPU ppc64le 128 GiB 1.0 TB HDD + 1.0 TB HDD 10 Gbps + 2 x 100 Gbps InfiniBand 4 x Nvidia Tesla P100 (16 GiB)
servan exotic job type 2021-12-15 2021-12-10 2 2 AMD EPYC 7352 24 cores/CPU x86_64 128 GiB 1.6 TB SSD + 1.6 TB SSD 25 Gbps + 2 x 100 Gbps FPGA/Ethernet Xilinx Alveo U200
troll exotic job type 2019-12-23 2019-11-21 4 2 Intel Xeon Gold 5218 16 cores/CPU x86_64 384 GiB + 1.5 TiB PMEM 480 GB SSD + 1.6 TB SSD 25 Gbps (SR‑IOV) + 100 Gbps Omni-Path
yeti exotic job type 2018-01-16 2017-12-26 4 4 Intel Xeon Gold 6130 16 cores/CPU x86_64 768 GiB 480 GB SSD + 3 x 2.0 TB HDD* + 2 x 1.6 TB SSD 10 Gbps + 100 Gbps Omni-Path

*: disk is reservable      **: crossed GPUs are not supported by Grid'5000 default environments      ***: OPA (Omni-Path Architecture) is currently not supported on Debian 12 environment

Testing queue resources

Cluster Access Condition Date of arrival Manufacturing date Nodes CPU Memory Storage Network Accelerators
# Name Cores Architecture
kinovis testing queue 2025-02-10 2024-06-26 6 2 Intel Xeon Gold 6442Y 24 cores/CPU x86_64 256 GiB 1.92 TB SSD 2 x 25 Gbps + 100 Gbps  2 x Nvidia Tesla L40S (45 GiB)
nessie testing queue 2024-11-08 2024-08-19 1 2 Intel Xeon Gold 6430 32 cores/CPU x86_64 64 GiB 1.92 TB SSD + 480 GB SSD 25 Gbps 

*: disk is reservable      **: crossed GPUs are not supported by Grid'5000 default environments      ***: OPA (Omni-Path Architecture) is currently not supported on Debian 12 environment

Clusters in the default queue

dahu

32 nodes, 64 cpus, 1024 cores, split as follows due to differences between nodes (json)

Reservation example:

Terminal.png fgrenoble:
oarsub -p dahu -I
dahu-[1,4-32] (30 nodes, 60 cpus, 960 cores)
Model: Dell PowerEdge C6420
Manufacturing date: 2017-12-12
Date of arrival: 2018-03-22
CPU: Intel Xeon Gold 6130 (Skylake-SP), x86_64, 2.10GHz, 2 CPUs/node, 16 cores/CPU
Memory: 192 GiB
Storage:
  • disk0, 240 GB SSD SATA Samsung MZ7KM240HMHQ0D3 (dev: /dev/disk0) (primary disk)
  • disk1, 480 GB SSD SATA Samsung MZ7KM480HMHQ0D3 (dev: /dev/disk1)
  • disk2, 4.0 TB HDD SATA Seagate ST4000NM0265-2DC (dev: /dev/disk2)
Network:
  • eth0/enp24s0f0, Ethernet, configured rate: 10 Gbps, model: Intel Ethernet Controller X710 for 10GbE SFP+, driver: i40e
  • eth1/enp24s0f1, Ethernet, model: Intel Ethernet Controller X710 for 10GbE SFP+, driver: i40e - unavailable for experiment
  • ib0, Omni-Path, configured rate: 100 Gbps, model: Intel Omni-Path HFI Silicon 100 Series [discrete], driver: hfi1
dahu-2 (1 node, 2 cpus, 32 cores)
Model: Dell PowerEdge C6420
Manufacturing date: 2017-12-12
Date of arrival: 2018-03-22
CPU: Intel Xeon Gold 6130 (Skylake-SP), x86_64, 2.10GHz, 2 CPUs/node, 16 cores/CPU
Memory: 192 GiB
Storage:
  • disk0, 240 GB SSD SATA Samsung MZ7KM240HMHQ0D3 (dev: /dev/disk0) (primary disk)
  • disk1, 480 GB SSD SATA Samsung MZ7KM480HMHQ0D3 (dev: /dev/disk1)
  • disk2, 4.0 TB HDD SATA Toshiba TOSHIBA MG08ADA4 (dev: /dev/disk2)
Network:
  • eth0/enp24s0f0, Ethernet, configured rate: 10 Gbps, model: Intel Ethernet Controller X710 for 10GbE SFP+, driver: i40e
  • eth1/enp24s0f1, Ethernet, model: Intel Ethernet Controller X710 for 10GbE SFP+, driver: i40e - unavailable for experiment
  • ib0, Omni-Path, configured rate: 100 Gbps, model: Intel Omni-Path HFI Silicon 100 Series [discrete], driver: hfi1
dahu-3 (1 node, 2 cpus, 32 cores)
Model: Dell PowerEdge C6420
Manufacturing date: 2017-12-12
Date of arrival: 2018-03-22
CPU: Intel Xeon Gold 6130 (Skylake-SP), x86_64, 2.10GHz, 2 CPUs/node, 16 cores/CPU
Memory: 192 GiB
Storage:
  • disk0, 240 GB SSD SATA Samsung MZ7KM240HMHQ0D3 (dev: /dev/disk0) (primary disk)
  • disk1, 480 GB SSD SATA Samsung MZ7KM480HMHQ0D3 (dev: /dev/disk1)
  • disk2, 4.0 TB HDD SATA Seagate ST4000NM018B-2TF (dev: /dev/disk2)
Network:
  • eth0/enp24s0f0, Ethernet, configured rate: 10 Gbps, model: Intel Ethernet Controller X710 for 10GbE SFP+, driver: i40e
  • eth1/enp24s0f1, Ethernet, model: Intel Ethernet Controller X710 for 10GbE SFP+, driver: i40e - unavailable for experiment
  • ib0, Omni-Path, configured rate: 100 Gbps, model: Intel Omni-Path HFI Silicon 100 Series [discrete], driver: hfi1

drac

12 nodes, 24 cpus, 240 cores (json)

Reservation example:

Terminal.png fgrenoble:
oarsub -t exotic -p drac -I
Access condition: exotic job type
Model: IBM PowerNV S822LC (8335-GTB)
Manufacturing date: 2016-10-17
Date of arrival: 2020-10-05
CPU: IBM POWER8NVL 1.0 (POWER8), ppc64le, 2 CPUs/node, 10 cores/CPU
Memory: 128 GiB
Storage:
  • disk0, 1.0 TB HDD SATA Seagate ST1000NX0313 (dev: /dev/disk0) (primary disk)
  • disk1, 1.0 TB HDD SATA Seagate ST1000NX0313 (dev: /dev/disk1)
Network:
  • eth0/enP1p1s0f0, Ethernet, configured rate: 10 Gbps, model: Broadcom Inc. and subsidiaries NetXtreme II BCM57800 1/10 Gigabit Ethernet, driver: bnx2x
  • eth1/enP1p1s0f1, Ethernet, model: Broadcom Inc. and subsidiaries NetXtreme II BCM57800 1/10 Gigabit Ethernet, driver: bnx2x - unavailable for experiment
  • eth2/enP1p1s0f2, Ethernet, model: Broadcom Inc. and subsidiaries NetXtreme II BCM57800 1/10 Gigabit Ethernet, driver: bnx2x - unavailable for experiment
  • eth3/enP1p1s0f3, Ethernet, model: Broadcom Inc. and subsidiaries NetXtreme II BCM57800 1/10 Gigabit Ethernet, driver: bnx2x - unavailable for experiment
  • eth4/enP9p7s0f0, Ethernet, configured rate: 1 Gbps, model: Broadcom Inc. and subsidiaries NetXtreme BCM5719 Gigabit Ethernet PCIe, driver: tg3 - unavailable for experiment
  • eth5/enP9p7s0f1, Ethernet, model: Broadcom Inc. and subsidiaries NetXtreme BCM5719 Gigabit Ethernet PCIe, driver: tg3 - unavailable for experiment
  • ib0, InfiniBand, configured rate: 100 Gbps, model: Mellanox Technologies MT27700 Family [ConnectX-4], driver: mlx5_core
  • ib1, InfiniBand, configured rate: 100 Gbps, model: Mellanox Technologies MT27700 Family [ConnectX-4], driver: mlx5_core
GPU: 4 x Nvidia Tesla P100-SXM2-16GB (16 GiB)
Compute capability: 6.0

Note: This cluster is defined as exotic. Please read the exotic page for more information.

servan

2 nodes, 4 cpus, 96 cores (json)

Reservation example:

Terminal.png fgrenoble:
oarsub -t exotic -p servan -I
Access condition: exotic job type
Model: Dell PowerEdge R7525
Manufacturing date: 2021-12-10
Date of arrival: 2021-12-15
CPU: AMD EPYC 7352 (Zen 2), x86_64, 2 CPUs/node, 24 cores/CPU
Memory: 128 GiB
Storage:
  • disk0, 1.6 TB SSD NVME Dell Dell Ent NVMe v2 AGN MU U.2 1.6TB (dev: /dev/disk0) (primary disk)
  • disk1, 1.6 TB SSD NVME Dell Dell Ent NVMe v2 AGN MU U.2 1.6TB (dev: /dev/disk1)
Network:
  • eth0/eno33, Ethernet, configured rate: 25 Gbps, model: Intel Ethernet Controller E810-C for SFP, driver: ice
  • eth1/eno34, Ethernet, model: Intel Ethernet Controller E810-C for SFP, driver: ice - unavailable for experiment
  • eth2/eno35, Ethernet, model: Intel Ethernet Controller E810-C for SFP, driver: ice - unavailable for experiment
  • eth3/eno36, Ethernet, model: Intel Ethernet Controller E810-C for SFP, driver: ice - unavailable for experiment
  • fpga0, FPGA/Ethernet, configured rate: 100 Gbps, model: Xilinx Alveo U200 QSFP28 (100GbE), driver: n/a
  • fpga1, FPGA/Ethernet, configured rate: 100 Gbps, model: Xilinx Alveo U200 QSFP28 (100GbE), driver: n/a
FPGA: Xilinx Alveo U200

Note: This cluster is defined as exotic. Please read the exotic page for more information.

troll

4 nodes, 8 cpus, 128 cores (json)

Reservation example:

Terminal.png fgrenoble:
oarsub -t exotic -p troll -I
Access condition: exotic job type
Model: Dell PowerEdge R640
Manufacturing date: 2019-11-21
Date of arrival: 2019-12-23
CPU: Intel Xeon Gold 5218 (Cascade Lake-SP), x86_64, 2.30GHz, 2 CPUs/node, 16 cores/CPU
Memory: 384 GiB + 1.5 TiB PMEM
Storage:
  • disk0, 480 GB SSD SATA Micron MTFDDAK480TDN (dev: /dev/disk0) (primary disk)
  • disk1, 1.6 TB SSD NVME Dell Samsung PM1735 (dev: /dev/disk1)
Network:
  • eth0/eno1, Ethernet, configured rate: 25 Gbps, model: Mellanox Technologies MT27710 Family [ConnectX-4 Lx], driver: mlx5_core, SR-IOV enabled
  • eth1/eno2, Ethernet, model: Mellanox Technologies MT27710 Family [ConnectX-4 Lx], driver: mlx5_core - unavailable for experiment
  • ib0, Omni-Path, configured rate: 100 Gbps, model: Intel Omni-Path HFI Silicon 100 Series [discrete], driver: hfi1

Note: This cluster is defined as exotic. Please read the exotic page for more information.

yeti

4 nodes, 16 cpus, 256 cores, split as follows due to differences between nodes (json)

Reservation example:

Terminal.png fgrenoble:
oarsub -t exotic -p yeti -I
yeti-1 (1 node, 4 cpus, 64 cores)
Access condition: exotic job type
Model: Dell PowerEdge R940
Manufacturing date: 2017-12-26
Date of arrival: 2018-01-16
CPU: Intel Xeon Gold 6130 (Skylake-SP), x86_64, 2.10GHz, 4 CPUs/node, 16 cores/CPU
Memory: 768 GiB
Storage:
  • disk0, 480 GB SSD SATA Intel SSDSC2KG480G7R (dev: /dev/disk0) (primary disk)
  • disk1, 2.0 TB HDD SAS Seagate ST2000NX0463 (dev: /dev/disk1*) (reservable)
  • disk2, 2.0 TB HDD SAS Seagate ST2000NX0463 (dev: /dev/disk2*) (reservable)
  • disk3, 2.0 TB HDD SAS Seagate ST2000NX0463 (dev: /dev/disk3*) (reservable)
  • disk4, 1.6 TB SSD NVME Dell Dell Express Flash PM1725b 1.6TB AIC (dev: /dev/disk4)
  • disk5, 1.6 TB SSD NVME Dell Dell Express Flash NVMe PM1725 1.6TB AIC (dev: /dev/disk5)
Network:
  • eth0/eno1, Ethernet, configured rate: 10 Gbps, model: Intel Ethernet Controller X710 for 10GbE SFP+, driver: i40e
  • eth1/eno2, Ethernet, model: Intel Ethernet Controller X710 for 10GbE SFP+, driver: i40e - unavailable for experiment
  • eth2/eno3, Ethernet, model: Intel Ethernet Controller X710 for 10GbE SFP+, driver: i40e - unavailable for experiment
  • eth3/eno4, Ethernet, model: Intel Ethernet Controller X710 for 10GbE SFP+, driver: i40e - unavailable for experiment
  • ib0, Omni-Path, configured rate: 100 Gbps, model: Intel Omni-Path HFI Silicon 100 Series [discrete], driver: hfi1
yeti-[2,4] (2 nodes, 8 cpus, 128 cores)
Access condition: exotic job type
Model: Dell PowerEdge R940
Manufacturing date: 2017-12-26
Date of arrival: 2018-01-16
CPU: Intel Xeon Gold 6130 (Skylake-SP), x86_64, 2.10GHz, 4 CPUs/node, 16 cores/CPU
Memory: 768 GiB
Storage:
  • disk0, 480 GB SSD SATA Intel SSDSC2KG480G7R (dev: /dev/disk0) (primary disk)
  • disk1, 2.0 TB HDD SAS Seagate ST2000NX0463 (dev: /dev/disk1*) (reservable)
  • disk2, 2.0 TB HDD SAS Seagate ST2000NX0463 (dev: /dev/disk2*) (reservable)
  • disk3, 2.0 TB HDD SAS Seagate ST2000NX0463 (dev: /dev/disk3*) (reservable)
  • disk4, 1.6 TB SSD NVME Dell Dell Express Flash NVMe PM1725 1.6TB AIC (dev: /dev/disk4)
  • disk5, 1.6 TB SSD NVME Dell Dell Express Flash NVMe PM1725 1.6TB AIC (dev: /dev/disk5)
Network:
  • eth0/eno1, Ethernet, configured rate: 10 Gbps, model: Intel Ethernet Controller X710 for 10GbE SFP+, driver: i40e
  • eth1/eno2, Ethernet, model: Intel Ethernet Controller X710 for 10GbE SFP+, driver: i40e - unavailable for experiment
  • eth2/eno3, Ethernet, model: Intel Ethernet Controller X710 for 10GbE SFP+, driver: i40e - unavailable for experiment
  • eth3/eno4, Ethernet, model: Intel Ethernet Controller X710 for 10GbE SFP+, driver: i40e - unavailable for experiment
  • ib0, Omni-Path, configured rate: 100 Gbps, model: Intel Omni-Path HFI Silicon 100 Series [discrete], driver: hfi1
yeti-3 (1 node, 4 cpus, 64 cores)
Access condition: exotic job type
Model: Dell PowerEdge R940
Manufacturing date: 2017-12-26
Date of arrival: 2018-01-16
CPU: Intel Xeon Gold 6130 (Skylake-SP), x86_64, 2.10GHz, 4 CPUs/node, 16 cores/CPU
Memory: 768 GiB
Storage:
  • disk0, 480 GB SSD SATA Intel SSDSC2KG480G8R (dev: /dev/disk0) (primary disk)
  • disk1, 2.0 TB HDD SAS Seagate ST2000NX0463 (dev: /dev/disk1*) (reservable)
  • disk2, 2.0 TB HDD SAS Seagate ST2000NX0463 (dev: /dev/disk2*) (reservable)
  • disk3, 2.0 TB HDD SAS Seagate ST2000NX0463 (dev: /dev/disk3*) (reservable)
  • disk4, 1.6 TB SSD NVME Dell Dell Express Flash NVMe PM1725 1.6TB AIC (dev: /dev/disk4)
  • disk5, 1.6 TB SSD NVME Dell Dell Express Flash NVMe PM1725 1.6TB AIC (dev: /dev/disk5)
Network:
  • eth0/eno1, Ethernet, configured rate: 10 Gbps, model: Intel Ethernet Controller X710 for 10GbE SFP+, driver: i40e
  • eth1/eno2, Ethernet, model: Intel Ethernet Controller X710 for 10GbE SFP+, driver: i40e - unavailable for experiment
  • eth2/eno3, Ethernet, model: Intel Ethernet Controller X710 for 10GbE SFP+, driver: i40e - unavailable for experiment
  • eth3/eno4, Ethernet, model: Intel Ethernet Controller X710 for 10GbE SFP+, driver: i40e - unavailable for experiment
  • ib0, Omni-Path, configured rate: 100 Gbps, model: Intel Omni-Path HFI Silicon 100 Series [discrete], driver: hfi1

Note: This cluster is defined as exotic. Please read the exotic page for more information.

Clusters in the testing queue

kinovis

6 nodes, 12 cpus, 288 cores (json)

Reservation example:

Terminal.png fgrenoble:
oarsub -q testing -p kinovis -I
Access condition: testing queue
Model: HPE Proliant DL380 Gen11
Manufacturing date: 2024-06-26
Date of arrival: 2025-02-10
CPU: Intel Xeon Gold 6442Y (Sapphire Rapids), x86_64, 2 CPUs/node, 24 cores/CPU
Memory: 256 GiB
Storage: disk0, 1.92 TB SSD SATA HPE MR416i-o Gen11 (dev: /dev/disk0) (primary disk)
Network:
  • eth0/enp1s0f0np0, Ethernet, configured rate: 25 Gbps, model: Broadcom Inc. and subsidiaries BCM57414 NetXtreme-E 10Gb/25Gb RDMA Ethernet Controller, driver: bnxt_en
  • eth1/ens15f1np1, Ethernet, configured rate: 25 Gbps, model: Broadcom Inc. and subsidiaries BCM57414 NetXtreme-E 10Gb/25Gb RDMA Ethernet Controller, driver: bnxt_en (multi NICs example)
  • eth2/ens4np0, Ethernet, configured rate: 100 Gbps, model: Mellanox Technologies MT28908 Family [ConnectX-6], driver: mlx5_core (multi NICs example)
GPU: 2 x Nvidia L40S (45 GiB)
Compute capability: 8.9

nessie

1 node, 2 cpus, 64 cores (json)

Reservation example:

Terminal.png fgrenoble:
oarsub -q testing -p nessie -I
Access condition: testing queue
Model: HPE ProLiant DL385 Gen10+ v2
Manufacturing date: 2024-08-19
Date of arrival: 2024-11-08
CPU: Intel Xeon Gold 6430 (Sapphire Rapids), x86_64, 2 CPUs/node, 32 cores/CPU
Memory: 64 GiB
Storage:
  • disk0, 1.92 TB SSD NVME HP VO001920KYDMT (dev: /dev/disk0) (primary disk)
  • disk1, 480 GB SSD SATA HP VK000480GZCNE (dev: /dev/disk1)
Network:
  • eth0, Ethernet, configured rate: 25 Gbps, model: Broadcom Inc. and subsidiaries BCM57414 NetXtreme-E 10Gb/25Gb RDMA Ethernet Controller, driver: bnxt_en - no KaVLAN
  • eth1, Ethernet, model: Broadcom Inc. and subsidiaries BCM57414 NetXtreme-E 10Gb/25Gb RDMA Ethernet Controller, driver: bnxt_en - unavailable for experiment

Last generated from the Grid'5000 Reference API on 2025-02-26 (commit b63c99645d)

Retrieved from "https://www.grid5000.fr/mediawiki/index.php?title=Grenoble:Hardware&oldid=105159"