Nancy:Hardware: Difference between revisions
No edit summary |
Lpouilloux (talk | contribs) No edit summary |
||
Line 207: | Line 207: | ||
== [https://intranet.grid5000.fr/oar/Nancy/drawgantt-svg/?filter=grouille%20only grouille] == | == [https://intranet.grid5000.fr/oar/Nancy/drawgantt-svg/?filter=grouille%20only grouille] == | ||
'''2 nodes, 4 cpus, 128 cores''' ([https://public-api.grid5000.fr/stable/sites/nancy/clusters/grouille/nodes.json?pretty=1 json]) | '''2 nodes, 4 cpus, 128 cores,''' split as follows due to differences between nodes ([https://public-api.grid5000.fr/stable/sites/nancy/clusters/grouille/nodes.json?pretty=1 json]) | ||
'''Reservation example:''' | '''Reservation example:''' | ||
Line 213: | Line 213: | ||
{{Term|location=fnancy|cmd=<code class="command">oarsub</code> <code class="replace">-t exotic</code> <code class="env">-p grouille</code> <code>-I</code>}} | {{Term|location=fnancy|cmd=<code class="command">oarsub</code> <code class="replace">-t exotic</code> <code class="env">-p grouille</code> <code>-I</code>}} | ||
; grouille-1 (1 node, 2 cpus, 64 cores) | |||
{| | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Access condition:''' | |||
| [[Exotic#Nancy:_grouille|exotic]] job type<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Model:''' | |||
| Dell PowerEdge R7525<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Manufacturing date:''' | |||
| 2020-12-07<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Date of arrival:''' | |||
| 2021-01-13<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''CPU:''' | |||
| AMD EPYC 7452 (Zen 2), x86_64, 2 CPUs/node, 32 cores/CPU<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Memory:''' | |||
| 128 GiB<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Storage:''' | |||
| | |||
* disk0, 1.92 TB SSD SAS Toshiba KRM5XVUG1T92 (dev: <code class="file">/dev/disk0</code>, by-path: <code class="file">/dev/disk/by-path/pci-0000:01:00.0-scsi-0:0:1:0</code>) (primary disk)<br /> | |||
* disk1, 960 GB SSD SATA Micron MTFDDAK960TDT (dev: <code class="file">/dev/disk1</code>[[Disk_reservation|*]], by-path: <code class="file">/dev/disk/by-path/pci-0000:01:00.0-scsi-0:0:0:0</code>) [[Disk_reservation|(reservable)]] <br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Network:''' | |||
| | |||
* <span style="color:grey">eth0/eno1, Ethernet, model: Broadcom Inc. and subsidiaries NetXtreme BCM5720 2-port Gigabit Ethernet PCIe, driver: tg3 - unavailable for experiment</span><br /> | |||
* <span style="color:grey">eth1/eno2, Ethernet, model: Broadcom Inc. and subsidiaries NetXtreme BCM5720 2-port Gigabit Ethernet PCIe, driver: tg3 - unavailable for experiment</span><br /> | |||
* eth2/eno33, Ethernet, configured rate: 25 Gbps, model: Mellanox Technologies MT27800 Family [ConnectX-5], driver: mlx5_core<br /> | |||
* <span style="color:grey">eth3/eno34, Ethernet, model: Mellanox Technologies MT27800 Family [ConnectX-5], driver: mlx5_core - unavailable for experiment</span><br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''GPU:''' | |||
| 2 x Nvidia A100-PCIE-40GB (40 GiB)<br>Compute capability: 8.0<br/> | |||
|- | |||
|} | |||
<hr style="height:10pt; visibility:hidden;" /> | |||
; grouille-2 (1 node, 2 cpus, 64 cores) | |||
{| | {| | ||
|- | |- | ||
Line 765: | Line 805: | ||
* ib0, Omni-Path, configured rate: 100 Gbps, model: Intel Omni-Path HFI Silicon 100 Series [discrete], driver: hfi1<br/> | * ib0, Omni-Path, configured rate: 100 Gbps, model: Intel Omni-Path HFI Silicon 100 Series [discrete], driver: hfi1<br/> | ||
|- | |- | ||
|}''<small>Last generated from the Grid'5000 Reference API on 2024-03- | |}''<small>Last generated from the Grid'5000 Reference API on 2024-03-12 ([https://gitlab.inria.fr/grid5000/reference-repository/commit/031d183d10 commit 031d183d10])</small>'' |
Revision as of 14:15, 12 March 2024
See also: Network topology for Nancy
Summary
- 12 clusters
- 291 nodes
- 6872 CPU cores
- 120 GPUs
- 547840 GPUs cores
- 44.13 TiB RAM
- 288 SSDs and 220 HDDs on nodes (total: 512.11 TB)
- 309.6 TFLOPS (excluding GPUs)
Clusters
Cluster | Access Condition | Date of arrival | Manufacturing date | Nodes | CPU | Memory | Storage | Network | Accelerators | |||
---|---|---|---|---|---|---|---|---|---|---|---|---|
# | Name | Cores | Architecture | |||||||||
graffiti | production queue | 2019-06-07 | 2019-05-27 | 13 | 2 | Intel Xeon Silver 4110 | 8 cores/CPU | x86_64 | 128 GiB | 479 GB HDD | 10 Gbps | [1-12]: 4 x Nvidia RTX 2080 Ti (11 GiB) 13: 4 x Nvidia Quadro RTX 6000 (23 GiB) |
grappe | production queue | 2020-08-20 | 2020-07-09 | 16 | 2 | Intel Xeon Gold 5218R | 20 cores/CPU | x86_64 | 96 GiB | 480 GB SSD + 8.0 TB HDD* | 25 Gbps | |
grat | production queue | 2022-09-07 | 2022-06-22 | 1 | 2 | AMD EPYC 7513 | 32 cores/CPU | x86_64 | 512 GiB | 3.84 TB SSD + 7 x 3.84 TB SSD | 25 Gbps (SR‑IOV) | 8 x Nvidia A100 (40 GiB) |
grele | production queue | 2017-06-26 | 2017-06-07 | 14 | 2 | Intel Xeon E5-2650 v4 | 12 cores/CPU | x86_64 | 128 GiB | 299 GB HDD + 299 GB HDD | 10 Gbps (SR‑IOV) + 100 Gbps Omni-Path | 2 x Nvidia GTX 1080 Ti (11 GiB) |
grisou | 2016-01-04 | 2015-10-23 | 46 | 2 | Intel Xeon E5-2630 v3 | 8 cores/CPU | x86_64 | 128 GiB | 600 GB HDD + 600 GB HDD | 1 Gbps + 4 x 10 Gbps (SR‑IOV) | ||
gros | 2019-09-04 | 2019-07-16 | 124 | 1 | Intel Xeon Gold 5220 | 18 cores/CPU | x86_64 | 96 GiB | 480 GB SSD + 960 GB SSD* | 2 x 25 Gbps (SR‑IOV) | ||
grosminet | production queue | 2023-12-05 | 2023-11-30 | 1 | 4 | Intel Xeon Gold 6240L | 18 cores/CPU | x86_64 | 6.0 TiB | 1.6 TB SSD + 7 x 1.6 TB SSD | 25 Gbps (SR‑IOV) | |
grostiti | production queue | 2024-01-10 | 2015-10-23 | 1 | 4 | Intel Xeon E7-4850 v3 | 14 cores/CPU | x86_64 | 1.5 TiB | 599 GB HDD + 4.0 TB HDD | 10 Gbps (SR‑IOV) + 100 Gbps Omni-Path | |
grouille | exotic job type | 2021-01-13 | 2020-12-07 | 2 | 2 | AMD EPYC 7452 | 32 cores/CPU | x86_64 | 128 GiB | 1.92 TB SSD + 960 GB SSD* | 25 Gbps | 2 x Nvidia A100 (40 GiB) |
grue | production queue | 2019-11-25 | 2019-11-15 | 5 | 2 | AMD EPYC 7351 | 16 cores/CPU | x86_64 | 128 GiB | 479 GB HDD | 10 Gbps | 4 x Nvidia Tesla T4 (15 GiB) |
gruss | production queue | 2021-08-26 | 2021-06-24 | 4 | 2 | AMD EPYC 7352 | 24 cores/CPU | x86_64 | 256 GiB | 1.92 TB SSD | 25 Gbps | 2 x Nvidia A40 (45 GiB) |
grvingt | production queue | 2018-04-11 | 2018-04-01 | 64 | 2 | Intel Xeon Gold 6130 | 16 cores/CPU | x86_64 | 192 GiB | 1.0 TB HDD | 10 Gbps + 100 Gbps Omni-Path |
*: disk is reservable
**: crossed GPUs are not supported by Grid'5000 default environments
Clusters in the default queue
grisou
46 nodes, 92 cpus, 736 cores (json)
Reservation example:
Model: | Dell PowerEdge R630 |
Manufacturing date: | 2015-10-23 |
Date of arrival: | 2016-01-04 |
CPU: | Intel Xeon E5-2630 v3 (Haswell), x86_64, 2.40GHz, 2 CPUs/node, 8 cores/CPU |
Memory: | 128 GiB |
Storage: |
|
Network: |
|
gros
124 nodes, 124 cpus, 2232 cores, split as follows due to differences between nodes (json)
Reservation example:
- gros-[1-26,
28-67, 69-124] (122 nodes, 122 cpus, 2196 cores)
Model: | Dell PowerEdge R640 |
Manufacturing date: | 2019-07-16 |
Date of arrival: | 2019-09-04 |
CPU: | Intel Xeon Gold 5220 (Cascade Lake-SP), x86_64, 2.20GHz, 1 CPU/node, 18 cores/CPU |
Memory: | 96 GiB |
Storage: |
|
Network: |
|
- gros-27 (1 node, 1 cpu, 18 cores)
Model: | Dell PowerEdge R640 |
Manufacturing date: | 2019-07-16 |
Date of arrival: | 2019-09-04 |
CPU: | Intel Xeon Gold 5220 (Cascade Lake-SP), x86_64, 2.20GHz, 1 CPU/node, 18 cores/CPU |
Memory: | 96 GiB |
Storage: |
|
Network: |
|
- gros-68 (1 node, 1 cpu, 18 cores)
Model: | Dell PowerEdge R640 |
Manufacturing date: | 2019-07-16 |
Date of arrival: | 2019-09-04 |
CPU: | Intel Xeon Gold 5220 (Cascade Lake-SP), x86_64, 2.20GHz, 1 CPU/node, 18 cores/CPU |
Memory: | 96 GiB |
Storage: |
|
Network: |
|
grouille
2 nodes, 4 cpus, 128 cores, split as follows due to differences between nodes (json)
Reservation example:
- grouille-1 (1 node, 2 cpus, 64 cores)
Access condition: | exotic job type |
Model: | Dell PowerEdge R7525 |
Manufacturing date: | 2020-12-07 |
Date of arrival: | 2021-01-13 |
CPU: | AMD EPYC 7452 (Zen 2), x86_64, 2 CPUs/node, 32 cores/CPU |
Memory: | 128 GiB |
Storage: |
|
Network: |
|
GPU: | 2 x Nvidia A100-PCIE-40GB (40 GiB) Compute capability: 8.0 |
- grouille-2 (1 node, 2 cpus, 64 cores)
Access condition: | exotic job type |
Model: | Dell PowerEdge R7525 |
Manufacturing date: | 2020-12-07 |
Date of arrival: | 2021-01-13 |
CPU: | AMD EPYC 7452 (Zen 2), x86_64, 2 CPUs/node, 32 cores/CPU |
Memory: | 128 GiB |
Storage: |
|
Network: |
|
GPU: | 2 x Nvidia A100-PCIE-40GB (40 GiB) Compute capability: 8.0 |
Note: This cluster is defined as exotic. Please read the exotic page for more information.
Clusters in the production queue
graffiti
13 nodes, 26 cpus, 208 cores, split as follows due to differences between nodes (json)
Reservation example:
Max walltime per nodes:
- graffiti-[1-3]: 24h
- graffiti-[4-6]: 48h
- graffiti-[7-13]: 168h
- graffiti-[1-12] (12 nodes, 24 cpus, 192 cores)
Access condition: | production queue |
Model: | Dell PowerEdge T640 |
Manufacturing date: | 2019-05-27 |
Date of arrival: | 2019-06-07 |
CPU: | Intel Xeon Silver 4110 (Skylake), x86_64, 2.10GHz, 2 CPUs/node, 8 cores/CPU |
Memory: | 128 GiB |
Storage: | disk0, 479 GB HDD SATA Dell PERC H330 Adp (dev: /dev/disk0 , by-path: /dev/disk/by-path/pci-0000:19:00.0-scsi-0:2:0:0 ) (primary disk) |
Network: |
|
GPU: | 4 x Nvidia GeForce RTX 2080 Ti (11 GiB) Compute capability: 7.5 |
- graffiti-13 (1 node, 2 cpus, 16 cores)
Access condition: | production queue |
Model: | Dell PowerEdge T640 |
Manufacturing date: | 2019-05-27 |
Date of arrival: | 2019-06-07 |
CPU: | Intel Xeon Silver 4110 (Skylake), x86_64, 2.10GHz, 2 CPUs/node, 8 cores/CPU |
Memory: | 128 GiB |
Storage: | disk0, 479 GB HDD SATA Dell PERC H330 Adp (dev: /dev/disk0 , by-path: /dev/disk/by-path/pci-0000:19:00.0-scsi-0:2:0:0 ) (primary disk) |
Network: |
|
GPU: | 4 x Nvidia Quadro RTX 6000 (23 GiB) Compute capability: 7.5 |
grappe
16 nodes, 32 cpus, 640 cores (json)
Reservation example:
Max walltime per nodes:
- grappe-[1-4]: 48h
- grappe-[5-8]: 96h
- grappe-[9-16]: 168h
Access condition: | production queue |
Model: | Dell PowerEdge R640 |
Manufacturing date: | 2020-07-09 |
Date of arrival: | 2020-08-20 |
CPU: | Intel Xeon Gold 5218R (Cascade Lake-SP), x86_64, 2.10GHz, 2 CPUs/node, 20 cores/CPU |
Memory: | 96 GiB |
Storage: |
|
Network: |
|
grat
1 node, 2 cpus, 64 cores (json)
Reservation example:
Max walltime per nodes:
- grat-1: 168h
Access condition: | production queue |
Model: | HPE Apollo 6500 |
Manufacturing date: | 2022-06-22 |
Date of arrival: | 2022-09-07 |
CPU: | AMD EPYC 7513 (Zen 3), x86_64, 2 CPUs/node, 32 cores/CPU |
Memory: | 512 GiB |
Storage: |
|
Network: |
|
GPU: | 8 x Nvidia A100-SXM4-40GB (40 GiB) Compute capability: 8.0 |
grele
14 nodes, 28 cpus, 336 cores (json)
Reservation example:
Max walltime per nodes:
- grele-[1-3]: 24h
- grele-[4-6]: 48h
- grele-[7-14]: 168h
Access condition: | production queue |
Model: | Dell PowerEdge R730 |
Manufacturing date: | 2017-06-07 |
Date of arrival: | 2017-06-26 |
CPU: | Intel Xeon E5-2650 v4 (Broadwell), x86_64, 2.20GHz, 2 CPUs/node, 12 cores/CPU |
Memory: | 128 GiB |
Storage: |
|
Network: |
|
GPU: | 2 x Nvidia GeForce GTX 1080 Ti (11 GiB) Compute capability: 6.1 |
grosminet
1 node, 4 cpus, 72 cores (json)
Reservation example:
Max walltime per nodes:
- grosminet-1: 24h
Access condition: | production queue |
Model: | Proliant DL560 Gen10 |
Manufacturing date: | 2023-11-30 |
Date of arrival: | 2023-12-05 |
CPU: | Intel Xeon Gold 6240L (Cascade Lake-SP), x86_64, 2.60GHz, 4 CPUs/node, 18 cores/CPU |
Memory: | 6.0 TiB |
Storage: |
|
Network: |
|
grostiti
1 node, 4 cpus, 56 cores (json)
Reservation example:
Max walltime per nodes:
- grostiti-1: 168h
Access condition: | production queue |
Model: | Dell PowerEdge R930 |
Manufacturing date: | 2015-10-23 |
Date of arrival: | 2024-01-10 |
CPU: | Intel Xeon E7-4850 v3 (Haswell), x86_64, 2.20GHz, 4 CPUs/node, 14 cores/CPU |
Memory: | 1.5 TiB |
Storage: |
|
Network: |
|
grue
5 nodes, 10 cpus, 160 cores (json)
Reservation example:
Max walltime per nodes:
- grue-[1-2]: 24h
- grue-[3-4]: 48h
- grue-5: 168h
Access condition: | production queue |
Model: | Dell PowerEdge R7425 |
Manufacturing date: | 2019-11-15 |
Date of arrival: | 2019-11-25 |
CPU: | AMD EPYC 7351 (Zen), x86_64, 2 CPUs/node, 16 cores/CPU |
Memory: | 128 GiB |
Storage: | disk0, 479 GB HDD SAS Dell PERC H730P Adp (dev: /dev/disk0 , by-path: /dev/disk/by-path/pci-0000:e1:00.0-scsi-0:2:0:0 ) (primary disk) |
Network: |
|
GPU: | 4 x Nvidia Tesla T4 (15 GiB) Compute capability: 7.5 |
gruss
4 nodes, 8 cpus, 192 cores, split as follows due to differences between nodes (json)
Reservation example:
Max walltime per nodes:
- gruss-[1-2]: 24h
- gruss-3: 48h
- gruss-4: 168h
- gruss-1 (1 node, 2 cpus, 48 cores)
Access condition: | production queue |
Model: | Dell PowerEdge R7525 |
Manufacturing date: | 2021-06-24 |
Date of arrival: | 2021-08-26 |
CPU: | AMD EPYC 7352 (Zen 2), x86_64, 2 CPUs/node, 24 cores/CPU |
Memory: | 256 GiB |
Storage: | disk0, 1.92 TB SSD SATA Samsung MZ7KH1T9HAJR0D3 (dev: /dev/disk0 , by-path: /dev/disk/by-path/pci-0000:01:00.0-scsi-0:0:1:0 ) (primary disk) |
Network: |
|
GPU: | 2 x Nvidia A40 (45 GiB) Compute capability: 8.6 |
- gruss-[2-4] (3 nodes, 6 cpus, 144 cores)
Access condition: | production queue |
Model: | Dell PowerEdge R7525 |
Manufacturing date: | 2021-06-24 |
Date of arrival: | 2021-08-26 |
CPU: | AMD EPYC 7352 (Zen 2), x86_64, 2 CPUs/node, 24 cores/CPU |
Memory: | 256 GiB |
Storage: | disk0, 1.92 TB SSD SATA Sk Hynix HFS1T9G32FEH-BA1 (dev: /dev/disk0 , by-path: /dev/disk/by-path/pci-0000:01:00.0-scsi-0:0:0:0 ) (primary disk) |
Network: |
|
GPU: | 2 x Nvidia A40 (45 GiB) Compute capability: 8.6 |
grvingt
64 nodes, 128 cpus, 2048 cores (json)
Reservation example:
Max walltime per nodes:
- grvingt-[1-8]: 4h
- grvingt-[9-16]: 12h
- grvingt-[17-64]: 168h
Access condition: | production queue |
Model: | Dell PowerEdge C6420 |
Manufacturing date: | 2018-04-01 |
Date of arrival: | 2018-04-11 |
CPU: | Intel Xeon Gold 6130 (Skylake), x86_64, 2.10GHz, 2 CPUs/node, 16 cores/CPU |
Memory: | 192 GiB |
Storage: | disk0, 1.0 TB HDD SATA Seagate ST1000NX0443 (dev: /dev/disk0 , by-path: /dev/disk/by-path/pci-0000:00:11.5-ata-3 ) (primary disk) |
Network: |
|
Last generated from the Grid'5000 Reference API on 2024-03-12 (commit 031d183d10)