Nancy:Hardware: Difference between revisions
No edit summary |
No edit summary |
||
Line 29: | Line 29: | ||
{| width="100%" | {| width="100%" | ||
=== Production queue resources === | === Production queue resources === | ||
* | * 11 clusters | ||
* | * 141 nodes | ||
* | * 8184 CPU cores | ||
* 128 GPUs | * 128 GPUs | ||
* 767488 GPUs cores | * 767488 GPUs cores | ||
* | * 45.88 TiB RAM | ||
* | * 59 SSDs and 127 HDDs on nodes (total: 343.56 TB) | ||
* | * 686.8 TFLOPS (excluding GPUs) | ||
|} | |} | ||
|} | |} | ||
Line 92: | Line 92: | ||
|- | |- | ||
|[[#grat|grat]]||<b>[[Grid5000:UsagePolicy#Rules_for_the_production_queue|production]]</b> queue||2022-09-07||2022-06-22||1||2||AMD EPYC 7513||32 cores/CPU||x86_64||data-sort-value="549755813888"|512 GiB||data-sort-value="28608"|<b>3.84 TB SSD</b> + 7 x 3.84 TB SSD||data-sort-value="25000"|25 Gbps (SR‑IOV) ||8 x Nvidia A100 (40 GiB) | |[[#grat|grat]]||<b>[[Grid5000:UsagePolicy#Rules_for_the_production_queue|production]]</b> queue||2022-09-07||2022-06-22||1||2||AMD EPYC 7513||32 cores/CPU||x86_64||data-sort-value="549755813888"|512 GiB||data-sort-value="28608"|<b>3.84 TB SSD</b> + 7 x 3.84 TB SSD||data-sort-value="25000"|25 Gbps (SR‑IOV) ||8 x Nvidia A100 (40 GiB) | ||
|- | |||
|[[#grdix|grdix]]||<b>[[Grid5000:UsagePolicy#Rules_for_the_production_queue|production]]</b> queue||2024-09-27||2024-09-02||16||2||AMD EPYC 9754||128 cores/CPU||x86_64||data-sort-value="1099511627776"|1.0 TiB||data-sort-value="1490"|<b>1.6 TB SSD</b>||data-sort-value="225000"|25 Gbps (SR‑IOV) + 200 Gbps InfiniBand|| | |||
|- | |- | ||
|[[#grele|grele]]||<b>[[Grid5000:UsagePolicy#Rules_for_the_production_queue|production]]</b> queue||2017-06-26||2017-06-07||13||2||Intel Xeon E5-2650 v4||12 cores/CPU||x86_64||data-sort-value="137438953472"|128 GiB||data-sort-value="556"|<b>299 GB HDD</b> + 299 GB HDD||data-sort-value="110000"|10 Gbps (SR‑IOV) + 100 Gbps Omni-Path||2 x Nvidia GTX 1080 Ti (11 GiB) | |[[#grele|grele]]||<b>[[Grid5000:UsagePolicy#Rules_for_the_production_queue|production]]</b> queue||2017-06-26||2017-06-07||13||2||Intel Xeon E5-2650 v4||12 cores/CPU||x86_64||data-sort-value="137438953472"|128 GiB||data-sort-value="556"|<b>299 GB HDD</b> + 299 GB HDD||data-sort-value="110000"|10 Gbps (SR‑IOV) + 100 Gbps Omni-Path||2 x Nvidia GTX 1080 Ti (11 GiB) | ||
Line 106: | Line 108: | ||
|- | |- | ||
|[[#grvingt|grvingt]]||<b>[[Grid5000:UsagePolicy#Rules_for_the_production_queue|production]]</b> queue||2018-04-11||2018-04-01||64||2||Intel Xeon Gold 6130||16 cores/CPU||x86_64||data-sort-value="206158430208"|192 GiB||data-sort-value="931"|<b>1.0 TB HDD</b>||data-sort-value="110000"|10 Gbps + 100 Gbps Omni-Path|| | |[[#grvingt|grvingt]]||<b>[[Grid5000:UsagePolicy#Rules_for_the_production_queue|production]]</b> queue||2018-04-11||2018-04-01||64||2||Intel Xeon Gold 6130||16 cores/CPU||x86_64||data-sort-value="206158430208"|192 GiB||data-sort-value="931"|<b>1.0 TB HDD</b>||data-sort-value="110000"|10 Gbps + 100 Gbps Omni-Path|| | ||
|- | |- | ||
|} | |} | ||
Line 469: | Line 447: | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''GPU:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''GPU:''' | ||
| 8 x Nvidia A100-SXM4-40GB (40 GiB)<br>Compute capability: 8.0<br/> | | 8 x Nvidia A100-SXM4-40GB (40 GiB)<br>Compute capability: 8.0<br/> | ||
|- | |||
|} | |||
== [https://intranet.grid5000.fr/oar/Nancy/drawgantt-svg-prod/?filter=grdix%20only grdix] == | |||
'''16 nodes, 32 cpus, 4096 cores''' ([https://public-api.grid5000.fr/stable/sites/nancy/clusters/grdix/nodes.json?pretty=1 json]) | |||
'''Reservation example:''' | |||
{{Term|location=fnancy|cmd=<code class="command">oarsub</code> <code class="replace">-q production</code> <code class="env">-p grdix</code> <code>-I</code>}} | |||
'''Max walltime per nodes:''' | |||
* grdix-[1-16]: 0h | |||
{| | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Access condition:''' | |||
| production queue<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Model:''' | |||
| ProLiant DL365 Gen11<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Manufacturing date:''' | |||
| 2024-09-02<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Date of arrival:''' | |||
| 2024-09-27<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''CPU:''' | |||
| AMD EPYC 9754 (Zen 4c), x86_64, 2 CPUs/node, 128 cores/CPU<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Memory:''' | |||
| 1.0 TiB<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Storage:''' | |||
| disk0, 1.6 TB SSD NVME Samsung MO001600KYDMU (dev: <code class="file">/dev/disk0</code>) (primary disk)<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Network:''' | |||
| | |||
* eth0/enp1s0f0np0, Ethernet, configured rate: 25 Gbps, model: Mellanox Technologies MT2894 Family [ConnectX-6 Lx], driver: mlx5_core, SR-IOV enabled<br /> | |||
* <span style="color:grey">eth1/ens22f1np1, Ethernet, model: Mellanox Technologies MT2894 Family [ConnectX-6 Lx], driver: mlx5_core - unavailable for experiment</span><br /> | |||
* ibs1, InfiniBand, configured rate: 200 Gbps, model: Mellanox Technologies MT28908 Family [ConnectX-6], driver: mlx5_core<br/> | |||
|- | |- | ||
|} | |} | ||
Line 841: | Line 861: | ||
* ib0, Omni-Path, configured rate: 100 Gbps, model: Intel Omni-Path HFI Silicon 100 Series [discrete], driver: hfi1<br/> | * ib0, Omni-Path, configured rate: 100 Gbps, model: Intel Omni-Path HFI Silicon 100 Series [discrete], driver: hfi1<br/> | ||
|- | |- | ||
|}''<small>Last generated from the Grid'5000 Reference API on 2024-12-17 ([https://gitlab.inria.fr/grid5000/reference-repository/commit/b57f23eab4 commit b57f23eab4])</small>'' | |||
|}''<small>Last generated from the Grid'5000 Reference API on 2024-12- |
Revision as of 08:55, 17 December 2024
See also: Network topology for Nancy
Summary
- 13 clusters
- 267 nodes
- 10544 CPU cores
- 132 GPUs
- 795136 GPUs cores
- 57.75 TiB RAM
- 311 SSDs and 127 HDDs on nodes (total: 527.92 TB)
- 769.7 TFLOPS (excluding GPUs)
Default queue resources
|
Production queue resources
|
Clusters summary
Default queue resources
Cluster | Access Condition | Date of arrival | Manufacturing date | Nodes | CPU | Memory | Storage | Network | Accelerators | |||
---|---|---|---|---|---|---|---|---|---|---|---|---|
# | Name | Cores | Architecture | |||||||||
gros | 2019-09-04 | 2019-07-16 | 124 | 1 | Intel Xeon Gold 5220 | 18 cores/CPU | x86_64 | 96 GiB | 480 GB SSD + 960 GB SSD* | 2 x 25 Gbps (SR‑IOV) | ||
grouille | exotic job type | 2021-01-13 | 2020-12-07 | 2 | 2 | AMD EPYC 7452 | 32 cores/CPU | x86_64 | 128 GiB | 1.92 TB SSD + 960 GB SSD* | 25 Gbps | 2 x Nvidia A100 (40 GiB) |
*: disk is reservable **: crossed GPUs are not supported by Grid'5000 default environments ***: OPA (Omni-Path Architecture) is currently not supported on Debian 12 environment
Production queue resources
Cluster | Access Condition | Date of arrival | Manufacturing date | Nodes | CPU | Memory | Storage | Network | Accelerators | |||
---|---|---|---|---|---|---|---|---|---|---|---|---|
# | Name | Cores | Architecture | |||||||||
graffiti | production queue | 2019-06-07 | 2019-05-27 | 13 | 2 | Intel Xeon Silver 4110 | 8 cores/CPU | x86_64 | 128 GiB | 479 GB HDD | 10 Gbps | [1-12]: 4 x Nvidia RTX 2080 Ti (11 GiB) 13: 4 x Nvidia Quadro RTX 6000 (23 GiB) |
grappe | production queue | 2020-08-20 | 2020-07-09 | 16 | 2 | Intel Xeon Gold 5218R | 20 cores/CPU | x86_64 | 96 GiB | 480 GB SSD + 8.0 TB HDD | 25 Gbps | |
grat | production queue | 2022-09-07 | 2022-06-22 | 1 | 2 | AMD EPYC 7513 | 32 cores/CPU | x86_64 | 512 GiB | 3.84 TB SSD + 7 x 3.84 TB SSD | 25 Gbps (SR‑IOV) | 8 x Nvidia A100 (40 GiB) |
grdix | production queue | 2024-09-27 | 2024-09-02 | 16 | 2 | AMD EPYC 9754 | 128 cores/CPU | x86_64 | 1.0 TiB | 1.6 TB SSD | 25 Gbps (SR‑IOV) + 200 Gbps InfiniBand | |
grele | production queue | 2017-06-26 | 2017-06-07 | 13 | 2 | Intel Xeon E5-2650 v4 | 12 cores/CPU | x86_64 | 128 GiB | 299 GB HDD + 299 GB HDD | 10 Gbps (SR‑IOV) + 100 Gbps Omni-Path | 2 x Nvidia GTX 1080 Ti (11 GiB) |
gres | production queue | 2024-08-23 | 2024-08-07 | 7 | 2 | AMD EPYC 9254 | 24 cores/CPU | x86_64 | 512 GiB | 6.4 TB SSD | 25 Gbps | 2 x Nvidia Tesla L40S (45 GiB) |
grosminet | production queue | 2023-12-05 | 2023-11-30 | 1 | 4 | Intel Xeon Gold 6240L | 18 cores/CPU | x86_64 | 6.0 TiB | 1.6 TB SSD + 7 x 1.6 TB SSD | 25 Gbps (SR‑IOV) | |
grostiti | production queue | 2024-01-10 | 2015-10-23 | 1 | 4 | Intel Xeon E7-4850 v3 | 14 cores/CPU | x86_64 | 1.5 TiB | 1.2 TB HDD + 4.0 TB HDD + 599 GB HDD | 10 Gbps (SR‑IOV) | |
grue | production queue | 2019-11-25 | 2019-11-15 | 5 | 2 | AMD EPYC 7351 | 16 cores/CPU | x86_64 | 128 GiB | 479 GB HDD | 10 Gbps | 4 x Nvidia Tesla T4 (15 GiB) |
gruss | production queue | 2021-08-26 | 2021-06-24 | 4 | 2 | AMD EPYC 7352 | 24 cores/CPU | x86_64 | 256 GiB | 1.92 TB SSD | 25 Gbps | 2 x Nvidia A40 (45 GiB) |
grvingt | production queue | 2018-04-11 | 2018-04-01 | 64 | 2 | Intel Xeon Gold 6130 | 16 cores/CPU | x86_64 | 192 GiB | 1.0 TB HDD | 10 Gbps + 100 Gbps Omni-Path |
*: disk is reservable **: crossed GPUs are not supported by Grid'5000 default environments ***: OPA (Omni-Path Architecture) is currently not supported on Debian 12 environment
Clusters in the default queue
gros
124 nodes, 124 cpus, 2232 cores, split as follows due to differences between nodes (json)
Reservation example:
- gros-[1-26,
28-67, 69-124] (122 nodes, 122 cpus, 2196 cores)
Model: | Dell PowerEdge R640 |
Manufacturing date: | 2019-07-16 |
Date of arrival: | 2019-09-04 |
CPU: | Intel Xeon Gold 5220 (Cascade Lake-SP), x86_64, 2.20GHz, 1 CPU/node, 18 cores/CPU |
Memory: | 96 GiB |
Storage: |
|
Network: |
|
- gros-27 (1 node, 1 cpu, 18 cores)
Model: | Dell PowerEdge R640 |
Manufacturing date: | 2019-07-16 |
Date of arrival: | 2019-09-04 |
CPU: | Intel Xeon Gold 5220 (Cascade Lake-SP), x86_64, 2.20GHz, 1 CPU/node, 18 cores/CPU |
Memory: | 96 GiB |
Storage: |
|
Network: |
|
- gros-68 (1 node, 1 cpu, 18 cores)
Model: | Dell PowerEdge R640 |
Manufacturing date: | 2019-07-16 |
Date of arrival: | 2019-09-04 |
CPU: | Intel Xeon Gold 5220 (Cascade Lake-SP), x86_64, 2.20GHz, 1 CPU/node, 18 cores/CPU |
Memory: | 96 GiB |
Storage: |
|
Network: |
|
grouille
2 nodes, 4 cpus, 128 cores (json)
Reservation example:
Access condition: | exotic job type |
Model: | Dell PowerEdge R7525 |
Manufacturing date: | 2020-12-07 |
Date of arrival: | 2021-01-13 |
CPU: | AMD EPYC 7452 (Zen 2), x86_64, 2 CPUs/node, 32 cores/CPU |
Memory: | 128 GiB |
Storage: |
|
Network: |
|
GPU: | 2 x Nvidia A100-PCIE-40GB (40 GiB) Compute capability: 8.0 |
Note: This cluster is defined as exotic. Please read the exotic page for more information.
Clusters in the production queue
graffiti
13 nodes, 26 cpus, 208 cores, split as follows due to differences between nodes (json)
Reservation example:
Max walltime per nodes:
- graffiti-[1-3]: 24h
- graffiti-[4-6]: 48h
- graffiti-[7-13]: 168h
- graffiti-[1-12] (12 nodes, 24 cpus, 192 cores)
Access condition: | production queue |
Model: | Dell PowerEdge T640 |
Manufacturing date: | 2019-05-27 |
Date of arrival: | 2019-06-07 |
CPU: | Intel Xeon Silver 4110 (Skylake-SP), x86_64, 2.10GHz, 2 CPUs/node, 8 cores/CPU |
Memory: | 128 GiB |
Storage: | disk0, 479 GB HDD SATA Dell PERC H330 Adp (dev: /dev/disk0 ) (primary disk) |
Network: |
|
GPU: | 4 x Nvidia GeForce RTX 2080 Ti (11 GiB) Compute capability: 7.5 |
- graffiti-13 (1 node, 2 cpus, 16 cores)
Access condition: | production queue |
Model: | Dell PowerEdge T640 |
Manufacturing date: | 2019-05-27 |
Date of arrival: | 2019-06-07 |
CPU: | Intel Xeon Silver 4110 (Skylake-SP), x86_64, 2.10GHz, 2 CPUs/node, 8 cores/CPU |
Memory: | 128 GiB |
Storage: | disk0, 479 GB HDD SATA Dell PERC H330 Adp (dev: /dev/disk0 ) (primary disk) |
Network: |
|
GPU: | 4 x Nvidia Quadro RTX 6000 (23 GiB) Compute capability: 7.5 |
grappe
16 nodes, 32 cpus, 640 cores (json)
Reservation example:
Max walltime per nodes:
- grappe-[1-4]: 48h
- grappe-[5-8]: 96h
- grappe-[9-16]: 168h
Access condition: | production queue |
Model: | Dell PowerEdge R640 |
Manufacturing date: | 2020-07-09 |
Date of arrival: | 2020-08-20 |
CPU: | Intel Xeon Gold 5218R (Cascade Lake-SP), x86_64, 2.10GHz, 2 CPUs/node, 20 cores/CPU |
Memory: | 96 GiB |
Storage: |
|
Network: |
|
grat
1 node, 2 cpus, 64 cores (json)
Reservation example:
Max walltime per nodes:
- grat-1: 168h
Access condition: | production queue |
Model: | HPE Apollo 6500 |
Manufacturing date: | 2022-06-22 |
Date of arrival: | 2022-09-07 |
CPU: | AMD EPYC 7513 (Zen 3), x86_64, 2 CPUs/node, 32 cores/CPU |
Memory: | 512 GiB |
Storage: |
|
Network: |
|
GPU: | 8 x Nvidia A100-SXM4-40GB (40 GiB) Compute capability: 8.0 |
grdix
16 nodes, 32 cpus, 4096 cores (json)
Reservation example:
Max walltime per nodes:
- grdix-[1-16]: 0h
Access condition: | production queue |
Model: | ProLiant DL365 Gen11 |
Manufacturing date: | 2024-09-02 |
Date of arrival: | 2024-09-27 |
CPU: | AMD EPYC 9754 (Zen 4c), x86_64, 2 CPUs/node, 128 cores/CPU |
Memory: | 1.0 TiB |
Storage: | disk0, 1.6 TB SSD NVME Samsung MO001600KYDMU (dev: /dev/disk0 ) (primary disk) |
Network: |
|
grele
13 nodes, 26 cpus, 312 cores (json)
Reservation example:
Max walltime per nodes:
- grele-[1-3]: 24h
- grele-[4-6]: 48h
- grele-[7-13]: 168h
Access condition: | production queue |
Model: | Dell PowerEdge R730 |
Manufacturing date: | 2017-06-07 |
Date of arrival: | 2017-06-26 |
CPU: | Intel Xeon E5-2650 v4 (Broadwell), x86_64, 2.20GHz, 2 CPUs/node, 12 cores/CPU |
Memory: | 128 GiB |
Storage: |
|
Network: |
|
GPU: | 2 x Nvidia GeForce GTX 1080 Ti (11 GiB) Compute capability: 6.1 |
gres
7 nodes, 14 cpus, 336 cores (json)
Reservation example:
Max walltime per nodes:
- gres-[1-7]: 168h
Access condition: | production queue |
Model: | ProLiant DL385 Gen11 |
Manufacturing date: | 2024-08-07 |
Date of arrival: | 2024-08-23 |
CPU: | AMD EPYC 9254 (Zen 4), x86_64, 2 CPUs/node, 24 cores/CPU |
Memory: | 512 GiB |
Storage: | disk0, 6.4 TB SSD NVME Samsung MO006400KYDND (dev: /dev/disk0 ) (primary disk) |
Network: |
|
GPU: | 2 x Nvidia L40S (45 GiB) Compute capability: 8.9 |
grosminet
1 node, 4 cpus, 72 cores (json)
Reservation example:
Max walltime per nodes:
- grosminet-1: 24h
Access condition: | production queue |
Model: | Proliant DL560 Gen10 |
Manufacturing date: | 2023-11-30 |
Date of arrival: | 2023-12-05 |
CPU: | Intel Xeon Gold 6240L (Cascade Lake-SP), x86_64, 2.60GHz, 4 CPUs/node, 18 cores/CPU |
Memory: | 6.0 TiB |
Storage: |
|
Network: |
|
grostiti
1 node, 4 cpus, 56 cores (json)
Reservation example:
Max walltime per nodes:
- grostiti-1: 168h
Access condition: | production queue |
Model: | Dell PowerEdge R930 |
Manufacturing date: | 2015-10-23 |
Date of arrival: | 2024-01-10 |
CPU: | Intel Xeon E7-4850 v3 (Haswell), x86_64, 2.20GHz, 4 CPUs/node, 14 cores/CPU |
Memory: | 1.5 TiB |
Storage: |
|
Network: |
|
grue
5 nodes, 10 cpus, 160 cores (json)
Reservation example:
Max walltime per nodes:
- grue-[1-2]: 24h
- grue-[3-4]: 48h
- grue-5: 168h
Access condition: | production queue |
Model: | Dell PowerEdge R7425 |
Manufacturing date: | 2019-11-15 |
Date of arrival: | 2019-11-25 |
CPU: | AMD EPYC 7351 (Zen), x86_64, 2 CPUs/node, 16 cores/CPU |
Memory: | 128 GiB |
Storage: | disk0, 479 GB HDD SAS Dell PERC H730P Adp (dev: /dev/disk0 ) (primary disk) |
Network: |
|
GPU: | 4 x Nvidia Tesla T4 (15 GiB) Compute capability: 7.5 |
gruss
4 nodes, 8 cpus, 192 cores, split as follows due to differences between nodes (json)
Reservation example:
Max walltime per nodes:
- gruss-[1-2]: 24h
- gruss-3: 48h
- gruss-4: 168h
- gruss-1 (1 node, 2 cpus, 48 cores)
Access condition: | production queue |
Model: | Dell PowerEdge R7525 |
Manufacturing date: | 2021-06-24 |
Date of arrival: | 2021-08-26 |
CPU: | AMD EPYC 7352 (Zen 2), x86_64, 2 CPUs/node, 24 cores/CPU |
Memory: | 256 GiB |
Storage: | disk0, 1.92 TB SSD SATA Samsung MZ7KH1T9HAJR0D3 (dev: /dev/disk0 ) (primary disk) |
Network: |
|
GPU: | 2 x Nvidia A40 (45 GiB) Compute capability: 8.6 |
- gruss-[2-4] (3 nodes, 6 cpus, 144 cores)
Access condition: | production queue |
Model: | Dell PowerEdge R7525 |
Manufacturing date: | 2021-06-24 |
Date of arrival: | 2021-08-26 |
CPU: | AMD EPYC 7352 (Zen 2), x86_64, 2 CPUs/node, 24 cores/CPU |
Memory: | 256 GiB |
Storage: | disk0, 1.92 TB SSD SATA Sk Hynix HFS1T9G32FEH-BA1 (dev: /dev/disk0 ) (primary disk) |
Network: |
|
GPU: | 2 x Nvidia A40 (45 GiB) Compute capability: 8.6 |
grvingt
64 nodes, 128 cpus, 2048 cores (json)
Reservation example:
Max walltime per nodes:
- grvingt-[1-8]: 4h
- grvingt-[9-16]: 12h
- grvingt-[17-64]: 168h
Access condition: | production queue |
Model: | Dell PowerEdge C6420 |
Manufacturing date: | 2018-04-01 |
Date of arrival: | 2018-04-11 |
CPU: | Intel Xeon Gold 6130 (Skylake-SP), x86_64, 2.10GHz, 2 CPUs/node, 16 cores/CPU |
Memory: | 192 GiB |
Storage: | disk0, 1.0 TB HDD SATA Seagate ST1000NX0443 (dev: /dev/disk0 ) (primary disk) |
Network: |
|
Last generated from the Grid'5000 Reference API on 2024-12-17 (commit b57f23eab4)