Grenoble:Hardware: Difference between revisions
Ddelabroye (talk | contribs) No edit summary |
No edit summary |
||
(84 intermediate revisions by 14 users not shown) | |||
Line 1: | Line 1: | ||
__NOEDITSECTION__ | __NOEDITSECTION__ | ||
{{Portal|User}} | {{Portal|User}} | ||
<div class="sitelink">[[Hardware|Global]] | [[Grenoble:Hardware|Grenoble]] | [[Lille:Hardware|Lille]] | [[Luxembourg:Hardware|Luxembourg]] | [[Lyon:Hardware|Lyon]] | [[Nancy:Hardware|Nancy]] | [[Nantes:Hardware|Nantes]] | [[Rennes:Hardware|Rennes]] | [[Sophia:Hardware|Sophia]]</div> | <div class="sitelink">Hardware: [[Hardware|Global]] | [[Grenoble:Hardware|Grenoble]] | [[Lille:Hardware|Lille]] | [[Luxembourg:Hardware|Luxembourg]] | [[Lyon:Hardware|Lyon]] | [[Nancy:Hardware|Nancy]] | [[Nantes:Hardware|Nantes]] | [[Rennes:Hardware|Rennes]] | [[Sophia:Hardware|Sophia]] | [[Strasbourg:Hardware|Strasbourg]] | [[Toulouse:Hardware|Toulouse]]</div> | ||
'''See also:''' [[Grenoble:Network|Network topology for Grenoble]] | |||
= Summary = | |||
* 5 clusters | |||
* 54 nodes | |||
* 1744 CPU cores | |||
* 48 GPUs | |||
* 172032 GPUs cores | |||
* 12.25 TiB RAM + 6.0 TiB PMEM | |||
* 88 SSDs and 68 HDDs on nodes (total: 228.53 TB) | |||
* 101.9 TFLOPS (excluding GPUs) | |||
= | = Clusters summary = | ||
{|class="wikitable sortable" | {|class="wikitable sortable" | ||
|- | |- | ||
!|Cluster | !rowspan=2|Cluster | ||
!|Access Condition | !rowspan=2|Access Condition | ||
!|Date of arrival | !rowspan=2|Date of arrival | ||
!data-sort-type="number"|Nodes | !rowspan=2|Manufacturing date | ||
!|CPU | !data-sort-type="number" rowspan=2|Nodes | ||
!colspan=4|CPU | |||
!data-sort-type="number" rowspan=2|Memory | |||
!data-sort-type="number" rowspan=2|Storage | |||
!data-sort-type="number" rowspan=2|Network | |||
!rowspan=2|Accelerators | |||
|- | |||
!data-sort-type="number"|# | |||
!|Name | |||
!data-sort-type="number"|Cores | !data-sort-type="number"|Cores | ||
!data-sort- | !|Architecture | ||
|- | |||
|[[#dahu|dahu]]||||2018-03-22||2017-12-12||32||2||Intel Xeon Gold 6130||16 cores/CPU||x86_64||data-sort-value="206158430208"|192 GiB||data-sort-value="4396"|<b>240 GB SSD</b> + 480 GB SSD + 4.0 TB HDD||data-sort-value="110000"|10 Gbps + 100 Gbps Omni-Path|| | |||
|- | |- | ||
|[[# | |[[#drac|drac]]||<b>[[Exotic#Grenoble:_drac|exotic]]</b> job type||2020-10-05||2016-10-17||12||2||IBM POWER8NVL 1.0||10 cores/CPU||ppc64le||data-sort-value="137438953472"|128 GiB||data-sort-value="1862"|<b>1.0 TB HDD</b> + 1.0 TB HDD||data-sort-value="210000"|10 Gbps + 2 x 100 Gbps InfiniBand||4 x Nvidia Tesla P100 (16 GiB) | ||
|- | |- | ||
|[[# | |[[#servan|servan]]||<b>[[Exotic#Grenoble:_servan|exotic]]</b> job type||2021-12-15||2021-12-10||2||2||AMD EPYC 7352||24 cores/CPU||x86_64||data-sort-value="137438953472"|128 GiB||data-sort-value="2980"|<b>1.6 TB SSD</b> + 1.6 TB SSD||data-sort-value="225000"|25 Gbps + 2 x 100 Gbps FPGA/Ethernet||Xilinx Alveo U200 | ||
|- | |||
|[[#troll|troll]]||<b>[[Exotic#Grenoble:_troll|exotic]]</b> job type||2019-12-23||2019-11-21||4||2||Intel Xeon Gold 5218||16 cores/CPU||x86_64||data-sort-value="412316860416"|384 GiB + 1.5 TiB [[PMEM]]||data-sort-value="1937"|<b>480 GB SSD</b> + 1.6 TB SSD||data-sort-value="125000"|25 Gbps (SR‑IOV) + 100 Gbps Omni-Path|| | |||
|- | |||
|[[#yeti|yeti]]||<b>[[Exotic#Grenoble:_yeti|exotic]]</b> job type||2018-01-16||2017-12-26||4||4||Intel Xeon Gold 6130||16 cores/CPU||x86_64||data-sort-value="824633720832"|768 GiB||data-sort-value="9016"|<b>480 GB SSD</b> + 3 x 2.0 TB HDD[[Disk_reservation|*]] + 2 x 1.6 TB SSD||data-sort-value="110000"|10 Gbps + 100 Gbps Omni-Path|| | |||
|- | |- | ||
|} | |} | ||
''*: disk is [[Disk_reservation|reservable]]'' | ''*: disk is [[Disk_reservation|reservable]]'' | ||
''**: crossed GPUs are not supported by Grid'5000 default environments'' | |||
'''32 nodes, 64 cpus, 1024 cores''' ([https://public-api.grid5000.fr/stable/sites/grenoble/clusters/dahu/nodes.json?pretty=1 json]) | ''***: OPA (Omni-Path Architecture) is currently not supported on Debian 12 environment'' | ||
= Clusters in the [https://intranet.grid5000.fr/oar/Grenoble/drawgantt-svg/ default queue] = | |||
== [https://intranet.grid5000.fr/oar/Grenoble/drawgantt-svg/?filter=dahu%20only dahu] == | |||
'''32 nodes, 64 cpus, 1024 cores,''' split as follows due to differences between nodes ([https://public-api.grid5000.fr/stable/sites/grenoble/clusters/dahu/nodes.json?pretty=1 json]) | |||
'''Reservation example:''' | |||
{{Term|location=fgrenoble|cmd=<code class="command">oarsub</code> <code class="env">-p dahu</code> <code>-I</code>}} | |||
; dahu-[1,<wbr>4-32] (30 nodes, 60 cpus, 960 cores) | |||
{| | {| | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Model:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Model:''' | ||
| Dell PowerEdge C6420<br/> | | Dell PowerEdge C6420<br/> | ||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Manufacturing date:''' | |||
| 2017-12-12<br/> | |||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Date of arrival:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Date of arrival:''' | ||
Line 40: | Line 72: | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''CPU:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''CPU:''' | ||
| Intel Xeon Gold 6130 (Skylake, 2.10GHz, 2 CPUs/node, 16 cores/CPU | | Intel Xeon Gold 6130 (Skylake), x86_64, 2.10GHz, 2 CPUs/node, 16 cores/CPU<br/> | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Memory:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Memory:''' | ||
Line 47: | Line 79: | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Storage:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Storage:''' | ||
| | | | ||
* 240 GB SSD SATA Samsung MZ7KM240HMHQ0D3 | * disk0, 240 GB SSD SATA Samsung MZ7KM240HMHQ0D3 (dev: <code class="file">/dev/disk0</code>) (primary disk)<br /> | ||
* 480 GB SSD SATA Samsung MZ7KM480HMHQ0D3 | * disk1, 480 GB SSD SATA Samsung MZ7KM480HMHQ0D3 (dev: <code class="file">/dev/disk1</code>) <br /> | ||
* 4.0 TB HDD SATA Seagate ST4000NM0265-2DC | * disk2, 4.0 TB HDD SATA Seagate ST4000NM0265-2DC (dev: <code class="file">/dev/disk2</code>) <br/> | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Network:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Network:''' | ||
Line 57: | Line 88: | ||
* <span style="color:grey">eth1/enp24s0f1, Ethernet, model: Intel Ethernet Controller X710 for 10GbE SFP+, driver: i40e - unavailable for experiment</span><br /> | * <span style="color:grey">eth1/enp24s0f1, Ethernet, model: Intel Ethernet Controller X710 for 10GbE SFP+, driver: i40e - unavailable for experiment</span><br /> | ||
* ib0, Omni-Path, configured rate: 100 Gbps, model: Intel Omni-Path HFI Silicon 100 Series [discrete], driver: hfi1<br/> | * ib0, Omni-Path, configured rate: 100 Gbps, model: Intel Omni-Path HFI Silicon 100 Series [discrete], driver: hfi1<br/> | ||
|- | |||
|} | |||
<hr style="height:10pt; visibility:hidden;" /> | |||
; dahu-2 (1 node, 2 cpus, 32 cores) | |||
{| | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Model:''' | |||
| Dell PowerEdge C6420<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Manufacturing date:''' | |||
| 2017-12-12<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Date of arrival:''' | |||
| 2018-03-22<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''CPU:''' | |||
| Intel Xeon Gold 6130 (Skylake), x86_64, 2.10GHz, 2 CPUs/node, 16 cores/CPU<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Memory:''' | |||
| 192 GiB<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Storage:''' | |||
| | |||
* disk0, 240 GB SSD SATA Samsung MZ7KM240HMHQ0D3 (dev: <code class="file">/dev/disk0</code>) (primary disk)<br /> | |||
* disk1, 480 GB SSD SATA Samsung MZ7KM480HMHQ0D3 (dev: <code class="file">/dev/disk1</code>) <br /> | |||
* disk2, 4.0 TB HDD SATA Toshiba TOSHIBA MG08ADA4 (dev: <code class="file">/dev/disk2</code>) <br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Network:''' | |||
| | |||
* eth0/enp24s0f0, Ethernet, configured rate: 10 Gbps, model: Intel Ethernet Controller X710 for 10GbE SFP+, driver: i40e<br /> | |||
* <span style="color:grey">eth1/enp24s0f1, Ethernet, model: Intel Ethernet Controller X710 for 10GbE SFP+, driver: i40e - unavailable for experiment</span><br /> | |||
* ib0, Omni-Path, configured rate: 100 Gbps, model: Intel Omni-Path HFI Silicon 100 Series [discrete], driver: hfi1<br/> | |||
|- | |||
|} | |||
<hr style="height:10pt; visibility:hidden;" /> | |||
; dahu-3 (1 node, 2 cpus, 32 cores) | |||
{| | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Model:''' | |||
| Dell PowerEdge C6420<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Manufacturing date:''' | |||
| 2017-12-12<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Date of arrival:''' | |||
| 2018-03-22<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''CPU:''' | |||
| Intel Xeon Gold 6130 (Skylake), x86_64, 2.10GHz, 2 CPUs/node, 16 cores/CPU<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Memory:''' | |||
| 192 GiB<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Storage:''' | |||
| | |||
* disk0, 240 GB SSD SATA Samsung MZ7KM240HMHQ0D3 (dev: <code class="file">/dev/disk0</code>) (primary disk)<br /> | |||
* disk1, 480 GB SSD SATA Samsung MZ7KM480HMHQ0D3 (dev: <code class="file">/dev/disk1</code>) <br /> | |||
* disk2, 4.0 TB HDD SATA Seagate ST4000NM018B-2TF (dev: <code class="file">/dev/disk2</code>) <br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Network:''' | |||
| | |||
* eth0/enp24s0f0, Ethernet, configured rate: 10 Gbps, model: Intel Ethernet Controller X710 for 10GbE SFP+, driver: i40e<br /> | |||
* <span style="color:grey">eth1/enp24s0f1, Ethernet, model: Intel Ethernet Controller X710 for 10GbE SFP+, driver: i40e - unavailable for experiment</span><br /> | |||
* ib0, Omni-Path, configured rate: 100 Gbps, model: Intel Omni-Path HFI Silicon 100 Series [discrete], driver: hfi1<br/> | |||
|- | |||
|} | |||
== [https://intranet.grid5000.fr/oar/Grenoble/drawgantt-svg/?filter=drac%20only drac] == | |||
'''12 nodes, 24 cpus, 240 cores''' ([https://public-api.grid5000.fr/stable/sites/grenoble/clusters/drac/nodes.json?pretty=1 json]) | |||
'''Reservation example:''' | |||
{{Term|location=fgrenoble|cmd=<code class="command">oarsub</code> <code class="replace">-t exotic</code> <code class="env">-p drac</code> <code>-I</code>}} | |||
{| | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Access condition:''' | |||
| [[Exotic#Grenoble:_drac|exotic]] job type<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Model:''' | |||
| IBM PowerNV S822LC (8335-GTB)<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Manufacturing date:''' | |||
| 2016-10-17<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Date of arrival:''' | |||
| 2020-10-05<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''CPU:''' | |||
| IBM POWER8NVL 1.0 (POWER8), ppc64le, 2 CPUs/node, 10 cores/CPU<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Memory:''' | |||
| 128 GiB<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Storage:''' | |||
| | |||
* disk0, 1.0 TB HDD SATA Seagate ST1000NX0313 (dev: <code class="file">/dev/disk0</code>) (primary disk)<br /> | |||
* disk1, 1.0 TB HDD SATA Seagate ST1000NX0313 (dev: <code class="file">/dev/disk1</code>) <br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Network:''' | |||
| | |||
* eth0/enP1p1s0f0, Ethernet, configured rate: 10 Gbps, model: Broadcom Inc. and subsidiaries NetXtreme II BCM57800 1/10 Gigabit Ethernet, driver: bnx2x<br /> | |||
* <span style="color:grey">eth1/enP1p1s0f1, Ethernet, model: Broadcom Inc. and subsidiaries NetXtreme II BCM57800 1/10 Gigabit Ethernet, driver: bnx2x - unavailable for experiment</span><br /> | |||
* <span style="color:grey">eth2/enP1p1s0f2, Ethernet, model: Broadcom Inc. and subsidiaries NetXtreme II BCM57800 1/10 Gigabit Ethernet, driver: bnx2x - unavailable for experiment</span><br /> | |||
* <span style="color:grey">eth3/enP1p1s0f3, Ethernet, model: Broadcom Inc. and subsidiaries NetXtreme II BCM57800 1/10 Gigabit Ethernet, driver: bnx2x - unavailable for experiment</span><br /> | |||
* <span style="color:grey">eth4/enP9p7s0f0, Ethernet, configured rate: 1 Gbps, model: Broadcom Inc. and subsidiaries NetXtreme BCM5719 Gigabit Ethernet PCIe, driver: tg3 - unavailable for experiment</span><br /> | |||
* <span style="color:grey">eth5/enP9p7s0f1, Ethernet, model: Broadcom Inc. and subsidiaries NetXtreme BCM5719 Gigabit Ethernet PCIe, driver: tg3 - unavailable for experiment</span><br /> | |||
* ib0, InfiniBand, configured rate: 100 Gbps, model: Mellanox Technologies MT27700 Family [ConnectX-4], driver: mlx5_core<br /> | |||
* ib1, InfiniBand, configured rate: 100 Gbps, model: Mellanox Technologies MT27700 Family [ConnectX-4], driver: mlx5_core<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''GPU:''' | |||
| 4 x Nvidia Tesla P100-SXM2-16GB (16 GiB)<br>Compute capability: 6.0<br/> | |||
|- | |- | ||
|} | |} | ||
== troll == | '''Note:''' This cluster is defined as exotic. Please read the '''[[Exotic#Grenoble:_drac|exotic]]''' page for more information.<br/> | ||
== [https://intranet.grid5000.fr/oar/Grenoble/drawgantt-svg/?filter=servan%20only servan] == | |||
'''2 nodes, 4 cpus, 96 cores''' ([https://public-api.grid5000.fr/stable/sites/grenoble/clusters/servan/nodes.json?pretty=1 json]) | |||
'''Reservation example:''' | |||
{{Term|location=fgrenoble|cmd=<code class="command">oarsub</code> <code class="replace">-t exotic</code> <code class="env">-p servan</code> <code>-I</code>}} | |||
{| | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Access condition:''' | |||
| [[Exotic#Grenoble:_servan|exotic]] job type<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Model:''' | |||
| Dell PowerEdge R7525<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Manufacturing date:''' | |||
| 2021-12-10<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Date of arrival:''' | |||
| 2021-12-15<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''CPU:''' | |||
| AMD EPYC 7352 (Zen 2), x86_64, 2 CPUs/node, 24 cores/CPU<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Memory:''' | |||
| 128 GiB<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Storage:''' | |||
| | |||
* disk0, 1.6 TB SSD NVME Dell Dell Ent NVMe v2 AGN MU U.2 1.6TB (dev: <code class="file">/dev/disk0</code>) (primary disk)<br /> | |||
* disk1, 1.6 TB SSD NVME Dell Dell Ent NVMe v2 AGN MU U.2 1.6TB (dev: <code class="file">/dev/disk1</code>) <br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Network:''' | |||
| | |||
* eth0/eno33, Ethernet, configured rate: 25 Gbps, model: Intel Ethernet Controller E810-C for SFP, driver: ice<br /> | |||
* <span style="color:grey">eth1/eno34, Ethernet, model: Intel Ethernet Controller E810-C for SFP, driver: ice - unavailable for experiment</span><br /> | |||
* <span style="color:grey">eth2/eno35, Ethernet, model: Intel Ethernet Controller E810-C for SFP, driver: ice - unavailable for experiment</span><br /> | |||
* <span style="color:grey">eth3/eno36, Ethernet, model: Intel Ethernet Controller E810-C for SFP, driver: ice - unavailable for experiment</span><br /> | |||
* fpga0, FPGA/Ethernet, configured rate: 100 Gbps, model: Xilinx Alveo U200 QSFP28 (100GbE), driver: n/a<br /> | |||
* fpga1, FPGA/Ethernet, configured rate: 100 Gbps, model: Xilinx Alveo U200 QSFP28 (100GbE), driver: n/a<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''FPGA:''' | |||
| Xilinx Alveo U200<br/> | |||
|- | |||
|} | |||
'''Note:''' This cluster is defined as exotic. Please read the '''[[Exotic#Grenoble:_servan|exotic]]''' page for more information.<br/> | |||
== [https://intranet.grid5000.fr/oar/Grenoble/drawgantt-svg/?filter=troll%20only troll] == | |||
'''4 nodes, 8 cpus, 128 cores''' ([https://public-api.grid5000.fr/stable/sites/grenoble/clusters/troll/nodes.json?pretty=1 json]) | '''4 nodes, 8 cpus, 128 cores''' ([https://public-api.grid5000.fr/stable/sites/grenoble/clusters/troll/nodes.json?pretty=1 json]) | ||
'''Reservation example:''' | |||
{{Term|location=fgrenoble|cmd=<code class="command">oarsub</code> <code class="replace">-t exotic</code> <code class="env">-p troll</code> <code>-I</code>}} | |||
{| | {| | ||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Access condition:''' | |||
| [[Exotic#Grenoble:_troll|exotic]] job type<br/> | |||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Model:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Model:''' | ||
| Dell PowerEdge R640<br/> | | Dell PowerEdge R640<br/> | ||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Manufacturing date:''' | |||
| 2019-11-21<br/> | |||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Date of arrival:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Date of arrival:''' | ||
Line 72: | Line 280: | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''CPU:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''CPU:''' | ||
| Intel Xeon Gold 5218 (Cascade Lake-SP, 2.30GHz, 2 CPUs/node, 16 cores/CPU | | Intel Xeon Gold 5218 (Cascade Lake-SP), x86_64, 2.30GHz, 2 CPUs/node, 16 cores/CPU<br/> | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Memory:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Memory:''' | ||
Line 79: | Line 287: | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Storage:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Storage:''' | ||
| | | | ||
* 480 GB SSD SATA Micron MTFDDAK480TDN | * disk0, 480 GB SSD SATA Micron MTFDDAK480TDN (dev: <code class="file">/dev/disk0</code>) (primary disk)<br /> | ||
* 1.6 TB SSD NVME Dell | * disk1, 1.6 TB SSD NVME Dell Samsung PM1735 (dev: <code class="file">/dev/disk1</code>) <br/> | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Network:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Network:''' | ||
| | | | ||
* eth0/eno1, Ethernet, configured rate: | * eth0/eno1, Ethernet, configured rate: 25 Gbps, model: Mellanox Technologies MT27710 Family [ConnectX-4 Lx], driver: mlx5_core, SR-IOV enabled<br /> | ||
* <span style="color:grey">eth1/eno2, Ethernet, model: Mellanox Technologies MT27710 Family [ConnectX-4 Lx], driver: mlx5_core - unavailable for experiment</span><br /> | * <span style="color:grey">eth1/eno2, Ethernet, model: Mellanox Technologies MT27710 Family [ConnectX-4 Lx], driver: mlx5_core - unavailable for experiment</span><br /> | ||
* ib0, Omni-Path, configured rate: 100 Gbps, model: Intel Omni-Path HFI Silicon 100 Series [discrete], driver: hfi1<br/> | * ib0, Omni-Path, configured rate: 100 Gbps, model: Intel Omni-Path HFI Silicon 100 Series [discrete], driver: hfi1<br/> | ||
Line 91: | Line 298: | ||
|} | |} | ||
== yeti == | '''Note:''' This cluster is defined as exotic. Please read the '''[[Exotic#Grenoble:_troll|exotic]]''' page for more information.<br/> | ||
== [https://intranet.grid5000.fr/oar/Grenoble/drawgantt-svg/?filter=yeti%20only yeti] == | |||
'''4 nodes, 16 cpus, 256 cores,''' split as follows due to differences between nodes ([https://public-api.grid5000.fr/stable/sites/grenoble/clusters/yeti/nodes.json?pretty=1 json]) | '''4 nodes, 16 cpus, 256 cores,''' split as follows due to differences between nodes ([https://public-api.grid5000.fr/stable/sites/grenoble/clusters/yeti/nodes.json?pretty=1 json]) | ||
; yeti-[1-2,<wbr>4] ( | |||
'''Reservation example:''' | |||
{{Term|location=fgrenoble|cmd=<code class="command">oarsub</code> <code class="replace">-t exotic</code> <code class="env">-p yeti</code> <code>-I</code>}} | |||
; yeti-1 (1 node, 4 cpus, 64 cores) | |||
{| | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Access condition:''' | |||
| [[Exotic#Grenoble:_yeti|exotic]] job type<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Model:''' | |||
| Dell PowerEdge R940<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Manufacturing date:''' | |||
| 2017-12-26<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Date of arrival:''' | |||
| 2018-01-16<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''CPU:''' | |||
| Intel Xeon Gold 6130 (Skylake), x86_64, 2.10GHz, 4 CPUs/node, 16 cores/CPU<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Memory:''' | |||
| 768 GiB<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Storage:''' | |||
| | |||
* disk0, 480 GB SSD SATA Intel SSDSC2KG480G7R (dev: <code class="file">/dev/disk0</code>) (primary disk)<br /> | |||
* disk1, 2.0 TB HDD SAS Seagate ST2000NX0463 (dev: <code class="file">/dev/disk1</code>[[Disk_reservation|*]]) [[Disk_reservation|(reservable)]] <br /> | |||
* disk2, 2.0 TB HDD SAS Seagate ST2000NX0463 (dev: <code class="file">/dev/disk2</code>[[Disk_reservation|*]]) [[Disk_reservation|(reservable)]] <br /> | |||
* disk3, 2.0 TB HDD SAS Seagate ST2000NX0463 (dev: <code class="file">/dev/disk3</code>[[Disk_reservation|*]]) [[Disk_reservation|(reservable)]] <br /> | |||
* disk4, 1.6 TB SSD NVME Dell Dell Express Flash PM1725b 1.6TB AIC (dev: <code class="file">/dev/disk4</code>) <br /> | |||
* disk5, 1.6 TB SSD NVME Dell Dell Express Flash NVMe PM1725 1.6TB AIC (dev: <code class="file">/dev/disk5</code>) <br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Network:''' | |||
| | |||
* eth0/eno1, Ethernet, configured rate: 10 Gbps, model: Intel Ethernet Controller X710 for 10GbE SFP+, driver: i40e<br /> | |||
* <span style="color:grey">eth1/eno2, Ethernet, model: Intel Ethernet Controller X710 for 10GbE SFP+, driver: i40e - unavailable for experiment</span><br /> | |||
* <span style="color:grey">eth2/eno3, Ethernet, model: Intel Ethernet Controller X710 for 10GbE SFP+, driver: i40e - unavailable for experiment</span><br /> | |||
* <span style="color:grey">eth3/eno4, Ethernet, model: Intel Ethernet Controller X710 for 10GbE SFP+, driver: i40e - unavailable for experiment</span><br /> | |||
* ib0, Omni-Path, configured rate: 100 Gbps, model: Intel Omni-Path HFI Silicon 100 Series [discrete], driver: hfi1<br/> | |||
|- | |||
|} | |||
<hr style="height:10pt; visibility:hidden;" /> | |||
; yeti-[2,<wbr>4] (2 nodes, 8 cpus, 128 cores) | |||
{| | {| | ||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Access condition:''' | |||
| [[Exotic#Grenoble:_yeti|exotic]] job type<br/> | |||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Model:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Model:''' | ||
| Dell PowerEdge R940<br/> | | Dell PowerEdge R940<br/> | ||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Manufacturing date:''' | |||
| 2017-12-26<br/> | |||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Date of arrival:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Date of arrival:''' | ||
Line 104: | Line 365: | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''CPU:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''CPU:''' | ||
| Intel Xeon Gold 6130 (Skylake, 2.10GHz, 4 CPUs/node, 16 cores/CPU | | Intel Xeon Gold 6130 (Skylake), x86_64, 2.10GHz, 4 CPUs/node, 16 cores/CPU<br/> | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Memory:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Memory:''' | ||
Line 111: | Line 372: | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Storage:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Storage:''' | ||
| | | | ||
* 480 GB SSD | * disk0, 480 GB SSD SATA Intel SSDSC2KG480G7R (dev: <code class="file">/dev/disk0</code>) (primary disk)<br /> | ||
* 2.0 TB HDD SAS Seagate ST2000NX0463 | * disk1, 2.0 TB HDD SAS Seagate ST2000NX0463 (dev: <code class="file">/dev/disk1</code>[[Disk_reservation|*]]) [[Disk_reservation|(reservable)]] <br /> | ||
* 2.0 TB HDD SAS Seagate ST2000NX0463 | * disk2, 2.0 TB HDD SAS Seagate ST2000NX0463 (dev: <code class="file">/dev/disk2</code>[[Disk_reservation|*]]) [[Disk_reservation|(reservable)]] <br /> | ||
* 2.0 TB HDD SAS Seagate ST2000NX0463 | * disk3, 2.0 TB HDD SAS Seagate ST2000NX0463 (dev: <code class="file">/dev/disk3</code>[[Disk_reservation|*]]) [[Disk_reservation|(reservable)]] <br /> | ||
* 1.6 TB SSD NVME Dell Dell Express Flash NVMe PM1725 1.6TB AIC | * disk4, 1.6 TB SSD NVME Dell Dell Express Flash NVMe PM1725 1.6TB AIC (dev: <code class="file">/dev/disk4</code>) <br /> | ||
* disk5, 1.6 TB SSD NVME Dell Dell Express Flash NVMe PM1725 1.6TB AIC (dev: <code class="file">/dev/disk5</code>) <br/> | |||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Network:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Network:''' | ||
Line 131: | Line 392: | ||
; yeti-3 (1 node, 4 cpus, 64 cores) | ; yeti-3 (1 node, 4 cpus, 64 cores) | ||
{| | {| | ||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Access condition:''' | |||
| [[Exotic#Grenoble:_yeti|exotic]] job type<br/> | |||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Model:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Model:''' | ||
| Dell PowerEdge R940<br/> | | Dell PowerEdge R940<br/> | ||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Manufacturing date:''' | |||
| 2017-12-26<br/> | |||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Date of arrival:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Date of arrival:''' | ||
Line 139: | Line 406: | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''CPU:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''CPU:''' | ||
| Intel Xeon Gold 6130 (Skylake, 2.10GHz, 4 CPUs/node, 16 cores/CPU | | Intel Xeon Gold 6130 (Skylake), x86_64, 2.10GHz, 4 CPUs/node, 16 cores/CPU<br/> | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Memory:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Memory:''' | ||
Line 146: | Line 413: | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Storage:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Storage:''' | ||
| | | | ||
* 480 GB SSD | * disk0, 480 GB SSD SATA Intel SSDSC2KG480G8R (dev: <code class="file">/dev/disk0</code>) (primary disk)<br /> | ||
* 2.0 TB HDD SAS Seagate ST2000NX0463 | * disk1, 2.0 TB HDD SAS Seagate ST2000NX0463 (dev: <code class="file">/dev/disk1</code>[[Disk_reservation|*]]) [[Disk_reservation|(reservable)]] <br /> | ||
* 2.0 TB HDD SAS Seagate ST2000NX0463 | * disk2, 2.0 TB HDD SAS Seagate ST2000NX0463 (dev: <code class="file">/dev/disk2</code>[[Disk_reservation|*]]) [[Disk_reservation|(reservable)]] <br /> | ||
* 2.0 TB HDD SAS Seagate ST2000NX0463 | * disk3, 2.0 TB HDD SAS Seagate ST2000NX0463 (dev: <code class="file">/dev/disk3</code>[[Disk_reservation|*]]) [[Disk_reservation|(reservable)]] <br /> | ||
* 1.6 TB SSD NVME Dell Dell Express Flash NVMe PM1725 1.6TB AIC | * disk4, 1.6 TB SSD NVME Dell Dell Express Flash NVMe PM1725 1.6TB AIC (dev: <code class="file">/dev/disk4</code>) <br /> | ||
* disk5, 1.6 TB SSD NVME Dell Dell Express Flash NVMe PM1725 1.6TB AIC (dev: <code class="file">/dev/disk5</code>) <br/> | |||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Network:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Network:''' | ||
Line 161: | Line 428: | ||
* ib0, Omni-Path, configured rate: 100 Gbps, model: Intel Omni-Path HFI Silicon 100 Series [discrete], driver: hfi1<br/> | * ib0, Omni-Path, configured rate: 100 Gbps, model: Intel Omni-Path HFI Silicon 100 Series [discrete], driver: hfi1<br/> | ||
|- | |- | ||
|}''<small>Last generated from the Grid'5000 Reference API on | |} | ||
'''Note:''' This cluster is defined as exotic. Please read the '''[[Exotic#Grenoble:_yeti|exotic]]''' page for more information.<br/>''<small>Last generated from the Grid'5000 Reference API on 2024-05-21 ([https://gitlab.inria.fr/grid5000/reference-repository/commit/385241ef09 commit 385241ef09])</small>'' |
Latest revision as of 10:52, 21 May 2024
See also: Network topology for Grenoble
Summary
- 5 clusters
- 54 nodes
- 1744 CPU cores
- 48 GPUs
- 172032 GPUs cores
- 12.25 TiB RAM + 6.0 TiB PMEM
- 88 SSDs and 68 HDDs on nodes (total: 228.53 TB)
- 101.9 TFLOPS (excluding GPUs)
Clusters summary
Cluster | Access Condition | Date of arrival | Manufacturing date | Nodes | CPU | Memory | Storage | Network | Accelerators | |||
---|---|---|---|---|---|---|---|---|---|---|---|---|
# | Name | Cores | Architecture | |||||||||
dahu | 2018-03-22 | 2017-12-12 | 32 | 2 | Intel Xeon Gold 6130 | 16 cores/CPU | x86_64 | 192 GiB | 240 GB SSD + 480 GB SSD + 4.0 TB HDD | 10 Gbps + 100 Gbps Omni-Path | ||
drac | exotic job type | 2020-10-05 | 2016-10-17 | 12 | 2 | IBM POWER8NVL 1.0 | 10 cores/CPU | ppc64le | 128 GiB | 1.0 TB HDD + 1.0 TB HDD | 10 Gbps + 2 x 100 Gbps InfiniBand | 4 x Nvidia Tesla P100 (16 GiB) |
servan | exotic job type | 2021-12-15 | 2021-12-10 | 2 | 2 | AMD EPYC 7352 | 24 cores/CPU | x86_64 | 128 GiB | 1.6 TB SSD + 1.6 TB SSD | 25 Gbps + 2 x 100 Gbps FPGA/Ethernet | Xilinx Alveo U200 |
troll | exotic job type | 2019-12-23 | 2019-11-21 | 4 | 2 | Intel Xeon Gold 5218 | 16 cores/CPU | x86_64 | 384 GiB + 1.5 TiB PMEM | 480 GB SSD + 1.6 TB SSD | 25 Gbps (SR‑IOV) + 100 Gbps Omni-Path | |
yeti | exotic job type | 2018-01-16 | 2017-12-26 | 4 | 4 | Intel Xeon Gold 6130 | 16 cores/CPU | x86_64 | 768 GiB | 480 GB SSD + 3 x 2.0 TB HDD* + 2 x 1.6 TB SSD | 10 Gbps + 100 Gbps Omni-Path |
*: disk is reservable
**: crossed GPUs are not supported by Grid'5000 default environments
***: OPA (Omni-Path Architecture) is currently not supported on Debian 12 environment
Clusters in the default queue
dahu
32 nodes, 64 cpus, 1024 cores, split as follows due to differences between nodes (json)
Reservation example:
- dahu-[1,
4-32] (30 nodes, 60 cpus, 960 cores)
Model: | Dell PowerEdge C6420 |
Manufacturing date: | 2017-12-12 |
Date of arrival: | 2018-03-22 |
CPU: | Intel Xeon Gold 6130 (Skylake), x86_64, 2.10GHz, 2 CPUs/node, 16 cores/CPU |
Memory: | 192 GiB |
Storage: |
|
Network: |
|
- dahu-2 (1 node, 2 cpus, 32 cores)
Model: | Dell PowerEdge C6420 |
Manufacturing date: | 2017-12-12 |
Date of arrival: | 2018-03-22 |
CPU: | Intel Xeon Gold 6130 (Skylake), x86_64, 2.10GHz, 2 CPUs/node, 16 cores/CPU |
Memory: | 192 GiB |
Storage: |
|
Network: |
|
- dahu-3 (1 node, 2 cpus, 32 cores)
Model: | Dell PowerEdge C6420 |
Manufacturing date: | 2017-12-12 |
Date of arrival: | 2018-03-22 |
CPU: | Intel Xeon Gold 6130 (Skylake), x86_64, 2.10GHz, 2 CPUs/node, 16 cores/CPU |
Memory: | 192 GiB |
Storage: |
|
Network: |
|
drac
12 nodes, 24 cpus, 240 cores (json)
Reservation example:
Access condition: | exotic job type |
Model: | IBM PowerNV S822LC (8335-GTB) |
Manufacturing date: | 2016-10-17 |
Date of arrival: | 2020-10-05 |
CPU: | IBM POWER8NVL 1.0 (POWER8), ppc64le, 2 CPUs/node, 10 cores/CPU |
Memory: | 128 GiB |
Storage: |
|
Network: |
|
GPU: | 4 x Nvidia Tesla P100-SXM2-16GB (16 GiB) Compute capability: 6.0 |
Note: This cluster is defined as exotic. Please read the exotic page for more information.
servan
2 nodes, 4 cpus, 96 cores (json)
Reservation example:
Access condition: | exotic job type |
Model: | Dell PowerEdge R7525 |
Manufacturing date: | 2021-12-10 |
Date of arrival: | 2021-12-15 |
CPU: | AMD EPYC 7352 (Zen 2), x86_64, 2 CPUs/node, 24 cores/CPU |
Memory: | 128 GiB |
Storage: |
|
Network: |
|
FPGA: | Xilinx Alveo U200 |
Note: This cluster is defined as exotic. Please read the exotic page for more information.
troll
4 nodes, 8 cpus, 128 cores (json)
Reservation example:
Access condition: | exotic job type |
Model: | Dell PowerEdge R640 |
Manufacturing date: | 2019-11-21 |
Date of arrival: | 2019-12-23 |
CPU: | Intel Xeon Gold 5218 (Cascade Lake-SP), x86_64, 2.30GHz, 2 CPUs/node, 16 cores/CPU |
Memory: | 384 GiB + 1.5 TiB PMEM |
Storage: |
|
Network: |
|
Note: This cluster is defined as exotic. Please read the exotic page for more information.
yeti
4 nodes, 16 cpus, 256 cores, split as follows due to differences between nodes (json)
Reservation example:
- yeti-1 (1 node, 4 cpus, 64 cores)
Access condition: | exotic job type |
Model: | Dell PowerEdge R940 |
Manufacturing date: | 2017-12-26 |
Date of arrival: | 2018-01-16 |
CPU: | Intel Xeon Gold 6130 (Skylake), x86_64, 2.10GHz, 4 CPUs/node, 16 cores/CPU |
Memory: | 768 GiB |
Storage: |
|
Network: |
|
- yeti-[2,
4] (2 nodes, 8 cpus, 128 cores)
Access condition: | exotic job type |
Model: | Dell PowerEdge R940 |
Manufacturing date: | 2017-12-26 |
Date of arrival: | 2018-01-16 |
CPU: | Intel Xeon Gold 6130 (Skylake), x86_64, 2.10GHz, 4 CPUs/node, 16 cores/CPU |
Memory: | 768 GiB |
Storage: |
|
Network: |
|
- yeti-3 (1 node, 4 cpus, 64 cores)
Access condition: | exotic job type |
Model: | Dell PowerEdge R940 |
Manufacturing date: | 2017-12-26 |
Date of arrival: | 2018-01-16 |
CPU: | Intel Xeon Gold 6130 (Skylake), x86_64, 2.10GHz, 4 CPUs/node, 16 cores/CPU |
Memory: | 768 GiB |
Storage: |
|
Network: |
|
Note: This cluster is defined as exotic. Please read the exotic page for more information.
Last generated from the Grid'5000 Reference API on 2024-05-21 (commit 385241ef09)