Grenoble:Hardware: Difference between revisions
Jump to navigation
Jump to search
No edit summary |
No edit summary |
||
(98 intermediate revisions by 15 users not shown) | |||
Line 1: | Line 1: | ||
__NOTOC__ | __NOTOC__ | ||
__NOEDITSECTION__ | __NOEDITSECTION__ | ||
<div class="sitelink">[[Hardware|Global]] | [[Grenoble:Hardware|Grenoble]] | [[Lille:Hardware|Lille]] | [[Luxembourg:Hardware|Luxembourg]] | [[Lyon:Hardware|Lyon]] | [[Nancy:Hardware|Nancy]] | [[Nantes:Hardware|Nantes]] | [[Rennes:Hardware|Rennes]] | [[Sophia:Hardware|Sophia]]</div> | {{Portal|User}} | ||
<div class="sitelink">Hardware: [[Hardware|Global]] | [[Grenoble:Hardware|Grenoble]] | [[Lille:Hardware|Lille]] | [[Luxembourg:Hardware|Luxembourg]] | [[Lyon:Hardware|Lyon]] | [[Nancy:Hardware|Nancy]] | [[Nantes:Hardware|Nantes]] | [[Rennes:Hardware|Rennes]] | [[Sophia:Hardware|Sophia]]</div> | |||
'''See also:''' [[Grenoble:Network|Network topology for Grenoble]] | |||
= Summary = | |||
* 5 clusters | |||
* 54 nodes | |||
* 1744 CPU cores | |||
* 48 GPUs | |||
* 12.25 TiB RAM + 6.0 TiB PMEM | |||
* 88 SSDs and 68 HDDs on nodes (total: 228.53 TB) | |||
* 101.9 TFLOPS (excluding GPUs) | |||
= | = Clusters = | ||
{|class="wikitable sortable" | {|class="wikitable sortable" | ||
|- | |- | ||
!|Cluster | !rowspan=2|Cluster | ||
!| | !rowspan=2|Access Condition | ||
!|Date of arrival | !rowspan=2|Date of arrival | ||
!data-sort-type="number"|Nodes | !data-sort-type="number" rowspan=2|Nodes | ||
!|CPU | !colspan=4|CPU | ||
!data-sort-type="number" rowspan=2|Memory | |||
!data-sort-type="number" rowspan=2|Storage | |||
!data-sort-type="number" rowspan=2|Network | |||
!rowspan=2|Accelerators | |||
|- | |||
!data-sort-type="number"|# | |||
!|Name | |||
!data-sort-type="number"|Cores | !data-sort-type="number"|Cores | ||
! | !|Architecture | ||
|- | |- | ||
|[[# | |||
|[[#dahu|dahu]]||||2018-03-22||32||2||Intel Xeon Gold 6130||16 cores/CPU||x86_64||192 GiB||data-sort-value="4396"|<b>240 GB SSD</b> + 480 GB SSD + 4.0 TB HDD||data-sort-value="110000"|10 Gbps + 100 Gbps Omni-Path|| | |||
|- | |- | ||
|[[# | |[[#drac|drac]]||<b>[[Getting_Started#Selecting_specific_resources|exotic]]</b> job type||2020-10-05||12||2||POWER8NVL 1.0||10 cores/CPU||ppc64le||128 GiB||data-sort-value="1862"|<b>1.0 TB HDD</b> + 1.0 TB HDD||data-sort-value="210000"|10 Gbps + 2 x 100 Gbps InfiniBand||4 x Nvidia Tesla P100 (16 GiB) | ||
|- | |- | ||
|[[# | |[[#servan|servan]]||<b>[[Getting_Started#Selecting_specific_resources|exotic]]</b> job type||2021-12-15||2||2||AMD EPYC 7352||24 cores/CPU||x86_64||128 GiB||data-sort-value="2980"|<b>1.6 TB SSD</b> + 1.6 TB SSD||data-sort-value="225000"|25 Gbps + 2 x 100 Gbps Ethernet-FPGA||Xilinx Alveo U200 | ||
|- | |- | ||
|[[# | |[[#troll|troll]]||<b>[[Getting_Started#Selecting_specific_resources|exotic]]</b> job type||2019-12-23||4||2||Intel Xeon Gold 5218||16 cores/CPU||x86_64||384 GiB + 1.5 TiB [[PMEM]]||data-sort-value="1937"|<b>480 GB SSD</b> + 1.6 TB SSD||data-sort-value="125000"|25 Gbps (SR‑IOV) + 100 Gbps Omni-Path|| | ||
|- | |||
|[[#yeti|yeti]]||<b>[[Getting_Started#Selecting_specific_resources|exotic]]</b> job type||2018-01-16||4||4||Intel Xeon Gold 6130||16 cores/CPU||x86_64||768 GiB||data-sort-value="9016"|<b>480 GB SSD</b> + 3 x 2.0 TB HDD[[Disk_reservation|*]] + 2 x 1.6 TB SSD||data-sort-value="110000"|10 Gbps + 100 Gbps Omni-Path|| | |||
|- | |||
|} | |||
''*: disk is [[Disk_reservation|reservable]]'' | |||
= Clusters in the [https://intranet.grid5000.fr/oar/Grenoble/drawgantt-svg/ default queue] = | |||
== [https://intranet.grid5000.fr/oar/Grenoble/drawgantt-svg/?filter=dahu%20only dahu] == | |||
'''32 nodes, 64 cpus, 1024 cores''' ([https://public-api.grid5000.fr/stable/sites/grenoble/clusters/dahu/nodes.json?pretty=1 json]) | |||
'''Reservation example:''' | |||
== | {{Term|location=fgrenoble|cmd=<code class="command">oarsub</code> <code class="env">-p dahu</code> <code>-I</code>}} | ||
{| | {| | ||
|- | |- | ||
Line 45: | Line 66: | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Memory:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Memory:''' | ||
| 192 | | 192 GiB<br/> | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Storage:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Storage:''' | ||
| | | | ||
* | * disk0, 240 GB SSD SATA Samsung MZ7KM240HMHQ0D3 (dev: <code class="file">/dev/disk0</code>, by-path: <code class="file">/dev/disk/by-path/pci-0000:00:11.5-ata-3</code>) (primary disk)<br /> | ||
* | * disk1, 480 GB SSD SATA Samsung MZ7KM480HMHQ0D3 (dev: <code class="file">/dev/disk1</code>, by-path: <code class="file">/dev/disk/by-path/pci-0000:00:11.5-ata-4</code>) <br /> | ||
* | * disk2, 4.0 TB HDD SATA Seagate ST4000NM0265-2DC (dev: <code class="file">/dev/disk2</code>, by-path: <code class="file">/dev/disk/by-path/pci-0000:00:11.5-ata-5</code>) <br/> | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Network:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Network:''' | ||
| | | | ||
* eth0/enp24s0f0, Ethernet, configured rate: 10 Gbps, model: Intel Ethernet Controller X710 for 10GbE SFP+, driver: i40e<br /> | * eth0/enp24s0f0, Ethernet, configured rate: 10 Gbps, model: Intel Ethernet Controller X710 for 10GbE SFP+, driver: i40e<br /> | ||
* <span style="color:grey">eth1/enp24s0f1, Ethernet, model: Intel Ethernet Controller X710 for 10GbE SFP+, driver: i40e - unavailable for experiment</span><br/> | * <span style="color:grey">eth1/enp24s0f1, Ethernet, model: Intel Ethernet Controller X710 for 10GbE SFP+, driver: i40e - unavailable for experiment</span><br /> | ||
* ib0, Omni-Path, configured rate: 100 Gbps, model: Intel Omni-Path HFI Silicon 100 Series [discrete], driver: hfi1<br/> | |||
|- | |- | ||
|} | |} | ||
== [https://intranet.grid5000.fr/oar/Grenoble/drawgantt-svg/?filter=drac%20only drac] == | |||
'''12 nodes, 24 cpus, 240 cores''' ([https://public-api.grid5000.fr/stable/sites/grenoble/clusters/drac/nodes.json?pretty=1 json]) | |||
'''Reservation example:''' | |||
{{Term|location=fgrenoble|cmd=<code class="command">oarsub</code> <code class="replace">-t exotic</code> <code class="env">-p drac</code> <code>-I</code>}} | |||
{| | {| | ||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Access condition:''' | |||
| exotic job type<br/> | |||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Model:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Model:''' | ||
| | | IBM PowerNV S822LC (8335-GTB)<br/> | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Date of arrival:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Date of arrival:''' | ||
| | | 2020-10-05<br/> | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''CPU:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''CPU:''' | ||
| | | POWER8NVL 1.0 (POWER8, 2 CPUs/node, 10 cores/CPU)<br/> | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Memory:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Memory:''' | ||
| | | 128 GiB<br/> | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Storage:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Storage:''' | ||
| | | | ||
* | * disk0, 1.0 TB HDD SATA Seagate ST1000NX0313 (dev: <code class="file">/dev/disk0</code>, by-path: <code class="file">/dev/disk/by-path/pci-0009:04:00.0-ata-1</code>) (primary disk)<br /> | ||
* disk1, 1.0 TB HDD SATA Seagate ST1000NX0313 (dev: <code class="file">/dev/disk1</code>, by-path: <code class="file">/dev/disk/by-path/pci-0009:04:00.0-ata-2</code>) <br/> | |||
* | |||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Network:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Network:''' | ||
| | | | ||
* | * eth0/enP1p1s0f0, Ethernet, configured rate: 10 Gbps, model: Broadcom Inc. and subsidiaries NetXtreme II BCM57800 1/10 Gigabit Ethernet, driver: bnx2x<br /> | ||
* | * <span style="color:grey">eth1/enP1p1s0f1, Ethernet, model: Broadcom Inc. and subsidiaries NetXtreme II BCM57800 1/10 Gigabit Ethernet, driver: bnx2x - unavailable for experiment</span><br /> | ||
* <span style="color:grey">eth2/enP1p1s0f2, Ethernet, model: Broadcom Inc. and subsidiaries NetXtreme II BCM57800 1/10 Gigabit Ethernet, driver: bnx2x - unavailable for experiment</span><br /> | |||
* <span style="color:grey">eth3/enP1p1s0f3, Ethernet, model: Broadcom Inc. and subsidiaries NetXtreme II BCM57800 1/10 Gigabit Ethernet, driver: bnx2x - unavailable for experiment</span><br /> | |||
* <span style="color:grey">eth4/enP9p7s0f0, Ethernet, configured rate: 1 Gbps, model: Broadcom Inc. and subsidiaries NetXtreme BCM5719 Gigabit Ethernet PCIe, driver: tg3 - unavailable for experiment</span><br /> | |||
* <span style="color:grey">eth5/enP9p7s0f1, Ethernet, model: Broadcom Inc. and subsidiaries NetXtreme BCM5719 Gigabit Ethernet PCIe, driver: tg3 - unavailable for experiment</span><br /> | |||
* ib0, InfiniBand, configured rate: 100 Gbps, model: Mellanox Technologies MT27700 Family [ConnectX-4], driver: mlx5_core<br /> | |||
* ib1, InfiniBand, configured rate: 100 Gbps, model: Mellanox Technologies MT27700 Family [ConnectX-4], driver: mlx5_core<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''GPU:''' | |||
| 4 x Nvidia Tesla P100-SXM2-16GB (16 GiB)<br>Compute capability: 6.0<br/> | |||
|- | |- | ||
|} | |} | ||
== [https://intranet.grid5000.fr/oar/Grenoble/drawgantt-svg/?filter=servan%20only servan] == | |||
'''2 nodes, 4 cpus, 96 cores''' ([https://public-api.grid5000.fr/stable/sites/grenoble/clusters/servan/nodes.json?pretty=1 json]) | |||
'''Reservation example:''' | |||
{{Term|location=fgrenoble|cmd=<code class="command">oarsub</code> <code class="replace">-t exotic</code> <code class="env">-p servan</code> <code>-I</code>}} | |||
{| | {| | ||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Access condition:''' | |||
| exotic job type<br/> | |||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Model:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Model:''' | ||
| Dell PowerEdge | | Dell PowerEdge R7525<br/> | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Date of arrival:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Date of arrival:''' | ||
| | | 2021-12-15<br/> | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''CPU:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''CPU:''' | ||
| | | AMD EPYC 7352 (Zen 2, 2 CPUs/node, 24 cores/CPU)<br/> | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Memory:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Memory:''' | ||
| | | 128 GiB<br/> | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Storage:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Storage:''' | ||
| | | | ||
* | * disk0, 1.6 TB SSD NVME Dell Dell Ent NVMe v2 AGN MU U.2 1.6TB (dev: <code class="file">/dev/disk0</code>, by-path: <code class="file">/dev/disk/by-path/pci-0000:c3:00.0-nvme-1</code>) (primary disk)<br /> | ||
* disk1, 1.6 TB SSD NVME Dell Dell Ent NVMe v2 AGN MU U.2 1.6TB (dev: <code class="file">/dev/disk1</code>, by-path: <code class="file">/dev/disk/by-path/pci-0000:c4:00.0-nvme-1</code>) <br/> | |||
* | |||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Network:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Network:''' | ||
| | | | ||
* | * eth0/eno33, Ethernet, configured rate: 25 Gbps, model: Intel Ethernet Controller E810-C for SFP, driver: ice<br /> | ||
* | * <span style="color:grey">eth1/eno34, Ethernet, model: Intel Ethernet Controller E810-C for SFP, driver: ice - unavailable for experiment</span><br /> | ||
* | * <span style="color:grey">eth2/eno35, Ethernet, model: Intel Ethernet Controller E810-C for SFP, driver: ice - unavailable for experiment</span><br /> | ||
* <span style="color:grey">eth3/eno36, Ethernet, model: Intel Ethernet Controller E810-C for SFP, driver: ice - unavailable for experiment</span><br /> | |||
* fpga0, Ethernet-FPGA, configured rate: 100 Gbps, model: Xilinx Alveo U200 QSFP28 (100GbE), driver: xrt<br /> | |||
* fpga1, Ethernet-FPGA, configured rate: 100 Gbps, model: Xilinx Alveo U200 QSFP28 (100GbE), driver: xrt<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''FPGA:''' | |||
| Xilinx Alveo U200<br/> | |||
|- | |- | ||
|} | |} | ||
== [https://intranet.grid5000.fr/oar/Grenoble/drawgantt-svg/?filter=troll%20only troll] == | |||
'''4 nodes, 8 cpus, 128 cores''' ([https://public-api.grid5000.fr/stable/sites/grenoble/clusters/troll/nodes.json?pretty=1 json]) | |||
'''Reservation example:''' | |||
{{Term|location=fgrenoble|cmd=<code class="command">oarsub</code> <code class="replace">-t exotic</code> <code class="env">-p troll</code> <code>-I</code>}} | |||
{| | {| | ||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Access condition:''' | |||
| exotic job type<br/> | |||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Model:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Model:''' | ||
| Dell PowerEdge | | Dell PowerEdge R640<br/> | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Date of arrival:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Date of arrival:''' | ||
| | | 2019-12-23<br/> | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''CPU:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''CPU:''' | ||
| Intel Xeon Gold | | Intel Xeon Gold 5218 (Cascade Lake-SP, 2.30GHz, 2 CPUs/node, 16 cores/CPU)<br/> | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Memory:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Memory:''' | ||
| | | 384 GiB + 1.5 TiB [[PMEM]]<br/> | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Storage:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Storage:''' | ||
| | | | ||
* | * disk0, 480 GB SSD SATA Micron MTFDDAK480TDN (dev: <code class="file">/dev/disk0</code>, by-path: <code class="file">/dev/disk/by-path/pci-0000:18:00.0-scsi-0:0:0:0</code>) (primary disk)<br /> | ||
* disk1, 1.6 TB SSD NVME Dell Samsung PM1735 (dev: <code class="file">/dev/disk1</code>, by-path: <code class="file">/dev/disk/by-path/pci-0000:d8:00.0-nvme-1</code>) <br/> | |||
* | |||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Network:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Network:''' | ||
| | | | ||
* eth0/ | * eth0/eno1, Ethernet, configured rate: 25 Gbps, model: Mellanox Technologies MT27710 Family [ConnectX-4 Lx], driver: mlx5_core, SR-IOV enabled<br /> | ||
* <span style="color:grey">eth1/ | * <span style="color:grey">eth1/eno2, Ethernet, model: Mellanox Technologies MT27710 Family [ConnectX-4 Lx], driver: mlx5_core - unavailable for experiment</span><br /> | ||
* ib0, Omni-Path, configured rate: 100 Gbps, model: Intel Omni-Path HFI Silicon 100 Series [discrete], driver: hfi1<br/> | |||
|- | |- | ||
|} | |} | ||
== [https://intranet.grid5000.fr/oar/Grenoble/drawgantt-svg/?filter=yeti%20only yeti] == | |||
'''4 nodes, 16 cpus, 256 cores,''' split as follows due to differences between nodes ([https://public-api.grid5000.fr/stable/sites/grenoble/clusters/yeti/nodes.json?pretty=1 json]) | |||
'''Reservation example:''' | |||
{{Term|location=fgrenoble|cmd=<code class="command">oarsub</code> <code class="replace">-t exotic</code> <code class="env">-p yeti</code> <code>-I</code>}} | |||
{| | |||
| | |||
; | ; yeti-1 (1 node, 4 cpus, 64 cores) | ||
{| | {| | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Access condition:''' | ||
| | | exotic job type<br/> | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Model:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Model:''' | ||
| | | Dell PowerEdge R940<br/> | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Date of arrival:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Date of arrival:''' | ||
| | | 2018-01-16<br/> | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''CPU:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''CPU:''' | ||
| Intel Xeon | | Intel Xeon Gold 6130 (Skylake, 2.10GHz, 4 CPUs/node, 16 cores/CPU)<br/> | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Memory:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Memory:''' | ||
| | | 768 GiB<br/> | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Storage:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Storage:''' | ||
| | | | ||
* | * disk0, 480 GB SSD SATA Intel SSDSC2KG480G7R (dev: <code class="file">/dev/disk0</code>, by-path: <code class="file">/dev/disk/by-path/pci-0000:18:00.0-scsi-0:0:0:0</code>) (primary disk)<br /> | ||
* disk1, 2.0 TB HDD SAS Seagate ST2000NX0463 (dev: <code class="file">/dev/disk1</code>[[Disk_reservation|*]], by-path: <code class="file">/dev/disk/by-path/pci-0000:18:00.0-scsi-0:0:1:0</code>) [[Disk_reservation|(reservable)]] <br /> | |||
* | * disk2, 2.0 TB HDD SAS Seagate ST2000NX0463 (dev: <code class="file">/dev/disk2</code>[[Disk_reservation|*]], by-path: <code class="file">/dev/disk/by-path/pci-0000:18:00.0-scsi-0:0:2:0</code>) [[Disk_reservation|(reservable)]] <br /> | ||
* < | * disk3, 2.0 TB HDD SAS Seagate ST2000NX0463 (dev: <code class="file">/dev/disk3</code>[[Disk_reservation|*]], by-path: <code class="file">/dev/disk/by-path/pci-0000:18:00.0-scsi-0:0:3:0</code>) [[Disk_reservation|(reservable)]] <br /> | ||
* disk4, 1.6 TB SSD NVME Dell Dell Express Flash PM1725b 1.6TB AIC (dev: <code class="file">/dev/disk4</code>, by-path: <code class="file">/dev/disk/by-path/pci-0000:59:00.0-nvme-1</code>) <br /> | |||
* disk5, 1.6 TB SSD NVME Dell Dell Express Flash NVMe PM1725 1.6TB AIC (dev: <code class="file">/dev/disk5</code>, by-path: <code class="file">/dev/disk/by-path/pci-0000:6d:00.0-nvme-1</code>) <br/> | |||
< | |||
| | |||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Network:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Network:''' | ||
| | | | ||
* eth0/ | * eth0/eno1, Ethernet, configured rate: 10 Gbps, model: Intel Ethernet Controller X710 for 10GbE SFP+, driver: i40e<br /> | ||
* <span style="color:grey">eth1/ | * <span style="color:grey">eth1/eno2, Ethernet, model: Intel Ethernet Controller X710 for 10GbE SFP+, driver: i40e - unavailable for experiment</span><br /> | ||
* | * <span style="color:grey">eth2/eno3, Ethernet, model: Intel Ethernet Controller X710 for 10GbE SFP+, driver: i40e - unavailable for experiment</span><br /> | ||
* <span style="color:grey"> | * <span style="color:grey">eth3/eno4, Ethernet, model: Intel Ethernet Controller X710 for 10GbE SFP+, driver: i40e - unavailable for experiment</span><br /> | ||
* ib0, Omni-Path, configured rate: 100 Gbps, model: Intel Omni-Path HFI Silicon 100 Series [discrete], driver: hfi1<br/> | |||
|- | |- | ||
|} | |} | ||
<hr style="height:10pt; visibility:hidden;" /> | <hr style="height:10pt; visibility:hidden;" /> | ||
; | ; yeti-[2,<wbr>4] (2 nodes, 8 cpus, 128 cores) | ||
{| | {| | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Access condition:''' | ||
| | | exotic job type<br/> | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Model:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Model:''' | ||
| | | Dell PowerEdge R940<br/> | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Date of arrival:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Date of arrival:''' | ||
| | | 2018-01-16<br/> | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''CPU:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''CPU:''' | ||
| Intel Xeon | | Intel Xeon Gold 6130 (Skylake, 2.10GHz, 4 CPUs/node, 16 cores/CPU)<br/> | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Memory:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Memory:''' | ||
| | | 768 GiB<br/> | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Storage:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Storage:''' | ||
| | | | ||
* | * disk0, 480 GB SSD SATA Intel SSDSC2KG480G7R (dev: <code class="file">/dev/disk0</code>, by-path: <code class="file">/dev/disk/by-path/pci-0000:18:00.0-scsi-0:0:0:0</code>) (primary disk)<br /> | ||
* disk1, 2.0 TB HDD SAS Seagate ST2000NX0463 (dev: <code class="file">/dev/disk1</code>[[Disk_reservation|*]], by-path: <code class="file">/dev/disk/by-path/pci-0000:18:00.0-scsi-0:0:1:0</code>) [[Disk_reservation|(reservable)]] <br /> | |||
* | * disk2, 2.0 TB HDD SAS Seagate ST2000NX0463 (dev: <code class="file">/dev/disk2</code>[[Disk_reservation|*]], by-path: <code class="file">/dev/disk/by-path/pci-0000:18:00.0-scsi-0:0:2:0</code>) [[Disk_reservation|(reservable)]] <br /> | ||
* < | * disk3, 2.0 TB HDD SAS Seagate ST2000NX0463 (dev: <code class="file">/dev/disk3</code>[[Disk_reservation|*]], by-path: <code class="file">/dev/disk/by-path/pci-0000:18:00.0-scsi-0:0:3:0</code>) [[Disk_reservation|(reservable)]] <br /> | ||
* disk4, 1.6 TB SSD NVME Dell Dell Express Flash NVMe PM1725 1.6TB AIC (dev: <code class="file">/dev/disk4</code>, by-path: <code class="file">/dev/disk/by-path/pci-0000:59:00.0-nvme-1</code>) <br /> | |||
* disk5, 1.6 TB SSD NVME Dell Dell Express Flash NVMe PM1725 1.6TB AIC (dev: <code class="file">/dev/disk5</code>, by-path: <code class="file">/dev/disk/by-path/pci-0000:6d:00.0-nvme-1</code>) <br/> | |||
< | |||
| | |||
|- | |||
| | |||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Network:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Network:''' | ||
| | | | ||
* eth0/ | * eth0/eno1, Ethernet, configured rate: 10 Gbps, model: Intel Ethernet Controller X710 for 10GbE SFP+, driver: i40e<br /> | ||
* <span style="color:grey">eth1/ | * <span style="color:grey">eth1/eno2, Ethernet, model: Intel Ethernet Controller X710 for 10GbE SFP+, driver: i40e - unavailable for experiment</span><br /> | ||
* | * <span style="color:grey">eth2/eno3, Ethernet, model: Intel Ethernet Controller X710 for 10GbE SFP+, driver: i40e - unavailable for experiment</span><br /> | ||
* <span style="color:grey"> | * <span style="color:grey">eth3/eno4, Ethernet, model: Intel Ethernet Controller X710 for 10GbE SFP+, driver: i40e - unavailable for experiment</span><br /> | ||
* ib0, Omni-Path, configured rate: 100 Gbps, model: Intel Omni-Path HFI Silicon 100 Series [discrete], driver: hfi1<br/> | |||
|- | |- | ||
|} | |} | ||
<hr style="height:10pt; visibility:hidden;" /> | <hr style="height:10pt; visibility:hidden;" /> | ||
; | ; yeti-3 (1 node, 4 cpus, 64 cores) | ||
{| | {| | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Access condition:''' | ||
| | | exotic job type<br/> | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Model:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Model:''' | ||
Line 476: | Line 310: | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Memory:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Memory:''' | ||
| 768 | | 768 GiB<br/> | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Storage:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Storage:''' | ||
| | | | ||
* | * disk0, 480 GB SSD SATA Intel SSDSC2KG480G8R (dev: <code class="file">/dev/disk0</code>, by-path: <code class="file">/dev/disk/by-path/pci-0000:18:00.0-scsi-0:0:0:0</code>) (primary disk)<br /> | ||
* | * disk1, 2.0 TB HDD SAS Seagate ST2000NX0463 (dev: <code class="file">/dev/disk1</code>[[Disk_reservation|*]], by-path: <code class="file">/dev/disk/by-path/pci-0000:18:00.0-scsi-0:0:1:0</code>) [[Disk_reservation|(reservable)]] <br /> | ||
* | * disk2, 2.0 TB HDD SAS Seagate ST2000NX0463 (dev: <code class="file">/dev/disk2</code>[[Disk_reservation|*]], by-path: <code class="file">/dev/disk/by-path/pci-0000:18:00.0-scsi-0:0:2:0</code>) [[Disk_reservation|(reservable)]] <br /> | ||
* | * disk3, 2.0 TB HDD SAS Seagate ST2000NX0463 (dev: <code class="file">/dev/disk3</code>[[Disk_reservation|*]], by-path: <code class="file">/dev/disk/by-path/pci-0000:18:00.0-scsi-0:0:3:0</code>) [[Disk_reservation|(reservable)]] <br /> | ||
* disk4, 1.6 TB SSD NVME Dell Dell Express Flash NVMe PM1725 1.6TB AIC (dev: <code class="file">/dev/disk4</code>, by-path: <code class="file">/dev/disk/by-path/pci-0000:59:00.0-nvme-1</code>) <br /> | |||
* disk5, 1.6 TB SSD NVME Dell Dell Express Flash NVMe PM1725 1.6TB AIC (dev: <code class="file">/dev/disk5</code>, by-path: <code class="file">/dev/disk/by-path/pci-0000:6d:00.0-nvme-1</code>) <br/> | |||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Network:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Network:''' | ||
| | | | ||
* eth0/ | * eth0/eno1, Ethernet, configured rate: 10 Gbps, model: Intel Ethernet Controller X710 for 10GbE SFP+, driver: i40e<br /> | ||
* <span style="color:grey">eth1/ | * <span style="color:grey">eth1/eno2, Ethernet, model: Intel Ethernet Controller X710 for 10GbE SFP+, driver: i40e - unavailable for experiment</span><br /> | ||
* <span style="color:grey">eth2/ | * <span style="color:grey">eth2/eno3, Ethernet, model: Intel Ethernet Controller X710 for 10GbE SFP+, driver: i40e - unavailable for experiment</span><br /> | ||
* <span style="color:grey">eth3/ | * <span style="color:grey">eth3/eno4, Ethernet, model: Intel Ethernet Controller X710 for 10GbE SFP+, driver: i40e - unavailable for experiment</span><br /> | ||
* ib0, Omni-Path, configured rate: 100 Gbps, model: Intel Omni-Path HFI Silicon 100 Series [discrete], driver: hfi1<br/> | |||
|- | |- | ||
|}''<small>Last generated from the Grid'5000 Reference API on | |}''<small>Last generated from the Grid'5000 Reference API on 2022-06-29 ([https://github.com/grid5000/reference-repository/commit/362f6514fa commit 362f6514fa])</small>'' |
Revision as of 18:10, 29 June 2022
See also: Network topology for Grenoble
Summary
- 5 clusters
- 54 nodes
- 1744 CPU cores
- 48 GPUs
- 12.25 TiB RAM + 6.0 TiB PMEM
- 88 SSDs and 68 HDDs on nodes (total: 228.53 TB)
- 101.9 TFLOPS (excluding GPUs)
Clusters
Cluster | Access Condition | Date of arrival | Nodes | CPU | Memory | Storage | Network | Accelerators | |||
---|---|---|---|---|---|---|---|---|---|---|---|
# | Name | Cores | Architecture | ||||||||
dahu | 2018-03-22 | 32 | 2 | Intel Xeon Gold 6130 | 16 cores/CPU | x86_64 | 192 GiB | 240 GB SSD + 480 GB SSD + 4.0 TB HDD | 10 Gbps + 100 Gbps Omni-Path | ||
drac | exotic job type | 2020-10-05 | 12 | 2 | POWER8NVL 1.0 | 10 cores/CPU | ppc64le | 128 GiB | 1.0 TB HDD + 1.0 TB HDD | 10 Gbps + 2 x 100 Gbps InfiniBand | 4 x Nvidia Tesla P100 (16 GiB) |
servan | exotic job type | 2021-12-15 | 2 | 2 | AMD EPYC 7352 | 24 cores/CPU | x86_64 | 128 GiB | 1.6 TB SSD + 1.6 TB SSD | 25 Gbps + 2 x 100 Gbps Ethernet-FPGA | Xilinx Alveo U200 |
troll | exotic job type | 2019-12-23 | 4 | 2 | Intel Xeon Gold 5218 | 16 cores/CPU | x86_64 | 384 GiB + 1.5 TiB PMEM | 480 GB SSD + 1.6 TB SSD | 25 Gbps (SR‑IOV) + 100 Gbps Omni-Path | |
yeti | exotic job type | 2018-01-16 | 4 | 4 | Intel Xeon Gold 6130 | 16 cores/CPU | x86_64 | 768 GiB | 480 GB SSD + 3 x 2.0 TB HDD* + 2 x 1.6 TB SSD | 10 Gbps + 100 Gbps Omni-Path |
*: disk is reservable
Clusters in the default queue
dahu
32 nodes, 64 cpus, 1024 cores (json)
Reservation example:
Model: | Dell PowerEdge C6420 |
Date of arrival: | 2018-03-22 |
CPU: | Intel Xeon Gold 6130 (Skylake, 2.10GHz, 2 CPUs/node, 16 cores/CPU) |
Memory: | 192 GiB |
Storage: |
|
Network: |
|
drac
12 nodes, 24 cpus, 240 cores (json)
Reservation example:
Access condition: | exotic job type |
Model: | IBM PowerNV S822LC (8335-GTB) |
Date of arrival: | 2020-10-05 |
CPU: | POWER8NVL 1.0 (POWER8, 2 CPUs/node, 10 cores/CPU) |
Memory: | 128 GiB |
Storage: |
|
Network: |
|
GPU: | 4 x Nvidia Tesla P100-SXM2-16GB (16 GiB) Compute capability: 6.0 |
servan
2 nodes, 4 cpus, 96 cores (json)
Reservation example:
Access condition: | exotic job type |
Model: | Dell PowerEdge R7525 |
Date of arrival: | 2021-12-15 |
CPU: | AMD EPYC 7352 (Zen 2, 2 CPUs/node, 24 cores/CPU) |
Memory: | 128 GiB |
Storage: |
|
Network: |
|
FPGA: | Xilinx Alveo U200 |
troll
4 nodes, 8 cpus, 128 cores (json)
Reservation example:
Access condition: | exotic job type |
Model: | Dell PowerEdge R640 |
Date of arrival: | 2019-12-23 |
CPU: | Intel Xeon Gold 5218 (Cascade Lake-SP, 2.30GHz, 2 CPUs/node, 16 cores/CPU) |
Memory: | 384 GiB + 1.5 TiB PMEM |
Storage: |
|
Network: |
|
yeti
4 nodes, 16 cpus, 256 cores, split as follows due to differences between nodes (json)
Reservation example:
- yeti-1 (1 node, 4 cpus, 64 cores)
Access condition: | exotic job type |
Model: | Dell PowerEdge R940 |
Date of arrival: | 2018-01-16 |
CPU: | Intel Xeon Gold 6130 (Skylake, 2.10GHz, 4 CPUs/node, 16 cores/CPU) |
Memory: | 768 GiB |
Storage: |
|
Network: |
|
- yeti-[2,
4] (2 nodes, 8 cpus, 128 cores)
Access condition: | exotic job type |
Model: | Dell PowerEdge R940 |
Date of arrival: | 2018-01-16 |
CPU: | Intel Xeon Gold 6130 (Skylake, 2.10GHz, 4 CPUs/node, 16 cores/CPU) |
Memory: | 768 GiB |
Storage: |
|
Network: |
|
- yeti-3 (1 node, 4 cpus, 64 cores)
Access condition: | exotic job type |
Model: | Dell PowerEdge R940 |
Date of arrival: | 2018-01-16 |
CPU: | Intel Xeon Gold 6130 (Skylake, 2.10GHz, 4 CPUs/node, 16 cores/CPU) |
Memory: | 768 GiB |
Storage: |
|
Network: |
|
Last generated from the Grid'5000 Reference API on 2022-06-29 (commit 362f6514fa)