Nancy:Hardware: Difference between revisions
No edit summary |
No edit summary |
||
(15 intermediate revisions by 6 users not shown) | |||
Line 1: | Line 1: | ||
__NOEDITSECTION__ | __NOEDITSECTION__ | ||
{{Portal|User}} | {{Portal|User}} | ||
<div class="sitelink">Hardware: [[Hardware|Global]] | [[Grenoble:Hardware|Grenoble]] | [[Lille:Hardware|Lille]] | [[Luxembourg:Hardware|Luxembourg]] | [[Lyon:Hardware|Lyon]] | [[Nancy:Hardware|Nancy]] | [[Nantes:Hardware|Nantes]] | [[Rennes:Hardware|Rennes]] | [[Sophia:Hardware|Sophia]] | [[Strasbourg:Hardware|Strasbourg]] | [[Toulouse:Hardware|Toulouse]]</div> | <div class="sitelink">Hardware: [[Hardware|Global]] | [[Grenoble:Hardware|Grenoble]] | [[Lille:Hardware|Lille]] | [[Louvain:Hardware|Louvain]] | [[Luxembourg:Hardware|Luxembourg]] | [[Lyon:Hardware|Lyon]] | [[Nancy:Hardware|Nancy]] | [[Nantes:Hardware|Nantes]] | [[Rennes:Hardware|Rennes]] | [[Sophia:Hardware|Sophia]] | [[Strasbourg:Hardware|Strasbourg]] | [[Toulouse:Hardware|Toulouse]]</div> | ||
'''See also:''' [[Nancy:Network|Network topology for Nancy]] | '''See also:''' [[Nancy:Network|Network topology for Nancy]] | ||
= Summary = | = Summary = | ||
* | * 14 clusters | ||
* | * 267 nodes | ||
* | * 10536 CPU cores | ||
* | * 133 GPUs | ||
* | * 800256 GPUs cores | ||
* | * 57.69 TiB RAM | ||
* | * 311 SSDs and 128 HDDs on nodes (total: 527.52 TB) | ||
* | * 768.4 TFLOPS (excluding GPUs) | ||
{|width="100%" border="0" | {|width="100%" border="0" | ||
|- | |- | ||
Line 28: | Line 28: | ||
| width="50%" valign="top" | | | width="50%" valign="top" | | ||
{| width="100%" | {| width="100%" | ||
=== | === Abaca queue resources === | ||
* | * 12 clusters | ||
* | * 141 nodes | ||
* | * 8176 CPU cores | ||
* | * 129 GPUs | ||
* | * 772608 GPUs cores | ||
* | * 45.81 TiB RAM | ||
* | * 59 SSDs and 128 HDDs on nodes (total: 343.16 TB) | ||
* | * 685.5 TFLOPS (excluding GPUs) | ||
|} | |} | ||
|} | |} | ||
Line 67: | Line 67: | ||
|} | |} | ||
''*: disk is [[Disk_reservation|reservable]]'' ''**: crossed GPUs are not supported by Grid'5000 default environments'' ''***: OPA (Omni-Path Architecture) is currently not supported on Debian 12 environment'' | ''*: disk is [[Disk_reservation|reservable]]'' ''**: crossed GPUs are not supported by Grid'5000 default environments'' ''***: OPA (Omni-Path Architecture) is currently not supported on Debian 12 environment'' | ||
== | == Abaca queue resources == | ||
{|class="wikitable sortable" | {|class="wikitable sortable" | ||
|- | |- | ||
Line 87: | Line 87: | ||
|- | |- | ||
|[[#graffiti|graffiti]]||<b>[[Grid5000:UsagePolicy#Rules_for_the_production_queue| | |[[#graffiti|graffiti]]||<b>[[Grid5000:UsagePolicy#Rules_for_the_production_queue|abaca]]</b> queue||2019-06-07||2019-05-27||13||2||Intel Xeon Silver 4110||8 cores/CPU||x86_64||data-sort-value="137438953472"|128 GiB||data-sort-value="446"|<b>479 GB HDD</b>||data-sort-value="10000"|10 Gbps ||[1-12]: 4 x Nvidia RTX 2080 Ti (11 GiB)<br />13: 4 x Nvidia Quadro RTX 6000 (23 GiB) | ||
|- | |- | ||
|[[#grappe|grappe]]||<b>[[Grid5000:UsagePolicy#Rules_for_the_production_queue| | |[[#grappe|grappe]]||<b>[[Grid5000:UsagePolicy#Rules_for_the_production_queue|abaca]]</b> queue||2020-08-20||2020-07-09||16||2||Intel Xeon Gold 5218R||20 cores/CPU||x86_64||data-sort-value="103079215104"|96 GiB||data-sort-value="7899"|<b>480 GB SSD</b> + 8.0 TB HDD||data-sort-value="25000"|25 Gbps || | ||
|- | |- | ||
|[[#grat|grat]]||<b>[[Grid5000:UsagePolicy#Rules_for_the_production_queue| | |[[#grat|grat]]||<b>[[Grid5000:UsagePolicy#Rules_for_the_production_queue|abaca]]</b> queue||2022-09-07||2022-06-22||1||2||AMD EPYC 7513||32 cores/CPU||x86_64||data-sort-value="549755813888"|512 GiB||data-sort-value="28608"|<b>3.84 TB SSD</b> + 7 x 3.84 TB SSD||data-sort-value="25000"|25 Gbps (SR‑IOV) ||8 x Nvidia A100 (40 GiB) | ||
|- | |- | ||
|[[# | |[[#gratouille|gratouille]]||<b>[[Grid5000:UsagePolicy#Rules_for_the_production_queue|abaca]]</b> queue||2017-06-26||2017-06-07||1||2||Intel Xeon E5-2650 v4||12 cores/CPU||x86_64||data-sort-value="137438953472"|128 GiB||data-sort-value="556"|<b>299 GB HDD</b> + 299 GB HDD||data-sort-value="10000"|10 Gbps (SR‑IOV) ||Nvidia Tesla V100 (16 GiB) | ||
|- | |- | ||
|[[# | |[[#grdix|grdix]]||<b>[[Grid5000:UsagePolicy#Rules_for_the_production_queue|abaca]]</b> queue||2024-09-27||2024-09-02||16||2||AMD EPYC 9754||128 cores/CPU||x86_64||data-sort-value="1099511627776"|1.0 TiB||data-sort-value="1490"|<b>1.6 TB SSD</b>||data-sort-value="225000"|25 Gbps (SR‑IOV) + 200 Gbps InfiniBand|| | ||
|- | |- | ||
|[[# | |[[#grele|grele]]||<b>[[Grid5000:UsagePolicy#Rules_for_the_production_queue|abaca]]</b> queue||2017-06-26||2017-06-07||13||2||Intel Xeon E5-2650 v4||12 cores/CPU||x86_64||data-sort-value="137438953472"|128 GiB||data-sort-value="556"|<b>299 GB HDD</b> + 299 GB HDD||data-sort-value="110000"|10 Gbps (SR‑IOV) + 100 Gbps Omni-Path||2 x Nvidia GTX 1080 Ti (11 GiB) | ||
|- | |- | ||
|[[# | |[[#gres|gres]]||<b>[[Grid5000:UsagePolicy#Rules_for_the_production_queue|abaca]]</b> queue||2024-08-23||2024-08-07||7||2||AMD EPYC 9254||24 cores/CPU||x86_64||data-sort-value="549755813888"|512 GiB||data-sort-value="5961"|<b>6.4 TB SSD</b>||data-sort-value="25000"|25 Gbps ||2 x Nvidia Tesla L40S (45 GiB) | ||
|- | |- | ||
|[[# | |[[#grosminet|grosminet]]||<b>[[Grid5000:UsagePolicy#Rules_for_the_production_queue|abaca]]</b> queue||2023-12-05||2023-11-30||1||4||Intel Xeon Gold 6240L||18 cores/CPU||x86_64||data-sort-value="6597069766656"|6.0 TiB||data-sort-value="11920"|<b>1.6 TB SSD</b> + 7 x 1.6 TB SSD||data-sort-value="25000"|25 Gbps (SR‑IOV) || | ||
|- | |- | ||
|[[#grvingt|grvingt]]||<b>[[Grid5000:UsagePolicy#Rules_for_the_production_queue| | |[[#grostiti|grostiti]]||<b>[[Grid5000:UsagePolicy#Rules_for_the_production_queue|abaca]]</b> queue||2024-01-10||2015-10-23||1||4||Intel Xeon E7-4850 v3||14 cores/CPU||x86_64||data-sort-value="1649267441664"|1.5 TiB||data-sort-value="5399"|<b>1.2 TB HDD</b> + 4.0 TB HDD + 599 GB HDD||data-sort-value="10000"|10 Gbps (SR‑IOV) || | ||
|- | |||
|[[#grue|grue]]||<b>[[Grid5000:UsagePolicy#Rules_for_the_production_queue|abaca]]</b> queue||2019-11-25||2019-11-15||5||2||AMD EPYC 7351||16 cores/CPU||x86_64||data-sort-value="137438953472"|128 GiB||data-sort-value="446"|<b>479 GB HDD</b>||data-sort-value="10000"|10 Gbps ||4 x Nvidia Tesla T4 (15 GiB) | |||
|- | |||
|[[#gruss|gruss]]||<b>[[Grid5000:UsagePolicy#Rules_for_the_production_queue|abaca]]</b> queue||2021-08-26||2021-06-24||4||2||AMD EPYC 7352||24 cores/CPU||x86_64||data-sort-value="274877906944"|256 GiB||data-sort-value="1788"|<b>1.92 TB SSD</b>||data-sort-value="25000"|25 Gbps ||2 x Nvidia A40 (45 GiB) | |||
|- | |||
|[[#grvingt|grvingt]]||<b>[[Grid5000:UsagePolicy#Rules_for_the_production_queue|abaca]]</b> queue||2018-04-11||2018-04-01||63||2||Intel Xeon Gold 6130||16 cores/CPU||x86_64||data-sort-value="206158430208"|192 GiB||data-sort-value="931"|<b>1.0 TB HDD</b>||data-sort-value="110000"|10 Gbps + 100 Gbps Omni-Path|| | |||
|- | |- | ||
|} | |} | ||
Line 116: | Line 122: | ||
'''Reservation example:''' | '''Reservation example:''' | ||
{{Term|location=fnancy|cmd=<code class="command">oarsub</code> <code class="env">-p gros</code> <code>-I</code>}} | {{Term|location=fnancy|cmd=<code class="command">oarsub</code> <code class="replace">-q default</code> <code class="env">-p gros</code> <code>-I</code>}} | ||
; gros-[1-26,<wbr>28-67,<wbr>69-124] (122 nodes, 122 cpus, 2196 cores) | ; gros-[1-26,<wbr>28-67,<wbr>69-124] (122 nodes, 122 cpus, 2196 cores) | ||
Line 216: | Line 222: | ||
'''Reservation example:''' | '''Reservation example:''' | ||
{{Term|location=fnancy|cmd=<code class="command">oarsub</code> <code class="replace">-t exotic</code> <code class="env">-p grouille</code> <code>-I</code>}} | {{Term|location=fnancy|cmd=<code class="command">oarsub</code> <code class="replace">-q default</code> <code class="replace">-t exotic</code> <code class="env">-p grouille</code> <code>-I</code>}} | ||
{| | {| | ||
Line 257: | Line 263: | ||
'''Note:''' This cluster is defined as exotic. Please read the '''[[Exotic#Nancy:_grouille|exotic]]''' page for more information.<br/> | '''Note:''' This cluster is defined as exotic. Please read the '''[[Exotic#Nancy:_grouille|exotic]]''' page for more information.<br/> | ||
= Clusters in the [https://intranet.grid5000.fr/oar/Nancy/drawgantt-svg-prod/ | = Clusters in the [https://intranet.grid5000.fr/oar/Nancy/drawgantt-svg-prod/ abaca queue] = | ||
== [https://intranet.grid5000.fr/oar/Nancy/drawgantt-svg-prod/?filter=graffiti%20only graffiti] == | == [https://intranet.grid5000.fr/oar/Nancy/drawgantt-svg-prod/?filter=graffiti%20only graffiti] == | ||
Line 265: | Line 271: | ||
'''Reservation example:''' | '''Reservation example:''' | ||
{{Term|location=fnancy|cmd=<code class="command">oarsub</code> <code class="replace">-q | {{Term|location=fnancy|cmd=<code class="command">oarsub</code> <code class="replace">-q abaca</code> <code class="env">-p graffiti</code> <code>-I</code>}} | ||
'''Max walltime per nodes:''' | '''Max walltime per nodes:''' | ||
Line 288: | Line 294: | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''CPU:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''CPU:''' | ||
| Intel Xeon Silver 4110 (Skylake), x86_64, 2.10GHz, 2 CPUs/node, 8 cores/CPU<br/> | | Intel Xeon Silver 4110 (Skylake-SP), x86_64, 2.10GHz, 2 CPUs/node, 8 cores/CPU<br/> | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Memory:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Memory:''' | ||
Line 325: | Line 331: | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''CPU:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''CPU:''' | ||
| Intel Xeon Silver 4110 (Skylake), x86_64, 2.10GHz, 2 CPUs/node, 8 cores/CPU<br/> | | Intel Xeon Silver 4110 (Skylake-SP), x86_64, 2.10GHz, 2 CPUs/node, 8 cores/CPU<br/> | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Memory:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Memory:''' | ||
Line 351: | Line 357: | ||
'''Reservation example:''' | '''Reservation example:''' | ||
{{Term|location=fnancy|cmd=<code class="command">oarsub</code> <code class="replace">-q | {{Term|location=fnancy|cmd=<code class="command">oarsub</code> <code class="replace">-q abaca</code> <code class="env">-p grappe</code> <code>-I</code>}} | ||
'''Max walltime per nodes:''' | '''Max walltime per nodes:''' | ||
Line 400: | Line 406: | ||
'''Reservation example:''' | '''Reservation example:''' | ||
{{Term|location=fnancy|cmd=<code class="command">oarsub</code> <code class="replace">-q | {{Term|location=fnancy|cmd=<code class="command">oarsub</code> <code class="replace">-q abaca</code> <code class="env">-p grat</code> <code>-I</code>}} | ||
'''Max walltime per nodes:''' | '''Max walltime per nodes:''' | ||
Line 443: | Line 449: | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''GPU:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''GPU:''' | ||
| 8 x Nvidia A100-SXM4-40GB (40 GiB)<br>Compute capability: 8.0<br/> | | 8 x Nvidia A100-SXM4-40GB (40 GiB)<br>Compute capability: 8.0<br/> | ||
|- | |||
|} | |||
== [https://intranet.grid5000.fr/oar/Nancy/drawgantt-svg-prod/?filter=gratouille%20only gratouille] == | |||
'''1 node, 2 cpus, 24 cores''' ([https://public-api.grid5000.fr/stable/sites/nancy/clusters/gratouille/nodes.json?pretty=1 json]) | |||
'''Reservation example:''' | |||
{{Term|location=fnancy|cmd=<code class="command">oarsub</code> <code class="replace">-q abaca</code> <code class="env">-p gratouille</code> <code>-I</code>}} | |||
'''Max walltime per nodes:''' | |||
* gratouille-1: 168h | |||
{| | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Access condition:''' | |||
| production queue<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Model:''' | |||
| Dell PowerEdge R730<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Manufacturing date:''' | |||
| 2017-06-07<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Date of arrival:''' | |||
| 2017-06-26<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''CPU:''' | |||
| Intel Xeon E5-2650 v4 (Broadwell), x86_64, 2.20GHz, 2 CPUs/node, 12 cores/CPU<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Memory:''' | |||
| 128 GiB<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Storage:''' | |||
| | |||
* disk0, 299 GB HDD SAS Dell PERC H730 Mini (dev: <code class="file">/dev/disk0</code>) (primary disk)<br /> | |||
* disk1, 299 GB HDD SAS Dell PERC H730 Mini (dev: <code class="file">/dev/disk1</code>) <br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Network:''' | |||
| | |||
* eth0/eno1, Ethernet, configured rate: 10 Gbps, model: Intel 82599ES 10-Gigabit SFI/SFP+ Network Connection, driver: ixgbe, SR-IOV enabled<br /> | |||
* <span style="color:grey">eth1/eno2, Ethernet, model: Intel 82599ES 10-Gigabit SFI/SFP+ Network Connection, driver: ixgbe - unavailable for experiment</span><br /> | |||
* <span style="color:grey">eth2/eno3, Ethernet, model: Intel I350 Gigabit Network Connection, driver: igb - unavailable for experiment</span><br /> | |||
* <span style="color:grey">eth3/eno4, Ethernet, model: Intel I350 Gigabit Network Connection, driver: igb - unavailable for experiment</span><br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''GPU:''' | |||
| Nvidia Tesla V100-PCIE-16GB (16 GiB)<br>Compute capability: 7.0<br/> | |||
|- | |||
|} | |||
== [https://intranet.grid5000.fr/oar/Nancy/drawgantt-svg-prod/?filter=grdix%20only grdix] == | |||
'''16 nodes, 32 cpus, 4096 cores''' ([https://public-api.grid5000.fr/stable/sites/nancy/clusters/grdix/nodes.json?pretty=1 json]) | |||
'''Reservation example:''' | |||
{{Term|location=fnancy|cmd=<code class="command">oarsub</code> <code class="replace">-q abaca</code> <code class="env">-p grdix</code> <code>-I</code>}} | |||
'''Max walltime per nodes:''' | |||
* grdix-[1-16]: 168h | |||
{| | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Access condition:''' | |||
| production queue<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Model:''' | |||
| ProLiant DL365 Gen11<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Manufacturing date:''' | |||
| 2024-09-02<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Date of arrival:''' | |||
| 2024-09-27<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''CPU:''' | |||
| AMD EPYC 9754 (Zen 4c), x86_64, 2 CPUs/node, 128 cores/CPU<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Memory:''' | |||
| 1.0 TiB<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Storage:''' | |||
| disk0, 1.6 TB SSD NVME Samsung MO001600KYDMU (dev: <code class="file">/dev/disk0</code>) (primary disk)<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Network:''' | |||
| | |||
* eth0/enp1s0f0np0, Ethernet, configured rate: 25 Gbps, model: Mellanox Technologies MT2894 Family [ConnectX-6 Lx], driver: mlx5_core, SR-IOV enabled<br /> | |||
* <span style="color:grey">eth1/ens22f1np1, Ethernet, model: Mellanox Technologies MT2894 Family [ConnectX-6 Lx], driver: mlx5_core - unavailable for experiment</span><br /> | |||
* ibs1, InfiniBand, configured rate: 200 Gbps, model: Mellanox Technologies MT28908 Family [ConnectX-6], driver: mlx5_core<br/> | |||
|- | |- | ||
|} | |} | ||
Line 448: | Line 544: | ||
== [https://intranet.grid5000.fr/oar/Nancy/drawgantt-svg-prod/?filter=grele%20only grele] == | == [https://intranet.grid5000.fr/oar/Nancy/drawgantt-svg-prod/?filter=grele%20only grele] == | ||
''' | '''13 nodes, 26 cpus, 312 cores''' ([https://public-api.grid5000.fr/stable/sites/nancy/clusters/grele/nodes.json?pretty=1 json]) | ||
'''Reservation example:''' | '''Reservation example:''' | ||
{{Term|location=fnancy|cmd=<code class="command">oarsub</code> <code class="replace">-q | {{Term|location=fnancy|cmd=<code class="command">oarsub</code> <code class="replace">-q abaca</code> <code class="env">-p grele</code> <code>-I</code>}} | ||
'''Max walltime per nodes:''' | '''Max walltime per nodes:''' | ||
* grele-[1-3]: 24h | * grele-[1-3]: 24h | ||
* grele-[4-6]: 48h | * grele-[4-6]: 48h | ||
* grele-[7- | * grele-[7-13]: 168h | ||
{| | {| | ||
Line 494: | Line 590: | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''GPU:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''GPU:''' | ||
| 2 x Nvidia GeForce GTX 1080 Ti (11 GiB)<br>Compute capability: 6.1<br/> | | 2 x Nvidia GeForce GTX 1080 Ti (11 GiB)<br>Compute capability: 6.1<br/> | ||
|- | |||
|} | |||
== [https://intranet.grid5000.fr/oar/Nancy/drawgantt-svg-prod/?filter=gres%20only gres] == | |||
'''7 nodes, 14 cpus, 336 cores''' ([https://public-api.grid5000.fr/stable/sites/nancy/clusters/gres/nodes.json?pretty=1 json]) | |||
'''Reservation example:''' | |||
{{Term|location=fnancy|cmd=<code class="command">oarsub</code> <code class="replace">-q abaca</code> <code class="env">-p gres</code> <code>-I</code>}} | |||
'''Max walltime per nodes:''' | |||
* gres-[1-7]: 168h | |||
{| | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Access condition:''' | |||
| production queue<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Model:''' | |||
| ProLiant DL385 Gen11<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Manufacturing date:''' | |||
| 2024-08-07<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Date of arrival:''' | |||
| 2024-08-23<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''CPU:''' | |||
| AMD EPYC 9254 (Zen 4), x86_64, 2 CPUs/node, 24 cores/CPU<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Memory:''' | |||
| 512 GiB<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Storage:''' | |||
| disk0, 6.4 TB SSD NVME Samsung MO006400KYDND (dev: <code class="file">/dev/disk0</code>) (primary disk)<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Network:''' | |||
| | |||
* eth0/ens22f0np0, Ethernet, configured rate: 25 Gbps, model: Broadcom Inc. and subsidiaries BCM57414 NetXtreme-E 10Gb/25Gb RDMA Ethernet Controller, driver: bnxt_en<br /> | |||
* <span style="color:grey">eth1/ens22f1np1, Ethernet, model: Broadcom Inc. and subsidiaries BCM57414 NetXtreme-E 10Gb/25Gb RDMA Ethernet Controller, driver: bnxt_en - unavailable for experiment</span><br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''GPU:''' | |||
| 2 x Nvidia L40S (45 GiB)<br>Compute capability: 8.9<br/> | |||
|- | |- | ||
|} | |} | ||
Line 503: | Line 643: | ||
'''Reservation example:''' | '''Reservation example:''' | ||
{{Term|location=fnancy|cmd=<code class="command">oarsub</code> <code class="replace">-q | {{Term|location=fnancy|cmd=<code class="command">oarsub</code> <code class="replace">-q abaca</code> <code class="env">-p grosminet</code> <code>-I</code>}} | ||
'''Max walltime per nodes:''' | '''Max walltime per nodes:''' | ||
Line 552: | Line 692: | ||
'''Reservation example:''' | '''Reservation example:''' | ||
{{Term|location=fnancy|cmd=<code class="command">oarsub</code> <code class="replace">-q | {{Term|location=fnancy|cmd=<code class="command">oarsub</code> <code class="replace">-q abaca</code> <code class="env">-p grostiti</code> <code>-I</code>}} | ||
'''Max walltime per nodes:''' | '''Max walltime per nodes:''' | ||
Line 601: | Line 741: | ||
'''Reservation example:''' | '''Reservation example:''' | ||
{{Term|location=fnancy|cmd=<code class="command">oarsub</code> <code class="replace">-q | {{Term|location=fnancy|cmd=<code class="command">oarsub</code> <code class="replace">-q abaca</code> <code class="env">-p grue</code> <code>-I</code>}} | ||
'''Max walltime per nodes:''' | '''Max walltime per nodes:''' | ||
Line 649: | Line 789: | ||
'''Reservation example:''' | '''Reservation example:''' | ||
{{Term|location=fnancy|cmd=<code class="command">oarsub</code> <code class="replace">-q | {{Term|location=fnancy|cmd=<code class="command">oarsub</code> <code class="replace">-q abaca</code> <code class="env">-p gruss</code> <code>-I</code>}} | ||
'''Max walltime per nodes:''' | '''Max walltime per nodes:''' | ||
Line 731: | Line 871: | ||
== [https://intranet.grid5000.fr/oar/Nancy/drawgantt-svg-prod/?filter=grvingt%20only grvingt] == | == [https://intranet.grid5000.fr/oar/Nancy/drawgantt-svg-prod/?filter=grvingt%20only grvingt] == | ||
''' | '''63 nodes, 126 cpus, 2016 cores''' ([https://public-api.grid5000.fr/stable/sites/nancy/clusters/grvingt/nodes.json?pretty=1 json]) | ||
'''Reservation example:''' | '''Reservation example:''' | ||
{{Term|location=fnancy|cmd=<code class="command">oarsub</code> <code class="replace">-q | {{Term|location=fnancy|cmd=<code class="command">oarsub</code> <code class="replace">-q abaca</code> <code class="env">-p grvingt</code> <code>-I</code>}} | ||
'''Max walltime per nodes:''' | '''Max walltime per nodes:''' | ||
* grvingt-[1-8]: 4h | * grvingt-[1-8]: 4h | ||
* grvingt-[9-16]: 12h | * grvingt-[9-16]: 12h | ||
* grvingt-[17-64]: 168h | * grvingt-[17-32,34-64]: 168h | ||
{| | {| | ||
Line 757: | Line 897: | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''CPU:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''CPU:''' | ||
| Intel Xeon Gold 6130 (Skylake), x86_64, 2.10GHz, 2 CPUs/node, 16 cores/CPU<br/> | | Intel Xeon Gold 6130 (Skylake-SP), x86_64, 2.10GHz, 2 CPUs/node, 16 cores/CPU<br/> | ||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Memory:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Memory:''' | ||
Line 771: | Line 911: | ||
* ib0, Omni-Path, configured rate: 100 Gbps, model: Intel Omni-Path HFI Silicon 100 Series [discrete], driver: hfi1<br/> | * ib0, Omni-Path, configured rate: 100 Gbps, model: Intel Omni-Path HFI Silicon 100 Series [discrete], driver: hfi1<br/> | ||
|- | |- | ||
|}''<small>Last generated from the Grid'5000 Reference API on | |}''<small>Last generated from the Grid'5000 Reference API on 2025-03-31 ([https://gitlab.inria.fr/grid5000/reference-repository/commit/84a18cccc2 commit 84a18cccc2])</small>'' |
Latest revision as of 15:17, 31 March 2025
See also: Network topology for Nancy
Summary
- 14 clusters
- 267 nodes
- 10536 CPU cores
- 133 GPUs
- 800256 GPUs cores
- 57.69 TiB RAM
- 311 SSDs and 128 HDDs on nodes (total: 527.52 TB)
- 768.4 TFLOPS (excluding GPUs)
Default queue resources
|
Abaca queue resources
|
Clusters summary
Default queue resources
Cluster | Access Condition | Date of arrival | Manufacturing date | Nodes | CPU | Memory | Storage | Network | Accelerators | |||
---|---|---|---|---|---|---|---|---|---|---|---|---|
# | Name | Cores | Architecture | |||||||||
gros | 2019-09-04 | 2019-07-16 | 124 | 1 | Intel Xeon Gold 5220 | 18 cores/CPU | x86_64 | 96 GiB | 480 GB SSD + 960 GB SSD* | 2 x 25 Gbps (SR‑IOV) | ||
grouille | exotic job type | 2021-01-13 | 2020-12-07 | 2 | 2 | AMD EPYC 7452 | 32 cores/CPU | x86_64 | 128 GiB | 1.92 TB SSD + 960 GB SSD* | 25 Gbps | 2 x Nvidia A100 (40 GiB) |
*: disk is reservable **: crossed GPUs are not supported by Grid'5000 default environments ***: OPA (Omni-Path Architecture) is currently not supported on Debian 12 environment
Abaca queue resources
Cluster | Access Condition | Date of arrival | Manufacturing date | Nodes | CPU | Memory | Storage | Network | Accelerators | |||
---|---|---|---|---|---|---|---|---|---|---|---|---|
# | Name | Cores | Architecture | |||||||||
graffiti | abaca queue | 2019-06-07 | 2019-05-27 | 13 | 2 | Intel Xeon Silver 4110 | 8 cores/CPU | x86_64 | 128 GiB | 479 GB HDD | 10 Gbps | [1-12]: 4 x Nvidia RTX 2080 Ti (11 GiB) 13: 4 x Nvidia Quadro RTX 6000 (23 GiB) |
grappe | abaca queue | 2020-08-20 | 2020-07-09 | 16 | 2 | Intel Xeon Gold 5218R | 20 cores/CPU | x86_64 | 96 GiB | 480 GB SSD + 8.0 TB HDD | 25 Gbps | |
grat | abaca queue | 2022-09-07 | 2022-06-22 | 1 | 2 | AMD EPYC 7513 | 32 cores/CPU | x86_64 | 512 GiB | 3.84 TB SSD + 7 x 3.84 TB SSD | 25 Gbps (SR‑IOV) | 8 x Nvidia A100 (40 GiB) |
gratouille | abaca queue | 2017-06-26 | 2017-06-07 | 1 | 2 | Intel Xeon E5-2650 v4 | 12 cores/CPU | x86_64 | 128 GiB | 299 GB HDD + 299 GB HDD | 10 Gbps (SR‑IOV) | Nvidia Tesla V100 (16 GiB) |
grdix | abaca queue | 2024-09-27 | 2024-09-02 | 16 | 2 | AMD EPYC 9754 | 128 cores/CPU | x86_64 | 1.0 TiB | 1.6 TB SSD | 25 Gbps (SR‑IOV) + 200 Gbps InfiniBand | |
grele | abaca queue | 2017-06-26 | 2017-06-07 | 13 | 2 | Intel Xeon E5-2650 v4 | 12 cores/CPU | x86_64 | 128 GiB | 299 GB HDD + 299 GB HDD | 10 Gbps (SR‑IOV) + 100 Gbps Omni-Path | 2 x Nvidia GTX 1080 Ti (11 GiB) |
gres | abaca queue | 2024-08-23 | 2024-08-07 | 7 | 2 | AMD EPYC 9254 | 24 cores/CPU | x86_64 | 512 GiB | 6.4 TB SSD | 25 Gbps | 2 x Nvidia Tesla L40S (45 GiB) |
grosminet | abaca queue | 2023-12-05 | 2023-11-30 | 1 | 4 | Intel Xeon Gold 6240L | 18 cores/CPU | x86_64 | 6.0 TiB | 1.6 TB SSD + 7 x 1.6 TB SSD | 25 Gbps (SR‑IOV) | |
grostiti | abaca queue | 2024-01-10 | 2015-10-23 | 1 | 4 | Intel Xeon E7-4850 v3 | 14 cores/CPU | x86_64 | 1.5 TiB | 1.2 TB HDD + 4.0 TB HDD + 599 GB HDD | 10 Gbps (SR‑IOV) | |
grue | abaca queue | 2019-11-25 | 2019-11-15 | 5 | 2 | AMD EPYC 7351 | 16 cores/CPU | x86_64 | 128 GiB | 479 GB HDD | 10 Gbps | 4 x Nvidia Tesla T4 (15 GiB) |
gruss | abaca queue | 2021-08-26 | 2021-06-24 | 4 | 2 | AMD EPYC 7352 | 24 cores/CPU | x86_64 | 256 GiB | 1.92 TB SSD | 25 Gbps | 2 x Nvidia A40 (45 GiB) |
grvingt | abaca queue | 2018-04-11 | 2018-04-01 | 63 | 2 | Intel Xeon Gold 6130 | 16 cores/CPU | x86_64 | 192 GiB | 1.0 TB HDD | 10 Gbps + 100 Gbps Omni-Path |
*: disk is reservable **: crossed GPUs are not supported by Grid'5000 default environments ***: OPA (Omni-Path Architecture) is currently not supported on Debian 12 environment
Clusters in the default queue
gros
124 nodes, 124 cpus, 2232 cores, split as follows due to differences between nodes (json)
Reservation example:
- gros-[1-26,
28-67, 69-124] (122 nodes, 122 cpus, 2196 cores)
Model: | Dell PowerEdge R640 |
Manufacturing date: | 2019-07-16 |
Date of arrival: | 2019-09-04 |
CPU: | Intel Xeon Gold 5220 (Cascade Lake-SP), x86_64, 2.20GHz, 1 CPU/node, 18 cores/CPU |
Memory: | 96 GiB |
Storage: |
|
Network: |
|
- gros-27 (1 node, 1 cpu, 18 cores)
Model: | Dell PowerEdge R640 |
Manufacturing date: | 2019-07-16 |
Date of arrival: | 2019-09-04 |
CPU: | Intel Xeon Gold 5220 (Cascade Lake-SP), x86_64, 2.20GHz, 1 CPU/node, 18 cores/CPU |
Memory: | 96 GiB |
Storage: |
|
Network: |
|
- gros-68 (1 node, 1 cpu, 18 cores)
Model: | Dell PowerEdge R640 |
Manufacturing date: | 2019-07-16 |
Date of arrival: | 2019-09-04 |
CPU: | Intel Xeon Gold 5220 (Cascade Lake-SP), x86_64, 2.20GHz, 1 CPU/node, 18 cores/CPU |
Memory: | 96 GiB |
Storage: |
|
Network: |
|
grouille
2 nodes, 4 cpus, 128 cores (json)
Reservation example:
Access condition: | exotic job type |
Model: | Dell PowerEdge R7525 |
Manufacturing date: | 2020-12-07 |
Date of arrival: | 2021-01-13 |
CPU: | AMD EPYC 7452 (Zen 2), x86_64, 2 CPUs/node, 32 cores/CPU |
Memory: | 128 GiB |
Storage: |
|
Network: |
|
GPU: | 2 x Nvidia A100-PCIE-40GB (40 GiB) Compute capability: 8.0 |
Note: This cluster is defined as exotic. Please read the exotic page for more information.
Clusters in the abaca queue
graffiti
13 nodes, 26 cpus, 208 cores, split as follows due to differences between nodes (json)
Reservation example:
Max walltime per nodes:
- graffiti-[1-3]: 24h
- graffiti-[4-6]: 48h
- graffiti-[7-13]: 168h
- graffiti-[1-12] (12 nodes, 24 cpus, 192 cores)
Access condition: | production queue |
Model: | Dell PowerEdge T640 |
Manufacturing date: | 2019-05-27 |
Date of arrival: | 2019-06-07 |
CPU: | Intel Xeon Silver 4110 (Skylake-SP), x86_64, 2.10GHz, 2 CPUs/node, 8 cores/CPU |
Memory: | 128 GiB |
Storage: | disk0, 479 GB HDD SATA Dell PERC H330 Adp (dev: /dev/disk0 ) (primary disk) |
Network: |
|
GPU: | 4 x Nvidia GeForce RTX 2080 Ti (11 GiB) Compute capability: 7.5 |
- graffiti-13 (1 node, 2 cpus, 16 cores)
Access condition: | production queue |
Model: | Dell PowerEdge T640 |
Manufacturing date: | 2019-05-27 |
Date of arrival: | 2019-06-07 |
CPU: | Intel Xeon Silver 4110 (Skylake-SP), x86_64, 2.10GHz, 2 CPUs/node, 8 cores/CPU |
Memory: | 128 GiB |
Storage: | disk0, 479 GB HDD SATA Dell PERC H330 Adp (dev: /dev/disk0 ) (primary disk) |
Network: |
|
GPU: | 4 x Nvidia Quadro RTX 6000 (23 GiB) Compute capability: 7.5 |
grappe
16 nodes, 32 cpus, 640 cores (json)
Reservation example:
Max walltime per nodes:
- grappe-[1-4]: 48h
- grappe-[5-8]: 96h
- grappe-[9-16]: 168h
Access condition: | production queue |
Model: | Dell PowerEdge R640 |
Manufacturing date: | 2020-07-09 |
Date of arrival: | 2020-08-20 |
CPU: | Intel Xeon Gold 5218R (Cascade Lake-SP), x86_64, 2.10GHz, 2 CPUs/node, 20 cores/CPU |
Memory: | 96 GiB |
Storage: |
|
Network: |
|
grat
1 node, 2 cpus, 64 cores (json)
Reservation example:
Max walltime per nodes:
- grat-1: 168h
Access condition: | production queue |
Model: | HPE Apollo 6500 |
Manufacturing date: | 2022-06-22 |
Date of arrival: | 2022-09-07 |
CPU: | AMD EPYC 7513 (Zen 3), x86_64, 2 CPUs/node, 32 cores/CPU |
Memory: | 512 GiB |
Storage: |
|
Network: |
|
GPU: | 8 x Nvidia A100-SXM4-40GB (40 GiB) Compute capability: 8.0 |
gratouille
1 node, 2 cpus, 24 cores (json)
Reservation example:
Max walltime per nodes:
- gratouille-1: 168h
Access condition: | production queue |
Model: | Dell PowerEdge R730 |
Manufacturing date: | 2017-06-07 |
Date of arrival: | 2017-06-26 |
CPU: | Intel Xeon E5-2650 v4 (Broadwell), x86_64, 2.20GHz, 2 CPUs/node, 12 cores/CPU |
Memory: | 128 GiB |
Storage: |
|
Network: |
|
GPU: | Nvidia Tesla V100-PCIE-16GB (16 GiB) Compute capability: 7.0 |
grdix
16 nodes, 32 cpus, 4096 cores (json)
Reservation example:
Max walltime per nodes:
- grdix-[1-16]: 168h
Access condition: | production queue |
Model: | ProLiant DL365 Gen11 |
Manufacturing date: | 2024-09-02 |
Date of arrival: | 2024-09-27 |
CPU: | AMD EPYC 9754 (Zen 4c), x86_64, 2 CPUs/node, 128 cores/CPU |
Memory: | 1.0 TiB |
Storage: | disk0, 1.6 TB SSD NVME Samsung MO001600KYDMU (dev: /dev/disk0 ) (primary disk) |
Network: |
|
grele
13 nodes, 26 cpus, 312 cores (json)
Reservation example:
Max walltime per nodes:
- grele-[1-3]: 24h
- grele-[4-6]: 48h
- grele-[7-13]: 168h
Access condition: | production queue |
Model: | Dell PowerEdge R730 |
Manufacturing date: | 2017-06-07 |
Date of arrival: | 2017-06-26 |
CPU: | Intel Xeon E5-2650 v4 (Broadwell), x86_64, 2.20GHz, 2 CPUs/node, 12 cores/CPU |
Memory: | 128 GiB |
Storage: |
|
Network: |
|
GPU: | 2 x Nvidia GeForce GTX 1080 Ti (11 GiB) Compute capability: 6.1 |
gres
7 nodes, 14 cpus, 336 cores (json)
Reservation example:
Max walltime per nodes:
- gres-[1-7]: 168h
Access condition: | production queue |
Model: | ProLiant DL385 Gen11 |
Manufacturing date: | 2024-08-07 |
Date of arrival: | 2024-08-23 |
CPU: | AMD EPYC 9254 (Zen 4), x86_64, 2 CPUs/node, 24 cores/CPU |
Memory: | 512 GiB |
Storage: | disk0, 6.4 TB SSD NVME Samsung MO006400KYDND (dev: /dev/disk0 ) (primary disk) |
Network: |
|
GPU: | 2 x Nvidia L40S (45 GiB) Compute capability: 8.9 |
grosminet
1 node, 4 cpus, 72 cores (json)
Reservation example:
Max walltime per nodes:
- grosminet-1: 24h
Access condition: | production queue |
Model: | Proliant DL560 Gen10 |
Manufacturing date: | 2023-11-30 |
Date of arrival: | 2023-12-05 |
CPU: | Intel Xeon Gold 6240L (Cascade Lake-SP), x86_64, 2.60GHz, 4 CPUs/node, 18 cores/CPU |
Memory: | 6.0 TiB |
Storage: |
|
Network: |
|
grostiti
1 node, 4 cpus, 56 cores (json)
Reservation example:
Max walltime per nodes:
- grostiti-1: 168h
Access condition: | production queue |
Model: | Dell PowerEdge R930 |
Manufacturing date: | 2015-10-23 |
Date of arrival: | 2024-01-10 |
CPU: | Intel Xeon E7-4850 v3 (Haswell), x86_64, 2.20GHz, 4 CPUs/node, 14 cores/CPU |
Memory: | 1.5 TiB |
Storage: |
|
Network: |
|
grue
5 nodes, 10 cpus, 160 cores (json)
Reservation example:
Max walltime per nodes:
- grue-[1-2]: 24h
- grue-[3-4]: 48h
- grue-5: 168h
Access condition: | production queue |
Model: | Dell PowerEdge R7425 |
Manufacturing date: | 2019-11-15 |
Date of arrival: | 2019-11-25 |
CPU: | AMD EPYC 7351 (Zen), x86_64, 2 CPUs/node, 16 cores/CPU |
Memory: | 128 GiB |
Storage: | disk0, 479 GB HDD SAS Dell PERC H730P Adp (dev: /dev/disk0 ) (primary disk) |
Network: |
|
GPU: | 4 x Nvidia Tesla T4 (15 GiB) Compute capability: 7.5 |
gruss
4 nodes, 8 cpus, 192 cores, split as follows due to differences between nodes (json)
Reservation example:
Max walltime per nodes:
- gruss-[1-2]: 24h
- gruss-3: 48h
- gruss-4: 168h
- gruss-1 (1 node, 2 cpus, 48 cores)
Access condition: | production queue |
Model: | Dell PowerEdge R7525 |
Manufacturing date: | 2021-06-24 |
Date of arrival: | 2021-08-26 |
CPU: | AMD EPYC 7352 (Zen 2), x86_64, 2 CPUs/node, 24 cores/CPU |
Memory: | 256 GiB |
Storage: | disk0, 1.92 TB SSD SATA Samsung MZ7KH1T9HAJR0D3 (dev: /dev/disk0 ) (primary disk) |
Network: |
|
GPU: | 2 x Nvidia A40 (45 GiB) Compute capability: 8.6 |
- gruss-[2-4] (3 nodes, 6 cpus, 144 cores)
Access condition: | production queue |
Model: | Dell PowerEdge R7525 |
Manufacturing date: | 2021-06-24 |
Date of arrival: | 2021-08-26 |
CPU: | AMD EPYC 7352 (Zen 2), x86_64, 2 CPUs/node, 24 cores/CPU |
Memory: | 256 GiB |
Storage: | disk0, 1.92 TB SSD SATA Sk Hynix HFS1T9G32FEH-BA1 (dev: /dev/disk0 ) (primary disk) |
Network: |
|
GPU: | 2 x Nvidia A40 (45 GiB) Compute capability: 8.6 |
grvingt
63 nodes, 126 cpus, 2016 cores (json)
Reservation example:
Max walltime per nodes:
- grvingt-[1-8]: 4h
- grvingt-[9-16]: 12h
- grvingt-[17-32,34-64]: 168h
Access condition: | production queue |
Model: | Dell PowerEdge C6420 |
Manufacturing date: | 2018-04-01 |
Date of arrival: | 2018-04-11 |
CPU: | Intel Xeon Gold 6130 (Skylake-SP), x86_64, 2.10GHz, 2 CPUs/node, 16 cores/CPU |
Memory: | 192 GiB |
Storage: | disk0, 1.0 TB HDD SATA Seagate ST1000NX0443 (dev: /dev/disk0 ) (primary disk) |
Network: |
|
Last generated from the Grid'5000 Reference API on 2025-03-31 (commit 84a18cccc2)