Sophia:Hardware: Difference between revisions

From Grid5000
Jump to navigation Jump to search
No edit summary
No edit summary
 
(One intermediate revision by the same user not shown)
Line 4: Line 4:
'''See also:''' [[Sophia:Network|Network topology for Sophia]]
'''See also:''' [[Sophia:Network|Network topology for Sophia]]
= Summary =
= Summary =
* 4 clusters
* 5 clusters
* 19 nodes
* 22 nodes
* 496 CPU cores
* 556 CPU cores
* 25 GPUs
* 250880 GPUs cores
* 6.25 TiB RAM
* 11 SSDs and 18 HDDs on nodes (total: 59.17 TB)
* 48.0 TFLOPS (excluding GPUs)
{|width="100%" border="0"
|-
| width="50%" valign="top" |
{| width="100%"
=== Default queue resources ===
* 1 cluster
* 4 nodes
* 48 CPU cores
* 384 GiB RAM
* 4 HDDs on nodes (total: 1.0 TB)
* 0.6 TFLOPS (excluding GPUs)
|}
| width="50%" valign="top" |
{| width="100%"
=== Production queue resources ===
* 3 clusters
* 15 nodes
* 448 CPU cores
* 14 GPUs
* 14 GPUs
* 211456 GPUs cores
* 211456 GPUs cores
* 5.88 TiB RAM
* 5.5 TiB RAM
* 8 SSDs and 12 HDDs on nodes (total: 50.77 TB)
* 8 SSDs and 8 HDDs on nodes (total: 49.77 TB)
* 45.9 TFLOPS (excluding GPUs)
* 45.3 TFLOPS (excluding GPUs)
|}
|}


= Clusters summary =
= Clusters summary =
Line 38: Line 63:
|}
|}
''**: crossed GPUs are not supported by Grid'5000 default environments''
''**: crossed GPUs are not supported by Grid'5000 default environments''
== Testing queue resources ==
== Production queue resources ==
{|class="wikitable sortable"
{|class="wikitable sortable"
|-
|-
Line 58: Line 83:
|-
|-


|[[#esterel41|esterel41]]||<b>testing</b>&nbsp;queue||2025-01-25||2024-03-01||1||2||Intel Xeon Gold 6426Y||16&nbsp;cores/CPU||x86_64||data-sort-value="549755813888"|512&nbsp;GiB||data-sort-value="3127"|<b>479&nbsp;GB&nbsp;SSD</b> +&nbsp;2.88&nbsp;TB&nbsp;SSD||data-sort-value="11000"|1&nbsp;Gbps&nbsp;+&nbsp;10&nbsp;Gbps&nbsp;InfiniBand||2&nbsp;x&nbsp;Nvidia L40&nbsp;(45&nbsp;GiB)
|[[#esterel41|esterel41]]||<b>[[Grid5000:UsagePolicy#Rules_for_the_production_queue|production]]</b>&nbsp;queue||2025-01-25||2024-03-01||1||2||Intel Xeon Gold 6426Y||16&nbsp;cores/CPU||x86_64||data-sort-value="549755813888"|512&nbsp;GiB||data-sort-value="3127"|<b>479&nbsp;GB&nbsp;SSD</b> +&nbsp;2.88&nbsp;TB&nbsp;SSD||data-sort-value="57000"|1&nbsp;Gbps&nbsp;+&nbsp;56&nbsp;Gbps&nbsp;InfiniBand||2&nbsp;x&nbsp;Nvidia L40&nbsp;(45&nbsp;GiB)
|-
|-
|[[#mercantour2|mercantour2]]||<b>testing</b>&nbsp;queue||2025-01-16||2015-09-01||8||2||Intel Xeon E5-2650 v2||8&nbsp;cores/CPU||x86_64||data-sort-value="274877906944"|256&nbsp;GiB||data-sort-value="931"|<b>1.0&nbsp;TB&nbsp;HDD</b>||data-sort-value="41000"|1&nbsp;Gbps&nbsp;(SR&#8209;IOV)&nbsp;+&nbsp;40&nbsp;Gbps&nbsp;InfiniBand||
|[[#mercantour2|mercantour2]]||<b>[[Grid5000:UsagePolicy#Rules_for_the_production_queue|production]]</b>&nbsp;queue||2025-01-16||2015-09-01||8||2||Intel Xeon E5-2650 v2||8&nbsp;cores/CPU||x86_64||data-sort-value="274877906944"|256&nbsp;GiB||data-sort-value="931"|<b>1.0&nbsp;TB&nbsp;HDD</b>||data-sort-value="41000"|1&nbsp;Gbps&nbsp;(SR&#8209;IOV)&nbsp;+&nbsp;40&nbsp;Gbps&nbsp;InfiniBand||
|-
|-
|[[#musa|musa]]||<b>testing</b>&nbsp;queue||2025-01-16||2024-12-09||6||2||AMD EPYC 9254||24&nbsp;cores/CPU||x86_64||data-sort-value="549755813888"|512&nbsp;GiB||data-sort-value="5961"|<b>6.4&nbsp;TB&nbsp;SSD</b>||data-sort-value="25000"|25&nbsp;Gbps&nbsp;||2&nbsp;x&nbsp;Nvidia Tesla&nbsp;H100&nbsp;(94&nbsp;GiB)
|[[#musa|musa]]||<b>[[Grid5000:UsagePolicy#Rules_for_the_production_queue|production]]</b>&nbsp;queue||2025-01-16||2024-12-09||6||2||AMD EPYC 9254||24&nbsp;cores/CPU||x86_64||data-sort-value="549755813888"|512&nbsp;GiB||data-sort-value="5961"|<b>6.4&nbsp;TB&nbsp;SSD</b>||data-sort-value="25000"|25&nbsp;Gbps&nbsp;||2&nbsp;x&nbsp;Nvidia Tesla&nbsp;H100&nbsp;(94&nbsp;GiB)
|-
|}
''**: crossed GPUs are not supported by Grid'5000 default environments''
== Testing queue resources ==
{|class="wikitable sortable"
|-
!rowspan=2|Cluster
!rowspan=2|Access Condition
!rowspan=2|Date of arrival
!rowspan=2|Manufacturing date
!data-sort-type="number" rowspan=2|Nodes
!colspan=4|CPU
!data-sort-type="number" rowspan=2|Memory
!data-sort-type="number" rowspan=2|Storage
!data-sort-type="number" rowspan=2|Network
!rowspan=2|Accelerators
|-
!data-sort-type="number"|#
!|Name
!data-sort-type="number"|Cores
!|Architecture
|-
 
|[[#esterel10|esterel10]]||<b>testing</b>&nbsp;queue||2024-12-19||2017-11-15||3||2||Intel Xeon E5-2630 v4||10&nbsp;cores/CPU||x86_64||data-sort-value="137438953472"|128&nbsp;GiB||data-sort-value="2606"|<b>1.6&nbsp;TB&nbsp;SSD</b> +&nbsp;2&nbsp;x&nbsp;600&nbsp;GB&nbsp;HDD||data-sort-value="57000"|1&nbsp;Gbps&nbsp;+&nbsp;56&nbsp;Gbps&nbsp;InfiniBand||[1-2]: 4&nbsp;x&nbsp;Nvidia GTX&nbsp;1080&nbsp;Ti&nbsp;(11&nbsp;GiB)<br />3: 3&nbsp;x&nbsp;Nvidia GTX&nbsp;1080&nbsp;Ti&nbsp;(11&nbsp;GiB)
|-
|-
|}
|}
Line 106: Line 155:
|}
|}


= Clusters in the testing queue =
= Clusters in the [https://intranet.grid5000.fr/oar/Sophia/drawgantt-svg-prod/ production queue] =


== [https://intranet.grid5000.fr/oar/Sophia/drawgantt-svg/?filter=esterel41%20only esterel41] ==
== [https://intranet.grid5000.fr/oar/Sophia/drawgantt-svg-prod/?filter=esterel41%20only esterel41] ==


'''1 node, 2 cpus, 32 cores''' ([https://public-api.grid5000.fr/stable/sites/sophia/clusters/esterel41/nodes.json?pretty=1 json])
'''1 node, 2 cpus, 32 cores''' ([https://public-api.grid5000.fr/stable/sites/sophia/clusters/esterel41/nodes.json?pretty=1 json])
Line 114: Line 163:
'''Reservation example:'''
'''Reservation example:'''


{{Term|location=fsophia|cmd=<code class="command">oarsub</code> <code class="replace">-q testing</code> <code class="env">-p esterel41</code> <code>-I</code>}}
{{Term|location=fsophia|cmd=<code class="command">oarsub</code> <code class="replace">-q production</code> <code class="env">-p esterel41</code> <code>-I</code>}}
 
'''Max walltime per nodes:'''
* esterel41-1: 168h


{|
{|
|-
|-
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Access condition:'''
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Access condition:'''
| testing queue<br/>
| production queue<br/>
|-
|-
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Model:'''
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Model:'''
Line 147: Line 199:
* <span style="color:grey">eth2/ens15f2, Ethernet, model: Broadcom Inc. and subsidiaries NetXtreme BCM5719 Gigabit Ethernet PCIe, driver: tg3 - unavailable for experiment</span><br />
* <span style="color:grey">eth2/ens15f2, Ethernet, model: Broadcom Inc. and subsidiaries NetXtreme BCM5719 Gigabit Ethernet PCIe, driver: tg3 - unavailable for experiment</span><br />
* <span style="color:grey">eth3/ens15f3, Ethernet, model: Broadcom Inc. and subsidiaries NetXtreme BCM5719 Gigabit Ethernet PCIe, driver: tg3 - unavailable for experiment</span><br />
* <span style="color:grey">eth3/ens15f3, Ethernet, model: Broadcom Inc. and subsidiaries NetXtreme BCM5719 Gigabit Ethernet PCIe, driver: tg3 - unavailable for experiment</span><br />
* ibs3, InfiniBand, configured rate: 10&nbsp;Gbps, model: Mellanox Technologies MT28908 Family [ConnectX-6], driver: mlx5_core<br/>
* ibs3, InfiniBand, configured rate: 56&nbsp;Gbps, model: Mellanox Technologies MT28908 Family [ConnectX-6], driver: mlx5_core<br/>
|-
|-
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''GPU:'''
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''GPU:'''
Line 154: Line 206:
|}
|}


== [https://intranet.grid5000.fr/oar/Sophia/drawgantt-svg/?filter=mercantour2%20only mercantour2] ==
== [https://intranet.grid5000.fr/oar/Sophia/drawgantt-svg-prod/?filter=mercantour2%20only mercantour2] ==


'''8 nodes, 16 cpus, 128 cores,''' split as follows due to differences between nodes ([https://public-api.grid5000.fr/stable/sites/sophia/clusters/mercantour2/nodes.json?pretty=1 json])
'''8 nodes, 16 cpus, 128 cores,''' split as follows due to differences between nodes ([https://public-api.grid5000.fr/stable/sites/sophia/clusters/mercantour2/nodes.json?pretty=1 json])
Line 160: Line 212:
'''Reservation example:'''
'''Reservation example:'''


{{Term|location=fsophia|cmd=<code class="command">oarsub</code> <code class="replace">-q testing</code> <code class="env">-p mercantour2</code> <code>-I</code>}}
{{Term|location=fsophia|cmd=<code class="command">oarsub</code> <code class="replace">-q production</code> <code class="env">-p mercantour2</code> <code>-I</code>}}
 
'''Max walltime per nodes:'''
* mercantour2-[1-8]: 168h


; mercantour2-[1-6,<wbr>8] (7 nodes, 14 cpus, 112 cores)
; mercantour2-[1-6,<wbr>8] (7 nodes, 14 cpus, 112 cores)
Line 166: Line 221:
|-
|-
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Access condition:'''
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Access condition:'''
| testing queue<br/>
| production queue<br/>
|-
|-
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Model:'''
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Model:'''
Line 188: Line 243:
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Network:'''
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Network:'''
|  
|  
* eth0/enp1s0f0np0, Ethernet, configured rate: 1&nbsp;Gbps, model: Intel I350 Gigabit Network Connection, driver: igb, SR-IOV enabled - no KaVLAN<br />
* eth0/enp1s0f0np0, Ethernet, configured rate: 1&nbsp;Gbps, model: Intel I350 Gigabit Network Connection, driver: igb, SR-IOV enabled<br />
* <span style="color:grey">eth1/eno2, Ethernet, model: Intel I350 Gigabit Network Connection, driver: igb - unavailable for experiment</span><br />
* <span style="color:grey">eth1/eno2, Ethernet, model: Intel I350 Gigabit Network Connection, driver: igb - unavailable for experiment</span><br />
* ib0, InfiniBand, configured rate: 40&nbsp;Gbps, model: Mellanox Technologies MT25408A0-FCC-QI ConnectX, Dual Port 40Gb/s InfiniBand / 10GigE Adapter IC with PCIe 2.0 x8 5.0GT/s Interface, driver: mlx4_core<br />
* ib0, InfiniBand, configured rate: 40&nbsp;Gbps, model: Mellanox Technologies MT25408A0-FCC-QI ConnectX, Dual Port 40Gb/s InfiniBand / 10GigE Adapter IC with PCIe 2.0 x8 5.0GT/s Interface, driver: mlx4_core<br />
Line 200: Line 255:
|-
|-
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Access condition:'''
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Access condition:'''
| testing queue<br/>
| production queue<br/>
|-
|-
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Model:'''
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Model:'''
Line 222: Line 277:
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Network:'''
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Network:'''
|  
|  
* eth0/enp1s0f0np0, Ethernet, configured rate: 1&nbsp;Gbps, model: Intel I350 Gigabit Network Connection, driver: igb, SR-IOV enabled - no KaVLAN<br />
* eth0/enp1s0f0np0, Ethernet, configured rate: 1&nbsp;Gbps, model: Intel I350 Gigabit Network Connection, driver: igb, SR-IOV enabled<br />
* <span style="color:grey">eth1/eno2, Ethernet, model: Intel I350 Gigabit Network Connection, driver: igb - unavailable for experiment</span><br />
* <span style="color:grey">eth1/eno2, Ethernet, model: Intel I350 Gigabit Network Connection, driver: igb - unavailable for experiment</span><br />
* ib0, InfiniBand, configured rate: 40&nbsp;Gbps, model: Mellanox Technologies MT25408A0-FCC-QI ConnectX, Dual Port 40Gb/s InfiniBand / 10GigE Adapter IC with PCIe 2.0 x8 5.0GT/s Interface, driver: mlx4_core<br />
* ib0, InfiniBand, configured rate: 40&nbsp;Gbps, model: Mellanox Technologies MT25408A0-FCC-QI ConnectX, Dual Port 40Gb/s InfiniBand / 10GigE Adapter IC with PCIe 2.0 x8 5.0GT/s Interface, driver: mlx4_core<br />
Line 229: Line 284:
|}
|}


== [https://intranet.grid5000.fr/oar/Sophia/drawgantt-svg/?filter=musa%20only musa] ==
== [https://intranet.grid5000.fr/oar/Sophia/drawgantt-svg-prod/?filter=musa%20only musa] ==


'''6 nodes, 12 cpus, 288 cores''' ([https://public-api.grid5000.fr/stable/sites/sophia/clusters/musa/nodes.json?pretty=1 json])
'''6 nodes, 12 cpus, 288 cores''' ([https://public-api.grid5000.fr/stable/sites/sophia/clusters/musa/nodes.json?pretty=1 json])
Line 235: Line 290:
'''Reservation example:'''
'''Reservation example:'''


{{Term|location=fsophia|cmd=<code class="command">oarsub</code> <code class="replace">-q testing</code> <code class="env">-p musa</code> <code>-I</code>}}
{{Term|location=fsophia|cmd=<code class="command">oarsub</code> <code class="replace">-q production</code> <code class="env">-p musa</code> <code>-I</code>}}
 
'''Max walltime per nodes:'''
* musa-[1-6]: 168h


{|
{|
|-
|-
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Access condition:'''
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Access condition:'''
| testing queue<br/>
| production queue<br/>
|-
|-
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Model:'''
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Model:'''
Line 268: Line 326:
| 2&nbsp;x&nbsp;Nvidia H100&nbsp;NVL&nbsp;(94&nbsp;GiB)<br>Compute&nbsp;capability:&nbsp;9.0<br/>
| 2&nbsp;x&nbsp;Nvidia H100&nbsp;NVL&nbsp;(94&nbsp;GiB)<br>Compute&nbsp;capability:&nbsp;9.0<br/>
|-
|-
|}''<small>Last generated from the Grid'5000 Reference API on 2025-01-24 ([https://gitlab.inria.fr/grid5000/reference-repository/commit/8df9a295b7 commit 8df9a295b7])</small>''
|}
 
= Clusters in the testing queue =
 
== [https://intranet.grid5000.fr/oar/Sophia/drawgantt-svg/?filter=esterel10%20only esterel10] ==
 
'''3 nodes, 6 cpus, 60 cores,''' split as follows due to differences between nodes ([https://public-api.grid5000.fr/stable/sites/sophia/clusters/esterel10/nodes.json?pretty=1 json])
 
'''Reservation example:'''
 
{{Term|location=fsophia|cmd=<code class="command">oarsub</code> <code class="replace">-q testing</code> <code class="env">-p esterel10</code> <code>-I</code>}}
 
; esterel10-[1-2] (2 nodes, 4 cpus, 40 cores)
{|
|-
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Access condition:'''
| testing queue<br/>
|-
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Model:'''
| Dell T630<br/>
|-
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Manufacturing date:'''
| 2017-11-15<br/>
|-
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Date of arrival:'''
| 2024-12-19<br/>
|-
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''CPU:'''
| Intel Xeon E5-2630 v4 (Broadwell), x86_64, 2.20GHz, 2&nbsp;CPUs/node, 10&nbsp;cores/CPU<br/>
|-
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Memory:'''
| 128&nbsp;GiB<br/>
|-
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Storage:'''
|
* disk0, 1.6&nbsp;TB SSD SAS Toshiba THNSF81D60CSE (dev: <code class="file">/dev/disk0</code>)  (primary disk)<br />
* disk1, 600&nbsp;GB HDD SAS Toshiba AL14SEB060NY (dev: <code class="file">/dev/disk1</code>)  <br />
* disk2, 600&nbsp;GB HDD SAS Toshiba AL14SEB060NY (dev: <code class="file">/dev/disk2</code>)  <br/>
|-
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Network:'''
|
* <span style="color:grey">eth0/eno1, Ethernet, model: Intel I350 Gigabit Network Connection, driver: igb - unavailable for experiment</span><br />
* eth1/enp1s0f1, Ethernet, configured rate: 1&nbsp;Gbps, model: Intel I350 Gigabit Network Connection, driver: igb - no KaVLAN<br />
* ib0, InfiniBand, configured rate: 56&nbsp;Gbps, model: Mellanox Technologies MT27500 Family [ConnectX-3], driver: mlx4_core<br/>
|-
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''GPU:'''
| 4&nbsp;x&nbsp;Nvidia GeForce&nbsp;GTX&nbsp;1080&nbsp;Ti&nbsp;(11&nbsp;GiB)<br>Compute&nbsp;capability:&nbsp;6.1<br/>
|-
|}
<hr style="height:10pt; visibility:hidden;" />
 
; esterel10-3 (1 node, 2 cpus, 20 cores)
{|
|-
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Access condition:'''
| testing queue<br/>
|-
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Model:'''
| Dell T630<br/>
|-
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Manufacturing date:'''
| 2017-11-15<br/>
|-
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Date of arrival:'''
| 2024-12-19<br/>
|-
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''CPU:'''
| Intel Xeon E5-2630 v4 (Broadwell), x86_64, 2.20GHz, 2&nbsp;CPUs/node, 10&nbsp;cores/CPU<br/>
|-
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Memory:'''
| 128&nbsp;GiB<br/>
|-
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Storage:'''
|
* disk0, 1.6&nbsp;TB SSD SAS Toshiba THNSF81D60CSE (dev: <code class="file">/dev/disk0</code>)  (primary disk)<br />
* disk1, 600&nbsp;GB HDD SAS Toshiba AL14SEB060NY (dev: <code class="file">/dev/disk1</code>)  <br />
* disk2, 600&nbsp;GB HDD SAS Toshiba AL14SEB060NY (dev: <code class="file">/dev/disk2</code>)  <br/>
|-
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Network:'''
|
* <span style="color:grey">eth0/eno1, Ethernet, model: Intel I350 Gigabit Network Connection, driver: igb - unavailable for experiment</span><br />
* eth1/enp1s0f1, Ethernet, configured rate: 1&nbsp;Gbps, model: Intel I350 Gigabit Network Connection, driver: igb - no KaVLAN<br />
* ib0, InfiniBand, configured rate: 56&nbsp;Gbps, model: Mellanox Technologies MT27500 Family [ConnectX-3], driver: mlx4_core<br/>
|-
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''GPU:'''
| 3&nbsp;x&nbsp;Nvidia GeForce&nbsp;GTX&nbsp;1080&nbsp;Ti&nbsp;(11&nbsp;GiB)<br>Compute&nbsp;capability:&nbsp;6.1<br/>
|-
|}''<small>Last generated from the Grid'5000 Reference API on 2025-02-13 ([https://gitlab.inria.fr/grid5000/reference-repository/commit/360740a910 commit 360740a910])</small>''

Latest revision as of 09:26, 13 February 2025

Hardware: Global | Grenoble | Lille | Louvain | Luxembourg | Lyon | Nancy | Nantes | Rennes | Sophia | Strasbourg | Toulouse

See also: Network topology for Sophia

Summary

  • 5 clusters
  • 22 nodes
  • 556 CPU cores
  • 25 GPUs
  • 250880 GPUs cores
  • 6.25 TiB RAM
  • 11 SSDs and 18 HDDs on nodes (total: 59.17 TB)
  • 48.0 TFLOPS (excluding GPUs)

Default queue resources

  • 1 cluster
  • 4 nodes
  • 48 CPU cores
  • 384 GiB RAM
  • 4 HDDs on nodes (total: 1.0 TB)
  • 0.6 TFLOPS (excluding GPUs)

Production queue resources

  • 3 clusters
  • 15 nodes
  • 448 CPU cores
  • 14 GPUs
  • 211456 GPUs cores
  • 5.5 TiB RAM
  • 8 SSDs and 8 HDDs on nodes (total: 49.77 TB)
  • 45.3 TFLOPS (excluding GPUs)

Clusters summary

Default queue resources

Cluster Access Condition Date of arrival Manufacturing date Nodes CPU Memory Storage Network Accelerators
# Name Cores Architecture
uvb 2011-01-04 2011-01-04 4 2 Intel Xeon X5670 6 cores/CPU x86_64 96 GiB 250 GB HDD 1 Gbps (SR‑IOV) + 40 Gbps InfiniBand

**: crossed GPUs are not supported by Grid'5000 default environments

Production queue resources

Cluster Access Condition Date of arrival Manufacturing date Nodes CPU Memory Storage Network Accelerators
# Name Cores Architecture
esterel41 production queue 2025-01-25 2024-03-01 1 2 Intel Xeon Gold 6426Y 16 cores/CPU x86_64 512 GiB 479 GB SSD + 2.88 TB SSD 1 Gbps + 56 Gbps InfiniBand 2 x Nvidia L40 (45 GiB)
mercantour2 production queue 2025-01-16 2015-09-01 8 2 Intel Xeon E5-2650 v2 8 cores/CPU x86_64 256 GiB 1.0 TB HDD 1 Gbps (SR‑IOV) + 40 Gbps InfiniBand
musa production queue 2025-01-16 2024-12-09 6 2 AMD EPYC 9254 24 cores/CPU x86_64 512 GiB 6.4 TB SSD 25 Gbps  2 x Nvidia Tesla H100 (94 GiB)

**: crossed GPUs are not supported by Grid'5000 default environments

Testing queue resources

Cluster Access Condition Date of arrival Manufacturing date Nodes CPU Memory Storage Network Accelerators
# Name Cores Architecture
esterel10 testing queue 2024-12-19 2017-11-15 3 2 Intel Xeon E5-2630 v4 10 cores/CPU x86_64 128 GiB 1.6 TB SSD + 2 x 600 GB HDD 1 Gbps + 56 Gbps InfiniBand [1-2]: 4 x Nvidia GTX 1080 Ti (11 GiB)
3: 3 x Nvidia GTX 1080 Ti (11 GiB)

**: crossed GPUs are not supported by Grid'5000 default environments

Clusters in the default queue

uvb

4 nodes, 8 cpus, 48 cores (json)

Reservation example:

Terminal.png fsophia:
oarsub -p uvb -I
Model: Dell PowerEdge C6100
Manufacturing date: 2011-01-04
Date of arrival: 2011-01-04
CPU: Intel Xeon X5670 (Westmere), x86_64, 2.93GHz, 2 CPUs/node, 6 cores/CPU
Memory: 96 GiB
Storage: disk0, 250 GB HDD SATA Western Digital WDC WD2502ABYS-1 (dev: /dev/disk0) (primary disk)
Network:
  • eth0/eno1, Ethernet, configured rate: 1 Gbps, model: Intel 82576 Gigabit Network Connection, driver: igb, SR-IOV enabled
  • eth1/eno2, Ethernet, model: Intel 82576 Gigabit Network Connection, driver: igb - unavailable for experiment
  • ib0, InfiniBand, configured rate: 40 Gbps, model: Mellanox Technologies MT25408A0-FCC-QI ConnectX, Dual Port 40Gb/s InfiniBand / 10GigE Adapter IC with PCIe 2.0 x8 5.0GT/s Interface, driver: mlx4_core
  • ib1, InfiniBand, model: Mellanox Technologies MT25408A0-FCC-QI ConnectX, Dual Port 40Gb/s InfiniBand / 10GigE Adapter IC with PCIe 2.0 x8 5.0GT/s Interface, driver: mlx4_core - unavailable for experiment

Clusters in the production queue

esterel41

1 node, 2 cpus, 32 cores (json)

Reservation example:

Terminal.png fsophia:
oarsub -q production -p esterel41 -I

Max walltime per nodes:

  • esterel41-1: 168h
Access condition: production queue
Model: DL380 Gen11
Manufacturing date: 2024-03-01
Date of arrival: 2025-01-25
CPU: Intel Xeon Gold 6426Y (Sapphire Rapids), x86_64, 2 CPUs/node, 16 cores/CPU
Memory: 512 GiB
Storage:
  • disk0, 479 GB SSD SATA HPE MR416i-o Gen11 (dev: /dev/disk0) (primary disk)
  • disk1, 2.88 TB SSD SATA HPE MR416i-o Gen11 (dev: /dev/disk1)
Network:
  • eth0/enp1s0f0np0, Ethernet, configured rate: 1 Gbps, model: Broadcom Inc. and subsidiaries NetXtreme BCM5719 Gigabit Ethernet PCIe, driver: tg3 - no KaVLAN
  • eth1/ens15f1, Ethernet, model: Broadcom Inc. and subsidiaries NetXtreme BCM5719 Gigabit Ethernet PCIe, driver: tg3 - unavailable for experiment
  • eth2/ens15f2, Ethernet, model: Broadcom Inc. and subsidiaries NetXtreme BCM5719 Gigabit Ethernet PCIe, driver: tg3 - unavailable for experiment
  • eth3/ens15f3, Ethernet, model: Broadcom Inc. and subsidiaries NetXtreme BCM5719 Gigabit Ethernet PCIe, driver: tg3 - unavailable for experiment
  • ibs3, InfiniBand, configured rate: 56 Gbps, model: Mellanox Technologies MT28908 Family [ConnectX-6], driver: mlx5_core
GPU: 2 x Nvidia L40 (45 GiB)
Compute capability: 8.9

mercantour2

8 nodes, 16 cpus, 128 cores, split as follows due to differences between nodes (json)

Reservation example:

Terminal.png fsophia:
oarsub -q production -p mercantour2 -I

Max walltime per nodes:

  • mercantour2-[1-8]: 168h
mercantour2-[1-6,8] (7 nodes, 14 cpus, 112 cores)
Access condition: production queue
Model: Dell PowerEdge C6220 II
Manufacturing date: 2015-09-01
Date of arrival: 2025-01-16
CPU: Intel Xeon E5-2650 v2 (Ivy Bridge), x86_64, 2.60GHz, 2 CPUs/node, 8 cores/CPU
Memory: 256 GiB
Storage: disk0, 1.0 TB HDD SATA Seagate ST1000NM0033-9ZM (dev: /dev/disk0) (primary disk)
Network:
  • eth0/enp1s0f0np0, Ethernet, configured rate: 1 Gbps, model: Intel I350 Gigabit Network Connection, driver: igb, SR-IOV enabled
  • eth1/eno2, Ethernet, model: Intel I350 Gigabit Network Connection, driver: igb - unavailable for experiment
  • ib0, InfiniBand, configured rate: 40 Gbps, model: Mellanox Technologies MT25408A0-FCC-QI ConnectX, Dual Port 40Gb/s InfiniBand / 10GigE Adapter IC with PCIe 2.0 x8 5.0GT/s Interface, driver: mlx4_core
  • ib1, InfiniBand, model: Mellanox Technologies MT25408A0-FCC-QI ConnectX, Dual Port 40Gb/s InfiniBand / 10GigE Adapter IC with PCIe 2.0 x8 5.0GT/s Interface, driver: mlx4_core - unavailable for experiment
mercantour2-7 (1 node, 2 cpus, 16 cores)
Access condition: production queue
Model: Dell PowerEdge C6220 II
Manufacturing date: 2015-09-01
Date of arrival: 2025-01-16
CPU: Intel Xeon E5-2650 v2 (Ivy Bridge), x86_64, 2.60GHz, 2 CPUs/node, 8 cores/CPU
Memory: 256 GiB
Storage: disk0, 1.0 TB HDD SATA Toshiba TOSHIBA MG03ACA1 (dev: /dev/disk0) (primary disk)
Network:
  • eth0/enp1s0f0np0, Ethernet, configured rate: 1 Gbps, model: Intel I350 Gigabit Network Connection, driver: igb, SR-IOV enabled
  • eth1/eno2, Ethernet, model: Intel I350 Gigabit Network Connection, driver: igb - unavailable for experiment
  • ib0, InfiniBand, configured rate: 40 Gbps, model: Mellanox Technologies MT25408A0-FCC-QI ConnectX, Dual Port 40Gb/s InfiniBand / 10GigE Adapter IC with PCIe 2.0 x8 5.0GT/s Interface, driver: mlx4_core
  • ib1, InfiniBand, model: Mellanox Technologies MT25408A0-FCC-QI ConnectX, Dual Port 40Gb/s InfiniBand / 10GigE Adapter IC with PCIe 2.0 x8 5.0GT/s Interface, driver: mlx4_core - unavailable for experiment

musa

6 nodes, 12 cpus, 288 cores (json)

Reservation example:

Terminal.png fsophia:
oarsub -q production -p musa -I

Max walltime per nodes:

  • musa-[1-6]: 168h
Access condition: production queue
Model: ProLiant DL385 Gen11
Manufacturing date: 2024-12-09
Date of arrival: 2025-01-16
CPU: AMD EPYC 9254 (Zen 4), x86_64, 2 CPUs/node, 24 cores/CPU
Memory: 512 GiB
Storage: disk0, 6.4 TB SSD NVME Samsung MO006400KYDND (dev: /dev/disk0) (primary disk)
Network:
  • eth0/enp1s0f0np0, Ethernet, configured rate: 25 Gbps, model: Broadcom Inc. and subsidiaries BCM57414 NetXtreme-E 10Gb/25Gb RDMA Ethernet Controller, driver: bnxt_en
  • eth1/ens22f1np1, Ethernet, model: Broadcom Inc. and subsidiaries BCM57414 NetXtreme-E 10Gb/25Gb RDMA Ethernet Controller, driver: bnxt_en - unavailable for experiment
GPU: 2 x Nvidia H100 NVL (94 GiB)
Compute capability: 9.0

Clusters in the testing queue

esterel10

3 nodes, 6 cpus, 60 cores, split as follows due to differences between nodes (json)

Reservation example:

Terminal.png fsophia:
oarsub -q testing -p esterel10 -I
esterel10-[1-2] (2 nodes, 4 cpus, 40 cores)
Access condition: testing queue
Model: Dell T630
Manufacturing date: 2017-11-15
Date of arrival: 2024-12-19
CPU: Intel Xeon E5-2630 v4 (Broadwell), x86_64, 2.20GHz, 2 CPUs/node, 10 cores/CPU
Memory: 128 GiB
Storage:
  • disk0, 1.6 TB SSD SAS Toshiba THNSF81D60CSE (dev: /dev/disk0) (primary disk)
  • disk1, 600 GB HDD SAS Toshiba AL14SEB060NY (dev: /dev/disk1)
  • disk2, 600 GB HDD SAS Toshiba AL14SEB060NY (dev: /dev/disk2)
Network:
  • eth0/eno1, Ethernet, model: Intel I350 Gigabit Network Connection, driver: igb - unavailable for experiment
  • eth1/enp1s0f1, Ethernet, configured rate: 1 Gbps, model: Intel I350 Gigabit Network Connection, driver: igb - no KaVLAN
  • ib0, InfiniBand, configured rate: 56 Gbps, model: Mellanox Technologies MT27500 Family [ConnectX-3], driver: mlx4_core
GPU: 4 x Nvidia GeForce GTX 1080 Ti (11 GiB)
Compute capability: 6.1
esterel10-3 (1 node, 2 cpus, 20 cores)
Access condition: testing queue
Model: Dell T630
Manufacturing date: 2017-11-15
Date of arrival: 2024-12-19
CPU: Intel Xeon E5-2630 v4 (Broadwell), x86_64, 2.20GHz, 2 CPUs/node, 10 cores/CPU
Memory: 128 GiB
Storage:
  • disk0, 1.6 TB SSD SAS Toshiba THNSF81D60CSE (dev: /dev/disk0) (primary disk)
  • disk1, 600 GB HDD SAS Toshiba AL14SEB060NY (dev: /dev/disk1)
  • disk2, 600 GB HDD SAS Toshiba AL14SEB060NY (dev: /dev/disk2)
Network:
  • eth0/eno1, Ethernet, model: Intel I350 Gigabit Network Connection, driver: igb - unavailable for experiment
  • eth1/enp1s0f1, Ethernet, configured rate: 1 Gbps, model: Intel I350 Gigabit Network Connection, driver: igb - no KaVLAN
  • ib0, InfiniBand, configured rate: 56 Gbps, model: Mellanox Technologies MT27500 Family [ConnectX-3], driver: mlx4_core
GPU: 3 x Nvidia GeForce GTX 1080 Ti (11 GiB)
Compute capability: 6.1

Last generated from the Grid'5000 Reference API on 2025-02-13 (commit 360740a910)

Retrieved from "https://www.grid5000.fr/mediawiki/index.php?title=Sophia:Hardware&oldid=105003"