Lille:Hardware: Difference between revisions
Jump to navigation
Jump to search
No edit summary |
No edit summary |
||
(5 intermediate revisions by 4 users not shown) | |||
Line 4: | Line 4: | ||
<div class="sitelink">Hardware: [[Hardware|Global]] | [[Grenoble:Hardware|Grenoble]] | [[Lille:Hardware|Lille]] | [[Luxembourg:Hardware|Luxembourg]] | [[Lyon:Hardware|Lyon]] | [[Nancy:Hardware|Nancy]] | [[Nantes:Hardware|Nantes]] | [[Rennes:Hardware|Rennes]] | [[Sophia:Hardware|Sophia]]</div> | <div class="sitelink">Hardware: [[Hardware|Global]] | [[Grenoble:Hardware|Grenoble]] | [[Lille:Hardware|Lille]] | [[Luxembourg:Hardware|Luxembourg]] | [[Lyon:Hardware|Lyon]] | [[Nancy:Hardware|Nancy]] | [[Nantes:Hardware|Nantes]] | [[Rennes:Hardware|Rennes]] | [[Sophia:Hardware|Sophia]]</div> | ||
'''See also:''' [[Lille:Network|Network topology for Lille]] | '''See also:''' [[Lille:Network|Network topology for Lille]] | ||
= Summary = | |||
* 4 clusters | |||
* 39 nodes | |||
* 972 CPU cores | |||
* 32 GPUs | |||
* 12.25 TiB RAM | |||
* 40 SSDs and 94 HDDs on nodes (total: 283.27 TB) | |||
* 39.6 TFLOPS (excluding GPUs) | |||
= | = Clusters = | ||
{|class="wikitable sortable" | {|class="wikitable sortable" | ||
|- | |- | ||
!|Cluster | !rowspan=2|Cluster | ||
!|Access Condition | !rowspan=2|Access Condition | ||
!|Date of arrival | !rowspan=2|Date of arrival | ||
!data-sort-type="number"|Nodes | !data-sort-type="number" rowspan=2|Nodes | ||
!|CPU | !colspan=4|CPU | ||
!data-sort-type="number" rowspan=2|Memory | |||
!data-sort-type="number" rowspan=2|Storage | |||
!data-sort-type="number" rowspan=2|Network | |||
!rowspan=2|Accelerators | |||
|- | |||
!data-sort-type="number"|# | |||
!|Name | |||
!data-sort-type="number"|Cores | !data-sort-type="number"|Cores | ||
!data-sort- | !|Architecture | ||
|- | |||
|[[#chetemi|chetemi]]||||2016-12-01||15||2||Intel Xeon E5-2630 v4||10 cores/CPU||x86_64||256 GiB||data-sort-value="558"|[1-9,<wbr>11-15]: <b>300 GB HDD</b> + 300 GB HDD<br />10: <b>600 GB HDD</b> + 300 GB HDD||data-sort-value="20000"|2 x 10 Gbps (SR‑IOV) || | |||
|- | |- | ||
|[[# | |[[#chiclet|chiclet]]||||2018-08-06||8||2||AMD EPYC 7301||16 cores/CPU||x86_64||128 GiB||data-sort-value="7899"|<b>480 GB SSD</b> + 2 x 4.0 TB HDD[[Disk_reservation|*]]||data-sort-value="50000"|2 x 25 Gbps || | ||
|- | |- | ||
|[[# | |[[#chifflet|chifflet]]||||2016-12-01||8||2||Intel Xeon E5-2680 v4||14 cores/CPU||x86_64||768 GiB||data-sort-value="8196"|<b>400 GB SSD</b> + 400 GB SSD[[Disk_reservation|*]] + 2 x 4.0 TB HDD[[Disk_reservation|*]]||data-sort-value="20000"|2 x 10 Gbps (SR‑IOV) ||2 x Nvidia GTX 1080 Ti (11 GiB) | ||
|- | |- | ||
|[[# | |[[#chifflot|chifflot]]||||2018-08-01||8||2||Intel Xeon Gold 6126||12 cores/CPU||x86_64||192 GiB||data-sort-value="15798"|<b>480 GB SSD</b> + 480 GB SSD[[Disk_reservation|*]] + 4 x 4.0 TB HDD[[Disk_reservation|*]]||data-sort-value="50000"|2 x 25 Gbps ||[1-6]: 2 x Nvidia Tesla P100 (16 GiB)<br />[7-8]: 2 x Nvidia Tesla V100 (32 GiB) | ||
|- | |- | ||
|} | |} | ||
''*: disk is [[Disk_reservation|reservable]]'' | ''*: disk is [[Disk_reservation|reservable]]'' | ||
== chetemi == | ''**: crossed GPUs are not supported by Grid'5000 default environments'' | ||
= Clusters in the [https://intranet.grid5000.fr/oar/Lille/drawgantt-svg/ default queue] = | |||
== [https://intranet.grid5000.fr/oar/Lille/drawgantt-svg/?filter=chetemi%20only chetemi] == | |||
'''15 nodes, 30 cpus, 300 cores,''' split as follows due to differences between nodes ([https://public-api.grid5000.fr/stable/sites/lille/clusters/chetemi/nodes.json?pretty=1 json]) | '''15 nodes, 30 cpus, 300 cores,''' split as follows due to differences between nodes ([https://public-api.grid5000.fr/stable/sites/lille/clusters/chetemi/nodes.json?pretty=1 json]) | ||
Line 38: | Line 52: | ||
'''Reservation example:''' | '''Reservation example:''' | ||
{{Term|location=flille|cmd=<code class="command">oarsub</code> <code class="env">-p | {{Term|location=flille|cmd=<code class="command">oarsub</code> <code class="env">-p chetemi</code> <code>-I</code>}} | ||
; chetemi-[1-9,<wbr>11-15] (14 nodes, 28 cpus, 280 cores) | ; chetemi-[1-9,<wbr>11-15] (14 nodes, 28 cpus, 280 cores) | ||
Line 99: | Line 113: | ||
|} | |} | ||
== chiclet == | == [https://intranet.grid5000.fr/oar/Lille/drawgantt-svg/?filter=chiclet%20only chiclet] == | ||
'''8 nodes, 16 cpus, 256 cores''' ([https://public-api.grid5000.fr/stable/sites/lille/clusters/chiclet/nodes.json?pretty=1 json]) | '''8 nodes, 16 cpus, 256 cores''' ([https://public-api.grid5000.fr/stable/sites/lille/clusters/chiclet/nodes.json?pretty=1 json]) | ||
Line 105: | Line 119: | ||
'''Reservation example:''' | '''Reservation example:''' | ||
{{Term|location=flille|cmd=<code class="command">oarsub</code> <code class="env">-p | {{Term|location=flille|cmd=<code class="command">oarsub</code> <code class="env">-p chiclet</code> <code>-I</code>}} | ||
{| | {| | ||
Line 138: | Line 152: | ||
|} | |} | ||
== chifflet == | == [https://intranet.grid5000.fr/oar/Lille/drawgantt-svg/?filter=chifflet%20only chifflet] == | ||
'''8 nodes, 16 cpus, 224 cores''' ([https://public-api.grid5000.fr/stable/sites/lille/clusters/chifflet/nodes.json?pretty=1 json]) | '''8 nodes, 16 cpus, 224 cores''' ([https://public-api.grid5000.fr/stable/sites/lille/clusters/chifflet/nodes.json?pretty=1 json]) | ||
Line 144: | Line 158: | ||
'''Reservation example:''' | '''Reservation example:''' | ||
{{Term|location=flille|cmd=<code class="command">oarsub</code> <code class="env">-p | {{Term|location=flille|cmd=<code class="command">oarsub</code> <code class="env">-p chifflet</code> <code>-I</code>}} | ||
{| | {| | ||
Line 179: | Line 193: | ||
|} | |} | ||
== chifflot == | == [https://intranet.grid5000.fr/oar/Lille/drawgantt-svg/?filter=chifflot%20only chifflot] == | ||
'''8 nodes, 16 cpus, 192 cores,''' split as follows due to differences between nodes ([https://public-api.grid5000.fr/stable/sites/lille/clusters/chifflot/nodes.json?pretty=1 json]) | '''8 nodes, 16 cpus, 192 cores,''' split as follows due to differences between nodes ([https://public-api.grid5000.fr/stable/sites/lille/clusters/chifflot/nodes.json?pretty=1 json]) | ||
Line 185: | Line 199: | ||
'''Reservation example:''' | '''Reservation example:''' | ||
{{Term|location=flille|cmd=<code class="command">oarsub</code> <code class="env">-p | {{Term|location=flille|cmd=<code class="command">oarsub</code> <code class="env">-p chifflot</code> <code>-I</code>}} | ||
; chifflot-[1,<wbr>4- | ; chifflot-[1,<wbr>4-5] (3 nodes, 6 cpus, 72 cores) | ||
{| | {| | ||
|- | |- | ||
Line 288: | Line 302: | ||
* disk4, 4.0 TB HDD SAS Toshiba MG04SCA40ENY (dev: <code class="file">/dev/disk4</code>[[Disk_reservation|*]], by-path: <code class="file">/dev/disk/by-path/pci-0000:18:00.0-scsi-0:0:4:0</code>) [[Disk_reservation|(reservable)]] <br /> | * disk4, 4.0 TB HDD SAS Toshiba MG04SCA40ENY (dev: <code class="file">/dev/disk4</code>[[Disk_reservation|*]], by-path: <code class="file">/dev/disk/by-path/pci-0000:18:00.0-scsi-0:0:4:0</code>) [[Disk_reservation|(reservable)]] <br /> | ||
* disk5, 4.0 TB HDD SAS Toshiba MG04SCA40ENY (dev: <code class="file">/dev/disk5</code>[[Disk_reservation|*]], by-path: <code class="file">/dev/disk/by-path/pci-0000:18:00.0-scsi-0:0:5:0</code>) [[Disk_reservation|(reservable)]] <br/> | * disk5, 4.0 TB HDD SAS Toshiba MG04SCA40ENY (dev: <code class="file">/dev/disk5</code>[[Disk_reservation|*]], by-path: <code class="file">/dev/disk/by-path/pci-0000:18:00.0-scsi-0:0:5:0</code>) [[Disk_reservation|(reservable)]] <br/> | ||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Network:''' | |||
| | |||
* eth0/ens5f0, Ethernet, configured rate: 25 Gbps, model: Intel Ethernet Controller XXV710 for 25GbE SFP28, driver: i40e<br /> | |||
* eth1/ens5f1, Ethernet, configured rate: 25 Gbps, model: Intel Ethernet Controller XXV710 for 25GbE SFP28, driver: i40e [[Advanced_KaVLAN#A_simple_multi_NICs_example|(multi NICs example)]]<br /> | |||
* <span style="color:grey">eth2/eno1, Ethernet, model: Intel I350 Gigabit Network Connection, driver: igb - unavailable for experiment</span><br /> | |||
* <span style="color:grey">eth3/eno2, Ethernet, model: Intel I350 Gigabit Network Connection, driver: igb - unavailable for experiment</span><br /> | |||
* <span style="color:grey">eth4/eno3, Ethernet, model: Intel I350 Gigabit Network Connection, driver: igb - unavailable for experiment</span><br /> | |||
* <span style="color:grey">eth5/eno4, Ethernet, model: Intel I350 Gigabit Network Connection, driver: igb - unavailable for experiment</span><br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''GPU:''' | |||
| 2 x Nvidia Tesla P100-PCIE-16GB (16 GiB)<br>Compute capability: 6.0<br/> | |||
|- | |||
|} | |||
<hr style="height:10pt; visibility:hidden;" /> | |||
; chifflot-6 (1 node, 2 cpus, 24 cores) | |||
{| | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Model:''' | |||
| Dell PowerEdge R740<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Date of arrival:''' | |||
| 2018-08-01<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''CPU:''' | |||
| Intel Xeon Gold 6126 (Skylake, 2.60GHz, 2 CPUs/node, 12 cores/CPU)<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Memory:''' | |||
| 192 GiB<br/> | |||
|- | |||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Storage:''' | |||
| | |||
* disk0, 480 GB SSD SATA Intel SSDSC2KG480G7R (dev: <code class="file">/dev/disk0</code>, by-path: <code class="file">/dev/disk/by-path/pci-0000:18:00.0-scsi-0:0:0:0</code>) (primary disk)<br /> | |||
* disk1, 480 GB SSD SATA Intel SSDSC2KG480G7R (dev: <code class="file">/dev/disk1</code>[[Disk_reservation|*]], by-path: <code class="file">/dev/disk/by-path/pci-0000:18:00.0-scsi-0:0:1:0</code>) [[Disk_reservation|(reservable)]] <br /> | |||
* disk2, 4.0 TB HDD SAS Seagate ST4000NM0295 (dev: <code class="file">/dev/disk2</code>[[Disk_reservation|*]], by-path: <code class="file">/dev/disk/by-path/pci-0000:18:00.0-scsi-0:0:2:0</code>) [[Disk_reservation|(reservable)]] <br /> | |||
* disk3, 4.0 TB HDD SAS Seagate ST4000NM0295 (dev: <code class="file">/dev/disk3</code>[[Disk_reservation|*]], by-path: <code class="file">/dev/disk/by-path/pci-0000:18:00.0-scsi-0:0:3:0</code>) [[Disk_reservation|(reservable)]] <br /> | |||
* disk4, 4.0 TB HDD SAS Seagate ST4000NM0295 (dev: <code class="file">/dev/disk4</code>[[Disk_reservation|*]], by-path: <code class="file">/dev/disk/by-path/pci-0000:18:00.0-scsi-0:0:4:0</code>) [[Disk_reservation|(reservable)]] <br /> | |||
* disk5, 4.0 TB HDD SAS Toshiba MG08SDA400NY (dev: <code class="file">/dev/disk5</code>[[Disk_reservation|*]], by-path: <code class="file">/dev/disk/by-path/pci-0000:18:00.0-scsi-0:0:5:0</code>) [[Disk_reservation|(reservable)]] <br/> | |||
|- | |- | ||
| valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Network:''' | | valign="top" style="background-color: #f9f9f9; padding: 0px 10px 0px 3px;" |'''Network:''' | ||
Line 340: | Line 393: | ||
| 2 x Nvidia Tesla V100-PCIE-32GB (32 GiB)<br>Compute capability: 7.0<br/> | | 2 x Nvidia Tesla V100-PCIE-32GB (32 GiB)<br>Compute capability: 7.0<br/> | ||
|- | |- | ||
|}''<small>Last generated from the Grid'5000 Reference API on | |}''<small>Last generated from the Grid'5000 Reference API on 2022-09-28 ([https://github.com/grid5000/reference-repository/commit/c2546dd443 commit c2546dd443])</small>'' |
Revision as of 10:06, 28 September 2022
See also: Network topology for Lille
Summary
- 4 clusters
- 39 nodes
- 972 CPU cores
- 32 GPUs
- 12.25 TiB RAM
- 40 SSDs and 94 HDDs on nodes (total: 283.27 TB)
- 39.6 TFLOPS (excluding GPUs)
Clusters
Cluster | Access Condition | Date of arrival | Nodes | CPU | Memory | Storage | Network | Accelerators | |||
---|---|---|---|---|---|---|---|---|---|---|---|
# | Name | Cores | Architecture | ||||||||
chetemi | 2016-12-01 | 15 | 2 | Intel Xeon E5-2630 v4 | 10 cores/CPU | x86_64 | 256 GiB | [1-9, 10: 600 GB HDD + 300 GB HDD |
2 x 10 Gbps (SR‑IOV) | ||
chiclet | 2018-08-06 | 8 | 2 | AMD EPYC 7301 | 16 cores/CPU | x86_64 | 128 GiB | 480 GB SSD + 2 x 4.0 TB HDD* | 2 x 25 Gbps | ||
chifflet | 2016-12-01 | 8 | 2 | Intel Xeon E5-2680 v4 | 14 cores/CPU | x86_64 | 768 GiB | 400 GB SSD + 400 GB SSD* + 2 x 4.0 TB HDD* | 2 x 10 Gbps (SR‑IOV) | 2 x Nvidia GTX 1080 Ti (11 GiB) | |
chifflot | 2018-08-01 | 8 | 2 | Intel Xeon Gold 6126 | 12 cores/CPU | x86_64 | 192 GiB | 480 GB SSD + 480 GB SSD* + 4 x 4.0 TB HDD* | 2 x 25 Gbps | [1-6]: 2 x Nvidia Tesla P100 (16 GiB) [7-8]: 2 x Nvidia Tesla V100 (32 GiB) |
*: disk is reservable
**: crossed GPUs are not supported by Grid'5000 default environments
Clusters in the default queue
chetemi
15 nodes, 30 cpus, 300 cores, split as follows due to differences between nodes (json)
Reservation example:
- chetemi-[1-9,
11-15] (14 nodes, 28 cpus, 280 cores)
Model: | Dell PowerEdge R630 |
Date of arrival: | 2016-12-01 |
CPU: | Intel Xeon E5-2630 v4 (Broadwell, 2.20GHz, 2 CPUs/node, 10 cores/CPU) |
Memory: | 256 GiB |
Storage: |
|
Network: |
|
- chetemi-10 (1 node, 2 cpus, 20 cores)
Model: | Dell PowerEdge R630 |
Date of arrival: | 2016-12-01 |
CPU: | Intel Xeon E5-2630 v4 (Broadwell, 2.20GHz, 2 CPUs/node, 10 cores/CPU) |
Memory: | 256 GiB |
Storage: |
|
Network: |
|
chiclet
8 nodes, 16 cpus, 256 cores (json)
Reservation example:
Model: | Dell PowerEdge R7425 |
Date of arrival: | 2018-08-06 |
CPU: | AMD EPYC 7301 (Zen, 2 CPUs/node, 16 cores/CPU) |
Memory: | 128 GiB |
Storage: |
|
Network: |
|
chifflet
8 nodes, 16 cpus, 224 cores (json)
Reservation example:
Model: | Dell PowerEdge R730 |
Date of arrival: | 2016-12-01 |
CPU: | Intel Xeon E5-2680 v4 (Broadwell, 2.40GHz, 2 CPUs/node, 14 cores/CPU) |
Memory: | 768 GiB |
Storage: |
|
Network: |
|
GPU: | 2 x Nvidia GeForce GTX 1080 Ti (11 GiB) Compute capability: 6.1 |
chifflot
8 nodes, 16 cpus, 192 cores, split as follows due to differences between nodes (json)
Reservation example:
- chifflot-[1,
4-5] (3 nodes, 6 cpus, 72 cores)
Model: | Dell PowerEdge R740 |
Date of arrival: | 2018-08-01 |
CPU: | Intel Xeon Gold 6126 (Skylake, 2.60GHz, 2 CPUs/node, 12 cores/CPU) |
Memory: | 192 GiB |
Storage: |
|
Network: |
|
GPU: | 2 x Nvidia Tesla P100-PCIE-16GB (16 GiB) Compute capability: 6.0 |
- chifflot-2 (1 node, 2 cpus, 24 cores)
Model: | Dell PowerEdge R740 |
Date of arrival: | 2018-08-01 |
CPU: | Intel Xeon Gold 6126 (Skylake, 2.60GHz, 2 CPUs/node, 12 cores/CPU) |
Memory: | 192 GiB |
Storage: |
|
Network: |
|
GPU: | 2 x Nvidia Tesla P100-PCIE-16GB (16 GiB) Compute capability: 6.0 |
- chifflot-3 (1 node, 2 cpus, 24 cores)
Model: | Dell PowerEdge R740 |
Date of arrival: | 2018-08-01 |
CPU: | Intel Xeon Gold 6126 (Skylake, 2.60GHz, 2 CPUs/node, 12 cores/CPU) |
Memory: | 192 GiB |
Storage: |
|
Network: |
|
GPU: | 2 x Nvidia Tesla P100-PCIE-16GB (16 GiB) Compute capability: 6.0 |
- chifflot-6 (1 node, 2 cpus, 24 cores)
Model: | Dell PowerEdge R740 |
Date of arrival: | 2018-08-01 |
CPU: | Intel Xeon Gold 6126 (Skylake, 2.60GHz, 2 CPUs/node, 12 cores/CPU) |
Memory: | 192 GiB |
Storage: |
|
Network: |
|
GPU: | 2 x Nvidia Tesla P100-PCIE-16GB (16 GiB) Compute capability: 6.0 |
- chifflot-[7-8] (2 nodes, 4 cpus, 48 cores)
Model: | Dell PowerEdge R740 |
Date of arrival: | 2018-08-01 |
CPU: | Intel Xeon Gold 6126 (Skylake, 2.60GHz, 2 CPUs/node, 12 cores/CPU) |
Memory: | 192 GiB |
Storage: |
|
Network: |
|
GPU: | 2 x Nvidia Tesla V100-PCIE-32GB (32 GiB) Compute capability: 7.0 |
Last generated from the Grid'5000 Reference API on 2022-09-28 (commit c2546dd443)