Guix: Difference between revisions

From Grid5000
Jump to navigation Jump to search
 
(5 intermediate revisions by 3 users not shown)
Line 15: Line 15:
'''Note''': For now, Guix is only available on Grid'5000 nodes with x86_64 architecture (not on POWER or ARM nodes).
'''Note''': For now, Guix is only available on Grid'5000 nodes with x86_64 architecture (not on POWER or ARM nodes).


{{Warning|text=Too use GUIX on Grid'5000, we strongly recommend not to use the <code>guix install</code> command, which can cause side effects since installed software will be directly available by default (in your $PATH), causing conflicts with system software and environment modules. We recommend using <code>guix shell</code> to set up isolated development environments (https://guix.gnu.org/manual/en/html_node/Invoking-guix-shell.html).}}
{{Warning|text=To use GUIX on Grid'5000, we strongly recommend not to use the <code>guix install</code> command, which can cause side effects since installed software will be directly available by default (in your $PATH), causing conflicts with system software and environment modules. We recommend using <code>guix shell</code> to set up isolated development environments (https://guix.gnu.org/manual/en/html_node/Invoking-guix-shell.html).}}
 
{{Warning|text=Guix doesn't provide support for C-Shell like tcsh. We provides some environment files for tcsh but you will have to execute specific commands to use guix in a C-shell context. These commands are listed in the documentation below..}}


= Quick start =
= Quick start =
Guix is available on frontends and nodes (using standard or deployed ''-nfs'' and ''-big'' environments) directly through the <code class="command">guix</code> command.
Guix is available on frontends and nodes (using standard or deployed ''-nfs'' and ''-big'' environments) directly through the <code class="command">guix</code> command.
== Initial setup ==
You should tell <code class="command">guix</code> to generate a default profile for you by running :
{{Term|location=fontend|cmd=<code class="command">guix pull</code>}}
The first execution can be quite long and the result of the command will be a profile stored in <code class="command">$HOME/.config/guix/current</code>.
Conveniently, Grid5000 defines the <code class="command">GUIX_PROFILE</code> to point to that location when connecting to the frontend.
{{Note|text=For C-Shell users (ie: tcsh) ignore the final message. Do the following instead:
<code> cat ~/.config/guix/current/etc/profile</code><br />
[...]<br />
Copy-Paste the hexadecimal string beetween "/gnu/store/" and "-profile" (something like "z2lyid73zcm06bbjkvs32cj3ankiyg3f") and run the following command:<br />
<code>set path=($GUIX_PROFILE/bin /gnu/store/"xxxxxxxxxxxxxxxxxxx"-profile/bin $path) </code>
}}
When pulling is over, simply log out/in again and then make sure everything looks good by checking both <code class="command">guix</code> and <code class="command">guix-hpc</code> are available in channels:
{{Term|location=fontend|cmd=<code class="command">guix describe</code>}}
  Generation 1 Aug 01 2023 16:50:17 (current)
    guix 5a293d0
      repository URL: https://git.savannah.gnu.org/git/guix.git
      branch: master
      commit: 5a293d0830aa9369e388d37fe767d5bf98af01b7
    guix-hpc 2e4749d
      repository URL: https://gitlab.inria.fr/guix-hpc/guix-hpc.git
      branch: master
      commit: 2e4749d63e4d2ad2317f37609052689b7ca49db4
Actual sha-s may vary but both channels should be there.
If you need to activate another channels (such as <code class="command">guix-hpc-non-free</code>), please take a look at [[#Adding additional channels]] section.


== guix shell (the recommended way)==
== guix shell (the recommended way)==
Line 43: Line 75:
{{Term|location=fontend|cmd=<code class="command"> guix</code> install <code class="replace">hello</code>}}  
{{Term|location=fontend|cmd=<code class="command"> guix</code> install <code class="replace">hello</code>}}  


{{Warning|text=Lucas> tu es sûr ? il me semble que le probleme avec guix install est justement que si on installe un truc, ça sera mis dans $PATH automatiquement par /etc/profile.d/guix}}
If it's the first time you're using GUIX, programs installed must be added to your PATH. You can do this by simply running :
 
To make the ''hello'' executable available, programs installed by Guix must be added to your PATH. You can do this by simply running:
{{Term|location=fontend|cmd=<code class="command"> export</code> <code class="replace">GUIX_PROFILE=~/.guix-profile</code>}}
{{Term|location=fontend|cmd=<code class="command"> export</code> <code class="replace">GUIX_PROFILE=~/.guix-profile</code>}}
{{Term|location=fontend|cmd=<code class="command"> .</code> <code class="replace">"$GUIX_PROFILE/etc/profile"</code>}}
{{Term|location=fontend|cmd=<code class="command"> .</code> <code class="replace">"$GUIX_PROFILE/etc/profile"</code>}}


(you can add these two commands to your '''.bashrc''' to make it persistent)
You can also simply disconnect and reconnect.


You can then use what you just installed:
You can then use what you just installed:
Line 131: Line 161:
   GUIX_DAEMON_SOCKET=guix://guix.sophia.grid5000.fr
   GUIX_DAEMON_SOCKET=guix://guix.sophia.grid5000.fr
   GUIX_LOCPATH=/var/guix/profiles/per-user/root/guix-profile/lib/locale
   GUIX_LOCPATH=/var/guix/profiles/per-user/root/guix-profile/lib/locale
   GUIX_PROFILE=/home/ddelabroye/.guix-profile
   GUIX_PROFILE=/home/ddelabroye/.config/guix/current


If those variables are not set you will have to manually source <code>/etc/profile.d/guix.sh</code>:
If those variables are not set you will have to manually source <code>/etc/profile.d/guix.sh</code>:
Line 142: Line 172:
* If not found, it will try to find it on all Grid'5000 Guix servers. (Each Grid'5000 site hosts a Guix server communicating with each other. If you install a package on Nancy and then decide to install the same package in Grenoble, it will be fetched from Nancy).
* If not found, it will try to find it on all Grid'5000 Guix servers. (Each Grid'5000 site hosts a Guix server communicating with each other. If you install a package on Nancy and then decide to install the same package in Grenoble, it will be fetched from Nancy).
* If it still not found, it will be compiled on a dedicated machine on Grid'5000, which is not accessible to the users
* If it still not found, it will be compiled on a dedicated machine on Grid'5000, which is not accessible to the users
== Adding additional channels ==
Additionally to the main channel <code class="command">guix</code>, the channel <code class="command">guix-hpc</code> is available by default.
If you would like to enable any additional channel, you just have to tell <code class="command">guix</code> about it; let's take the example of the [[https://gitlab.inria.fr/guix-hpc/guix-hpc-non-free guix-hpc-non-free]] channel, you can activate it by creating the <code class="command">~/.config/guix/channels.scm</code> with the following content:
(cons (channel
        (name 'guix-hpc-non-free)
        (url "https://gitlab.inria.fr/guix-hpc/guix-hpc-non-free.git"))
      %default-channels)
You can then run <code class="command">guix pull</code>, and using <code class="command">guix describe</code> should now show the channel (actual output may vary):
Generation 2 Aug 01 2023 17:50:46 (current)
  guix 5a293d0
    repository URL: https://git.savannah.gnu.org/git/guix.git
    branch: master
    commit: 5a293d0830aa9369e388d37fe767d5bf98af01b7
  guix-hpc-non-free 76fe75f
    repository URL: https://gitlab.inria.fr/guix-hpc/guix-hpc-non-free.git
    branch: master
    commit: 76fe75ffcadc171054336b62aed7b2d491ba4c61
  guix-science-nonfree ae46295
    repository URL: https://github.com/guix-science/guix-science-nonfree.git
    branch: master
    commit: ae46295bc81f35bdc7d5afe3ab0019af58bd509f
  guix-hpc 2e4749d
    repository URL: https://gitlab.inria.fr/guix-hpc/guix-hpc.git
    branch: master
    commit: 2e4749d63e4d2ad2317f37609052689b7ca49db4

Latest revision as of 16:58, 3 October 2023

Note.png Note

This page is actively maintained by the Grid'5000 team. If you encounter problems, please report them (see the Support page). Additionally, as it is a wiki page, you are free to make minor corrections yourself if needed. If you would like to suggest a more fundamental change, please contact the Grid'5000 team.

The Guix package manager is available on Grid'5000 to provide an alternative way of installing software on nodes.

This page describes the use of Guix on Grid'5000. For additional information about Guix, see:


Note: For now, Guix is only available on Grid'5000 nodes with x86_64 architecture (not on POWER or ARM nodes).

Warning.png Warning

To use GUIX on Grid'5000, we strongly recommend not to use the guix install command, which can cause side effects since installed software will be directly available by default (in your $PATH), causing conflicts with system software and environment modules. We recommend using guix shell to set up isolated development environments (https://guix.gnu.org/manual/en/html_node/Invoking-guix-shell.html).

Warning.png Warning

Guix doesn't provide support for C-Shell like tcsh. We provides some environment files for tcsh but you will have to execute specific commands to use guix in a C-shell context. These commands are listed in the documentation below..

Quick start

Guix is available on frontends and nodes (using standard or deployed -nfs and -big environments) directly through the guix command.

Initial setup

You should tell guix to generate a default profile for you by running :

Terminal.png fontend:
guix pull

The first execution can be quite long and the result of the command will be a profile stored in $HOME/.config/guix/current. Conveniently, Grid5000 defines the GUIX_PROFILE to point to that location when connecting to the frontend.

Note.png Note

For C-Shell users (ie: tcsh) ignore the final message. Do the following instead:

cat ~/.config/guix/current/etc/profile
[...]
Copy-Paste the hexadecimal string beetween "/gnu/store/" and "-profile" (something like "z2lyid73zcm06bbjkvs32cj3ankiyg3f") and run the following command:

set path=($GUIX_PROFILE/bin /gnu/store/"xxxxxxxxxxxxxxxxxxx"-profile/bin $path)

When pulling is over, simply log out/in again and then make sure everything looks good by checking both guix and guix-hpc are available in channels:

Terminal.png fontend:
guix describe
 Generation 1	Aug 01 2023 16:50:17	(current)
   guix 5a293d0
     repository URL: https://git.savannah.gnu.org/git/guix.git
     branch: master
     commit: 5a293d0830aa9369e388d37fe767d5bf98af01b7
   guix-hpc 2e4749d
     repository URL: https://gitlab.inria.fr/guix-hpc/guix-hpc.git
     branch: master
     commit: 2e4749d63e4d2ad2317f37609052689b7ca49db4

Actual sha-s may vary but both channels should be there. If you need to activate another channels (such as guix-hpc-non-free), please take a look at #Adding additional channels section.

guix shell (the recommended way)

You can find all you need here : https://guix.gnu.org/manual/en/html_node/Invoking-guix-shell.html

For exemple, to use openmpi, you can do :

Terminal.png fontend:
guix shell openmpi -- mpirun --version

It is also possible to create manifest with command below. For example with openmpi, automake and bash :

Terminal.png fontend:
guix shell --export-manifest openmpi automake bash > manifest.scm

And we can after use this manifest to load our environment :

Terminal.png fontend:
guix shell -m manifest.scm -- which mpirun automake

It's also possible to open a shell within this environment if desired :

Terminal.png fontend:
guix shell -m manifest.scm -- bash

guix install (not recommended)

guix install command is not recommended on Grid'5000 because it causes side effects since installed software will be directly available by default (in your $PATH), causing conflicts with system software and environment modules.

For example to install hello command :

Terminal.png fontend:
guix install hello

If it's the first time you're using GUIX, programs installed must be added to your PATH. You can do this by simply running :

Terminal.png fontend:
export GUIX_PROFILE=~/.guix-profile
Terminal.png fontend:
. "$GUIX_PROFILE/etc/profile"

You can also simply disconnect and reconnect.

You can then use what you just installed:

Terminal.png fontend:
hello
 > Hello, world!

To update available packages from Guix channels, use:

Terminal.png fontend:
guix pull

(be patient it will take a while the first time)

To list installed packages :

Terminal.png fontend:
guix package -I

To remove package installed with guix install commmand :

Terminal.png fontend:
guix remove hello

Advanced usage examples

Generating image to use with singularity

You can use Guix to create container image for singularity with guix pack. For instance, to build an image containing emacs and bash, use:

Terminal.png $:
guix pack -f squashfs bash emacs gcc
 ....
 /gnu/store/7pzhxrkjrfk3sh9i4dn2k61gvi1slr7g-squashfs-pack.gz.squashfs

The built image can be run using singularity. For instance, from a node:

Terminal.png node:
singularity run /gnu/store/7pzhxrkjrfk3sh9i4dn2k61gvi1slr7g-squashfs-pack.gz.squashfs
  WARNING: passwd file doesn't exist in container, not updating
  WARNING: group file doesn't exist in container, not updating
  /bin/sh: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
  No runscript found in container, executing /bin/sh
  sh: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
 Singularity> gcc -v
 Using built-in specs.
 COLLECT_GCC=gcc
 COLLECT_LTO_WRAPPER=/gnu/store/xpdy9vvqym9xv7praqkwsm3gdzn2kv1p-gcc- 
 10.2.0/libexec/gcc/x86_64-unknown-linux-gnu/10.2.0/lto-wrapper
 Target: x86_64-unknown-linux-gnu
 Configured with:
 Thread model: posix
 Supported LTO compression algorithms: zlib
 gcc version 10.2.0 (GCC)

Note: If you forget the bash (or similar) package, singularity run and singularity exec will fail with an unhelpful “no such file or directory” message.

For more information of singularity usage under Grid'5000, see Singularity page.

Container

Guix shell command, with --container option, can be used to easily isolate the execution of your program. For instance, this will instanciate an isolated container containing coreutils, gcc and procps

Terminal.png node:
guix shell --container coreutils gcc procps
Terminal.png node~[env]:
gcc -v
 Using built-in specs.
 COLLECT_GCC=gcc
 COLLECT_LTO_WRAPPER=/gnu/store/j3zyz07nivflink7ix0cqasl1aa52vd8-gcc-12.2.0/libexec/gcc/x86_64-unknown-linux-gnu/12.2.0/lto-wrapper
 Target: x86_64-unknown-linux-gnu
 Configured with: 
 Thread model: posix
 Supported LTO compression algorithms: zlib
 gcc version 12.2.0 (GCC)

Configuration details

Here are some details of Guix deployment and configuration under Grid'5000 that may help in case of problem or for your customized deployment.

Guix profile

Note: To work properly, a .guix directory is created in your home directory (it's a Grid'5000-specific directory). If you remove it, Guix will stop working since your profile is stored there.

Warning.png Warning

If you have a guix profile because you have used guix before you should move it somewhere else or it won't work with the guix we provide

Environment variables

In order to use Guix on Grid'5000, some environment variables are correctly set:

Terminal.png $:
env | grep GUIX
 GUIX_DAEMON_SOCKET=guix://guix.sophia.grid5000.fr
 GUIX_LOCPATH=/var/guix/profiles/per-user/root/guix-profile/lib/locale
 GUIX_PROFILE=/home/ddelabroye/.config/guix/current

If those variables are not set you will have to manually source /etc/profile.d/guix.sh:

Terminal.png node:
. /etc/profile.d/guix.sh

Package installation steps

When you try to install or compile a package, the following steps happen:

  • Guix first tries to fetch the compiled package from ci.guix.gnu.org
  • If not found, it will try to find it on all Grid'5000 Guix servers. (Each Grid'5000 site hosts a Guix server communicating with each other. If you install a package on Nancy and then decide to install the same package in Grenoble, it will be fetched from Nancy).
  • If it still not found, it will be compiled on a dedicated machine on Grid'5000, which is not accessible to the users

Adding additional channels

Additionally to the main channel guix, the channel guix-hpc is available by default. If you would like to enable any additional channel, you just have to tell guix about it; let's take the example of the [guix-hpc-non-free] channel, you can activate it by creating the ~/.config/guix/channels.scm with the following content:

(cons (channel
        (name 'guix-hpc-non-free)
        (url "https://gitlab.inria.fr/guix-hpc/guix-hpc-non-free.git"))
      %default-channels)

You can then run guix pull, and using guix describe should now show the channel (actual output may vary):

Generation 2	Aug 01 2023 17:50:46	(current)
 guix 5a293d0
   repository URL: https://git.savannah.gnu.org/git/guix.git
   branch: master
   commit: 5a293d0830aa9369e388d37fe767d5bf98af01b7
 guix-hpc-non-free 76fe75f
   repository URL: https://gitlab.inria.fr/guix-hpc/guix-hpc-non-free.git
   branch: master
   commit: 76fe75ffcadc171054336b62aed7b2d491ba4c61
 guix-science-nonfree ae46295
   repository URL: https://github.com/guix-science/guix-science-nonfree.git
   branch: master
   commit: ae46295bc81f35bdc7d5afe3ab0019af58bd509f
 guix-hpc 2e4749d
   repository URL: https://gitlab.inria.fr/guix-hpc/guix-hpc.git
   branch: master
   commit: 2e4749d63e4d2ad2317f37609052689b7ca49db4