Title: How to use cpan or pip packages on Nix and NixOS | |
Author: Solène | |
Date: 18 September 2021 | |
Tags: nixos nix perl python | |
Description: | |
# Introduction | |
When using Nix/NixOS and requiring some development libraries available | |
in pip (for python) or cpan (for perl) but not available as package, it | |
can be extremely complicated to get those on your system because the | |
usual way won't work. | |
# Nix-shell | |
The command nix-shell will be our friend here, we will define a new | |
environment in which we will have to create the package for the | |
libraries we need. If you really think this library is useful, it may | |
be time to contribute to nixpkgs so everyone can enjoy it :) | |
The simple way to invoke nix-shell is to use packages, for example the | |
command ` nix-shell -p python38Packages.pyyaml` will give you access to | |
the python library pyyaml for Python 3.8 as long as you run python from | |
this current shell. | |
The same way for Perl, we can start a shell with some packages | |
available for databases access, multiples packages can be passed to | |
"nix-shell -p" like this: `nix-shell -p perl532Packages.DBI | |
perl532Packages.DBDSQLite`. | |
# Defining a nix-shell | |
Reading the explanations found on a blog and help received on Mastodon, | |
I've been able to understand how to use a simple nix-shell definition | |
file to declare new cpan or pip packages. | |
Mattia Gheda's blog: Introduction to nix-shell | |
Mastodon toot from @[email protected] how to declare a python package on the … | |
What we want is to create a file that will define the state of the | |
shell, it will contain new packages needed but also the list of | |
packages. | |
# Skeleton | |
Create a file with the nix extension (or really, whatever the file name | |
you want), special file name "shell.nix" will be automatically picked | |
up when using "nix-shell" instead of passing the file name as | |
parameter. | |
```nix configuration file | |
with (import <nixpkgs> {}); | |
let | |
# we will declare new packages here | |
in | |
mkShell { | |
buildInputs = [ ]; # we will declare package list here | |
} | |
``` | |
Now we will see how to declare a python or perl library. | |
## Python | |
For python, we need to know the package name on pypi.org and its | |
version. Reusing the previous template, the code would look like this | |
for the package Crossplane | |
```nix configuration file | |
with (import <nixpkgs> {}).pkgs; | |
let | |
crossplane = python37.pkgs.buildPythonPackage rec { | |
pname = "crossplane"; | |
version = "0.5.7"; | |
src = python37.pkgs.fetchPypi { | |
inherit pname version; | |
sha256 = "a3d3ee1776bcccebf7a58cefeb365775374ab38bd544408117717ccd9f264f6… | |
}; | |
meta = { }; | |
}; | |
in | |
mkShell { | |
buildInputs = [ crossplane python37 ]; | |
} | |
``` | |
If you need another library, replace crossplane variable name but also | |
pname value by the new name, don't forget to update that name in | |
buildInputs at the end of the file. Use the correct version value too. | |
There are two references to python37 here, this implies we need python | |
3.7, adapt to the version you want. | |
The only tricky part is the sha256 value, the only way I found to find | |
it easily is the following. | |
1. declare the package with a random sha256 value (like echo hello | | |
sha256) | |
2. run nix-shell on the file, see it complaining about the wrong | |
checksum | |
3. get the url of the file, download it and run sha256 on it | |
4. update the file with the new value | |
## Perl | |
For perl, it is required to use a script available in the official git | |
repository when packages are made. We will only download the latest | |
checkout because it's quite huge. | |
In this example I will generate a package for Data::Traverse. | |
```shell instructions | |
$ git clone --depth 1 https://github.com/nixos/nixpkgs | |
$ cd nixpkgs/maintainers/scripts | |
$ nix-shell -p perlPackages.{CPANPLUS,perl,GetoptLongDescriptive,LogLog4perl,Re… | |
$ ./nix-generate-from-cpan.pl Data::Traverse | |
attribute name: DataTraverse | |
module: Data::Traverse | |
version: 0.03 | |
package: Data-Traverse-0.03.tar.gz (Data-Traverse-0.03, DataTraverse) | |
path: authors/id/F/FR/FRIEDO | |
downloaded to: /home/solene/.cpanplus/authors/id/F/FR/FRIEDO/Data-Traverse-0.03… | |
sha-256: dd992ad968bcf698acf9fd397601ef23d73c59068a6227ba5d3055fd186af16f | |
unpacked to: /home/solene/.cpanplus/5.34.0/build/EB15LXwI8e/Data-Traverse-0.03 | |
runtime deps: | |
build deps: | |
description: Unknown | |
license: unknown | |
License 'unknown' is ambiguous, please verify | |
RSS feed: https://metacpan.org/feed/distribution/Data-Traverse | |
=== | |
DataTraverse = buildPerlPackage { | |
pname = "Data-Traverse"; | |
version = "0.03"; | |
src = fetchurl { | |
url = "mirror://cpan/authors/id/F/FR/FRIEDO/Data-Traverse-0.03.tar.gz"; | |
sha256 = "dd992ad968bcf698acf9fd397601ef23d73c59068a6227ba5d3055fd186af16… | |
}; | |
meta = { | |
}; | |
}; | |
``` | |
We will only reuse the part after the ===, this is nix code that | |
defines a package named DataTraverse. | |
The shell definition will look like this: | |
```nix shell definition code | |
with (import <nixpkgs> {}); | |
let | |
DataTraverse = buildPerlPackage { | |
pname = "Data-Traverse"; | |
version = "0.03"; | |
src = fetchurl { | |
url = "mirror://cpan/authors/id/F/FR/FRIEDO/Data-Traverse-0.03.tar.gz"; | |
sha256 = "dd992ad968bcf698acf9fd397601ef23d73c59068a6227ba5d3055fd186af16… | |
}; | |
meta = { }; | |
}; | |
in | |
mkShell { | |
buildInputs = [ DataTraverse perl ]; | |
# putting perl here is only required when not using NixOS, this tell you want… | |
} | |
``` | |
Then, run "nix-shell myfile.nix" and run you perl script using | |
Data::Traverse, it should work! | |
# Conclusion | |
Using not packaged libraries is not that bad once you understand the | |
logic of declaring it properly as a new package that you keep locally | |
and then hook it to your current shell session. | |
Finding the syntax, the logic and the method when you are not a Nix | |
guru made me despair. I've been struggling a lot with this, trying to | |
install from cpan or pip (even if it wouldn't work after next update of | |
my system and I didn't even got it to work. |