How to Package Your Software in Nix or Write Your Own Package Derivation for Nixpkgs

Welcome to a comprehensive guide on packaging your software using Nix, and subsequently, contributing it to the expansive Nixpkgs collection. This guide is tailored for developers, sysadmins, and anyone seeking to leverage the power of Nix for reproducible and declarative package management. This is a revWhiteShadow production for kts’ personal blog site.

Understanding Nix Derivations: The Building Blocks of Packages

At the heart of Nix lies the concept of a derivation. A derivation is essentially a recipe written in the Nix expression language, outlining precisely how to build a software package. It specifies the source code, dependencies, build commands, and other crucial details needed to produce a reproducible and hermetic build. In simpler terms, it’s a function that tells Nix how to turn source code into a usable package.

Crafting a Basic Nix Derivation: A Step-by-Step Approach

Let’s embark on creating a simple derivation for a hypothetical “hello world” program written in C. We’ll assume the program consists of a single file, hello.c, with the following content:

#include <stdio.h>

int main() {
    printf("Hello, world!\n");
    return 0;
}

And a simple Makefile to build it:

hello: hello.c
	gcc -o hello hello.c

Here’s how we can create a Nix derivation to package this program:

Create a default.nix File: This file will contain our Nix expression, the derivation that describes how to build the package. Place it in the same directory as your hello.c and Makefile.
Define the Derivation: Open default.nix and add the following code:

{ pkgs ? import <nixpkgs> {} }:

pkgs.stdenv.mkDerivation {
  name = "hello";
  version = "1.0";
  src = ./.; # Current directory containing source code

  buildPhase = ''
    make
  '';

  installPhase = ''
    mkdir -p $out/bin
    cp hello $out/bin
  '';

  meta = with pkgs.lib; {
    description = "A simple hello world program";
    homepage = "https://revwhiteshadow.gitlab.io";
    license = licenses.mit; # Replace with the appropriate license
    maintainers = [ maintainers.revWhiteShadow ]; # Replace with your Nixpkgs username
  };
}

Let’s break down this derivation:

{ pkgs ? import <nixpkgs> {} }:: This imports the Nixpkgs collection, providing access to pre-built packages and utility functions. pkgs is a common convention for the Nixpkgs set.
pkgs.stdenv.mkDerivation { ... }:: This utilizes the mkDerivation function from stdenv (standard environment), a crucial part of Nixpkgs. mkDerivation is the workhorse for creating most packages.
name = "hello";: Sets the name of the package.
version = "1.0";: Specifies the version of the package.
src = ./.;: Indicates the source code location. Here, ./. means the current directory. For larger projects, you’d typically specify a tarball or Git repository.
buildPhase = '' make '';: Defines the build phase. In this simple case, we just run make. This is a shell script that Nix executes.
installPhase = '' mkdir -p $out/bin; cp hello $out/bin; '';: Defines the installation phase. $out is a special variable representing the output directory where the package will be installed. We create a bin directory within $out and copy the compiled hello executable into it.
meta = { ... };: Contains metadata about the package, such as its description, homepage, license, and maintainers. This information is valuable for users searching and understanding the package. The licenses and maintainers attributes come from Nixpkgs. Ensure you choose the correct license and add your Nixpkgs username (if contributing).

Building the Package: In the directory containing your default.nix, hello.c, and Makefile, run the following command in your terminal:

nix-build

This command will build the package and create a result symlink pointing to the output directory. Inside the result directory, you’ll find the bin directory, which contains your hello executable.

Running the Executable: You can execute the program by navigating to the result/bin directory and running ./hello.

Enhancing Your Derivation: Handling Dependencies

Real-world software often relies on external libraries and dependencies. Nix provides robust mechanisms to manage these dependencies. Let’s say our “hello world” program now uses the libunistring library. First, install the development packages for libunistring on your build system, if you haven’t done so already. Now, we need to modify our hello.c:

#include <stdio.h>
#include <unistring.h>

int main() {
    printf("Hello, world! Unicode version: %s\n", u8_strup("hello world",0));
    return 0;
}

Modify the Makefile:

hello: hello.c
	gcc -o hello hello.c -lunistring

We need to declare libunistring as a build input in our default.nix file:

{ pkgs ? import <nixpkgs> {} }:

pkgs.stdenv.mkDerivation {
  name = "hello";
  version = "1.0";
  src = ./.;

  buildInputs = [ pkgs.libunistring ]; # Declare libunistring as a build input

  buildPhase = ''
    make
  '';

  installPhase = ''
    mkdir -p $out/bin
    cp hello $out/bin
  '';

  meta = with pkgs.lib; {
    description = "A simple hello world program with libunistring dependency";
    homepage = "https://revwhiteshadow.gitlab.io";
    license = licenses.mit;
    maintainers = [ maintainers.revWhiteShadow ];
  };
}

buildInputs = [ pkgs.libunistring ];: This line is crucial. It tells Nix that the build process depends on the libunistring package from Nixpkgs. Nix will ensure that libunistring (and its dependencies) are available during the build.

When you run nix-build again, Nix will automatically fetch and make libunistring available during the compilation process.

Leveraging Fetchers: Handling External Source Code

In many cases, your software’s source code might reside in a remote location, such as a Git repository or a tarball. Nix provides fetcher functions to simplify downloading and managing external source code.

Let’s assume our “hello world” program is hosted on GitHub at https://github.com/example/hello-world.git. We can use the fetchFromGitHub function to retrieve the source code:

{ pkgs ? import <nixpkgs> {} }:

pkgs.stdenv.mkDerivation {
  name = "hello";
  version = "1.0";

  src = pkgs.fetchFromGitHub {
    owner = "example";
    repo = "hello-world";
    rev = "main"; # Replace with a specific commit hash or branch
    sha256 = "replace-with-valid-sha256"; # Replace with the correct SHA256 hash
  };

  buildPhase = ''
    make
  '';

  installPhase = ''
    mkdir -p $out/bin
    cp hello $out/bin
  '';

  meta = with pkgs.lib; {
    description = "A simple hello world program from GitHub";
    homepage = "https://github.com/example/hello-world";
    license = licenses.mit;
    maintainers = [ maintainers.revWhiteShadow ];
  };
}

src = pkgs.fetchFromGitHub { ... };: This replaces the ./. with a call to fetchFromGitHub.
owner = "example"; repo = "hello-world";: Specifies the GitHub repository.
rev = "main";: Specifies the branch or commit to fetch. Important: It’s highly recommended to use a specific commit hash instead of a branch to ensure reproducibility.
sha256 = "replace-with-valid-sha256";: A crucial security measure. Nix will verify that the downloaded source code matches the specified SHA256 hash. You can obtain this hash using the nix-hash --flat --base32 <path-to-file> command after downloading the source code manually. Nix will also show you what the hash is expected to be if the hash doesn’t match.

Nix provides other fetcher functions for different source code locations, such as fetchurl for downloading from URLs and fetchFromGitLab for GitLab repositories.

Contributing Your Package to Nixpkgs: A Path to Wider Adoption

Once you have a working Nix derivation, you might want to contribute it to Nixpkgs, making it available to a wider audience. This involves creating a pull request (PR) on the Nixpkgs repository.

Fork the Nixpkgs Repository: Go to the Nixpkgs GitHub repository (https://github.com/NixOS/nixpkgs) and fork it to your own GitHub account.
Clone Your Fork: Clone your forked repository to your local machine:

git clone git@github.com:your-username/nixpkgs.git
cd nixpkgs

Create a New Branch: Create a new branch for your package:

git checkout -b add-hello

Find a Suitable Location: Determine the appropriate location for your package within the Nixpkgs directory structure. For simple applications, a common location is pkgs/applications/misc/.
Create a Package Directory: Create a new directory for your package within the chosen location (e.g., pkgs/applications/misc/hello).
Place Your default.nix: Copy your default.nix file into the newly created package directory. You might also need to create other files, such as a patches directory for applying patches to the source code.
Update pkgs/top-level/all-packages.nix: This file lists all packages available in Nixpkgs. You need to add your package to this file to make it discoverable. Open pkgs/top-level/all-packages.nix and add the following line within the appropriate section (usually alphabetically):

hello = callPackage ../pkgs/applications/misc/hello { };

Test Your Package: Before submitting a PR, it’s essential to test your package thoroughly. Run nix-build within the Nixpkgs directory to ensure that the package builds correctly within the Nixpkgs environment.
Commit Your Changes: Commit your changes with descriptive messages:

git add pkgs/applications/misc/hello/default.nix pkgs/top-level/all-packages.nix
git commit -m "Add hello package"

Push to Your Fork: Push your branch to your forked repository:

git push origin add-hello

Create a Pull Request: Go to your forked repository on GitHub and create a new pull request targeting the nixpkgs:master branch. Provide a clear and concise description of your changes, including the purpose of the package and any relevant information.
Address Review Comments: Nixpkgs maintainers will review your PR and provide feedback. Be prepared to address any comments or suggestions and make necessary changes to your package.
Follow the Nixpkgs Style Guide: Adhering to the Nixpkgs style guide is crucial for a successful PR. Pay attention to naming conventions, code formatting, and other guidelines. The Nix community cares about the quality of the package.
Patience is Key: The Nixpkgs review process can take time. Be patient and responsive to feedback.

Advanced Derivation Techniques: Going Beyond the Basics

Using `configurePhase` for Complex Builds

Some software requires a configuration step before building. Nix provides a configurePhase attribute for this purpose. This is useful for projects that use configure scripts.

{ pkgs ? import <nixpkgs> {} }:

pkgs.stdenv.mkDerivation {
  name = "my-complex-package";
  version = "1.0";
  src = ./.;

  configurePhase = ''
    ./configure --prefix=$out
  '';

  buildPhase = ''
    make
  '';

  installPhase = ''
    make install
  '';

  meta = with pkgs.lib; {
    description = "A more complex package with a configure script";
    license = licenses.gpl3;
    maintainers = [ maintainers.revWhiteShadow ];
  };
}

configurePhase = '' ./configure --prefix=$out '';: This runs the configure script with the --prefix=$out option, ensuring that the package is installed to the correct output directory.

Patching Source Code

Sometimes, you need to apply patches to the source code before building. Nix provides a convenient way to manage patches.

Create a patches Directory: Create a directory named patches in the same directory as your default.nix.
Place Your Patches: Place your patch files (e.g., fix-build.patch) in the patches directory.
Use the patches Attribute: Add the patches attribute to your derivation:

{ pkgs ? import <nixpkgs> {} }:

pkgs.stdenv.mkDerivation {
  name = "my-package-with-patches";
  version = "1.0";
  src = ./.;

  patches = [ ./patches/fix-build.patch ]; # Apply the patch

  buildPhase = ''
    make
  '';

  installPhase = ''
    mkdir -p $out/bin
    cp my-program $out/bin
  '';

  meta = with pkgs.lib; {
    description = "A package with patches applied";
    license = licenses.bsd3;
    maintainers = [ maintainers.revWhiteShadow ];
  };
}

Nix will automatically apply the patches before the build phase.

Overlays: Extending Nixpkgs Without Direct Modification

Overlays provide a mechanism to extend or modify Nixpkgs without directly editing the Nixpkgs repository. This is useful for personal customizations or for providing packages that are not yet ready for inclusion in Nixpkgs.

Create an Overlay File: Create a file (e.g., overlay.nix) containing your overlay definition.
Define Your Overlay:

self: super: {
  hello = super.hello.overrideAttrs (oldAttrs: {
    version = "1.1";
    src = ./.; # Or a different source
    meta = oldAttrs.meta // {
      description = "Modified hello package";
    };
  });
}

This overlay modifies the existing hello package by overriding its version, source code, and description. If hello does not exist, this will create the hello package.

Apply the Overlay: You can apply the overlay using the -I option with nix-build or by configuring it in your nix.conf file. For example:

nix-build -I nixpkgs=/path/to/nixpkgs -I nixpkgs=/path/to/your/overlay

Conclusion: Mastering Nix Packaging

Packaging software with Nix empowers you with reproducible, declarative, and reliable builds. By understanding derivations, utilizing fetchers, and contributing to Nixpkgs, you can become a proficient Nix user and contribute to the vibrant Nix ecosystem. Remember to consult the official Nix documentation and Nixpkgs manual for further details and advanced techniques. Good luck!

How to package my software in nix or write my own package derivation for nixpkgs

How to Package Your Software in Nix or Write Your Own Package Derivation for Nixpkgs #

Understanding Nix Derivations: The Building Blocks of Packages #

Crafting a Basic Nix Derivation: A Step-by-Step Approach #

Enhancing Your Derivation: Handling Dependencies #

Leveraging Fetchers: Handling External Source Code #

Contributing Your Package to Nixpkgs: A Path to Wider Adoption #

Advanced Derivation Techniques: Going Beyond the Basics #

Using configurePhase for Complex Builds #

Patching Source Code #

Overlays: Extending Nixpkgs Without Direct Modification #

Conclusion: Mastering Nix Packaging #