Nix Derivation

From nice yt video

Recap

In Nix, file paths are treated as first-class citizens, not just regular strings. This means that paths have their own syntax and behave differently from typical strings in other languages.

File Paths in Nix

Paths without quotes:
- In most languages, paths are enclosed in quotes as strings, but in Nix, if you omit the quotes, it’s recognized as a Nix path.
- Example:
```
./some/file
```
REPL Display vs. Actual Value:
- In the Nix REPL, paths are shown as absolute paths to the referenced file or folder. However, this is just a display value.
- The actual value of the path becomes visible when you use string interpolation or builtin.toString. This reveals the Nix store path.

Nix Store

Nix store path:
- When a path is used in a Nix expression, the file (or folder) is copied into the Nix store, located by default at /nix/store.
- The store path follows the format:
```
/nix/store/<hash>-<name>
```
- This path contains a hash that is based on the file’s content and other factors, like the file name. This ensures that files with different content get different paths, but even files with the same content may have different store paths due to other metadata.

Content-Addressable Store

Hash-based storage:
- The Nix store is a content-addressable, immutable data store, similar to how Git works. Once a file is placed in the Nix store, it’s given a unique, read-only path based on its content.
- This design ensures deterministic builds, where the same inputs always produce the same output.

Theoretical Flexibility

While theoretically the Nix store could be placed anywhere, it is strongly recommended to keep it at the default location /nix/store for consistency and compatibility.

Hashing and File Uniqueness

The hash included in the store path is not just based on file contents but also includes other factors, like the file name. This means that:
- Files with the same content but different names may have different store paths.
- Hash uniqueness helps ensure that builds are reproducible and isolated.

Important Takeaway

Every time a path is evaluated in a Nix expression (whether it’s a single file or a whole directory), it gets copied to the Nix store, made read-only, and assigned a unique hash-based path.
The Nix store design plays a crucial role in Nix’s ability to provide reliable and reproducible software builds, which is key to Nix’s approach to software deployment.

Derivations in Nix

In Nix, a derivation is a built-in function used to execute commands and capture the resulting output. Derivations solve challenges in software building, ensuring reproducibility and purity.

Key Concepts

Basic Structure of a Derivation:
- Derivations require certain attributes:
  - name: The name of the derivation, which will be reflected in the Nix store.
  - system: Specifies the target system (e.g., the architecture and OS).
  - builder: Defines how the derivation is built, often using bash or another script.
  - args: Arguments passed to the builder.
- Additional attributes are passed to the derivation and made available as environment variables during the build.
Example:
```
derivation {
  name = "example";
  system = "x86_64-linux";
  builder = "/bin/bash";
  args = [ "-c" '' echo "Building..." ''; ];
}
```
Example: Compiling a C Program:
- You can use a derivation to compile a main.c file with Clang. The file will be copied to the Nix store.
- To ensure Clang runs in the correct environment, you might write a bash script as the builder:
```
derivation {
  name = "compile-hello";
  system = "x86_64-linux";
  builder = "/bin/bash";
  args = [ "-c" '' clang -o $out ./main.c ''; ];
  src = ./main.c;
}
```
- The compiled output would go into a folder in the Nix store.
Serialization of a Derivation:
- Derivations are serialized using nix-instantiate, which stores the derivation recipe in the Nix store. This doesn’t execute the build but ensures that the derivation is deterministic and can be re-evaluated later.
- Nix derivation show provides a pretty-printed, human-readable view of the serialized derivation as JSON.
Realizing a Derivation:
- To actually build the derivation, you run nix-store --realize. The build happens in a sandbox, preventing external system dependencies from being accidentally used.
- All output files must be placed in the $out environment variable’s folder (otherwise the build fails).
Using nix-build:
- nix-build is a shortcut that combines the serialization and realization steps, creating a result symlink pointing to the build output in the Nix store.
Ensuring Reproducibility:
- Hardcoded paths to system binaries like /bin/bash can break reproducibility. The proper way to handle this is to use Nix to specify versions of tools like Clang explicitly, ensuring the same version is always used.
- Example of using Clang 16:
```
derivation {
  name = "clang";
  src = fetchurl {
    url = "https://github.com/llvm/llvm-project/archive/refs/tags/llvmorg-16.0.0.tar.gz";
    sha256 = "...";
  };
}
```
- This ensures that Clang is built from a specific source and version, independent of the host system.
Handling Impurities:
- If a derivation relies on system libraries (like libc), it is considered impure, because the result might change across systems.
- The solution is to nest derivations: use a well-defined version of the compiler to build its own dependencies (like glibc), ensuring everything is built from pure sources in the Nix store.
- Example:
```
derivation {
  name = "glibc";
  src = fetchurl {
    url = "https://ftp.gnu.org/gnu/libc/glibc-2.40.tar.gz";
    sha256 = "...";
  };
  buildInputs = [ clang ];
}
```
Merkle Tree Structure:
- Nix uses a Merkle tree structure for derivations. The hash of each derivation includes the hashes of its dependencies. This ensures that if a dependency like Clang changes, all derivations depending on it (e.g., glibc) also change, ensuring consistency.
- Different versions of the same package (e.g., glibc with SIMD support and without) can coexist in the Nix store without conflict due to unique hashes.
Standard Environment (stdenv):
- Once you’ve built enough tools (like a pure Clang, Bash, etc.), Nix provides a collection of pure build tools known as the standard environment.
- Nix packages rely on this standard environment to ensure that builds are independent of the host system.