zfs

Identity

• Kernel module: zfs.ko (loaded via modprobe zfs; auto-loaded on most distros)
• Main CLI tools: zpool (pool management), zfs (dataset/snapshot management), zdb (low-level diagnostics)
• Config: /etc/zfs/ (import cache, key files), /etc/zfs/zpool.cache (auto-import list)
• Service: zfs-import-cache.service, zfs-mount.service, zfs-share.service (systemd units)
• Logs: journalctl -u zfs-import-cache / journalctl -u zfs-mount / dmesg | grep -i zfs
• Distro install: apt install zfsutils-linux (Debian/Ubuntu) / dnf install zfs after adding OpenZFS repo (RHEL/Fedora)
• Version check: zpool version / zfs version

Key Operations

Expected State

• All pools report ONLINE under zpool status — DEGRADED means redundancy is lost, FAULTED means data may be at risk.
• No checksum or read errors in zpool status output (errors: No known data errors).
• All datasets mounted at expected mountpoints: zfs mount shows no unmounted datasets that should be online.
• Scrub completed within the last 30 days with zero errors.
• ARC hit rate above 80% under normal workloads: arc_summary or /proc/spl/kstat/zfs/arcstats.

Health Checks

1. zpool status — all pools ONLINE, no errors, scrub date and result visible
2. zpool list — verify free space; pools above 80% capacity show fragmentation increases
3. zfs mount — lists currently mounted datasets; cross-check against zfs list
4. awk '/^hits|^misses/ {sum+=$3} END {print "ARC total accesses:", sum}' /proc/spl/kstat/zfs/arcstats — then compute hit rate: hits / (hits + misses)

Common Failures

Pain Points

• ARC is not a memory leak: ZFS ARC uses all available RAM by design — it releases memory to other processes on demand via the kernel's memory pressure mechanism. Only cap it if RAM pressure is causing issues.
• Deduplication is memory-expensive: The deduplication table (DDT) requires roughly 300–500 bytes of RAM per unique block. A 10 TB deduplicated pool can require 5–30 GB of RAM just for the DDT. Use compression=lz4 or compression=zstd instead — similar space savings at negligible cost.
• Cannot shrink a vdev: Once a vdev is added to a pool, its device count is permanent. You cannot remove a RAIDZ vdev (only mirrors can be removed in recent OpenZFS versions). Plan pool layout before creation.
• Pool feature flags are one-way upgrades: zpool upgrade enables new features but the pool then requires the same or newer OpenZFS version to import. Never upgrade if the pool might need to be read by an older system.
• ECC RAM — required or recommended: ZFS does not require ECC RAM; it runs fine without it. ECC protects against RAM bit errors corrupting data before ZFS writes it. On servers with large datasets and high write rates, ECC is strongly recommended. On a home NAS it is a judgment call.
• L2ARC and SLOG devices: L2ARC (SSD read cache) rarely helps unless your working set is larger than RAM. SLOG (separate ZFS Intent Log) only speeds up synchronous writes — databases, NFS with sync enabled. Cheap SSDs as SLOG devices are dangerous: a failed SLOG can cause a pool to need recovery. Use enterprise or power-loss-protected SSDs for SLOG.
• recordsize matters for databases: Default recordsize=128K is good for large sequential files. PostgreSQL and MySQL benefit from recordsize=16K or recordsize=8K to match their page sizes. Set before writing data — changing recordsize applies to new writes only.
• Snapshot space accounting: zfs list -t snapshot shows USED for each snapshot, but this only reflects blocks that are unique to that snapshot. Deleting a snapshot transfers its blocks to the next newer snapshot until the last one is destroyed. Space is not freed until the last snapshot holding unique blocks is destroyed.

References

See references/ for:

• zfs-properties.md — pool and dataset property reference organized by category
• common-patterns.md — pool creation, snapshots, send/receive, encryption, and tuning examples
• docs.md — official documentation and man page links