Learn Zig Series):Exercise 1: DirectoryWalker with .ignore file support
const std = @import("std");
const IgnoreWalker = struct {
allocator: std.mem.Allocator,
entries: std.ArrayList([]const u8),
max_depth: usize,
fn init(allocator: std.mem.Allocator, max_depth: usize) IgnoreWalker {
return .{
.allocator = allocator,
.entries = std.ArrayList([]const u8).init(allocator),
.max_depth = max_depth,
};
}
fn deinit(self: *IgnoreWalker) void {
for (self.entries.items) |p| self.allocator.free(p);
self.entries.deinit();
}
fn walkRecursive(self: *IgnoreWalker, dir_path: []const u8, depth: usize) !void {
if (depth > self.max_depth) return;
var dir = std.fs.cwd().openDir(dir_path, .{ .iterate = true }) catch return;
defer dir.close();
// check for .ignore -- if it exists, skip entire subtree
if (dir.access(".ignore", .{})) |_| {
return;
} else |_| {}
var iter = dir.iterate();
while (try iter.next()) |entry| {
const full_path = try std.fs.path.join(self.allocator, &.{ dir_path, entry.name });
try self.entries.append(full_path);
if (entry.kind == .directory) {
try self.walkRecursive(full_path, depth + 1);
}
}
}
};
The key insight: check for .ignore right after opening the directory, before iterating its contents. The access call is cheap -- just a stat under the hood.
Exercise 2: findFiles with basic glob matching
const std = @import("std");
fn globMatch(pattern: []const u8, name: []const u8) bool {
if (std.mem.eql(u8, pattern, "*")) return true;
// *.ext -- match suffix
if (pattern[0] == '*' and pattern.len > 1) {
const suffix = pattern[1..];
if (name.len < suffix.len) return false;
return std.mem.eql(u8, name[name.len - suffix.len ..], suffix);
}
// prefix* -- match prefix
if (pattern[pattern.len - 1] == '*') {
const prefix = pattern[0 .. pattern.len - 1];
if (name.len < prefix.len) return false;
return std.mem.eql(u8, name[0..prefix.len], prefix);
}
return std.mem.eql(u8, pattern, name);
}
fn findFilesRecursive(
allocator: std.mem.Allocator,
dir_path: []const u8,
pattern: []const u8,
results: *std.ArrayList([]const u8),
depth: usize,
) !void {
if (depth > 20) return;
var dir = std.fs.cwd().openDir(dir_path, .{ .iterate = true }) catch return;
defer dir.close();
var iter = dir.iterate();
while (try iter.next()) |entry| {
const full_path = try std.fs.path.join(allocator, &.{ dir_path, entry.name });
if (entry.kind == .file and globMatch(pattern, entry.name)) {
try results.append(full_path);
} else if (entry.kind == .directory) {
try findFilesRecursive(allocator, full_path, pattern, results, depth + 1);
allocator.free(full_path);
} else {
allocator.free(full_path);
}
}
}
The glob matcher handles *.ext (suffix), prefix* (prefix), and * (match all). Not a full glob, but covers the common cases.
Exercise 3: du (disk usage) command
const std = @import("std");
fn calculateDirSize(allocator: std.mem.Allocator, dir_path: []const u8) !u64 {
var total: u64 = 0;
var dir = std.fs.cwd().openDir(dir_path, .{ .iterate = true }) catch return 0;
defer dir.close();
var iter = dir.iterate();
while (try iter.next()) |entry| {
if (entry.kind == .file) {
const stat = dir.statFile(entry.name) catch continue;
total += stat.size;
} else if (entry.kind == .directory) {
const sub_path = try std.fs.path.join(allocator, &.{ dir_path, entry.name });
defer allocator.free(sub_path);
total += try calculateDirSize(allocator, sub_path);
}
}
return total;
}
fn formatSize(bytes: u64, buf: *[32]u8) []const u8 {
if (bytes >= 1024 * 1024) {
const mb = @as(f64, @floatFromInt(bytes)) / (1024.0 * 1024.0);
return std.fmt.bufPrint(buf, "{d:.1} MB", .{mb}) catch "???";
} else if (bytes >= 1024) {
const kb = @as(f64, @floatFromInt(bytes)) / 1024.0;
return std.fmt.bufPrint(buf, "{d:.1} KB", .{kb}) catch "???";
}
return std.fmt.bufPrint(buf, "{d} B", .{bytes}) catch "???";
}
Walk each subdirectory recursively, sum file sizes, sort largest-first and print with formatSize. The recursive calculateDirSize is the core -- main just collects top-level entries into an ArrayList, sorts, and prints.
Last episode we dug into file system fundamentals -- listing directories, reading metadata, following symlinks, building a tree command. All that was about inspecting the filesystem at one point in time. Today we go one step further: watching the filesystem for changes as they happen. This is the foundation for development tools like zig build --watch, hot-reload servers, file sync tools, and pretty much any application that needs to react when the user saves a file.
Here we go!
The simplest way to detect file changes is polling: check the modification timestamp every N milliseconds, and if it changed, do something. You can build this in about 20 lines of Zig using what we already know from the stat call in episode 62:
const std = @import("std");
fn pollForChanges(path: []const u8, interval_ms: u64) !void {
const stdout = std.io.getStdOut().writer();
var last_mtime: i128 = 0;
while (true) {
const stat = std.fs.cwd().statFile(path) catch |err| {
try stdout.print("cannot stat {s}: {}\n", .{ path, err });
std.time.sleep(interval_ms * std.time.ns_per_ms);
continue;
};
if (stat.mtime != last_mtime and last_mtime != 0) {
try stdout.print("CHANGED: {s}\n", .{path});
}
last_mtime = stat.mtime;
std.time.sleep(interval_ms * std.time.ns_per_ms);
}
}
This works. And for watching a single file it's honestly fine. But it has a fundamental problem: it scales terribly. If you want to watch a project with 500 source files, you need to stat all 500 files every cycle. That's 500 syscalls per poll interval. At 100ms polling, that's 5000 syscalls per second -- just to check if anything changed. On a laptop running on battery, this is measurably bad.
The operating system already knows when files change. Every write, rename, and delete goes through the kernel. So instead of asking the kernel 5000 times per second "did anything change?", we can ask once: "tell me when something changes." That's what inotify (Linux) and kqueue (macOS/BSD) do. They're event-driven file change notification APIs. You register watches, then block (or poll with epoll/kqueue) until the kernel wakes you up with events. Zero CPU usage while nothing is happening. Instant notification when something does.
inotify has been in Linux since kernel 2.6.13 (2005). It works through a special file descriptor -- you create one with inotify_init, add watches to it with inotify_add_watch, and then read events from it like you'd read from any file. Zig's standard library exposes the inotify syscalls through std.os.linux:
const std = @import("std");
const linux = std.os.linux;
const InotifyWatcher = struct {
fd: i32,
watch_fds: std.AutoHashMap(i32, []const u8),
allocator: std.mem.Allocator,
fn init(allocator: std.mem.Allocator) !InotifyWatcher {
const fd = linux.inotify_init1(linux.IN.NONBLOCK | linux.IN.CLOEXEC);
if (@as(isize, @bitCast(@as(usize, fd))) < 0) {
return error.InotifyInitFailed;
}
return .{
.fd = @intCast(fd),
.watch_fds = std.AutoHashMap(i32, []const u8).init(allocator),
.allocator = allocator,
};
}
fn deinit(self: *InotifyWatcher) void {
var it = self.watch_fds.valueIterator();
while (it.next()) |v| {
self.allocator.free(v.*);
}
self.watch_fds.deinit();
_ = std.posix.close(@intCast(self.fd));
}
fn addWatch(self: *InotifyWatcher, path: []const u8) !i32 {
// we need a sentinel-terminated path for the syscall
const c_path = try self.allocator.dupeZ(u8, path);
defer self.allocator.free(c_path);
const mask = linux.IN.MODIFY | linux.IN.CREATE |
linux.IN.DELETE | linux.IN.MOVED_FROM |
linux.IN.MOVED_TO;
const wd = linux.inotify_add_watch(self.fd, c_path, mask);
if (@as(isize, @bitCast(@as(usize, wd))) < 0) {
return error.WatchAddFailed;
}
const wd_int: i32 = @intCast(wd);
const path_copy = try self.allocator.dupe(u8, path);
try self.watch_fds.put(wd_int, path_copy);
return wd_int;
}
fn processEvents(self: *InotifyWatcher, writer: anytype) !bool {
var buf: [4096]u8 align(@alignOf(linux.inotify_event)) = undefined;
const bytes_read = linux.read(@intCast(self.fd), &buf, buf.len);
// EAGAIN means no events available (non-blocking mode)
if (@as(isize, @bitCast(bytes_read)) < 0) return false;
if (bytes_read == 0) return false;
var offset: usize = 0;
while (offset < bytes_read) {
const event: *const linux.inotify_event = @alignCast(
@ptrCast(&buf[offset]),
);
const name = if (event.len > 0)
std.mem.sliceTo(@as([*:0]const u8, @ptrCast(&buf[offset + @sizeOf(linux.inotify_event)])), 0)
else
"(dir itself)";
const dir_path = self.watch_fds.get(event.wd) orelse "(unknown)";
if (event.mask & linux.IN.CREATE != 0) {
try writer.print("[CREATE] {s}/{s}\n", .{ dir_path, name });
}
if (event.mask & linux.IN.MODIFY != 0) {
try writer.print("[MODIFY] {s}/{s}\n", .{ dir_path, name });
}
if (event.mask & linux.IN.DELETE != 0) {
try writer.print("[DELETE] {s}/{s}\n", .{ dir_path, name });
}
if (event.mask & linux.IN.MOVED_FROM != 0) {
try writer.print("[MOVE_FROM] {s}/{s}\n", .{ dir_path, name });
}
if (event.mask & linux.IN.MOVED_TO != 0) {
try writer.print("[MOVE_TO] {s}/{s}\n", .{ dir_path, name });
}
offset += @sizeOf(linux.inotify_event) + event.len;
}
return true;
}
};
A few things to unpack here. The inotify_init1 call with IN.NONBLOCK means reads won't block when there are no events -- they return -EAGAIN instead. We need this because we'll be checking for events in a loop and we don't want to get stuck waiting. The IN.CLOEXEC flag means the file descriptor gets closed automatically if we exec another process (a good habit for any fd you don't intend to pass to child processes).
The event mask on inotify_add_watch controls which events we care about. IN.MODIFY fires when file contents change. IN.CREATE fires when a new file appears in the watched directory. IN.DELETE fires when a file is removed. The MOVED_FROM and MOVED_TO pair handles renames -- a rename within the same directory produces both events, while a move across directories produces MOVED_FROM in the source and MOVED_TO in the destination.
The event reading is the most involved part. inotify packs events contiguously in the read buffer, each with a variable-length name field. The inotify_event struct has a len field that tells you how many bytes follow the struct for the filename. When the watch is on a directory, the name tells you which file inside that directory changed. When the watch is on a specific file, len is 0 and there's no name.
One big inotify limitation: it does NOT watch recursively. Adding a watch on /home/user/project only watches that one directory. Files in project/src/ won't trigger events. You have to add a separate watch for every subdirectory. We'll solve this later.
macOS and the BSDs use kqueue instead of inotify. The concept is similar -- register interest, get events -- but the API is different. kqueue is actually more general than inotify; it can watch file descriptors, processes, signals, and timers in addition to filesystem changes:
const std = @import("std");
const builtin = @import("builtin");
const KqueueWatcher = struct {
kq: i32,
watched_fds: std.ArrayList(WatchedEntry),
allocator: std.mem.Allocator,
const WatchedEntry = struct {
fd: i32,
path: []const u8,
};
fn init(allocator: std.mem.Allocator) !KqueueWatcher {
const kq = try std.posix.kqueue();
return .{
.kq = kq,
.watched_fds = std.ArrayList(WatchedEntry).init(allocator),
.allocator = allocator,
};
}
fn deinit(self: *KqueueWatcher) void {
for (self.watched_fds.items) |entry| {
std.posix.close(entry.fd);
self.allocator.free(entry.path);
}
self.watched_fds.deinit();
std.posix.close(self.kq);
}
fn addWatch(self: *KqueueWatcher, path: []const u8) !void {
// kqueue needs an open file descriptor for the target
const c_path = try self.allocator.dupeZ(u8, path);
defer self.allocator.free(c_path);
const fd = std.posix.open(c_path, .{ .ACCMODE = .RDONLY }, 0) catch {
return error.OpenFailed;
};
// register the fd with kqueue for vnode events
var changelist = [1]std.posix.Kevent{.{
.ident = @intCast(fd),
.filter = std.posix.system.EVFILT.VNODE,
.flags = std.posix.system.EV.ADD | std.posix.system.EV.CLEAR,
.fflags = std.posix.system.NOTE.WRITE | std.posix.system.NOTE.DELETE |
std.posix.system.NOTE.RENAME | std.posix.system.NOTE.ATTRIB,
.data = 0,
.udata = 0,
}};
_ = try std.posix.kevent(self.kq, &changelist, &.{}, null);
const path_copy = try self.allocator.dupe(u8, path);
try self.watched_fds.append(.{ .fd = fd, .path = path_copy });
}
fn waitForEvents(self: *KqueueWatcher, writer: anytype) !void {
var events: [16]std.posix.Kevent = undefined;
const timeout = std.posix.timespec{ .sec = 1, .nsec = 0 };
const n = try std.posix.kevent(self.kq, &.{}, &events, &timeout);
for (events[0..n]) |ev| {
const fd: i32 = @intCast(ev.ident);
// find the path for this fd
var path: []const u8 = "(unknown)";
for (self.watched_fds.items) |entry| {
if (entry.fd == fd) {
path = entry.path;
break;
}
}
if (ev.fflags & std.posix.system.NOTE.WRITE != 0) {
try writer.print("[WRITE] {s}\n", .{path});
}
if (ev.fflags & std.posix.system.NOTE.DELETE != 0) {
try writer.print("[DELETE] {s}\n", .{path});
}
if (ev.fflags & std.posix.system.NOTE.RENAME != 0) {
try writer.print("[RENAME] {s}\n", .{path});
}
if (ev.fflags & std.posix.system.NOTE.ATTRIB != 0) {
try writer.print("[ATTRIB] {s}\n", .{path});
}
}
}
};
The biggest difference between kqueue and inotify is that kqueue requires an open file descriptor for each watched path. inotify uses watch descriptors that are kernel-managed -- you don't need to hold an fd open for every watched file. This means kqueue can hit the per-process file descriptor limit faster if you're watching thousands of files. On macOS, the default soft limit is 256 fds (you can raise it with ulimit -n). On Linux, inotify has a separate limit (/proc/sys/fs/inotify/max_user_watches, typically 65536 or higher).
The EV.CLEAR flag is important -- it means the event is automatically re-enabled after being retrieved. Without it, you'd get the event once and then never again (one-shot mode). For a file watcher, you always want continuous notification.
kqueue's NOTE.WRITE is roughly equivalent to inotify's IN.MODIFY. But kqueue doesn't distinguish between create-in-directory and modify-file the way inotify does. If you're watching a directory, kqueue fires NOTE.WRITE whenever anything inside changes -- you'd need to rescan the directory to figure out what specifically happened.
Now for the fun part. Zig's comptime makes cross-platform abstraction almost free. We can write a single FileWatcher type that uses inotify on Linux and kqueue on macOS, with the switch happening at compile time -- no runtime overhead, no vtable, no dynamic dispatch:
const std = @import("std");
const builtin = @import("builtin");
pub const FileEvent = struct {
kind: EventKind,
path: []const u8,
name: []const u8,
const EventKind = enum {
created,
modified,
deleted,
renamed,
attribute_changed,
};
};
pub const FileWatcher = struct {
allocator: std.mem.Allocator,
// platform-specific handle
handle: PlatformHandle,
paths: std.StringHashMap(void),
const PlatformHandle = switch (builtin.os.tag) {
.linux => struct {
inotify_fd: i32,
wd_to_path: std.AutoHashMap(i32, []const u8),
},
.macos => struct {
kq_fd: i32,
file_fds: std.ArrayList(i32),
},
else => @compileError("unsupported OS for file watching"),
};
pub fn init(allocator: std.mem.Allocator) !FileWatcher {
const handle: PlatformHandle = switch (builtin.os.tag) {
.linux => blk: {
const fd = std.os.linux.inotify_init1(
std.os.linux.IN.NONBLOCK | std.os.linux.IN.CLOEXEC,
);
if (@as(isize, @bitCast(@as(usize, fd))) < 0)
return error.InotifyInitFailed;
break :blk .{
.inotify_fd = @intCast(fd),
.wd_to_path = std.AutoHashMap(i32, []const u8).init(allocator),
};
},
.macos => blk: {
const kq = try std.posix.kqueue();
break :blk .{
.kq_fd = kq,
.file_fds = std.ArrayList(i32).init(allocator),
};
},
else => unreachable,
};
return .{
.allocator = allocator,
.handle = handle,
.paths = std.StringHashMap(void).init(allocator),
};
}
pub fn deinit(self: *FileWatcher) void {
switch (builtin.os.tag) {
.linux => {
var it = self.handle.wd_to_path.valueIterator();
while (it.next()) |v| self.allocator.free(v.*);
self.handle.wd_to_path.deinit();
std.posix.close(@intCast(self.handle.inotify_fd));
},
.macos => {
for (self.handle.file_fds.items) |fd| std.posix.close(fd);
self.handle.file_fds.deinit();
std.posix.close(self.handle.kq_fd);
},
else => unreachable,
}
var path_it = self.paths.keyIterator();
while (path_it.next()) |k| self.allocator.free(k.*);
self.paths.deinit();
}
pub fn watch(self: *FileWatcher, path: []const u8) !void {
const path_copy = try self.allocator.dupe(u8, path);
switch (builtin.os.tag) {
.linux => {
const c_path = try self.allocator.dupeZ(u8, path);
defer self.allocator.free(c_path);
const mask = std.os.linux.IN.MODIFY |
std.os.linux.IN.CREATE |
std.os.linux.IN.DELETE |
std.os.linux.IN.MOVED_FROM |
std.os.linux.IN.MOVED_TO;
const wd = std.os.linux.inotify_add_watch(
self.handle.inotify_fd,
c_path,
mask,
);
if (@as(isize, @bitCast(@as(usize, wd))) < 0) {
self.allocator.free(path_copy);
return error.WatchFailed;
}
try self.handle.wd_to_path.put(@intCast(wd), path_copy);
},
.macos => {
const c_path = try self.allocator.dupeZ(u8, path);
defer self.allocator.free(c_path);
const fd = std.posix.open(c_path, .{ .ACCMODE = .RDONLY }, 0) catch {
self.allocator.free(path_copy);
return error.WatchFailed;
};
try self.handle.file_fds.append(fd);
},
else => unreachable,
}
try self.paths.put(path_copy, {});
}
pub fn pollEvents(
self: *FileWatcher,
callback: *const fn (FileEvent) void,
) !void {
switch (builtin.os.tag) {
.linux => try self.pollLinux(callback),
.macos => try self.pollMacos(callback),
else => unreachable,
}
}
fn pollLinux(self: *FileWatcher, callback: *const fn (FileEvent) void) !void {
const linux = std.os.linux;
var buf: [4096]u8 align(@alignOf(linux.inotify_event)) = undefined;
const n = linux.read(@intCast(self.handle.inotify_fd), &buf, buf.len);
if (@as(isize, @bitCast(n)) <= 0) return;
var offset: usize = 0;
while (offset < n) {
const event: *const linux.inotify_event = @alignCast(
@ptrCast(&buf[offset]),
);
const name = if (event.len > 0)
std.mem.sliceTo(@as([*:0]const u8, @ptrCast(
&buf[offset + @sizeOf(linux.inotify_event)],
)), 0)
else
"";
const dir = self.handle.wd_to_path.get(event.wd) orelse "";
const kind: FileEvent.EventKind = if (event.mask & linux.IN.CREATE != 0)
.created
else if (event.mask & linux.IN.MODIFY != 0)
.modified
else if (event.mask & linux.IN.DELETE != 0)
.deleted
else if (event.mask & (linux.IN.MOVED_FROM | linux.IN.MOVED_TO) != 0)
.renamed
else
.modified;
callback(.{ .kind = kind, .path = dir, .name = name });
offset += @sizeOf(linux.inotify_event) + event.len;
}
}
fn pollMacos(self: *FileWatcher, callback: *const fn (FileEvent) void) !void {
_ = self;
_ = callback;
// kqueue implementation would go here -- same pattern as above
}
};
The PlatformHandle type is a comptime switch -- on Linux the struct contains an inotify fd and a watch-descriptor-to-path map, on macOS it's a kqueue fd and a list of open file descriptors. The compiler only generates code for the current target. On Linux, the macOS branch doesn't even get compiled -- it's dead code, eliminated before code generation. This is the same pattern we used in episode 35 for target-specific compilation, but applied to runtime behavior selection.
The @compileError on the else branch is important. If someone tries to compile this for Windows or WASI, they get a clear compile error instead of a mysterious runtime crash. Explicit failure over silent breakage.
Let's build a more complete event handler that properly classifies filesystem events and tracks renames (which are actually two events on Linux):
const EventClassifier = struct {
// track move cookies to match MOVED_FROM with MOVED_TO
pending_moves: std.AutoHashMap(u32, MoveInfo),
allocator: std.mem.Allocator,
const MoveInfo = struct {
dir_path: []const u8,
name: []const u8,
timestamp: i64,
};
fn init(allocator: std.mem.Allocator) EventClassifier {
return .{
.pending_moves = std.AutoHashMap(u32, MoveInfo).init(allocator),
.allocator = allocator,
};
}
fn deinit(self: *EventClassifier) void {
var it = self.pending_moves.valueIterator();
while (it.next()) |v| {
self.allocator.free(v.name);
}
self.pending_moves.deinit();
}
fn classify(self: *EventClassifier, mask: u32, cookie: u32, dir: []const u8, name: []const u8) !?ClassifiedEvent {
const linux = std.os.linux;
// renames produce a MOVED_FROM/MOVED_TO pair with the same cookie
if (mask & linux.IN.MOVED_FROM != 0) {
const name_copy = try self.allocator.dupe(u8, name);
try self.pending_moves.put(cookie, .{
.dir_path = dir,
.name = name_copy,
.timestamp = std.time.milliTimestamp(),
});
return null; // wait for the MOVED_TO
}
if (mask & linux.IN.MOVED_TO != 0) {
if (self.pending_moves.fetchRemove(cookie)) |kv| {
defer self.allocator.free(kv.value.name);
return .{
.kind = .renamed,
.path = dir,
.name = name,
.old_name = kv.value.name,
};
}
// MOVED_TO without MOVED_FROM means file moved in from outside
return .{ .kind = .created, .path = dir, .name = name, .old_name = null };
}
if (mask & linux.IN.CREATE != 0) {
return .{ .kind = .created, .path = dir, .name = name, .old_name = null };
}
if (mask & linux.IN.MODIFY != 0) {
return .{ .kind = .modified, .path = dir, .name = name, .old_name = null };
}
if (mask & linux.IN.DELETE != 0) {
return .{ .kind = .deleted, .path = dir, .name = name, .old_name = null };
}
return null;
}
fn expireStaleEntries(self: *EventClassifier, max_age_ms: i64) void {
const now = std.time.milliTimestamp();
var to_remove = std.ArrayList(u32).init(self.allocator);
defer to_remove.deinit();
var it = self.pending_moves.iterator();
while (it.next()) |entry| {
if (now - entry.value_ptr.timestamp > max_age_ms) {
to_remove.append(entry.key_ptr.*) catch continue;
}
}
for (to_remove.items) |key| {
if (self.pending_moves.fetchRemove(key)) |kv| {
self.allocator.free(kv.value.name);
}
}
}
const ClassifiedEvent = struct {
kind: Kind,
path: []const u8,
name: []const u8,
old_name: ?[]const u8,
const Kind = enum { created, modified, deleted, renamed };
};
};
The rename handling is the tricky bit. On Linux, a rename within a watched directory produces two inotify events: IN_MOVED_FROM with the old name and IN_MOVED_TO with the new name. They share a cookie value so you can match them up. But if a file is moved to a directory you're NOT watching, you'll only see MOVED_FROM -- it looks like a delete. And if a file is moved FROM an unwatched directory into yours, you'll only see MOVED_TO -- it looks like a create. The EventClassifier buffers MOVED_FROM events briefly, waiting for a matching MOVED_TO with the same cookie. If no match arrives within a timeout, the expireStaleEntries function treats it as a regular delete.
This buffering is a design pattern you'll see in every real file watcher. The Node.js chokidar library, Python's watchdog, Rust's notify crate -- they all do some version of this "wait a bit to see if a rename completes" logic. The timeout is typically 100-300ms.
When you save a file in your editor, the OS might generate multiple events: a truncate, then a write, then a metadata update. Some editors (like vim) write to a temp file and rename it over the original, which generates delete + create + modify events. If your file watcher triggers a rebuild on every event, you'll rebuild 3-5 times per save. Debouncing coalesces these rapid-fire events into a single callback:
const Debouncer = struct {
pending: std.StringHashMap(PendingEvent),
delay_ms: i64,
allocator: std.mem.Allocator,
const PendingEvent = struct {
last_event_time: i64,
fired: bool,
};
fn init(allocator: std.mem.Allocator, delay_ms: i64) Debouncer {
return .{
.pending = std.StringHashMap(PendingEvent).init(allocator),
.delay_ms = delay_ms,
.allocator = allocator,
};
}
fn deinit(self: *Debouncer) void {
var it = self.pending.keyIterator();
while (it.next()) |k| self.allocator.free(k.*);
self.pending.deinit();
}
fn recordEvent(self: *Debouncer, path: []const u8) !void {
const now = std.time.milliTimestamp();
if (self.pending.getPtr(path)) |entry| {
entry.last_event_time = now;
entry.fired = false;
} else {
const path_copy = try self.allocator.dupe(u8, path);
try self.pending.put(path_copy, .{
.last_event_time = now,
.fired = false,
});
}
}
fn getReady(self: *Debouncer, out: *std.ArrayList([]const u8)) !void {
const now = std.time.milliTimestamp();
var it = self.pending.iterator();
while (it.next()) |entry| {
if (!entry.value_ptr.fired and
now - entry.value_ptr.last_event_time >= self.delay_ms)
{
entry.value_ptr.fired = true;
try out.append(entry.key_ptr.*);
}
}
}
};
test "debouncer coalesces rapid events" {
const allocator = std.testing.allocator;
var debouncer = Debouncer.init(allocator, 50);
defer debouncer.deinit();
// simulate rapid events on the same file
try debouncer.recordEvent("src/main.zig");
try debouncer.recordEvent("src/main.zig");
try debouncer.recordEvent("src/main.zig");
// nothing ready yet (delay not elapsed)
var ready = std.ArrayList([]const u8).init(allocator);
defer ready.deinit();
try debouncer.getReady(&ready);
try std.testing.expectEqual(@as(usize, 0), ready.items.len);
// wait for the debounce delay
std.time.sleep(60 * std.time.ns_per_ms);
try debouncer.getReady(&ready);
try std.testing.expectEqual(@as(usize, 1), ready.items.len);
}
The approach is simple: for every event, record the timestamp. When getReady is called, only return paths where the last event was at least delay_ms ago and we haven't already fired. This means rapid-fire events keep pushing the "last event" timestamp forward, and the callback only fires once things have been quiet for the full delay period.
50-200ms is the sweet spot for the delay. Below 50ms you'll still get duplicate triggers from editor save patterns. Above 200ms and the user starts noticing the lag between saving and rebuilding. Most development tools use 100ms. The zig build --watch command in Zig's own build system uses a similar debounce mechanism internally.
As I mentioned, inotify doesn't recurse automatically. To watch an entire project tree, you need to add a watch for every directory. And when new directories are created, you need to add watches for those too. Here's a recursive watcher that handles all of this:
const RecursiveWatcher = struct {
allocator: std.mem.Allocator,
inotify_fd: i32,
wd_map: std.AutoHashMap(i32, []const u8),
debouncer: Debouncer,
ignore_patterns: std.ArrayList([]const u8),
fn init(allocator: std.mem.Allocator) !RecursiveWatcher {
const fd = std.os.linux.inotify_init1(
std.os.linux.IN.NONBLOCK | std.os.linux.IN.CLOEXEC,
);
if (@as(isize, @bitCast(@as(usize, fd))) < 0)
return error.InotifyInitFailed;
return .{
.allocator = allocator,
.inotify_fd = @intCast(fd),
.wd_map = std.AutoHashMap(i32, []const u8).init(allocator),
.debouncer = Debouncer.init(allocator, 100),
.ignore_patterns = std.ArrayList([]const u8).init(allocator),
};
}
fn deinit(self: *RecursiveWatcher) void {
var it = self.wd_map.valueIterator();
while (it.next()) |v| self.allocator.free(v.*);
self.wd_map.deinit();
self.debouncer.deinit();
self.ignore_patterns.deinit();
std.posix.close(@intCast(self.inotify_fd));
}
fn addIgnore(self: *RecursiveWatcher, pattern: []const u8) !void {
try self.ignore_patterns.append(pattern);
}
fn shouldIgnore(self: *RecursiveWatcher, name: []const u8) bool {
for (self.ignore_patterns.items) |pattern| {
if (std.mem.eql(u8, name, pattern)) return true;
// support dotfile patterns
if (std.mem.eql(u8, pattern, ".*") and name.len > 0 and name[0] == '.') {
return true;
}
}
return false;
}
fn watchRecursive(self: *RecursiveWatcher, root_path: []const u8) !void {
try self.addSingleWatch(root_path);
try self.scanAndWatchSubdirs(root_path);
}
fn addSingleWatch(self: *RecursiveWatcher, path: []const u8) !void {
const c_path = try self.allocator.dupeZ(u8, path);
defer self.allocator.free(c_path);
const mask = std.os.linux.IN.MODIFY | std.os.linux.IN.CREATE |
std.os.linux.IN.DELETE | std.os.linux.IN.MOVED_FROM |
std.os.linux.IN.MOVED_TO | std.os.linux.IN.CREATE;
const wd = std.os.linux.inotify_add_watch(self.inotify_fd, c_path, mask);
if (@as(isize, @bitCast(@as(usize, wd))) < 0) return;
const path_copy = try self.allocator.dupe(u8, path);
try self.wd_map.put(@intCast(wd), path_copy);
}
fn scanAndWatchSubdirs(self: *RecursiveWatcher, dir_path: []const u8) !void {
var dir = std.fs.cwd().openDir(dir_path, .{ .iterate = true }) catch return;
defer dir.close();
var iter = dir.iterate();
while (try iter.next()) |entry| {
if (entry.kind != .directory) continue;
if (self.shouldIgnore(entry.name)) continue;
const sub_path = try std.fs.path.join(self.allocator, &.{
dir_path, entry.name,
});
defer self.allocator.free(sub_path);
try self.addSingleWatch(sub_path);
try self.scanAndWatchSubdirs(sub_path);
}
}
fn processEvents(self: *RecursiveWatcher) !void {
const linux = std.os.linux;
var buf: [4096]u8 align(@alignOf(linux.inotify_event)) = undefined;
const n = linux.read(@intCast(self.inotify_fd), &buf, buf.len);
if (@as(isize, @bitCast(n)) <= 0) return;
var offset: usize = 0;
while (offset < n) {
const event: *const linux.inotify_event = @alignCast(
@ptrCast(&buf[offset]),
);
const name = if (event.len > 0)
std.mem.sliceTo(@as([*:0]const u8, @ptrCast(
&buf[offset + @sizeOf(linux.inotify_event)],
)), 0)
else
"";
const dir = self.wd_map.get(event.wd) orelse "";
// if a new directory was created, add a watch for it
if (event.mask & linux.IN.CREATE != 0 and
event.mask & linux.IN.ISDIR != 0)
{
if (!self.shouldIgnore(name)) {
const new_path = try std.fs.path.join(self.allocator, &.{
dir, name,
});
defer self.allocator.free(new_path);
self.addSingleWatch(new_path) catch {};
}
}
// build full path and debounce
if (name.len > 0 and !self.shouldIgnore(name)) {
const full_path = try std.fs.path.join(self.allocator, &.{
dir, name,
});
defer self.allocator.free(full_path);
try self.debouncer.recordEvent(full_path);
}
offset += @sizeOf(linux.inotify_event) + event.len;
}
}
};
The IN.ISDIR flag in the event mask tells us whether a newly created entry is a directory. When we detect a new directory, we immediately add a watch for it -- otherwise files created inside it would be invisible to us. This is a race condition techincally: files could be created in the new directory between it appearing and us adding the watch. In practice the window is microseconds and it rarely matters, but production file watchers sometimes do a directory scan after adding the watch to catch anything they missed.
The ignore patterns are essential for real use. Without them, watching a project directory would also watch .git/ (which churns constantly during operations), node_modules/ (potentially tens of thousands of directories), build output directories, and other things you don't care about. The shouldIgnore check runs on every directory name during the initial scan AND on every event. Keeping the check fast matters.
Let's put everything together into a tool that watches your project's source directory and runs zig build whenever something changes. This is the killer application for file watching in development:
const std = @import("std");
const linux = std.os.linux;
const AutoRebuilder = struct {
allocator: std.mem.Allocator,
inotify_fd: i32,
wd_map: std.AutoHashMap(i32, []const u8),
build_command: []const []const u8,
watch_extensions: []const []const u8,
last_build_time: i64,
cooldown_ms: i64,
fn init(
allocator: std.mem.Allocator,
build_cmd: []const []const u8,
extensions: []const []const u8,
) !AutoRebuilder {
const fd = linux.inotify_init1(linux.IN.NONBLOCK | linux.IN.CLOEXEC);
if (@as(isize, @bitCast(@as(usize, fd))) < 0)
return error.InotifyInitFailed;
return .{
.allocator = allocator,
.inotify_fd = @intCast(fd),
.wd_map = std.AutoHashMap(i32, []const u8).init(allocator),
.build_command = build_cmd,
.watch_extensions = extensions,
.last_build_time = 0,
.cooldown_ms = 200,
};
}
fn deinit(self: *AutoRebuilder) void {
var it = self.wd_map.valueIterator();
while (it.next()) |v| self.allocator.free(v.*);
self.wd_map.deinit();
std.posix.close(@intCast(self.inotify_fd));
}
fn addWatchRecursive(self: *AutoRebuilder, path: []const u8) !void {
const c_path = try self.allocator.dupeZ(u8, path);
defer self.allocator.free(c_path);
const mask = linux.IN.MODIFY | linux.IN.CREATE | linux.IN.DELETE |
linux.IN.MOVED_FROM | linux.IN.MOVED_TO;
const wd = linux.inotify_add_watch(self.inotify_fd, c_path, mask);
if (@as(isize, @bitCast(@as(usize, wd))) >= 0) {
const copy = try self.allocator.dupe(u8, path);
try self.wd_map.put(@intCast(wd), copy);
}
// recurse into subdirectories
var dir = std.fs.cwd().openDir(path, .{ .iterate = true }) catch return;
defer dir.close();
var iter = dir.iterate();
while (try iter.next()) |entry| {
if (entry.kind != .directory) continue;
// skip hidden dirs, zig-cache, zig-out
if (entry.name[0] == '.') continue;
if (std.mem.eql(u8, entry.name, "zig-cache")) continue;
if (std.mem.eql(u8, entry.name, "zig-out")) continue;
const sub = try std.fs.path.join(self.allocator, &.{ path, entry.name });
defer self.allocator.free(sub);
try self.addWatchRecursive(sub);
}
}
fn matchesExtension(self: *AutoRebuilder, name: []const u8) bool {
for (self.watch_extensions) |ext| {
if (name.len > ext.len and std.mem.eql(u8, name[name.len - ext.len ..], ext)) {
return true;
}
}
return false;
}
fn runBuild(self: *AutoRebuilder) !void {
const stdout = std.io.getStdOut().writer();
try stdout.print("\x1b[33m--- rebuild triggered ---\x1b[0m\n", .{});
var child = std.process.Child.init(self.build_command, self.allocator);
child.stderr_behavior = .Inherit;
child.stdout_behavior = .Inherit;
try child.spawn();
const result = try child.wait();
if (result.Exited == 0) {
try stdout.print("\x1b[32m--- build succeeded ---\x1b[0m\n", .{});
} else {
try stdout.print("\x1b[31m--- build failed (exit {d}) ---\x1b[0m\n", .{
result.Exited,
});
}
self.last_build_time = std.time.milliTimestamp();
}
fn run(self: *AutoRebuilder) !void {
const stdout = std.io.getStdOut().writer();
try stdout.print("Watching for changes... (press Ctrl+C to stop)\n", .{});
// initial build
try self.runBuild();
while (true) {
var buf: [4096]u8 align(@alignOf(linux.inotify_event)) = undefined;
const n = linux.read(@intCast(self.inotify_fd), &buf, buf.len);
if (@as(isize, @bitCast(n)) > 0) {
var triggered = false;
var offset: usize = 0;
while (offset < n) {
const event: *const linux.inotify_event = @alignCast(
@ptrCast(&buf[offset]),
);
if (event.len > 0) {
const name = std.mem.sliceTo(
@as([*:0]const u8, @ptrCast(
&buf[offset + @sizeOf(linux.inotify_event)],
)),
0,
);
if (self.matchesExtension(name)) {
triggered = true;
}
// add watch for new directories
if (event.mask & linux.IN.CREATE != 0 and
event.mask & linux.IN.ISDIR != 0)
{
const dir = self.wd_map.get(event.wd) orelse "";
const new_path = try std.fs.path.join(
self.allocator,
&.{ dir, name },
);
defer self.allocator.free(new_path);
self.addWatchRecursive(new_path) catch {};
}
}
offset += @sizeOf(linux.inotify_event) + event.len;
}
// debounce: only rebuild if cooldown has elapsed
if (triggered) {
const now = std.time.milliTimestamp();
if (now - self.last_build_time >= self.cooldown_ms) {
try self.runBuild();
}
}
}
// sleep briefly to avoid busy-waiting in non-blocking mode
std.time.sleep(10 * std.time.ns_per_ms);
}
}
};
pub fn main() !void {
var gpa = std.heap.GeneralPurposeAllocator(.{}){};
defer _ = gpa.deinit();
const allocator = gpa.allocator();
const build_cmd = &[_][]const u8{ "zig", "build" };
const extensions = &[_][]const u8{ ".zig", ".zon" };
var rebuilder = try AutoRebuilder.init(allocator, build_cmd, extensions);
defer rebuilder.deinit();
try rebuilder.addWatchRecursive("src");
try rebuilder.run();
}
The auto-rebuilder ties together everything from this episode. It watches recursively, filters by file extension (so editing a README doesn't trigger a rebuild), debounces with a 200ms cooldown, automatically picks up new directories, and skips zig-cache and zig-out to avoid watching build artifacts. The ANSI escape codes give colored output -- yellow for "rebuilding", green for success, red for failure. The std.process.Child API spawns the build command as a child process with inherited stdout/stderr so you see the compiler output directly.
The 10ms sleep in the main loop is a compromise. In a production tool you'd use epoll (or poll/select) to block until the inotify fd has data, which is more efficient. But for a development tool where we're already spending hundreds of milliseconds on compilation, the 10ms poll overhead is negligible. Sometimes simpler is better ;-) Having said that, if you wanted to make this really clean you could use std.posix.poll on the inotify fd -- we covered poll semantics implicitly in the TCP server episodes (51 and 52).
Add a --filter flag to the AutoRebuilder that accepts a list of directories to exclude from watching (on top of the hardcoded zig-cache and .git). Parse the flag from std.process.argsAlloc, store the exclusion list, and test it by creating a directory structure where one branch should be watched and another should be ignored. Verify that changes in the excluded directory do NOT trigger a rebuild.
Implement a file change log that writes events to a JSON file. Each entry should have a timestamp, event type (create/modify/delete/rename), file path, and file size (if applicable). Use the AtomicWriter pattern from episode 62 so the log file is never corrupted if the watcher crashes mid-write. Write a test that generates events and verifies the log contents.
Build a simple file sync tool that watches a source directory and copies changed files to a destination directory, preserving the directory structure. When a file is modified or created, copy it to the matching path in the destination. When a file is deleted, delete it from the destination. Handle the case where the destination directory structure doesn't exist yet (create intermediate directories). Test with a temp directory setup.
stat in a loop works but wastes CPU cycles, especially when watching many files -- the OS already knows when files changeswitch (builtin.os.tag) lets you write a single FileWatcher type that uses the right backend on each platform, with zero runtime overheadMOVED_FROM + MOVED_TO) with a shared cookie -- you need to buffer and match them, with a timeout for moves to unwatched directoriesshouldIgnore filter for .git, node_modules etc is essentialFile watching is one of those OS interfaces that sounds simple but has a lot of subtlety when you dig in. The rename pairing, the recursive directory management, the debouncing -- real file watchers like fswatch, watchman, and chokidar all implement these same patterns because the underlying kernel APIs are relativly primitive. Zig gives us the tools to work with those APIs directly, without layers of abstraction hiding what's actually happening.
Next time we're going deeper into OS interaction. The concepts we've been building -- file descriptors, kernel events, process management -- all feed into what's coming.
Bedankt en tot de volgende keer!
