# Embedding Deft
Deft is designed to be embedded in a host application as a scripting
engine. There are two API surfaces, both backed by the same code:
- **C API** — link `libdeft.a`, `#include "deft.h"`. Values are passed
as `uint64_t` (NaN-boxed). Use this from C/C++/Rust/any language with
a C FFI.
- **Zig API** — `@import` the `kernel` module and use the `Kernel`
struct directly. Values are the native `Value` union. Use this when
your host is itself a Zig project (you get richer types and can
register native functions without crossing an ABI boundary).
Both APIs expose the same operations: create a VM, evaluate source,
call functions by name, read/write globals, register host functions,
and enforce resource limits.
## Architecture
```
┌─────────────────────────────────────────────┐
│ Host application (C, C++, Rust, Zig, ...) │
│ │
│ deft.h / deft_vm_t* Kernel struct │
│ │ │ │
│ ▼ ▼ │
│ ┌─────────────────┐ ┌────────────────────┐ │
│ │ src/host/ffi/ │ │ src/kernel/ │ │
│ │ capi.zig │──▶│ kernel.zig │ │
│ │ (C ABI shim) │ │ (embedding API) │ │
│ └─────────────────┘ └────────┬───────────┘ │
│ │ │
│ ┌──────────────▼──────────┐ │
│ │ VM + compiler + parser │ │
│ │ + GC heap (NaN-boxing) │ │
│ └─────────────────────────┘ │
└─────────────────────────────────────────────┘
```
The C shim (`capi.zig`) is a thin wrapper over `Kernel`. Every
`deft_*` function maps 1:1 to a `Kernel` method, so behaviour is
identical across both surfaces.
---
## Building the Library
### C / static library
```
$ zig build lib # builds + installs libdeft.a and deft.h
$ zig build # installs the vendored dependency libs too
```
`zig build lib` produces:
```
zig-out/lib/libdeft.a
zig-out/include/deft.h
```
`libdeft.a` is built position-independent (`-fPIC`) so it can link
into a PIE executable or shared library. It **references** — but does
not bundle — the vendored SQLite, PCRE2, and mbedTLS static libraries.
Those land in `zig-out/lib/` when you run the default `zig build`
(which builds the executables and their dependency artifacts):
```
zig-out/lib/libdeft.a libsqlite3.a libpcre2.a libmbedtls.a
```
Link your program against all four, `deft` first then its deps:
```
$ cc -o myapp myapp.c -Izig-out/include -Lzig-out/lib -ldeft \
-lsqlite3 -lpcre2 -lmbedtls -lm -lpthread
```
> **Build gap:** the `lib` step currently only installs `libdeft.a`
> and the header; it does not yet install the dependency archives it
> needs to link. Running the default `zig build` (or any executable
> target) alongside `lib` is what populates `zig-out/lib/` with the
> vendored deps. A self-contained `lib` install step is a known
> pending improvement.
### Cross-compiling
Target the usual Zig triples:
```
$ zig build lib -Dtarget=x86_64-linux-gnu
$ zig build lib -Dtarget=aarch64-linux-gnu
$ zig build lib -Dtarget=x86_64-linux-musl # fully static
```
### Zig dependency
If your host is a Zig project, add `deft_zig` to your `build.zig.zon`:
```zig
.dependencies = .{
.deft_zig = .{
.url = "https://...",
.hash = "...",
},
},
```
Then in `build.zig`:
```zig
const deft = b.dependency("deft_zig", .{
.target = target,
.optimize = optimize,
});
const mod = deft.module("deft_zig"); // the root module
exe_mod.addImport("deft_zig", mod);
```
And in your code:
```zig
const deft = @import("deft_zig");
const Kernel = deft.kernel.Kernel;
```
---
## The C API
The full header is `include/deft.h`. Everything operates on an opaque
`deft_vm_t*` handle and passes values as `uint64_t`.
### Lifecycle
```c
deft_vm_t *vm = deft_vm_init(print_fn, import_fn, max_fuel);
// ... use vm ...
deft_vm_deinit(vm);
```
| Argument | Type | Purpose |
|---|---|---|
| `print_fn` | `deft_print_fn` | Receives `echo`/`puts` output. `NULL` = stdout. |
| `import_fn` | `deft_import_fn` | Resolves `import` paths to source. `NULL` = filesystem. |
| `max_fuel` | `uint64_t` | Operation budget. `0` = unlimited. |
Returns `NULL` on allocation failure.
#### Print callback
```c
void my_print(const char *data, size_t len) {
fwrite(data, 1, len, stdout); // or your own log/buffer
}
deft_vm_t *vm = deft_vm_init(my_print, NULL, 0);
```
The callback receives raw bytes — **not** null-terminated. Use the
`len` argument; `printf("%s", data)` is unsafe.
#### Import callback
Intercept `import "path" as alias` so the embedded VM never touches
the filesystem:
```c
const char *my_import(const char *path, size_t path_len, size_t *out_len) {
if (memcmp(path, "mathy", path_len) == 0) {
static const char src[] = "def pi 3.14159";
*out_len = sizeof(src) - 1;
return src;
}
return NULL; // not found
}
deft_vm_t *vm = deft_vm_init(NULL, my_import, 0);
```
The returned pointer must remain valid until the next import call or
`deft_vm_deinit`.
### Evaluation
```c
uint64_t deft_eval(deft_vm_t *vm, const char *source, size_t len);
uint64_t deft_eval_cstr(deft_vm_t *vm, const char *source);
```
Evaluate a source string. `deft_eval` takes an explicit length (the
source need not be null-terminated); `deft_eval_cstr` is the
convenience wrapper for C strings.
Returns the result as a NaN-boxed `uint64_t`, or `DEFT_NIL` on error.
```c
uint64_t r = deft_eval_cstr(vm, "(1 + 2)");
if (deft_is_number(r)) {
printf("%f\n", deft_get_number(r)); // 3.0
}
```
### Calling functions
```c
uint64_t deft_call(deft_vm_t *vm, const char *name,
const uint64_t *args, size_t nargs);
```
Call a Deft function by name with pre-built argument values. Useful
when the host holds function definitions in the VM and wants to invoke
them repeatedly without re-parsing source.
```c
// Define a function, then call it from C
deft_eval_cstr(vm, "def greet {|name| str \"hello \" $name }");
uint64_t arg = deft_string(vm, "world", 5);
uint64_t result = deft_call(vm, "greet", &arg, 1);
size_t len = 0;
const char *s = deft_get_string(result, &len);
// s == "hello world"
```
### Value constructors and accessors
Values are NaN-boxed `uint64_t`. Always construct and inspect them
through the provided functions — never bit-cast by hand.
**Constructors:**
| Function | Makes |
|---|---|
| `deft_number(double)` | a number |
| `deft_string(vm, ptr, len)` | a GC-heap string (requires live `vm`) |
| `deft_bool(bool)` | a boolean |
| `deft_nil()` | the nil sentinel |
**Predicates:**
| Function | Returns |
|---|---|
| `deft_is_number(v)` | true if `v` is a number |
| `deft_is_string(v)` | true if `v` is a string |
| `deft_is_bool(v)` | true if `v` is a boolean |
| `deft_is_nil(v)` | true if `v` is nil |
**Accessors:**
| Function | Returns |
|---|---|
| `deft_get_number(v)` | the `double`, or `0.0` if not a number |
| `deft_get_bool(v)` | the `bool`, or `false` if not a boolean |
| `deft_get_string(v, &len)` | pointer to bytes, or `NULL` if not a string |
```c
uint64_t s = deft_string(vm, "hello", 5);
size_t len = 0;
const char *bytes = deft_get_string(s, &len); // valid until next GC
```
> **GC lifetime:** pointers returned by `deft_get_string` are valid
> only until the next GC cycle or `deft_vm_deinit`. Copy the bytes if
> you need them longer.
### Globals
```c
void deft_set_global(deft_vm_t *vm, const char *name, uint64_t val);
uint64_t deft_get_global(deft_vm_t *vm, const char *name);
```
Read/write top-level Deft bindings from the host. `deft_get_global`
returns `DEFT_NIL` for unknown names.
```c
// Push a value into the VM
deft_set_global(vm, "host_version", deft_string(vm, "1.2.3", 5));
// Read a value the script set
deft_eval_cstr(vm, "set computed ($x * 2)");
uint64_t v = deft_get_global(vm, "computed");
```
### Resource limits (fuel)
Fuel is an operation counter. Each VM instruction decrements it; when
it hits zero, execution aborts and the call returns `DEFT_NIL`. This
is the primary defence against untrusted or runaway scripts.
```c
deft_vm_t *vm = deft_vm_init(NULL, NULL, 1000000); // 1M ops budget
// ... eval untrusted script ...
uint64_t left = deft_remaining_fuel(vm); // check what's left
deft_set_fuel(vm, 5000000); // top up for the next run
```
| Function | Purpose |
|---|---|
| `deft_set_fuel(vm, n)` | Set the budget. `0` disables fuel (unlimited). |
| `deft_remaining_fuel(vm)` | Ops remaining. `0` if fuel is disabled. |
### A complete C example
```c
#include
#include "deft.h"
static void on_print(const char *data, size_t len) {
fwrite(data, 1, len, stdout);
}
int main(void) {
deft_vm_t *vm = deft_vm_init(on_print, NULL, 1000000);
if (!vm) { fprintf(stderr, "init failed\n"); return 1; }
// Feed in a value from the host
deft_set_global(vm, "greeting", deft_string(vm, "world", 5));
// Evaluate Deft source
uint64_t r = deft_eval_cstr(vm, "echo \"hello $greeting\"");
// Call a defined function
deft_eval_cstr(vm, "def double {|n^number| ($n * 2) }");
uint64_t arg = deft_number(21.0);
uint64_t doubled = deft_call(vm, "double", &arg, 1);
printf("doubled = %f\n", deft_get_number(doubled)); // 42.0
deft_vm_deinit(vm);
return 0;
}
```
Build (see [Building the Library](#building-the-library) for the
dependency libs):
```
$ zig build lib && zig build
$ cc -o demo demo.c -Izig-out/include -Lzig-out/lib -ldeft \
-lsqlite3 -lpcre2 -lmbedtls -lm -lpthread
```
---
## The Zig API
When the host is a Zig project, skip the C ABI and use `Kernel`
directly. You get the native `Value` type (not raw `u64`), can
register native functions as plain Zig fns, and avoid the
thread-local callback adapter dance.
```zig
const std = @import("std");
const deft = @import("deft_zig");
const Kernel = deft.kernel.Kernel;
const KernelConfig = deft.kernel.KernelConfig;
const Value = deft.value_mod.Value;
const DeftCtx = deft.vm.DeftCtx;
```
Note that `DeftCtx` lives at the `deft.vm` module level (it is
re-exported from the plugin ABI module), not as a member of the `VM`
struct.
### Configuration
`KernelConfig` controls the three host hooks:
```zig
pub const KernelConfig = struct {
print_fn: ?PrintFn = null, // stdout if null
import_fn: ?ImportFn = null, // filesystem if null
max_fuel: u64 = 0, // 0 = unlimited
max_call_depth: u32 = 0, // 0 = default (256)
};
```
The callback signatures:
```zig
pub const PrintFn = *const fn ([]const u8) void;
pub const ImportFn = *const fn (path: []const u8) ?[]const u8;
```
### Lifecycle and evaluation
```zig
var kernel = Kernel.init(allocator, .{
.max_fuel = 1_000_000,
});
defer kernel.deinit();
const result = try kernel.eval("(1 + 2)");
std.debug.print("{d}\n", .{result.asF64()}); // 3.0
```
> **Stable address:** `Kernel.init` returns by value, but the struct
> contains an `ArenaAllocator` that captures a pointer to itself.
> Keep the `Kernel` at a fixed address: assign it to a `var` binding
> (not pass-by-value after init) and call `eval`/`call` on `&kernel`.
> The first `eval` lazily wires the arena pointer. The C shim
> heap-allocates a `StableKernel` wrapper for this reason.
### Calling functions
```zig
try kernel.eval("def sq {|n| ($n * $n) }");
const r = try kernel.call("sq", &.{Value.number(7.0)});
std.debug.print("{d}\n", .{r.asF64()}); // 49.0
```
`call` returns `error.UndefinedFunction` if the name isn't bound,
`error.NotCallable` if it isn't a function, or `error.RuntimeError`
if execution faults.
### Registering host functions
This is the main advantage of the Zig API. Register a plain Zig fn as
a Deft stdlib function callable from scripts:
```zig
fn native_greet(ctx: *DeftCtx, args: []const Value) Value {
const name = ctx.str(0, args) orelse "stranger";
return ctx.newString(name); // returns a heap string
}
// name, arity (-1 = variadic), function pointer
kernel.registerFn("greet", 1, &native_greet);
```
Now Deft code can call it:
echo [greet "alice"] # prints alice
`DeftCtx` is the per-call context. It provides typed argument
extractors and value constructors:
| Method | Extracts / Builds |
|---|---|
| `ctx.str(i, args)` | `?[]const u8` (string arg) |
| `ctx.num(i, args)` | `?f64` (number arg) |
| `ctx.int(i, args)` | `?i64` (integer arg, range-checked) |
| `ctx.boolean(i, args)` | `?bool` |
| `ctx.kw(i, args)` | `?[]const u8` (keyword arg) |
| `ctx.newString(s)` | heap string `Value` |
| `ctx.newNumber(n)` | number `Value` |
| `ctx.newBool(b)` | bool `Value` |
| `ctx.newKeyword(k)` | heap keyword `Value` |
| `ctx.ok(v)` | `%Ok{:value v}` tagged map |
| `ctx.err(msg)` | `%Err{:message msg}` tagged map |
Extractors return `null` on count or type mismatch — handle that as
an error with `ctx.err(...)`.
```zig
fn native_add(ctx: *DeftCtx, args: []const Value) Value {
const a = ctx.num(0, args) orelse return ctx.err("add: expected number");
const b = ctx.num(1, args) orelse return ctx.err("add: expected number");
return ctx.ok(ctx.newNumber(a + b));
}
```
For the richer `plugin.decl()` / `plugin.derive()` registration used
by the built-in stdlib functions (automatic arity derivation, param docs,
feature gating), see [Zig Plugins](01-zig-plugins.md). `registerFn`
is the lightweight path suitable for ad-hoc host functions; the
plugin system is for building a reusable stdlib function library.
### Globals
```zig
kernel.setGlobal("host_name", Value.fromF64(42.0));
const v = kernel.getGlobal("computed") orelse Value.nil;
```
`getGlobal` returns `?Value` — `null` if the name is unbound. It
resolves through the full var chain (locals → current namespace →
globals), so it finds both `def` and `set` bindings.
### Fuel
```zig
kernel.setFuel(5_000_000); // top up
const left = kernel.remainingFuel(); // ops remaining (0 if disabled)
```
### The Value type
`Value` is a NaN-boxed `u64` exposed as a Zig struct. Key operations:
```zig
// Construction (no allocation needed for scalars)
const n = Value.fromF64(3.14);
const b = Value.fromBool(true);
const nil = Value.nil;
// Heap values need the GC heap
const s = try deft.value_mod.makeString(&kernel.vm.heap, "hi");
const k = try deft.value_mod.makeKeyword(&kernel.vm.heap, "tag");
// Predicates
if (v.isNumber()) { ... }
if (v.isString()) { ... }
if (v.isBool()) { ... }
if (v.isNull()) { ... }
// Accessors (check the predicate first)
const f = v.asF64();
const bool_val = v.asBool();
const slice = v.stringSlice(); // []const u8
```
### A complete Zig example
```zig
const std = @import("std");
const deft = @import("deft_zig");
const Kernel = deft.kernel.Kernel;
const Value = deft.value_mod.Value;
const DeftCtx = deft.vm.DeftCtx;
fn native_celsius(ctx: *DeftCtx, args: []const Value) Value {
const f = ctx.num(0, args) orelse return ctx.err("celsius: expected number");
return ctx.newNumber((f - 32.0) * 5.0 / 9.0);
}
pub fn main() !void {
var kernel = Kernel.init(std.heap.page_allocator, .{ .max_fuel = 1_000_000 });
defer kernel.deinit();
kernel.registerFn("celsius", 1, &native_celsius);
// Call the host function from Deft source
const r = try kernel.eval("[celsius 98.6]");
std.debug.print("{d}\n", .{r.asF64()}); // 37.0
}
```
---
## NaN-Boxing
Deft values are **NaN-boxed**: every value — numbers, strings, maps,
closures, nil, bool — fits in a 64-bit word. The C API exposes this
as `uint64_t`; the Zig API wraps it in a `Value` struct whose sole
field is `.bits`.
Practical consequences:
- **No object pointers leak across the ABI.** You pass `uint64_t`s
around; the VM owns the heap.
- **Scalars are immediate.** Numbers (`f64`), bools, and nil need no
allocation — they're encoded directly in the bits.
- **Heap values are pointers in disguise.** A string/`Value` carries
a pointer to a GC-managed `ObjString`. That pointer is stable for
the value's lifetime, but the GC may move or collect the underlying
object — keep the `Value` (not a raw pointer you extracted from it)
if you need to hold a reference across calls.
The `DEFT_NIL` constant (`0x7FFC000000000001`) is the canonical nil.
Use `deft_nil()` / `Value.nil` rather than writing the literal.
---
## What's Available vs. In Progress
The embedding API covers the core loop: init, eval, call, globals,
fuel, host functions, and I/O callbacks. The following are **available
now**:
- Synchronous evaluation (`eval`, `eval_cstr`, `call`)
- All scalar and string value marshalling
- Host function registration (Zig API: `registerFn`)
- Resource limits (fuel, call depth)
- Pluggable print and import callbacks
- The pure/synchronous stdlib functions on the VM — `str`, `math/*`,
collections, `json/*`, `sqlite/*` (local files), regex, crypto.
These are registered automatically by `VM.init`.
The following are **not yet exposed through the embedding API**:
- **The host-platform layer.** The `Kernel` gives you a VM, not a
`Runtime`. Stdlib functions that depend on the Runtime — `pub/*`, `rpc/*`,
`config/*`, `runtime/*`, the HTTP server, async `task`s, `vwait` —
are registered as functions but will fail when called because no
Runtime is bound to their execution context. Embedding the full
Runtime (event loop, pub/sub, child runtimes) via C is not yet
wired; from Zig you can construct a `Runtime` directly.
- **The prelude.** `Kernel.init` does not load `.deft/prelude.dft`,
so `defmod`-based module namespaces and any prelude definitions are
absent unless the host evaluates them explicitly.
- **C-side host function registration.** There is no `deft_register_fn`
in `deft.h` yet — host functions can only be added from Zig (the
trampolines and signature-marshalling for arbitrary C fns are the
missing piece; track `src/host/ffi/capi.zig`).
- **Error introspection.** On failure, `eval`/`call` return `DEFT_NIL`
(C) or an `error{...}` (Zig). There is no structured error value
returned to the host — error detail goes to the print callback.
- **Async / coroutine control from the host.** Coroutines run inside
`eval` but the host cannot step or resume them across calls.
- **Object lifetime pinning from C.** The Zig FFI plugin's `ffi/pin`
exists, but the C embedding API has no equivalent; long-lived
references must be held as `Value`s and re-extracted per use.
If you need one of these, the underlying VM supports it — the gap is
purely in the embedding surface.
---
## Quick Reference
### C
```c
deft_vm_t *vm = deft_vm_init(print_fn, import_fn, fuel);
deft_vm_deinit(vm);
uint64_t r = deft_eval_cstr(vm, "(1 + 2)");
uint64_t r2 = deft_eval(vm, src, len);
uint64_t r3 = deft_call(vm, "fn", args, nargs);
uint64_t num = deft_number(3.14);
uint64_t str = deft_string(vm, "hi", 2);
uint64_t b = deft_bool(true);
uint64_t nil = deft_nil();
deft_is_number(r); deft_get_number(r);
deft_is_string(s); deft_get_string(s, &len);
deft_is_bool(b); deft_get_bool(b);
deft_is_nil(nil);
deft_set_global(vm, "x", val);
deft_get_global(vm, "x");
deft_set_fuel(vm, n);
deft_remaining_fuel(vm);
```
### Zig
```zig
var kernel = Kernel.init(allocator, .{ .max_fuel = 1_000_000 });
defer kernel.deinit();
const r = try kernel.eval("(1 + 2)");
const r2 = try kernel.call("fn", &.{ Value.number(1.0) });
kernel.registerFn("greet", 1, &native_greet);
kernel.setGlobal("x", Value.fromF64(42.0));
const v = kernel.getGlobal("x") orelse Value.nil;
kernel.setFuel(5_000_000);
_ = kernel.remainingFuel();
```