Evaluation

The evaluator (crates/jlens-core/src/eval.rs) is a tree-walking interpreter over the processed syntax tree. It is a function-by-function port of src/jsonata.js in jsonata-js. That code effectively defines JSONata’s subtler behaviour, such as when a result is a sequence or an array, so jlens keeps its structure recognisable. Unlike jsonata-js, evaluation is synchronous.

Expression::evaluate(input, bindings)
  └─ eval::evaluate          converts input JSON, wraps an input array, binds $$ and bindings
       └─ Evaluator::eval    one call per node: dispatch, predicates, group-by, sequence unwrapping
            ├─ path / step / tuple_step / filter / stages      location paths
            ├─ binary / array_constructor / group / sort       operators and constructors
            └─ call / apply / partial / apply_expression       functions

Values

See ADR 0003 for the reasoning. The short version:

Sequences

A sequence is the result of navigating a path. After evaluating any node, a sequence of length 0 becomes undefined, and a sequence of length 1 becomes its only item, unless KEEP_SINGLETON is set by the [] modifier. Arrays that come from the input, and arrays built with [...], are not sequences, so they never collapse. This single rule, applied at the end of Evaluator::eval, is why Account.Order.Product.Price returns a number for one product and an array for several.

When the input document is itself an array, it is wrapped in a one-item sequence flagged OUTER_WRAPPER, so that the array counts as a single context value. $ unwraps it again.

Paths and tuple streams

path evaluates its steps left to right. Each step maps over the current sequence and flattens the results by one level. Array constructors flagged CONS are not flattened.

Steps that bind variables switch the path into tuple-stream mode from that step on. These are @$focus, #$index, and steps marked as the ancestor for a %. In this mode each item is carried as a tuple, an object holding @ (the current value) and the bound variables. Later steps, predicates, sort keys and group-by expressions are evaluated in a frame created from the tuple. At the end of the path the tuples are mapped back to their @ values, unless an enclosing path still needs the ancestry.

Functions and the trampoline

A lambda value captures its definition (Arc<LambdaDef>, shared with the tree), the environment frame in which it was created, and the context value. Applying it:

1. validates the arguments against the lambda’s signature, if it has one;
2. creates a child frame of the captured environment and binds the parameters (missing ones are bound to undefined, so they shadow outer variables);
3. evaluates the body.

When post-processing marked a call as being in tail position, the body evaluates to a thunk instead of making the call. Evaluator::apply runs thunks in a loop, so tail-recursive functions run in constant stack depth. $loop(100000, 0) in the test suite exercises this.

Built-in functions are Native entries in functions/mod.rs, each with its signature and declared parameters. Host functions (HostFunction, from the bindings) have an optional signature and an arity; their arguments and results are converted to and from JSON (ADR 0006). Partial application ($f(?, 2)) works for lambdas, built-ins and host functions.

Signature validation

Signatures such as <s-nn?:s> are checked by turning the arguments into a string of type symbols (s, n, b, l, a, o, f, or m for undefined). That string is matched against the parameter patterns by a small greedy backtracking matcher. The matcher is equivalent to the regular expression jsonata-js builds, but needs no regex engine. The match then:

• fills in the context value for - parameters (T0411 if its type is wrong);
• wraps single values for a parameters;
• checks array subtypes (T0412).

Environments and memory

Frames form a chain from the innermost scope to the root; the built-ins sit behind the root. A closure stored in the frame that it captured creates a reference cycle, as in $f := function(){ $f() }. Reference counting cannot reclaim such a cycle on its own. The evaluator therefore records every frame that receives a function, array or object binding, and clears those frames when the evaluation ends.

Guardrails

jsonata-js has no depth limit of its own; it stops when the JavaScript stack overflows. jlens needs limits so that it fails with an error instead of crashing the process.

• Native targets. The parser, the post-processing pass, the evaluator and the schema typing walker grow their stack on demand (with the stacker crate), so the default depth limit is generous. The CLI also runs on a thread with a 64 MiB stack, which covers the other recursive walks of the tree (printing it as JSON, for example).
• WebAssembly. The stack cannot grow, and an overflow traps, which leaves the module instance unusable. jlens links the module with a 4 MiB stack and checks the space left before each recursive step (jlens_core::stack), failing with D1011 (evaluation) or U1001 (parsing) instead. WebAssembly frames also use the JavaScript engine’s own stack, which is about 1 MiB in V8 and cannot be measured from inside the module; the lower default depth limit (500) keeps evaluations inside it. The WebAssembly guide has the measurements behind it. A larger maxDepth can raise a JavaScript RangeError and break the instance.
• Calls from native functions. A lambda whose body is a tail call leaves no expression on the stack, so a recursion through a higher-order function (function($n){ $map([$n], function($v){ $f($v) }) }) deepens only through the native function’s calls back. Each of those calls counts as a level of depth and checks the stack too.

The nesting limit bounds every recursive walk of the syntax tree, including the ones that do not grow the stack, such as cloning and dropping it. Both nested subexpressions and the operators of a chain count: 1 + 2 + 3 is a tree two levels deep, built by a loop rather than by recursion. 1,000 levels is far more than any hand-written expression uses.

The size limits stop an expression that doubles a string or an array on each step, or a path whose steps multiply the length of a sequence, before it runs out of memory, which would abort the process. They are checked by the operators that build strings and arrays (&, ranges, array constructors, path steps) and on the value that each built-in or host function returns; $join checks the length before joining. The operators check before they build the value; a function’s result is checked once it has returned. * and **, which flatten nested arrays, count the items as they go.

The string and array limits apply to each string and array on its own, and values are shared rather than copied: an object bound to a variable can be nested in a new object twice, then that object twice, and so on. Thirty such steps stand for about a billion copies of the first object while using little memory, until something writes them all out. So everything that writes a value out is limited too:

• The result, which becomes JSON when the evaluation ends, is measured first: its size is the length of its compact JSON text, counting a shared value each time it appears. The walk stops once the size passes Options::max_result_size, and the evaluation fails with U1001, before anything is converted. Escaped characters count as one byte, so the size is slightly below the length of the text that format_json prints.
• The arguments of a host function are measured together in the same way, and so is its context value (HostCall::input).
• $string, $clone (and the transform operator, which clones its input) and & write JSON text, which is a string: the text is checked against Options::max_string_length as it is written.
• The value of an error (T2001, T0410 and others) is left out when it is larger than the default size of a result.

Tracing hooks

The evaluator reports its work to an optional Tracer; ADR 0004 records why. The hooks sit at these points:

When no tracer is attached, eval calls the untraced code directly. The traced variant is a separate function marked cold, so the hot path does not change. The audit guide describes the tracers built on these hooks.