Skip to main content

MAX_YAML_EXPANDED_BYTES

Constant MAX_YAML_EXPANDED_BYTES 

Source
pub const MAX_YAML_EXPANDED_BYTES: usize = _; // 33_554_432usize
Expand description

Maximum total byte weight check_yaml_expansion allows a document to expand to (counting anchor/alias-driven duplication) before rejecting it.

yaml-rust2 0.12’s YamlLoader::on_event_impl deep-clones the whole anchored subtree once per Event::Alias reference (anchor_map.get(&id) => v.clone()), and again into anchor_map itself for every anchored node. Nesting depth (bounded by MAX_YAML_NESTING_DEPTH) is irrelevant to this: a shallow document with a handful of anchors, each aliased a handful of times, expands exponentially in the memory actually allocated. Critically, this must be a byte budget, not a node-count budget: a single large scalar anchor (e.g. a 1 MB string) aliased many times allocates megabytes per alias while costing only one node each, so a node-count budget lets it through cheaply — a document under 3 MB can exhaust hundreds of gigabytes this way. YamlLoader exposes no budget/config hook, so callers must reject pathological input before handing it to the loader.

32 MiB (32 * 1024 * 1024 = 33,554,432) is the charged byte budget — not an exact bound on YamlLoader’s real peak allocation. Charged bytes track YAML_NODE_OVERHEAD_BYTES’s per-node floor plus scalar content, which undercounts two real costs that floor doesn’t model: Hash’s LinkedHashMap prev/next link pointers and hash-table slots (hash-heavy documents, e.g. a pubspec.lock), and String/Vec capacity-doubling slack — the scanner builds every scalar via String::new() + repeated push, so a single large scalar whose length lands just past a power-of-two capacity boundary (e.g. 1,048,577 bytes) wastes nearly its own length again in unused capacity, and the same growth pattern applies to a Vec backing a long sequence. Measured with a counting allocator: real peak allocation runs about 1.16x-1.74x the charged total depending on document shape (steadier ~1.56x for hash-heavy lockfiles, up to ~2x for a large scalar or long sequence whose length lands right past a capacity-doubling boundary), so an accepted document charged right at this limit really allocates roughly 50-65 MB, not 32 MB. Bytes charged do not compound with nesting, so ~2x is the ceiling on that ratio, not a growing multiplier — the budget stays a bounded, linear function of input size either way, which is what actually matters for this guard (an unbounded multiplier, as with the pre-fix node-count budget’s O(2^depth) blowup, is the failure mode this guards against).

Measured against real payloads (exact charged totals, reproducible against the shape scaled up from test_check_yaml_expansion_few_hundred_package_lockfile_accepted): a 1.9 MB / 12,500-package synthetic pubspec.lock charges 12,416,870 bytes (2.70x headroom); a 757 KB / 5,000-package one charges 4,961,870 bytes (6.76x headroom); the doubling-chain attack payload (see check_yaml_expansion’s doctest) charges 16,907,046 bytes at N=14 (accepted) and 33,815,273 at N=15 (rejected, in well under a millisecond); and a single 1 MB anchor aliased 31 times (33,002,370 bytes charged, ~1 MB source) is accepted while 32 times (34,002,434 bytes) is rejected rather than allocating unboundedly.