Memory & ownership

Documentation · API · Migration

A collection is a frozen JavaScript handle over persistent nodes in a WebAssembly memory arena. An update allocates new storage and reuses unchanged nodes. The earlier handle keeps its root and remains readable.

Source map#

The runtime is kept in flat, focused modules. Tests sit next to the modules they exercise; browser examples and checks live under demo/.

The small legacy collection-specific .as.ts entry points are not separate current implementations. Read persistent-core.as.ts for the engine.

Storage choices#

Maps use persistent hash tries. Sets reuse map storage with encoded keys. Lists and queues use block vectors with tail storage; queues also track a read offset. Linked-list interfaces use indexed block sequences. Ordered maps keep a persistent insertion-order log. Sorted maps use a sorted-key index, not the red-black tree described by early documentation. Custom comparators sort entries locally during iteration. Priority queues use persistent heap nodes.

These choices have different costs. A scalar update can allocate path nodes, a string write needs encoding, and a JSON read can allocate decoded values. Repeated reads can hit a process-local cache. First-use construction includes memory and WASM-instance setup. None of these costs disappears because transport shares memory.

Ownership and publication#

Each arena has one allocating owner. Worker attachment creates a read-only view. The owner can create new versions while workers read old published nodes. Application messages publish complete snapshot descriptors; a reader does not follow the owner's latest JavaScript variable automatically.

The immutable contract applies to supported collection APIs. Allocation scratch, caches, and unpublished construction are mutable implementation details. Raw memory is writable by code that holds it. Shared memory is not an authentication or isolation boundary; use it only with trusted application workers.

The current transport format is 4. Producer and reader must use compatible bundles. A descriptor needs both its matching arena and the correct collection layout. A numerical root alone is not a portable reference or a saved collection.

Memory initialization and limits#

Importing the collection, state, or worker entry point does not allocate a default arena. A collection type creates its default arena when first used. Read-only workers attach the supplied memory without allocating an unused writer for each registered type.

A new writable arena starts with 128 KiB of backing memory and has a default maximum of 256 MiB. The maximum is a growth ceiling, not the initial buffer size. Some browser engines reserve address space for that ceiling at creation. Requesting many large ceilings can therefore fail even for small datasets.

Applications that need a different ceiling can configure future arenas in each allocating thread:

import { configureMemory, SharedList } from 'zerocopy';

configureMemory({ maximumBytes: 512 * 1024 * 1024 });
const events = new SharedList('number');

Call configureMemory() before that thread creates its collections. It does not resize existing arenas or change attached memory. The limit must be a whole number of 64 KiB pages, from 128 KiB through 0x7fff0000 bytes. Higher ceilings can still fail under browser or operating-system limits. Read-only copy attachments allocate only enough pages for the received snapshot.

The source-module legacy sharedMemory and sharedBuffer bindings are populated on first collection use, explicit reset, or attachment. Use collection handles and the transport APIs instead of reading those bindings before initialization.

Arena lifetime#

Arenas are append-only and have an allocation ceiling below 2 GiB. There is no per-node reclamation. Retaining a single snapshot can retain the full arena, including unreachable intermediate nodes. Nested dependencies can retain other arenas. Queues can retain consumed prefixes, and ordered collections can retain old log entries.

An arena becomes eligible for garbage collection only when every holder releases it. Holders include collection handles, worker views, transport payloads, selectors, nested dependencies, and module-default references. Garbage collection is not immediate or guaranteed at a particular time.

resetMap() and the other reset functions select fresh default arenas for future collections. They do not compact existing collections or invalidate earlier snapshots. dispose() and configureAutoGC() are deprecated no-ops; they do not free individual snapshots.

Compact at an application boundary#

import { SharedMap, compact, compactMany } from 'zerocopy';

const original = new SharedMap('number').set('headphones', 12);
const owned = compact(original);
const updated = owned.set('headphones', 0);
const group = compactMany({ original, updated });

original.get('headphones');       // 12
owned.get('headphones');          // 12
group.updated.get('headphones');  // 0

Compaction copies live data into a fresh writable arena. Group compaction can retain shared live blobs and nested snapshots within the copied group. The source remains valid. Compaction can temporarily retain both arenas and requires CPU time, so do not put it on every Redux dispatch or frame.

After replacing live state, release old history, worker views, and payloads when the application no longer needs them. Replace relevant default arena references where applicable. Dropping a local variable alone does not prove that all other holders are gone.

A store can start with compact(new SharedMap('number')) to give its data a separate arena lifetime. This is a deliberate allocation, not free initialization.

Measuring memory#

Distinguish allocated payload bytes, retained backing buffers, JavaScript heap, peak temporary memory, and total process RSS. The README measurements count post-GC heap plus full retained buffers, not payload bytes alone. Compaction results exclude the time and temporary peak of the copy.