FFI & Memory Model

Last Updated March 24, 2026

When you bypass traditional serialization and pass raw memory pointers across language boundaries (FFI), you inherit a massive responsibility: Memory Management.

JavaScript and Python use Garbage Collection (GC) to automatically clean up unused objects. Rust, however, has no Garbage Collector; it uses a strict Ownership model evaluated at compile time.

So, when Rust creates an OCP Buffer and passes its pointer to Node.js or Python, who owns that memory, and who is responsible for freeing it?

The Ownership Dilemma

If Rust drops (deallocates) the memory too early, the host language will try to read a dangling pointer, resulting in a catastrophic Segmentation Fault that instantly crashes your server or application.

If the host language forgets to tell Rust to drop the memory, the memory stays allocated forever. Parse enough MDX files, and your server will eventually run out of RAM (a classic Memory Leak).

The Omni-MDX Memory Lifecycle

To guarantee memory safety and zero leaks, Omni-Core implements a deterministic, hybrid memory model based on the host language’s capabilities.

Here is the exact lifecycle of an Omni-MDX payload:

1. Allocation (Rust)

When you call parse(), Rust allocates a Vec<u8> on its heap to store the OCP Binary payload. To prevent Rust’s ownership rules from instantly destroying this vector when the function ends, we use Box::into_raw(). This effectively “leaks” the memory intentionally, converting it into a raw C-pointer (*mut u8).

2. The Handoff (FFI)

Rust returns this raw pointer, along with the length of the buffer, to the host language. The host creates a zero-copy view (like a Uint8Array in JS or a memoryview in Python) that points directly to this address.

3. Automatic Deallocation (Host-Driven)

The host language is now effectively “borrowing” the memory. But how do we ensure it gets cleaned up? We tie the Rust memory pointer to the lifecycle of a native host object.

In Python (PyO3):
The memory view is wrapped in a Python class. When the Python Garbage Collector notices that the AST object is no longer used, it calls the __del__ method. Inside this method, we call a specific Rust FFI function (free_ocp_buffer(ptr, len)) that reconstructs the Vec<u8> from the pointer and safely drops it.

In JavaScript / WebAssembly:
We utilize the modern FinalizationRegistry API. When the JavaScript engine garbage-collects the JS object holding the AST, the registry fires a callback. This callback calls the exported WASM function free_buffer(ptr, len), returning the memory to the WASM allocator.

Safety Guarantees

By intertwining Rust’s manual memory management with the host’s Garbage Collector, Omni-Core achieves the holy grail of FFI:

• Zero-Copy Performance: Data is never cloned.
• Zero Memory Leaks: The host GC deterministically triggers Rust’s deallocation.
• Zero Segfaults: The host retains a strong reference to the wrapper object as long as the data is needed, preventing Rust from freeing the memory prematurely.