NativeSpanFeed

Zero-copy streaming interface for processing terminal output

Overview

NativeSpanFeed is a zero-copy wrapper over native Zig memory for efficient streaming of terminal output. It manages chunks of data in native memory and provides spans (slices) to TypeScript without copying.

This is not a full stream interface - it’s optimized for zero-copy access to native memory buffers.

Creating a Stream

NativeSpanFeed.create · function

Create a new native span feed stream

static create(options?: NativeSpanFeedOptions): NativeSpanFeed

options · NativeSpanFeedOptions

Stream configuration options

NativeSpanFeed.attach · function

Attach to an existing native stream by pointer

static attach(
  streamPtr: bigint | number,
  options?: NativeSpanFeedOptions
): NativeSpanFeed

streamPtr · bigint | number · required

Native pointer to existing stream

Options

NativeSpanFeedOptions · object

Configuration for span feed behavior

interface NativeSpanFeedOptions {
  initialChunkSize?: number
  maxChunkSize?: number
  chunkGrowthPolicy?: GrowthPolicy
}

initialChunkSize · number

Initial size for memory chunks

maxChunkSize · number

Maximum size for memory chunks

chunkGrowthPolicy · GrowthPolicy

How chunks grow when more space is needed

Event Handlers

onData · function

Register a handler for data availability

onData(handler: DataHandler): () => void

handler · (data: Uint8Array) => void | Promise<void> · required

Callback invoked when data is available. Receives a zero-copy view into native memory.

Returns: Cleanup function to unregister the handler

The Uint8Array provided to your handler is a view into native memory. Do not retain references to it after the handler returns. If you need the data later, copy it.

onError · function

Register an error handler

onError(handler: (code: number) => void): () => void

handler · (code: number) => void · required

Callback invoked on error with error code

Returns: Cleanup function to unregister the handler

Stream Control

drainAll · function

Manually drain all available data

drainAll(): void

Calls registered data handlers for all pending spans. Usually not needed as data is drained automatically when handlers are registered.

close · function

Close the stream and free resources

close(): void

After calling close(), the stream cannot be reused. Do not call any methods on a closed stream.

Properties

streamPtr · Pointer

Native pointer to the underlying stream

Types

DataHandler

type DataHandler = (data: Uint8Array) => void | Promise<void>

Data handlers can be synchronous or asynchronous. Async handlers keep the underlying memory chunk pinned until the promise resolves.

GrowthPolicy

Controls how memory chunks grow:

• "linear" - Add fixed amount each time
• "exponential" - Double size each time
• "fibonacci" - Use Fibonacci sequence

NativeSpanFeedStats

interface NativeSpanFeedStats {
  totalBytesWritten: number
  totalChunks: number
  activeChunks: number
  drainedSpans: number
}

Example: Basic Usage

import { NativeSpanFeed } from "@opentui/core"

// Create stream with custom options
const stream = NativeSpanFeed.create({
  initialChunkSize: 4096,
  maxChunkSize: 1024 * 1024,
  chunkGrowthPolicy: "exponential"
})

// Register data handler
const unsubscribe = stream.onData((data) => {
  // Process the data
  const text = new TextDecoder().decode(data)
  console.log("Received:", text)
  
  // Don't keep references to 'data' after this returns!
})

// Register error handler
stream.onError((code) => {
  console.error("Stream error:", code)
})

// When done
unsubscribe()
stream.close()

Example: Async Handler

const stream = NativeSpanFeed.create()

stream.onData(async (data) => {
  // The chunk is pinned while this promise is pending
  const text = new TextDecoder().decode(data)
  await processTextAsync(text)
  // Chunk is released after promise resolves
})

Example: Copy Data for Later Use

const chunks: Uint8Array[] = []

stream.onData((data) => {
  // Must copy if you want to retain the data
  const copy = new Uint8Array(data.length)
  copy.set(data)
  chunks.push(copy)
})

Memory Management

The native span feed uses a sophisticated memory management system:

1. Chunks: Data is allocated in chunks in native memory
2. Spans: Each data callback receives a span (slice) of a chunk
3. Reference Counting: Chunks are reference counted and freed when no longer needed
4. Zero-Copy: Data is never copied from native memory to JavaScript unless you explicitly copy it

Lifecycle

1. Data is written to a chunk in native memory
2. When data is available, the stream emits a “data available” event
3. Your handler receives a Uint8Array view into the chunk
4. After your handler returns (or async handler resolves), the reference count is decremented
5. When a chunk’s reference count reaches zero, it’s freed

Critical: The Uint8Array passed to your handler is only valid during the handler execution. Do not store references to it. If you need the data later, copy it to a new array.

Performance Tips

• Use synchronous handlers when possible (async handlers keep chunks pinned longer)
• Process data immediately in the handler rather than queuing it
• If you must store data, copy only what you need
• Consider using drainAll() if you want to batch process multiple spans
• Unregister handlers when no longer needed to avoid memory leaks