# tywrap — Full Documentation > Complete tywrap documentation for AI agent frameworks. > See /llms.txt for the structured index. --- --- layout: home --- ## Try it Install the TypeScript package and Python generator. Then create a config and generate wrappers for the Python modules you want to call. ```bash npm install tywrap pip install tywrap-ir # Python component for code generation npx tywrap init npx tywrap generate ``` `tywrap` and `tywrap-ir` are versioned independently. Install the latest published release of each package unless you need to pin them explicitly. ```typescript import { NodeBridge } from 'tywrap/node'; import { setRuntimeBridge } from 'tywrap/runtime'; import * as math from './generated/math.generated.js'; setRuntimeBridge(new NodeBridge({ pythonPath: 'python3' })); const result = await math.sqrt(16); // 4 ``` See the [getting started guide](https://bbopen.github.io/tywrap/guide/getting-started) to configure your own Python module and choose a runtime bridge. > Experimental: APIs may change before v1.0.0. See [Releases](https://github.com/bbopen/tywrap/releases) for breaking changes. --- # Getting Started with tywrap This guide covers installation, basic configuration, and creating your first TypeScript wrapper for a Python library. ## Prerequisites - Node.js ≥20.0.0, Bun ≥1.1.0, or Deno ≥1.46.0 (Deno subprocess support is experimental and untested in CI) - Python 3.10+ with target libraries installed - Basic TypeScript knowledge ## Installation Choose your preferred package manager: ```bash # npm npm install tywrap # pnpm pnpm add tywrap # yarn yarn add tywrap # bun bun add tywrap # deno deno add npm:tywrap ``` Install the Python IR extractor in the environment that will run code generation: ```bash pip install tywrap-ir ``` `tywrap` and `tywrap-ir` are versioned independently. Install the latest published release of each package unless you need to pin them explicitly. ## Configuration Create a `tywrap.config.ts` in your project root (or run `npx tywrap init`): ```ts import { defineConfig } from 'tywrap'; export default defineConfig({ pythonModules: { math: { typeHints: 'strict', }, }, output: { dir: './generated', format: 'esm', declaration: false, sourceMap: false, }, runtime: { node: { pythonPath: 'python3', }, }, }); ``` ## Generation Run the tywrap CLI to generate TypeScript wrappers: ```bash # Using npx npx tywrap generate # Using the CLI directly (if installed globally) tywrap generate # Using programmatic API node -e "import('tywrap').then(async tw => { const cfg = await tw.resolveConfig({ configFile: './tywrap.config.ts' }); await tw.generate(cfg); })" ``` Run `tywrap generate --help` to see all available options and defaults. The CLI searches for `tywrap.config.ts`, `.mts`, `.js`, `.mjs`, `.cjs`, and `.json` when `--config` is not provided. The config file drives code generation. Your application still creates a runtime bridge with `NodeBridge`, `PyodideBridge`, or `HttpBridge`. ## Usage After generation, import and use your Python library with full TypeScript support: ```typescript // Import the generated wrapper import { NodeBridge } from 'tywrap/node'; import { setRuntimeBridge } from 'tywrap/runtime'; import * as math from './generated/math.generated.js'; const bridge = new NodeBridge({ pythonPath: 'python3' }); setRuntimeBridge(bridge); // Types are precise where tywrap can resolve them. async function example() { const result = await math.sqrt(16); // TypeScript knows this returns Promise const power = await math.pow(2, 3); // Full autocompletion and type checking console.log(`√16 = ${result}`); // 4 console.log(`2³ = ${power}`); // 8 } example().catch(console.error); ``` ## Runtime return validation Generated wrappers validate returns at runtime after decoding. If a Python function returns a value that does not match its annotation, tywrap throws `BridgeValidationError` with the wrapped call site and received shape. Fix the Python annotation or function. Use `-> Any` only when that return is intentionally untyped. ## Custom Module Example Create a wrapper for a custom Python module: ### Python Module Create `my_utils.py`: ```python """Custom utility functions with type hints.""" def greet(name: str, excited: bool = False) -> str: """Generate a greeting message.""" message = f"Hello, {name}!" return message + " 🎉" if excited else message def calculate_area(width: float, height: float) -> float: """Calculate rectangular area.""" return width * height def add(a: float, b: float, precision: int = 2) -> float: """Add two numbers.""" return round(a + b, precision) def multiply(a: float, b: float, precision: int = 2) -> float: """Multiply two numbers.""" return round(a * b, precision) ``` ### Configuration Update your `tywrap.config.ts`: ```json { "pythonImportPath": [], "pythonModules": { "my_utils": { "runtime": "node", "typeHints": "strict" } }, "output": { "dir": "./generated", "format": "esm", "declaration": true, "sourceMap": true } } ``` If `my_utils` is not importable from your current working directory or your Python environment, add the directory that contains it to `pythonImportPath`: ```json { "pythonImportPath": ["./python"], "pythonModules": { "my_utils": { "runtime": "node", "typeHints": "strict" } } } ``` ### Generate and Import ```bash npx tywrap generate ``` ```typescript import { NodeBridge } from 'tywrap/node'; import { setRuntimeBridge } from 'tywrap/runtime'; import { greet, calculate_area, add, multiply, } from './generated/my_utils.generated.js'; const bridge = new NodeBridge({ pythonPath: 'python3' }); setRuntimeBridge(bridge); async function demo() { // Function calls with type safety const greeting = await greet('World', true); console.log(greeting); // "Hello, World! 🎉" const area = await calculate_area(10.5, 8.2); console.log(`Area: ${area}`); // Area: 86.1 // Use value-returning module functions instead of live class handles. const sum = await add(3.14159, 2.71828, 3); const product = await multiply(sum, 2, 3); console.log(`Sum: ${sum}`); // Sum: 5.86 console.log(`Product: ${product}`); // Product: 11.72 } ``` ## Runtime Options ### Node.js (Default) ```json { "pythonModules": { "numpy": { "runtime": "node" } }, "runtime": { "node": { "pythonPath": "/usr/local/bin/python3", "virtualEnv": "./venv", "timeout": 30000 } } } ``` ### Browser (Pyodide) ```json { "pythonModules": { "numpy": { "runtime": "pyodide" } }, "runtime": { "pyodide": { "indexURL": "https://cdn.jsdelivr.net/pyodide/v0.28.0/full/", "packages": ["numpy", "scipy"] } } } ``` ### HTTP API ```json { "pythonModules": { "my_api": { "runtime": "http" } }, "runtime": { "http": { "baseURL": "https://api.example.com/python", "timeout": 10000, "headers": { "Authorization": "Bearer token" } } } } ``` ## Development Workflow (Hot Reload) Use `tywrap/dev` for development hot reload: wrapper regeneration plus bridge replacement. ```typescript import { startNodeWatchSession } from 'tywrap/dev'; import { NodeBridge } from 'tywrap/node'; const session = await startNodeWatchSession({ configFile: './tywrap.config.ts', createBridge: async config => new NodeBridge({ pythonPath: config.runtime.node?.pythonPath ?? 'python3', timeoutMs: config.runtime.node?.timeout ?? 30000, }), }); ``` For Pyodide, use `createBridgeReloader(...)` from `tywrap/dev` for manual bridge replacement. For HTTP, restart or redeploy the remote server outside tywrap. `startNodeWatchSession(...)` watches local package directories as directory trees and refreshes them when nested directories change. A structured generation failure leaves the last known good wrappers and bridge live. See [Watch & Reload](https://bbopen.github.io/tywrap/guide/dev-reload.md) for the reload lifecycle events and the full failure / recovery contract. ### Build Integration Run `tywrap generate --check` in CI to ensure generated wrappers are committed and up to date. Structured generation failures exit with code `1`; out-of-date generated files exit with code `3`. After upgrading Python dependencies, run `tywrap generate` to refresh the generated surface. If you ran `tywrap init` in a Node project, it will also add `tywrap:generate` and `tywrap:check` scripts to `package.json` (disable with `--no-scripts`). ## Performance Tips 1. Set `"caching": true` to reuse generation IR across rebuilds. 2. Limit `functions` and `classes` when a wrapper needs only part of a module. `batching` and `compression` are accepted config fields in 0.10.0, but no current generator or runtime code applies them. ```json { "pythonModules": { "numpy": { "functions": ["array", "zeros", "ones"], "classes": ["ndarray"] } }, "performance": { "caching": true, "batching": false, "compression": "none" } } ``` ## Common Patterns ### Error Handling ```typescript try { const result = await math.sqrt(-1); } catch (error) { if (error.message.includes('ValueError')) { console.error('Invalid input for sqrt'); } } ``` ### Working with Arrays ```typescript import { array, zeros } from './generated/numpy.generated.js'; // Assumes setRuntimeBridge(...) has been called during app startup. // Create arrays with proper typing const arr = await array([1, 2, 3, 4, 5]); const empty = await zeros([3, 3]); ``` ### Async/Await Best Practices ```typescript // Good: Handle promises properly const results = await Promise.all([ math.sin(Math.PI / 2), math.cos(0), math.tan(Math.PI / 4), ]); // Group independent calls with Promise.all const [sinValue, cosValue, tanValue] = await Promise.all([ math.sin(Math.PI / 2), math.cos(0), math.tan(Math.PI / 4), ]); ``` ## Next Steps - [Configuration Guide](https://bbopen.github.io/tywrap/guide/configuration) - Complete configuration reference - [Runtime Guides](https://bbopen.github.io/tywrap/guide/runtimes/node) - Platform-specific setup - [Examples](https://bbopen.github.io/tywrap/examples/) - Usage patterns and examples - [Troubleshooting](https://bbopen.github.io/tywrap/troubleshooting/) - Common issues and solutions ## Support - [Troubleshooting Guide](https://bbopen.github.io/tywrap/troubleshooting/) - [GitHub Issues](https://github.com/bbopen/tywrap/issues) - [GitHub Discussions](https://github.com/bbopen/tywrap/discussions) --- # Migrating to 0.11 This guide covers tywrap 0.11.0 and tywrap-ir 0.3.1. The documentation deployment requires both versions to be available on their public package registries. ## Large Python integers Python `int` results still use TypeScript `number`. The supported range is `-9007199254740991` through `9007199254740991`, inclusive. Earlier versions could round larger integers when JavaScript parsed the response. The new serializer rejects those values before writing JSON. This applies to root results, nested containers, supported NumPy scalars, and model dumps. For example, a result containing `{"items": [9007199254740993]}` now fails at `result.items[0]`. The error identifies the unsafe value and suggests an explicit string or an Arrow integer column. Return a string when the value represents an identifier or must remain an exact decimal integer: ```python def large_identifier() -> str: return str(9007199254740993) ``` The generated return type is `Promise`. Convert that string with JavaScript `BigInt` if your application needs integer arithmetic. Validate any string that comes from an untrusted source before using it. Keep integer columns in Arrow tables when you need the existing Arrow int64 representation. Arrow int64 remains exact and can decode as JavaScript `bigint`. JSON fallback does not provide that representation and rejects unsafe integers. Booleans and finite floats remain supported. Converting an integer to a Python `float` can lose precision before tywrap receives it. Use a string when exactness matters. Tagged bigint transport remains a separate design proposal. ## NumPy and Torch float16 The Arrow decoder now converts float16 storage words into their numeric values before reshaping the result. For example, `[15872, 49280]` becomes the intended `[1.5, -2.25]`. Remove application code that manually decodes those storage words. The conversion preserves signed zero, finite subnormal values, and finite extrema. NumPy arrays and nested Torch tensors use the same conversion. Non-finite results continue to reject under the existing policy. The scientific envelope format, JSON fallback, and Arrow int64 behavior stay compatible. Torch bfloat16 keeps its existing float32 transport conversion. ## Return types that cannot be validated An unresolved Python `object` result now produces `Promise`. Python objects can include strings and numbers, which TypeScript's `object` type excludes. An unresolved generic result also becomes `unknown`. Earlier wrappers could declare `Promise` while accepting any returned value. A generic input remains generic, but its type alone cannot prove that Python returned the same type. This also applies when the unresolved type parameter appears in a nested result. Generation reports the unresolved conversion. `--fail-on-warn` rejects that diagnostic. Use a supported concrete return annotation or overload when the Python API has a known result type. Otherwise, check the value before using it: ```ts const result = await generatedFunction(input); if (typeof result !== 'string') { throw new TypeError('Expected a string result'); } console.log(result.toUpperCase()); ``` The current value contract does not resolve every Python annotation. For example, `Literal` results become `unknown`, while their existing literal checks remain. Other unresolved returns retain available marker, container or record checks. These partial checks do not prove a precise TypeScript return type. Type-only Protocol methods keep their declared generic signatures. Supported string and integer overloads retain their corresponding return types. Use Python 3.11 or later to extract registered overloads. Python 3.10 keeps the implementation signature when its typing module cannot expose the overloads. ## Coroutine results The updated Node and Pyodide bridges await Python coroutine results. Generated wrappers use the function's resolved result type, such as `Promise`. Upgrade the generator and runtime together, then regenerate wrappers. An HTTP client still depends on its server. Use a compatible server that awaits Python results; upgrading the TypeScript client does not update that server. ## Regenerate wrappers Upgrade both packages: ```bash npm install tywrap@0.11.0 python3 -m pip install --upgrade tywrap-ir==0.3.1 ``` Use the Python executable selected by `runtime.node.pythonPath` and `runtime.node.virtualEnv` instead of `python3` when your configuration differs. The new extractor captures generic parameters used only in overload annotations. Its IR schema remains `0.4.0`, so the schema check alone cannot detect an older extractor or an older saved contract. Extract fresh IR when regenerating: ```bash npx tywrap generate --no-cache ``` This bypasses the on-disk IR cache. It does not replace existing cache entries. If you enable caching again, first remove the old generated IR files from `.tywrap/cache` so generation extracts and caches the updated contracts. If you use `contractInput`, temporarily omit it to extract fresh contracts with the updated Python package. Review each new `.contract.json` in your output directory. Replace the corresponding pinned input file with that contract, or point `contractInput` to the new file. Then restore the setting. The `--no-cache` option does not replace a pinned input contract. Regenerate wrappers with the upgraded generator. Python `int` return validators then check the safe integer range as well as the JavaScript value type. Regeneration also applies the reviewed callable-contract diagnostics. Update any tests that expected a rounded integer or float16 storage words. Assert the corrected value or the specific conversion error instead. Run your generated calls against the upgraded runtime before deploying them. An old wrapper cannot add the new generated return checks by itself, although the upgraded Python serializer still rejects unsafe integer results. --- # Agent Adoption Use `NodeBridge` for a TypeScript project that can start a local Python process. Use `PyodideBridge` for browser code. Use `HttpBridge` when Python runs on another host. See the [runtime comparison](https://bbopen.github.io/tywrap/guide/runtimes/comparison) for constraints such as Deno Deploy and Arrow support. ## Recipe Run these commands from the TypeScript project root. Replace `math` with the Python module the project must call. ```bash npm install tywrap ``` Expected output includes an added `tywrap` dependency in `package.json`. ```bash pip install tywrap-ir ``` Expected output ends with a successful `tywrap-ir` installation. This `pip` must install into the same Python environment configured below. ```bash npx tywrap init ``` Expected output: ```text Created /absolute/path/to/project/tywrap.config.ts ``` The command writes this starter configuration and adds `tywrap:generate` and `tywrap:check` scripts when `package.json` exists: ```ts import { defineConfig } from 'tywrap'; export default defineConfig({ pythonModules: { "math": { typeHints: 'strict' }, }, output: { dir: './generated', format: 'esm', declaration: false, sourceMap: false }, runtime: { node: { pythonPath: 'python3' } }, types: { presets: ['stdlib'] } }); ``` Edit `pythonModules` and `runtime.node.pythonPath` as needed. A local module also needs its parent directory in `pythonImportPath`. ```bash npx tywrap generate ``` Expected output for one module: ```text Generated: 2 files from 1 modules (warnings: 1). - generated/math.generated.ts - generated/math.contract.json ``` A nonzero warning count is normal for C modules such as `math`: functions without Python return annotations degrade to `unknown` and the warning names them. Generation still succeeds. The wrapper is `.generated.ts`. The pinned extractor output is `.contract.json`. Import the compiled wrapper, set the bridge once, then call the wrapper: ```ts import { NodeBridge } from 'tywrap/node'; import { setRuntimeBridge } from 'tywrap/runtime'; import * as math from './generated/math.generated.js'; setRuntimeBridge(new NodeBridge({ pythonPath: 'python3' })); const value = await math.sqrt(16); console.log(value); // 4 ``` Expected output: ```text 4 ``` ## Verify generated output Commit the wrapper and contract files, then check them in CI: ```bash npx tywrap generate --check ``` Expected output when current: ```text Generated wrappers are up to date. ``` Exit code `0` means all generated files match. Exit code `1` means generation failed, including a missing configuration or Python import failure. Exit code `2` means `--fail-on-warn` found warnings. Exit code `3` means generated files or contracts differ; run `npx tywrap generate` and commit the result. See the [CLI reference](https://bbopen.github.io/tywrap/reference/cli) for the complete command surface. ## Failure signatures | Error prefix or text | Fix | | --- | --- | | `No IR produced for module ... tywrap_ir failed.` with `ModuleNotFoundError` | Install the target Python module in the same interpreter named by `runtime.node.pythonPath`. For a local module, add its parent directory to `pythonImportPath`. | | `No IR produced for module` or `tywrap_ir not found on PYTHONPATH.` | Run `pip install tywrap-ir` in that configured Python environment, then rerun generation. | | `IR version mismatch:` or `ir-version-mismatch` | Upgrade `tywrap-ir` to match `tywrap`, then regenerate the contract. | | `contract-invalid` or `Contract ... is missing` | Replace the invalid or stale `contractInput` with a regenerated contract. | | `Generated wrappers are out of date:` | Run `npx tywrap generate`, review the wrapper and contract changes, and commit them. | | `Received an Arrow-encoded payload but no Arrow decoder is available.` | Run `npm install apache-arrow`, or set `TYWRAP_CODEC_FALLBACK=json` on the Python side when its documented JSON domain is acceptable. | | `JSON pandas.DataFrame encoding requires an unnamed RangeIndex` or `JSON pandas.Series encoding requires an unnamed RangeIndex` | Follow the recipe in the error, such as `.reset_index(drop=True)`, or use Arrow. Other JSON codec preflight errors also state their conversion recipe. Apply that recipe rather than suppressing the error. | | `Return validation failed for` (`BridgeValidationError`) | Compare the Python return value with the generated declared type. The message gives the call site and describes the expected type and received shape. Fix the annotation or returned value, then regenerate. | | `Response payload is ... exceeds TYWRAP_CODEC_MAX_BYTES=` | Raise `TYWRAP_CODEC_MAX_BYTES` only for a response size the process can handle, or return a smaller result. | --- # Configuration A `tywrap.config.ts` file configures wrapper generation. Use `defineConfig()` for editor completion, or use JSON, JavaScript, or TypeScript. Without `--config`, the CLI looks for `tywrap.config.ts`, `.mts`, `.js`, `.mjs`, `.cjs`, then `.json`. ```ts import { defineConfig } from 'tywrap'; export default defineConfig({ pythonModules: { math: { typeHints: 'strict' }, numpy: { functions: ['array', 'zeros', 'ones'], alias: 'np', typeHints: 'strict', }, }, pythonImportPath: ['./python'], output: { dir: './src/generated', format: 'esm', declaration: true, sourceMap: false, annotatedJSDoc: true, }, runtime: { node: { pythonPath: 'python3', timeout: 30000, }, }, performance: { caching: true, batching: false, compression: 'none', }, types: { presets: ['stdlib'], }, }); ``` ## Top-level fields | Field | Type | Default | Purpose | | --- | --- | --- | --- | | `pythonModules` | `Record` | `{}` | Modules to analyze and generate. | | `pythonImportPath` | `string[]` | `[]` | Directories prepended to `PYTHONPATH` for discovery and IR extraction. | | `contractInput` | `string \| Record` | unset | A pinned contract file, or one path per module, used instead of starting Python. | | `output` | `OutputConfig` | See below | Generated file location and format. | | `runtime` | `RuntimeConfig` | Node settings | Runtime-related settings. Generation reads the Node settings. | | `performance` | `PerformanceConfig` | See below | Generation cache controls and accepted compatibility settings. | | `types` | `TypeMappingConfig` | `{ presets: [] }` | Extra mappings for supported library types. | | `debug` | `boolean` | `false` | Enables debug logging. | Unknown top-level fields fail validation. The loader also validates the value types for the documented sections. ## Modules Each `pythonModules` key is the Python module name passed to the IR extractor. For local modules, add their parent directory to `pythonImportPath`. ```ts export default defineConfig({ pythonModules: { 'my_package.statistics': { functions: ['mean', 'percentile'], classes: ['Summary'], exclude: ['internal_helper'], excludePatterns: ['^experimental_'], alias: 'stats', typeHints: 'strict', }, }, pythonImportPath: ['./python'], }); ``` | Field | Type | Default | Purpose | | --- | --- | --- | --- | | `version` | `string` | unset | Version value recorded in the IR cache key. | | `functions` | `string[]` | all exports | Limits generated module functions. | | `classes` | `string[]` | all exports | Limits generated classes. | | `exclude` | `string[]` | `[]` | Excludes exact export names after selection. | | `excludePatterns` | `string[]` | `[]` | Excludes exports matching JavaScript regular-expression source. Invalid patterns produce a generation warning. | | `alias` | `string` | module name | Sets the generated module alias. | | `typeHints` | `'strict' \| 'loose' \| 'ignore'` | `'strict'` at generation | Accepted for configuration compatibility and included in the IR cache key. It does not currently change analysis or emitted types. | The former per-module `runtime` field is not part of new configurations. Set runtime options under the top-level `runtime` field. ## Output ```ts output: { dir: './generated', format: 'both', declaration: true, sourceMap: false, annotatedJSDoc: true, } ``` | Field | Type | Default | Purpose | | --- | --- | --- | --- | | `dir` | `string` | `'./generated'` | Output directory. | | `format` | `'esm' \| 'cjs' \| 'both'` | `'esm'` | Generated module format. | | `declaration` | `boolean` | `false` | Writes a matching `.generated.d.ts` file. | | `sourceMap` | `boolean` | `false` | Writes a `.generated.ts.map` file. | | `annotatedJSDoc` | `boolean` | `false` | Adds source annotation strings to generated function JSDoc. | Generation also writes `.contract.json` beside each generated wrapper. The contract is byte-stable and lets `generate --check` detect contract drift. ## Runtime settings The configuration loader accepts Node, Pyodide, and HTTP settings. During generation, the CLI reads `runtime.node.pythonPath` and `runtime.node.virtualEnv` to run the Python IR extractor. Create `NodeBridge`, `PyodideBridge`, or `HttpBridge` in application code for runtime calls. ```ts runtime: { node: { pythonPath: 'python3', virtualEnv: './.venv', timeout: 30000, }, pyodide: { indexURL: 'https://cdn.jsdelivr.net/pyodide/v0.28.0/full/', packages: ['numpy'], }, http: { baseURL: 'https://api.example.com/python', timeout: 10000, headers: { Authorization: 'Bearer token' }, }, } ``` | Section | Fields | Notes | | --- | --- | --- | | `runtime.node` | `pythonPath?: string`, `virtualEnv?: string`, `timeout?: number` | Defaults to the platform Python command and a 30,000 ms timeout. | | `runtime.pyodide` | `indexURL?: string`, `packages?: string[]` | Stored and type-checked by the configuration loader. | | `runtime.http` | `baseURL: string`, `timeout?: number`, `headers?: Record` | `baseURL` must be a non-empty string. | ## Performance settings ```ts performance: { caching: true, batching: false, compression: 'none', } ``` | Field | Type | Default | Current effect | | --- | --- | --- | --- | | `caching` | `boolean` | `false` | Reuses cached Python IR during generation. It does not cache function call results. | | `batching` | `boolean` | `false` | Validated and retained in the resolved config, but no current generator or runtime code reads it. | | `compression` | `'auto' \| 'gzip' \| 'brotli' \| 'none'` | `'none'` | Validated and included in the IR cache key, but no current generator or runtime code performs compression for it. | ## Type presets Use `types.presets` to enable mappings for supported third-party annotations. ```ts types: { presets: ['stdlib', 'pandas', 'scipy'], } ``` Accepted presets are `numpy`, `pandas`, `pydantic`, `stdlib`, `scipy`, `torch`, and `sklearn`. `numpy` is accepted as a no-op in 0.10.0. The other presets map the library types implemented by the generator, such as `DataFrame`, sparse matrix classes, `Tensor`, and `BaseEstimator`. ## Pinned contracts Use `contractInput` to generate from a saved IR contract without starting a Python process. A single path applies to every configured module. A record selects an input path for each module. ```ts export default defineConfig({ pythonModules: { math: { typeHints: 'strict' }, }, contractInput: { math: './generated/math.contract.json', }, }); ``` Contracts must declare IR version `0.4.0`. Generation reports the mismatch when the contract version does not match the TypeScript generator. ## Configuration checks Use `generate --check` in CI after committing generated wrappers and their contracts. It compares the generated TypeScript, declaration files when enabled, source maps when enabled, and contract files without writing them. ```bash npx tywrap generate --check ``` See the [CLI reference](https://bbopen.github.io/tywrap/reference/cli) for command-line overrides and exit codes. --- # Runtime Comparison tywrap supports five runtime configurations. Choose based on your environment. ## Feature Matrix | Feature | Node.js | Bun | Deno (experimental) | Browser (Pyodide) | HTTP | | ------------------------------ | :-----: | :-: | :-----------------: | :------------------: | :---------: | | Python subprocess | yes | yes | yes | no | no | | Deno Deploy / serverless | no | no | no | yes | yes | | Apache Arrow transport | yes | yes | yes | no (JSON only) | yes | | Virtual environment support | yes | yes | yes | no | Server-side | | Process pooling (experimental) | yes | yes | yes | no | no | | Development hot reload | yes | no | no | Manual bridge reload | External | | Tested in CI | yes | yes | no (untested) | yes | yes | ## Import Paths | Runtime | Import | | ----------------------------- | ------------------------------------------------ | | Node.js | `import { NodeBridge } from 'tywrap/node'` | | Bun | `import { NodeBridge } from 'tywrap/node'` | | Deno (experimental, untested) | `import { NodeBridge } from 'npm:tywrap'` | | Browser | `import { PyodideBridge } from 'tywrap/pyodide'` | | HTTP | `import { HttpBridge } from 'tywrap/http'` | ## Decision Guide ``` Do you need subprocess-based Python execution? ├── Yes → Does your environment support subprocess? │ ├── Node.js or Bun → Use NodeBridge (import from 'tywrap/node' or 'tywrap') │ ├── Deno (experimental, untested) → Use NodeBridge with --allow-run=python3 │ └── Deno Deploy / serverless → Continue ↓ └── No (browser, edge, serverless) → ├── Can you load ~50MB WebAssembly? → Use PyodideBridge └── Need heavy Python libs or can't load WASM → Use HttpBridge ``` ## Bridge Reference | Bridge | Export | Guide | | --------------- | ---------------- | ------------------------------------------------- | | `NodeBridge` | `tywrap/node` | [Node.js](https://bbopen.github.io/tywrap/guide/runtimes/node) · [Bun](https://bbopen.github.io/tywrap/guide/runtimes/bun) · [Deno](https://bbopen.github.io/tywrap/guide/runtimes/deno) | | `PyodideBridge` | `tywrap/pyodide` | [Browser](https://bbopen.github.io/tywrap/guide/runtimes/browser) | | `HttpBridge` | `tywrap/http` | [HTTP](https://bbopen.github.io/tywrap/guide/runtimes/http) | --- # Node.js Runtime Guide The Node.js runtime is tywrap's default and most feature-complete runtime environment. It uses child processes to execute Python code, providing excellent performance and compatibility. ## Overview The Node.js runtime: - **Executes Python via subprocess** - Spawns Python processes for code execution - **High Performance** - Direct process communication with minimal overhead - **Full Feature Support** - Supports all tywrap features and Python libraries - **Development Friendly** - Excellent debugging and error reporting - **Failure handling** - Reports Python errors and enforces configured timeouts ## Bridge Selection - **NodeBridge (default)**: the public Node bridge. It supports both single-process execution and pooled execution through its constructor options. - **OptimizedNodeBridge**: deprecated compatibility alias for older deep imports. It is not part of the package exports and should not be used in new code. Both bridges share the same JSONL core for protocol validation, timeouts, and stderr buffering. ## Development Hot Reload Node hot reload in tywrap means wrapper regeneration plus bridge replacement. It lives in `tywrap/dev`, not in `tywrap.config.*`. ```typescript import { startNodeWatchSession } from 'tywrap/dev'; import { NodeBridge } from 'tywrap/node'; const session = await startNodeWatchSession({ configFile: './tywrap.config.ts', createBridge: async config => new NodeBridge({ pythonPath: config.runtime.node?.pythonPath ?? 'python3', timeoutMs: config.runtime.node?.timeout ?? 30000, }), }); ``` Use `reloadNow()` for an explicit rebuild or `close()` to stop watching and dispose the active bridge. The watch session manages directory trees internally for local package roots and directory-valued `extraWatchPaths`, ignores Python cache directories, and keeps the last known good wrappers plus bridge live if a reload returns structured generation failures. See [Watch & Reload](https://bbopen.github.io/tywrap/guide/dev-reload.md) for the reload lifecycle events and the full failure / recovery contract. ## Basic Setup ### Installation ```bash npm install tywrap ``` ### Configuration ```json { "pythonModules": { "numpy": { "runtime": "node" }, "pandas": { "runtime": "node" } }, "runtime": { "node": { "pythonPath": "python3", "timeout": 30000 } } } ``` ### Usage ```typescript import { NodeBridge } from 'tywrap/node'; import { setRuntimeBridge } from 'tywrap/runtime'; import { array, zeros } from './generated/numpy.generated.js'; const bridge = new NodeBridge({ pythonPath: 'python3' }); setRuntimeBridge(bridge); async function example() { const arr = await array([1, 2, 3, 4, 5]); const empty = await zeros([3, 3]); console.log('Array created:', arr); } ``` ### Bridge Diagnostics ```typescript const info = await bridge.getBridgeInfo({ refresh: true }); console.log(info.protocol, info.pythonVersion, info.instances); ``` ## Configuration Options `tywrap.config.*` and `NodeBridge` do different jobs: - `tywrap.config.*` controls wrapper generation. - `new NodeBridge(...)` controls the live subprocess bridge in your app. ### `tywrap.config.*` fields ```json { "runtime": { "node": { "pythonPath": "/usr/local/bin/python3", "virtualEnv": "./venv", "timeout": 30000 } } } ``` ### `NodeBridge` constructor options ```typescript const bridge = new NodeBridge({ pythonPath: '/usr/local/bin/python3.11', virtualEnv: './venv', scriptPath: './custom_bridge.py', cwd: './python_src', timeoutMs: 60000, queueTimeoutMs: 60000, inheritProcessEnv: true, env: { PYTHONPATH: './additional_modules', OMP_NUM_THREADS: '4', }, codec: { bytesHandling: 'base64', }, }); ``` | Option | Type | Default | Description | | ------------------------- | ---------------------------------------- | --------------- | --------------------------------------------------------------------------------------- | | `pythonPath` | `string` | auto-detect | Path to the Python executable | | `scriptPath` | `string` | built-in bridge | Custom `python_bridge.py` path | | `virtualEnv` | `string` | not set | Virtual environment root | | `cwd` | `string` | `process.cwd()` | Working directory for the subprocess | | `timeoutMs` | `number` | `30000` | Per-call timeout | | `queueTimeoutMs` | `number` | `30000` | Queue timeout when the pool is saturated | | `minProcesses` | `number` | `1` | Minimum worker count | | `maxProcesses` | `number` | `1` | Maximum worker count | | `maxConcurrentPerProcess` | `number` | `1` | Concurrent requests per serial Python worker; use more worker processes for concurrency | | `inheritProcessEnv` | `boolean` | `false` | Pass the full parent environment through | | `env` | `Record` | `{}` | Extra subprocess env vars | | `codec` | `CodecOptions` | not set | Codec validation and byte handling | | `warmupCommands` | `Array<{ module, functionName, args? }>` | `[]` | Commands to run when each worker starts | Deprecated compatibility fields still exist on the interface: `maxIdleTime`, `maxRequestsPerProcess`, `enableJsonFallback`, and `maxLineLength`. Avoid them in new code. By default, the subprocess environment is minimal (PATH/PYTHON*/TYWRAP\_* only). Set `inheritProcessEnv: true` to pass through the full environment when needed. ## Python Environment Setup ### Using System Python ```json { "runtime": { "node": { "pythonPath": "/usr/bin/python3" } } } ``` ### Using Virtual Environment ```json { "runtime": { "node": { "pythonPath": "./venv/bin/python", "virtualEnv": "./venv" } } } ``` ### Using Conda Environment ```bash # Activate conda environment first conda activate myenv # Then use conda's python ``` ```json { "runtime": { "node": { "pythonPath": "/opt/miniconda3/envs/myenv/bin/python" } } } ``` ### Using pyenv ```bash # Set Python version with pyenv pyenv local 3.11.0 ``` ```json { "runtime": { "node": { "pythonPath": "python3" // pyenv will provide the right version } } } ``` ## Data Transport and Performance ### Arrow Transport (Default) For optimal performance with NumPy/Pandas data: ```bash # Install Apache Arrow for Python pip install pyarrow # Install Apache Arrow for Node.js (optional, for decoding) npm install apache-arrow ``` Auto path (when apache-arrow is installed): ```typescript import { autoRegisterArrowDecoder } from 'tywrap'; await autoRegisterArrowDecoder(); ``` Manual path (customize decoding outside NodeBridge): ```typescript import { registerArrowDecoder } from 'tywrap'; import { tableFromIPC } from 'apache-arrow'; registerArrowDecoder(bytes => tableFromIPC(bytes)); ``` ### JSON Fallback For environments without Arrow support, set the environment variable: ```bash export TYWRAP_CODEC_FALLBACK=json ``` ### Payload Size Limit The subprocess bridge writes one JSONL response per call; a response larger than the line ceiling is split with `tywrap-frame/1` chunking. To prevent oversized payloads: ```bash export TYWRAP_CODEC_MAX_BYTES=10485760 # 10 MB cap ``` If a response exceeds `TYWRAP_CODEC_MAX_BYTES`, the call fails with an explicit error. Use this instead of older line-length knobs. ### Request Size Limit To cap incoming request payloads (JSONL request size in bytes): ```bash export TYWRAP_REQUEST_MAX_BYTES=1048576 # 1 MB cap ``` If a request exceeds `TYWRAP_REQUEST_MAX_BYTES`, the call fails with an explicit error. ### Torch Tensor Copy Opt-in For GPU tensors or non-contiguous tensors, enable explicit CPU/copy conversion: ```bash export TYWRAP_TORCH_ALLOW_COPY=1 ``` ## Error Handling and Debugging ### Error Types ```typescript try { const result = await math.sqrt(-1); } catch (error) { console.error('Error type:', error.name); // ValueError console.error('Error message:', error.message); // math domain error console.error('Python traceback:', error.traceback); } ``` ### Debugging Configuration Use the CLI's debug flag when you are troubleshooting wrapper generation: ```bash npx tywrap generate --debug ``` Use runtime log env vars for subprocess diagnostics: ```bash export TYWRAP_LOG_LEVEL=DEBUG export TYWRAP_LOG_JSON=1 ``` If you need to disable timeouts or pass extra Python env vars while debugging, do it on the bridge instance: ```typescript const bridge = new NodeBridge({ pythonPath: 'python3', timeoutMs: 0, env: { PYTHONUNBUFFERED: '1', }, }); ``` ### Common Error Scenarios **Module Import Error**: ```typescript // Error: ModuleNotFoundError: No module named 'numpy' // Solution: Install module or check PYTHONPATH ``` **Timeout Error**: ```typescript // Error: Python call timed out // Solution: Increase timeoutMs or optimize Python code ``` **Process Exit Error**: ```typescript // Error: Python process exited // Solution: Check Python path and permissions ``` ## Performance Optimization ### Process Reuse tywrap automatically reuses Python processes for better performance: ```typescript // These calls will reuse the same Python process const a1 = await numpy.array([1, 2, 3]); const a2 = await numpy.array([4, 5, 6]); const result = await numpy.add(a1, a2); ``` ### Batching Operations ```typescript // Instead of multiple round trips const sin1 = await math.sin(1); const sin2 = await math.sin(2); const sin3 = await math.sin(3); // Use Promise.all for concurrent execution const [sin1, sin2, sin3] = await Promise.all([ math.sin(1), math.sin(2), math.sin(3), ]); ``` ### Memory Management Pass Python-specific tuning through `env` on the bridge: ```typescript const bridge = new NodeBridge({ pythonPath: 'python3', env: { PYTHONMALLOC: 'malloc', OMP_NUM_THREADS: '4', }, }); ``` ## Production Deployment ### Process Management ```typescript // Graceful shutdown process.on('SIGTERM', async () => { // tywrap automatically cleans up Python processes process.exit(0); }); ``` ### Resource Monitoring ```typescript import { NodeBridge } from 'tywrap/node'; const bridge = new NodeBridge({ pythonPath: 'python3', timeoutMs: 30000, }); // Monitor process health setInterval(async () => { try { await bridge.call('math', 'sqrt', [4]); // Health check } catch (error) { console.error('Python process unhealthy:', error); // Restart or alert } }, 60000); ``` ### Docker Configuration ```dockerfile # Dockerfile FROM node:20-slim # Install Python and dependencies RUN apt-get update && apt-get install -y python3 python3-pip COPY requirements.txt . RUN pip3 install -r requirements.txt # Install Node.js dependencies COPY package*.json ./ RUN npm ci --only=production # Copy application COPY . . # Generate wrappers at build time RUN npm run build EXPOSE 3000 CMD ["node", "dist/index.js"] ``` ```yaml # docker-compose.yml version: '3.8' services: app: build: . environment: - TYWRAP_CODEC_FALLBACK=json # For smaller containers - TYWRAP_LOG_LEVEL=INFO ports: - '3000:3000' ``` ### Environment Variables ```bash # Production environment export NODE_ENV=production export TYWRAP_CODEC_MAX_BYTES=10485760 export TYWRAP_REQUEST_MAX_BYTES=1048576 export TYWRAP_LOG_LEVEL=INFO # Security export PYTHONDONTWRITEBYTECODE=1 export PYTHONUNBUFFERED=1 ``` Set the Python executable in config or when you construct the bridge: ```typescript const bridge = new NodeBridge({ pythonPath: '/usr/local/bin/python3', }); ``` ## Security Considerations ### Subprocess Security ```typescript const bridge = new NodeBridge({ cwd: '/safe/directory', timeoutMs: 10000, env: { PATH: '/usr/bin:/bin', PYTHONPATH: '/safe/python/libs', }, }); ``` ### Input Validation ```typescript // Validate inputs before passing to Python function validateInput(value: unknown): boolean { // Add your validation logic return typeof value === 'number' && isFinite(value); } async function safeSqrt(value: number) { if (!validateInput(value) || value < 0) { throw new Error('Invalid input for sqrt'); } return await math.sqrt(value); } ``` ## Troubleshooting ### Common Issues **"Python not found"**: ```bash # Check Python installation which python3 python3 --version # Update configuration { "runtime": { "node": { "pythonPath": "/usr/local/bin/python3" } } } ``` **"Module not found"**: ```bash # Check module installation python3 -c "import numpy; print(numpy.__version__)" # Check PYTHONPATH python3 -c "import sys; print(sys.path)" ``` **"Permission denied"**: ```bash # Check executable permissions ls -la /usr/local/bin/python3 # Fix permissions if needed chmod +x /usr/local/bin/python3 ``` **"Process timeout"**: ```typescript const bridge = new NodeBridge({ pythonPath: 'python3', timeoutMs: 60000, env: { OMP_NUM_THREADS: '1', }, }); ``` ### Debug Mode ```bash # Wrapper-generation diagnostics npx tywrap generate --debug # Runtime bridge diagnostics export TYWRAP_LOG_LEVEL=DEBUG export TYWRAP_LOG_JSON=1 # Run with debug output node --trace-warnings your-app.js ``` ## Advanced Usage ### Custom Bridge Script Create your own Python bridge for specialized needs: ```python # custom_bridge.py import sys import json import traceback def handle_request(request): try: # Your custom handling logic result = process_request(request) return {'result': result} except Exception as e: return { 'error': { 'type': type(e).__name__, 'message': str(e), 'traceback': traceback.format_exc() } } if __name__ == '__main__': # Bridge implementation pass ``` ```typescript const bridge = new NodeBridge({ pythonPath: 'python3', scriptPath: './custom_bridge.py', }); ``` ### Process Pooling ```typescript import { NodeBridge } from 'tywrap/node'; // Create multiple bridges for load balancing const bridges = Array.from( { length: 4 }, () => new NodeBridge({ pythonPath: 'python3' }) ); let currentBridge = 0; function getNextBridge() { const bridge = bridges[currentBridge]; currentBridge = (currentBridge + 1) % bridges.length; return bridge; } ``` ## Next Steps - [Configuration Guide](https://bbopen.github.io/tywrap/guide/configuration) - Complete configuration reference - [Examples](https://bbopen.github.io/tywrap/examples/) - Usage examples and patterns - [Troubleshooting](https://bbopen.github.io/tywrap/troubleshooting/) - Common issues and solutions - [API Reference](https://bbopen.github.io/tywrap/reference/api/) - Complete API documentation --- # Bun Runtime Guide tywrap works with [Bun](https://bun.sh/) 1.1+ using the same `NodeBridge` as Node.js. No separate bridge is needed. ## Installation ```bash bun add tywrap pip install tywrap-ir ``` ## Basic Setup ```typescript import { NodeBridge } from 'tywrap/node'; import { setRuntimeBridge } from 'tywrap/runtime'; setRuntimeBridge(new NodeBridge({ pythonPath: 'python3', virtualEnv: '.venv', timeoutMs: 30000, })); ``` Both `'tywrap/node'` and `'tywrap'` work on Bun. Use `'tywrap/node'` for smaller bundles through tree-shaking. ## Configuration Options `NodeBridge` accepts the same options under Bun as under Node.js. See the [Node.js guide](https://bbopen.github.io/tywrap/guide/runtimes/node) for the full option reference. Key options: | Option | Default | Description | |--------|---------|-------------| | `pythonPath` | auto-detect | Path to `python3` executable | | `virtualEnv` | not set | Path to virtual environment directory | | `timeoutMs` | `30000` | Request timeout in milliseconds | | `inheritProcessEnv` | `false` | Set `true` to pass full `process.env` to subprocess | ## bunfig.toml The repository's `bunfig.toml` contains these build and run settings: ```toml [build] target = "bun" format = "esm" splitting = true sourcemap = "external" external = ["pyodide"] [dev] hot = true ``` ## Running Bun-Specific Tests ```bash npm run test:bun ``` Or directly: ```bash LC_ALL=C LANG=C bun run vitest --run test/runtime_bun.test.ts ``` ## Virtual Environments ```typescript setRuntimeBridge(new NodeBridge({ pythonPath: '.venv/bin/python', virtualEnv: '.venv', })); ``` ## Environment Variables The same `TYWRAP_*` env vars work under Bun. See the [environment variables reference](https://bbopen.github.io/tywrap/reference/env-vars). ## Troubleshooting `python3: command not found`: set `pythonPath` explicitly or ensure Python is on `PATH`. For a subprocess timeout, increase `timeoutMs`. Verify `pip install tywrap-ir` ran in the correct environment. The [Node.js troubleshooting guide](https://bbopen.github.io/tywrap/guide/runtimes/node) has more patterns that also apply to Bun. --- # Deno Runtime Guide tywrap's Deno subprocess support is experimental and untested in CI. It uses the same `NodeBridge` as Node.js, and Deno requires the `npm:` prefix for npm imports. ## Deno Deploy limitation **Deno Deploy does NOT support subprocess execution.** Because `NodeBridge` spawns a Python subprocess, it cannot run in Deno Deploy. **Alternatives for Deno Deploy:** - Use [`PyodideBridge`](https://bbopen.github.io/tywrap/guide/runtimes/browser), which runs Python in-browser via WebAssembly (no subprocess) - Use [`HttpBridge`](https://bbopen.github.io/tywrap/guide/runtimes/http), which connects to a remote Python server over HTTP ## Installation ```bash deno add npm:tywrap pip install tywrap-ir ``` ## Basic Setup ```typescript import { NodeBridge } from 'npm:tywrap'; import { setRuntimeBridge } from 'npm:tywrap/runtime'; setRuntimeBridge( new NodeBridge({ pythonPath: 'python3', timeoutMs: 30000, }) ); ``` ## Required Permissions Deno requires explicit permission flags for subprocess execution: ```bash deno run \ --allow-run=python3 \ --allow-read \ --allow-env \ your-script.ts ``` | Flag | Reason | | --------------------- | ------------------------------------------------ | | `--allow-run=python3` | Spawn the Python subprocess | | `--allow-read` | Read Python scripts and config files | | `--allow-env` | Read `TYWRAP_*` and `PATH` environment variables | ## Type Checking ```bash deno check src/index.ts ``` ## Configuration Options See the [Node.js guide](https://bbopen.github.io/tywrap/guide/runtimes/node) for the full `NodeBridgeOptions` reference. All options work identically in Deno. ## When to Use Each Bridge in Deno | Scenario | Bridge | Notes | | ------------------------------- | --------------- | -------------------------------- | | Local Deno script | `NodeBridge` | Needs `--allow-run` | | Deno Deploy | `PyodideBridge` | WebAssembly, no subprocess | | Deno Deploy + heavy Python libs | `HttpBridge` | Python runs on a separate server | ## Environment Variables The same `TYWRAP_*` env vars work under Deno. See the [environment variables reference](https://bbopen.github.io/tywrap/reference/env-vars). ## Troubleshooting `PermissionDenied: Requires run access to "python3"`: add `--allow-run=python3` to your `deno run` command. `NotSupported: Subprocess access is not allowed`: you are running in Deno Deploy. Switch to [`PyodideBridge`](https://bbopen.github.io/tywrap/guide/runtimes/browser) or [`HttpBridge`](https://bbopen.github.io/tywrap/guide/runtimes/http). See the [Node.js troubleshooting guide](https://bbopen.github.io/tywrap/guide/runtimes/node) for additional patterns. --- # Browser runtime guide Run Python in the browser using Pyodide WebAssembly. ## Overview - WebAssembly Python in the browser - No server required for execution - Pre-built packages via Pyodide - Works well for notebooks, demos, and client-side analysis ## Quick start ### Install ```bash npm install tywrap pyodide ``` ### Configure ```json { "pythonModules": { "numpy": { "runtime": "pyodide" }, "matplotlib": { "runtime": "pyodide" } }, "runtime": { "pyodide": { "indexURL": "https://cdn.jsdelivr.net/pyodide/v0.28.1/full/", "packages": ["numpy", "matplotlib", "scipy"] } } } ``` ### Use ```ts import { PyodideBridge } from 'tywrap/pyodide'; import { setRuntimeBridge } from 'tywrap/runtime'; import { array } from './generated/numpy.generated.js'; const bridge = new PyodideBridge({ indexURL: 'https://cdn.jsdelivr.net/pyodide/v0.28.1/full/', packages: ['numpy'], }); setRuntimeBridge(bridge); const arr = await array([1, 2, 3]); console.log(arr); ``` ## Configuration options ```ts interface PyodideBridgeOptions { indexURL?: string; packages?: string[]; } ``` The bridge loads the `packages` array during initialization. For additional package loading, rely on Pyodide directly. ## Data transport PyodideBridge uses JSON transport. It converts supported NumPy arrays and pandas tables to JSON envelopes. It does not send Arrow payloads. See [codec envelopes](https://bbopen.github.io/tywrap/codec-envelopes.md) for supported values and transport limits. ## Build integration Run `tywrap generate` during your build. Load Pyodide at runtime from a CDN or your own server. ## Test coverage The unit tests mock the Pyodide loader. The cross-backend suite runs the shared Python bootstrap under CPython. Neither check proves browser execution. Required CI also runs the pinned Pyodide package in Node WebAssembly and a Chromium smoke test. The browser test loads local Pyodide assets and executes a generated wrapper. --- # HTTP Bridge Guide `HttpBridge` connects to a Python server over HTTP. Use it when Python must run separately, including Deno Deploy, serverless environments, or distributed architectures. ## When to Use HttpBridge - You cannot spawn subprocesses (Deno Deploy, Cloudflare Workers, serverless) - Python must run on a dedicated server for resource reasons - You want to share one Python server across multiple TypeScript clients ## Installation ```bash npm install tywrap pip install tywrap-ir ``` ## TypeScript Setup ```typescript import { HttpBridge } from 'tywrap/http'; import { setRuntimeBridge } from 'tywrap/runtime'; setRuntimeBridge( new HttpBridge({ baseURL: 'http://localhost:8080', timeoutMs: 30000, headers: { Authorization: 'Bearer your-token', }, }) ); ``` ## Options | Option | Required | Default | Description | | ----------- | -------- | ------- | ------------------------------------------ | | `baseURL` | Yes | none | Base URL of the Python bridge server | | `headers` | No | `{}` | Additional HTTP headers (auth, CORS, etc.) | | `timeoutMs` | No | `30000` | Request timeout in milliseconds | | `codec` | No | Arrow | Codec options | ## Running the Python Server `HttpBridge` expects a server that accepts POST requests with JSON/Arrow payloads. You must provide a compatible server endpoint. Each call is an independent POST request because the protocol is stateless. Generated async callables have typed results when their return annotations have supported value conversions. The in-repo Node and Pyodide bridges await Python coroutines. For `HttpBridge`, your server must await coroutines before it sends the result. `HttpBridge` cannot verify a remote server's coroutine support. tywrap does not include an HTTP server. See the [API reference](https://bbopen.github.io/tywrap/reference/api/) for the expected request and response format. ## Apache Arrow Arrow binary transport works over HTTP. Register a decoder to enable it: ```typescript import { registerArrowDecoder } from 'tywrap'; import { tableFromIPC } from 'apache-arrow'; registerArrowDecoder(bytes => tableFromIPC(bytes)); ``` ## Environment Variables | Var | Purpose | | ---------------------------- | ---------------------------- | | `TYWRAP_CODEC_FALLBACK=json` | Disable Arrow, use JSON only | | `TYWRAP_CODEC_MAX_BYTES` | Cap max response size | See the [environment variables reference](https://bbopen.github.io/tywrap/reference/env-vars). ## Security - Always use HTTPS in production - Set `Authorization` headers for server access control - Consider rate-limiting the Python server endpoint --- # Watch & Reload (Failure / Recovery Contract) `tywrap/dev` regenerates wrappers and replaces the runtime bridge during development. Application-level hot module reloading remains the application's job. The helper keeps generated wrappers and the active Python bridge in sync with Python sources while the process stays up. This page documents the reload lifecycle and failure behavior: what happens when a reload fails, and what stays live so your app keeps working. > Reload is configured in code through `tywrap/dev`, never in `tywrap.config.*`. > The legacy `development` block and `pythonModules..watch` fields were > removed in 0.4.0. Using them now raises an explicit migration error. ## The two entry points | Helper | Use it for | | --- | --- | | `startNodeWatchSession(...)` | Node-only. Watches the config and local Python sources. After regenerating wrappers, it swaps the active bridge in place. | | `createBridgeReloader(...)` | Cross-runtime manual primitive (e.g. Pyodide). You call `reload()` yourself. It has no filesystem watcher. | ```typescript import { startNodeWatchSession } from 'tywrap/dev'; import { NodeBridge } from 'tywrap/node'; const session = await startNodeWatchSession({ configFile: './tywrap.config.ts', createBridge: async config => new NodeBridge({ pythonPath: config.runtime.node?.pythonPath ?? 'python3', timeoutMs: config.runtime.node?.timeout ?? 30000, }), }); // Force a rebuild now (resolves to `true` on success, `false` on failure): const ok = await session.reloadNow(); // Stop watching and dispose the active bridge: await session.close(); ``` `createBridge` receives the **freshly resolved config for that reload cycle**, so config edits (e.g. a changed `timeout`) are picked up on the next reload. By default the newly created bridge is published to the global runtime registry, so existing generated wrappers transparently route through the swapped bridge. No imports change and the process never restarts. ## Reload lifecycle events Pass an `onEvent` callback to observe the session. Events are emitted in this order for a successful cycle: | Event | Meaning | | --- | --- | | `watchPaths` | The set of paths the session is currently watching (config file, resolved Python package trees, and `extraWatchPaths`). Re-emitted whenever the watched set changes. | | `change` | A watched path changed (`manual: false`), so a reload has been scheduled. | | `reload-start` | A reload cycle began (`manual: true` for `reloadNow()` / initial startup). | | `reload-success` | Regeneration and bridge swap completed. Carries `written` (relative paths of the generated files now on disk) and `warnings`. | | `reload-error` | The reload failed. Carries the underlying `error: Error`. | A reload performs these steps atomically with respect to the live state: 1. **Generate** the next wrapper set into a temporary staging directory. 2. **Promote** the staged files into the real output directory (writing new files, removing managed files that no longer exist). 3. **Warm and activate** the next bridge, then dispose the previous one. 4. **Commit** the refreshed watcher set and emit `reload-success`. `reloadNow()` and filesystem changes are serialized: an in-flight reload finishes before the next one starts, and rapid edits are debounced (`debounceMs`, default `100`). ## Failure / recovery contract If any step of a reload throws, the session stays up. The last-known-good state stays live: - The **previously generated wrappers remain on disk unchanged.** Staging happens in a temp directory; the real output directory is only touched once generation succeeds. If promotion itself fails partway, the previous file contents are restored. - The **previously active bridge stays active** in the runtime registry and is not disposed. A bridge that was warmed for the failed reload is disposed instead, so you never leak the half-prepared one. - A structured `reload-error` event is emitted with the underlying `Error`, and the failing call (`reloadNow()` or the debounced auto-reload) resolves to `false`. The session keeps watching and will attempt the next reload normally. Generation and bridge construction failures surface through the same event: | Source | What the `error` looks like | | --- | --- | | **Generation failure** (a watched module's IR can't be produced, such as after a syntax error). This is the `GenerateFailure` path. | `error.message` begins with `Generation failed for N module(s):` followed by per-module detail. | | **Bridge construction failure** (your `createBridge` throws). | The error your factory threw, propagated verbatim. | ```typescript const session = await startNodeWatchSession({ configFile: './tywrap.config.ts', createBridge, onEvent: event => { if (event.type === 'reload-error') { // Last-good wrappers + bridge are still live here. console.error('[tywrap] reload failed, keeping last-good state:', event.error.message); } if (event.type === 'reload-success') { console.log('[tywrap] reloaded:', event.written.join(', ')); } }, }); ``` > **Startup is the exception.** There is no last-known-good state on the very > first reload, so if the *initial* `startNodeWatchSession(...)` setup fails, the > promise rejects (watchers are closed and any partial bridge disposed). After a > successful start, subsequent failures keep the last known good state live. ## Other notes - Watched Python package trees are watched per-directory; new nested directories are picked up automatically and `__pycache__` / `.pytest_cache` / `.mypy_cache` / `.ruff_cache` churn is ignored. - Writes into the output directory and `.tywrap/{cache,reports}` are ignored so generation does not retrigger itself. - For Pyodide use `createBridgeReloader(...)` and drive `reload()` from your own trigger. For the HTTP runtime, reload by restarting/redeploying the remote server. That reload is external to tywrap. --- # CLI Reference The `tywrap` CLI has two commands: `init` and `generate`. ## Installation ```bash npm install tywrap ``` Or run it with `npx`: ```bash npx tywrap ``` ## `tywrap init` Creates a starter config file in the current directory. - The default file is `tywrap.config.ts`. - If you do not pass `--modules`, the starter config wraps `math`. - If a `package.json` is present, tywrap adds `tywrap:generate` and `tywrap:check` scripts unless you pass `--no-scripts`. ```bash npx tywrap init npx tywrap init --format json --modules math,numpy ``` | Flag | Description | | ------------------------------------- | ------------------------------------------------------ | | `--config`, `-c` | Path for the new config file | | `--format ts\|json` | Output format for the starter config | | `--modules` | Comma-separated Python modules to seed into the config | | `--runtime node\|pyodide\|http\|auto` | Accepted for compatibility. Set the active runtime under top-level `runtime` in the generated config. | | `--output-dir` | Generated wrapper directory in the starter config | | `--force` | Overwrite an existing config file | | `--scripts`, `--no-scripts` | Add or skip recommended `package.json` scripts | ## `tywrap generate` Reads config and writes generated wrapper files from Python IR. When `--config` is omitted, the CLI searches in this order: 1. `tywrap.config.ts` 2. `tywrap.config.mts` 3. `tywrap.config.js` 4. `tywrap.config.mjs` 5. `tywrap.config.cjs` 6. `tywrap.config.json` If no config file is found, you can still generate wrappers by passing `--modules`. ```bash npx tywrap generate npx tywrap generate --config ./tywrap.config.json npx tywrap generate --modules math,statistics --runtime node ``` | Flag | Description | | ------------------------------------- | ------------------------------------------------------------------- | | `--config`, `-c` | Config file path | | `--modules` | Comma-separated Python modules to wrap | | `--runtime node\|pyodide\|http\|auto` | Accepted for compatibility. Set the active runtime under top-level `runtime` in the config. | | `--python` | Python executable path override | | `--output-dir` | Override `output.dir` | | `--format esm\|cjs\|both` | Override `output.format` | | `--declaration` | Override `output.declaration` and emit matching `.d.ts` files | | `--source-map` | Override `output.sourceMap` | | `--cache`, `--no-cache` | Enable or disable on-disk IR caching | | `--debug` | Enable debug logging | | `--verbose`, `-v` | Alias for `--debug` | | `--fail-on-warn` | Exit non-zero when generation emits warnings | | `--check` | Compare generated output with what is on disk without writing files | ## `tywrap generate --check` `--check` is for CI and upgrade verification. It does not write files. A normal generation writes `.contract.json` beside the generated wrapper. Check mode compares against it without writing. The contract is byte-stable across machines and Python processes. `--check` compares that file as well as generated TypeScript, declaration files, and source maps when those outputs are enabled, so it also reports contract drift. Set `contractInput` in the config to regenerate from a pinned contract without starting Python. The TypeScript generator and the Python extractor use IR version `0.4.0`. Generation fails with a version-mismatch error that names the expected and received versions when they differ. ```bash npx tywrap generate --check ``` Exit codes: - `0`: generated files are up to date - `2`: generation succeeded but warnings were present and `--fail-on-warn` was set - `3`: generated files are out of date - `1`: general failure, including missing config or import failure Typical CI step: ```yaml - name: Check generated wrappers run: npx tywrap generate --check ``` ## Starter Config Example ```ts import { defineConfig } from 'tywrap'; export default defineConfig({ pythonModules: { math: { typeHints: 'strict' }, numpy: { typeHints: 'strict', alias: 'np' }, }, output: { dir: './generated', format: 'esm', declaration: false, sourceMap: false, }, runtime: { node: { pythonPath: 'python3', }, }, types: { presets: ['stdlib'], }, }); ``` See the [Configuration guide](https://bbopen.github.io/tywrap/guide/configuration) for the full config surface. --- # Environment Variables This page lists the implemented `TYWRAP_*` environment variables in the repo today. ## Bridge and Codec These affect the Python bridge or decoded runtime behavior. | Variable | Scope | Default | Purpose | | ---------------------------- | ------------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `TYWRAP_CODEC_FALLBACK` | Python bridge | unset | Set to `json` to allow JSON fallback when Arrow encoding is unavailable | | `TYWRAP_CODEC_MAX_BYTES` | Python bridge | unset | Reject response payloads larger than this byte count (applied post-reassembly) | | `TYWRAP_REQUEST_MAX_BYTES` | Python bridge | unset | Reject request payloads larger than this byte count (applied post-reassembly) | | `TYWRAP_TORCH_ALLOW_COPY` | Python bridge | off | Allow GPU-to-CPU or contiguous-copy conversion when serializing `torch.Tensor` | | `TYWRAP_ALLOWED_MODULES` | Python bridge | unset | Comma- and/or whitespace-separated import allowlist; blank is unset, while a non-empty value permits only named modules plus bridge-required stdlib modules. See [SECURITY.md](https://github.com/bbopen/tywrap/blob/main/SECURITY.md). | | `TYWRAP_ALLOW_PRIVATE_ATTRS` | Python bridge | off | Set to `1`, `true`, or `yes` to allow underscore-prefixed attribute access; otherwise it is blocked | ## Logging These control the structured logger in the JavaScript runtime. | Variable | Default | Purpose | | ------------------ | ------- | ------------------------------------------------- | | `TYWRAP_LOG_LEVEL` | `WARN` | One of `DEBUG`, `INFO`, `WARN`, `ERROR`, `SILENT` | | `TYWRAP_LOG_JSON` | `false` | Set to `1` or `true` for JSON log output | ## Repo Test and Benchmark Knobs These are used by the tywrap test suite and maintenance workflows. They are not required for normal library use. | Variable | Purpose | | ------------------------------------ | ------------------------------------------------ | | `TYWRAP_CODEC_PYTHON` | Python executable for codec-heavy tests | | `TYWRAP_PERF_BUDGETS` | Enable performance budget suites | | `TYWRAP_PERF_TIME_BUDGET_MS` | Time budget for generator performance tests | | `TYWRAP_PERF_MEMORY_BUDGET_MB` | Memory budget for generator performance tests | | `TYWRAP_CODEC_PERF_ITERATIONS` | Iteration count for codec performance tests | | `TYWRAP_CODEC_PERF_TIME_BUDGET_MS` | Time budget for codec performance tests | | `TYWRAP_CODEC_PERF_MEMORY_BUDGET_MB` | Memory budget for codec performance tests | | `TYWRAP_ADVERSARIAL` | Enable adversarial bridge tests | | `TYWRAP_ADVERSARIAL_PYTHON` | Python executable override for adversarial tests | ## Common Examples Use JSON fallback: ```bash export TYWRAP_CODEC_FALLBACK=json ``` Cap response and request payload size: ```bash export TYWRAP_CODEC_MAX_BYTES=10485760 export TYWRAP_REQUEST_MAX_BYTES=1048576 ``` Enable JSON logs: ```bash export TYWRAP_LOG_LEVEL=INFO export TYWRAP_LOG_JSON=1 ``` ## Not Configured Through Env Vars Python executable and virtual environment selection are configured through code or config files today: ```ts new NodeBridge({ pythonPath: '/usr/bin/python3', virtualEnv: './venv' }); ``` ```ts import { defineConfig } from 'tywrap'; export default defineConfig({ runtime: { node: { pythonPath: 'python3', virtualEnv: './venv', timeout: 30000, }, }, }); ``` --- # Python to TypeScript Type Mapping This page describes the TypeScript shapes tywrap generates and returns today. ## Core Annotation Mapping ### Primitives | Python | TypeScript | | ------------------------- | ---------- | | `int` | `number` | | `float` | `number` | | `str` | `string` | | `bool` | `boolean` | | `bytes` | `string` | | `None` in value position | `null` | | `None` in return position | `void` | ### Collections | Python | TypeScript | Notes | | -------------------------- | ----------------- | -------------------------------------------- | | `list[T]`, `List[T]` | `Array` | | | `Sequence[T]` | `Array` | Normalized to a list-like shape | | `tuple[T1, T2]` | `[T1, T2]` | Exact tuple shape | | `Tuple[T, ...]` | `Array` | Variable-length tuple | | `tuple[()]` | `[undefined]` | Empty tuple representation | | `set[T]`, `frozenset[T]` | `Set` | | | `dict[K, V]`, `Dict[K, V]` | `{ [key: K]: V }` | Index keys fall back to `string` when needed | | `Mapping[K, V]` | `{ [key: K]: V }` | Normalized to a dict-like shape | ### Unions and Literals | Python | TypeScript | | ---------------------- | --------------- | | `Union[A, B]` | `A \| B` | | `A \| B` | `A \| B` | | `Optional[T]` | `T \| null` | | `T \| None` | `T \| null` | | `Literal["a", "b"]` | `"a" \| "b"` | | `Literal[1, 2]` | `1 \| 2` | | `Literal[True, False]` | `true \| false` | ### Callables and Wrappers | Python | TypeScript | Notes | | ------------------------------- | --------------------------- | ------------------------------------------- | | `Callable[[A, B], R]` | `(arg0: A, arg1: B) => R` | | | `Callable[..., R]` | `(...args: unknown[]) => R` | | | `Annotated[T, ...]` | `T` | Metadata is stripped for the generated type | | `ClassVar[T]` | `T` | | | `Final[T]` | `T` | | | `Required[T]`, `NotRequired[T]` | `T` | | ### Special Names | Python | TypeScript | Notes | | ------------------------- | ------------------ | ----- | | `Any` | `unknown` | | | `Never`, `NoReturn` | `never` | | | `LiteralString`, `AnyStr` | `string` | | | `object` | `object` | | | `Awaitable` | `Promise` | | | `Coroutine` | `Promise` | | | `TypeVar('T')` | `T` | Preserved for simple unconstrained/invariant declarations; otherwise falls back to `unknown` | | `ParamSpec('P')` | `P extends unknown[]` | Preserved for callable parameter packs when tywrap can emit a matching generic declaration | | `TypeVarTuple('Ts')` | `unknown` | Variadic generic parameters still fall back conservatively | | `Unpack[Ts]` | `unknown` | Variadic tuple unpacking still falls back conservatively | ## Preset Mappings Enable presets with `types.presets` in your config. ### `stdlib` | Python | TypeScript | | ----------------------------------------------------- | ---------- | | `datetime.datetime`, `datetime.date`, `datetime.time` | `string` | | `datetime.timedelta` | `number` | | `decimal.Decimal` | `string` | | `uuid.UUID` | `string` | | `pathlib.Path` and related path types | `string` | ### `pandas` These are the decoded JavaScript shapes that tywrap exposes to TypeScript. | Python | TypeScript | | ------------------ | ----------------------------------------------------------- | | `pandas.DataFrame` | `Record \| Array>` | | `pandas.Series` | `unknown[] \| Record` | ### `pydantic` | Python | TypeScript | Notes | | -------------------- | ------------------------- | ---------------------------------------------------------------------------------- | | `pydantic.BaseModel` | `Record` | Runtime serialization uses `model_dump(by_alias=True, mode='json')` when available | ### `scipy` ```ts type SparseMatrix = | { format: 'csr' | 'csc'; shape: number[]; data: unknown[]; indices: number[]; indptr: number[]; dtype?: string; } | { format: 'coo'; shape: number[]; data: unknown[]; row: number[]; col: number[]; dtype?: string; }; ``` ### `torch` ```ts interface TorchTensor { data: unknown; shape: number[]; dtype?: string; device?: string; /** Original dtype when the bridge upcast the tensor (e.g. torch.bfloat16). */ sourceDtype?: string; /** Original device when TYWRAP_TORCH_ALLOW_COPY=1 moved the tensor to CPU. */ sourceDevice?: string; } ``` ### `sklearn` ```ts interface SklearnEstimator { className: string; module: string; version?: string; params: Record; } ``` ## Limits and Fallbacks - tywrap does not preserve Python numeric distinctions beyond `number`. - tywrap preserves simple unconstrained `TypeVar`s, generic classes, generic type aliases, and callable `ParamSpec` packs in generated `.ts` and `.d.ts` output when they can be rendered safely. - Complex user-defined generics that are not explicitly normalized stay as custom TypeScript names. - Bound, constrained, or variant `TypeVar`s, plus variadic generics such as `TypeVarTuple` and `Unpack`, still lower conservatively to `unknown`-based shapes. - `P.args` and `P.kwargs` lower conservatively in runtime wrapper signatures as `unknown[]` and `Record`. - `Annotated[...]` metadata is not reflected in the generated type. - Runtime serialization can still shape values more narrowly than the static annotation alone. `torch.Tensor` and `sklearn` values are examples of this. ## Generic Example ```python from typing import Callable, Generic, TypeVar try: from typing import ParamSpec except ImportError: from typing_extensions import ParamSpec T = TypeVar("T") P = ParamSpec("P") Pair = tuple[T, T] Transform = Callable[P, T] class Container(Generic[T]): def get(self) -> T: ... ``` Generates TypeScript shaped like: ```ts export type Pair = [T, T]; export type Transform

= (...args: P) => T; export class Container { // NOTE: Instance members are not generated in v0.10; migrate this API to value-returning module functions. } ``` ## Basic Example ```python from typing import Literal, Optional, Sequence def summarize(values: Sequence[int], mode: Literal["min", "max"]) -> Optional[int]: ... ``` Generates a TypeScript signature shaped like: ```ts export async function summarize( values: Array, mode: 'min' | 'max' ): Promise; ``` --- # Scientific Codec Envelopes tywrap serializes supported SciPy, Torch, and Sklearn values with versioned envelopes. The Python bridge produces them and the JavaScript codec validates them before exposing decoded values to an application. ## Support and failures ### SciPy sparse matrices | Case | Behavior | | --- | --- | | Supported | `csr`, `csc`, and `coo` matrices with integer, floating-point, or boolean dtypes. Empty matrices and any two-dimensional shape work. The dtype and shape are preserved. | | Python rejection | Other formats, including `dia`, `bsr`, and `lil`, fail with the supported formats in the error. Complex dtypes fail rather than being coerced. | | JavaScript validation | The decoder rejects an invalid `indptr` length, mismatched index and data lengths, non-integer or out-of-range indexes, and a shape that does not contain two items. | ### Torch tensors | Case | Behavior | | --- | --- | | Supported | CPU, contiguous, strided tensors with integer or floating-point dtypes. Scalars and tensors of any dimension use an ndarray envelope. Shape, dtype, and device metadata are preserved. | | Opt-in copy | Non-CPU tensors and non-contiguous layouts fail unless `TYWRAP_TORCH_ALLOW_COPY=1`. With that setting, the bridge creates a CPU, contiguous copy. | | Python rejection | Sparse layouts, quantized tensors, `meta` tensors, and complex dtypes always fail. `TYWRAP_TORCH_ALLOW_COPY` does not change those cases. | | JavaScript validation | The decoder rejects negative or non-integer dimensions, a nested ndarray shape that disagrees with the outer shape, an empty device string, and a non-ndarray nested value. | ### Sklearn estimators | Case | Behavior | | --- | --- | | Supported | Metadata only: `className`, `module`, `version`, and `get_params(deep=False)` when every parameter is plain JSON. | | Python rejection | A callable, nested estimator, NumPy array, non-finite number, or other non-JSON parameter fails with the parameter name. | | JavaScript validation | The decoder requires `params` to be a plain JSON object and rejects functions, symbols, bigints, class instances, and non-finite numbers within it. | | Excluded | The envelope never contains `pickle` or `joblib` data. | ## Envelope conventions Every scientific envelope has these fields: - `__tywrap__` identifies the envelope type. - `codecVersion` is `1`. Decoders treat a missing version as legacy version `0`. - `encoding` identifies the payload representation. Scientific values can appear at the root of a result or inside nested Python dictionaries, lists, and tuples. The Python bridge replaces each recognized value with its envelope. The JavaScript codec then walks plain objects and arrays and decodes recognized envelopes within the traversal bounds. A decoded envelope is a terminal value, so fields inside its decoded output are not scanned again. A `torch.tensor` envelope is the exception because its `value` field is a required nested ndarray envelope. Serialization accepts a maximum nesting depth of 900 and at most 1,000,000 visited containers. Decoding accepts a maximum depth of 2048 and the same 1,000,000-container limit. Scientific envelopes count as containers, including the ndarray nested inside a tensor. Primitive leaves and the elements of a decoded envelope payload do not consume the container limit. The bridge reports the result path when a serialization bound is exceeded, and the JavaScript codec does the same for decoding bounds. The bridge also rejects circular Python containers. ### SciPy sparse Supported targets are `scipy.sparse.csr_matrix`, `csc_matrix`, and `coo_matrix`. The block below is a schema sketch, not a literal payload: ```jsonc { "__tywrap__": "scipy.sparse", "codecVersion": 1, "encoding": "json", "format": "csr" | "csc" | "coo", "dtype": "float64", "shape": [rows, cols], "data": [ ... ], // csr/csc only: "indices": [ ... ], "indptr": [ ... ], // coo only: "row": [ ... ], "col": [ ... ] } ``` There is no dense fallback. Convert a matrix explicitly before returning it if your consumer needs a dense value. ### Torch tensor The target is `torch.Tensor`. CPU tensors work by default. Schema sketch: ```jsonc { "__tywrap__": "torch.tensor", "codecVersion": 1, "encoding": "ndarray", "shape": [ ... ], "dtype": "torch.float32", "device": "cpu" | "cuda:0" | ..., "sourceDtype": "torch.bfloat16", // optional; original dtype before conversion "sourceDevice": "cuda:0", // optional; original device before CPU copy // Nested ndarray envelope: "value": { "__tywrap__": "ndarray", "codecVersion": 1, "encoding": "arrow" | "json", "b64": "...", // when encoding="arrow" "data": [ ... ], // when encoding="json" "shape": [ ... ], "dtype": "float32" } } ``` `torch.bfloat16` tensors are upcast exactly to `torch.float32` for transport, with the original dtype recorded in `sourceDtype`. Set `TYWRAP_TORCH_ALLOW_COPY=1` only when the CPU transfer or contiguous copy is acceptable for the call. ### Sklearn estimator Sklearn prediction and score values use the existing NumPy and pandas codecs. An estimator itself can use this metadata envelope (schema sketch): ```jsonc { "__tywrap__": "sklearn.estimator", "codecVersion": 1, "encoding": "json", "className": "LinearRegression", "module": "sklearn.linear_model._base", "version": "1.4.2", "params": { ... } } ``` ## Arrow and JSON Arrow is the default encoding for ndarrays, DataFrames, and Series when both sides support it: the Python bridge needs `pyarrow` (reported as `arrowAvailable` in `BridgeInfo`) and the JavaScript runtime needs the optional `apache-arrow` peer dependency. The Pyodide transport is JSON-only because pyarrow is not available in WASM. If an Arrow payload arrives without the dependency, decoding fails with an installation hint. Set `TYWRAP_CODEC_FALLBACK=json` on the Python side when a JSON representation is acceptable. An ndarray JSON envelope includes `shape`, `dtype`, and plain JSON `data`. The decoder checks that the data matches the declared shape and supported dtype domain. It returns plain JavaScript arrays, or a scalar for a zero-dimensional array. The `dtype` field records provenance for validation; it does not create a typed JavaScript array. JSON encoding rejects ndarray dtypes and values that it cannot represent safely, including complex, structured, object, byte-string, unicode, datetime, timedelta, big-endian, and unsafe integer cases. DataFrame and Series JSON envelopes are values-only representations. Before encoding, the bridge requires an unnamed `RangeIndex` starting at 0 with step 1. It rejects a `MultiIndex`, categorical dtype, unsupported cell values, unsafe integers, non-finite floats, duplicate DataFrame column labels, and labels that collide after JSON object-key coercion. `None`, `pd.NA`, and `pd.NaT` become JSON `null`. Supported booleans, numbers, strings, empty values, and nulls keep their value semantics, but pandas dtype and index metadata are not retained as they are with Arrow. ## Size limits and transport framing `TYWRAP_CODEC_MAX_BYTES` limits the serialized response size emitted by the Python bridge. It accepts a byte count in UTF-8. A response that exceeds the limit fails instead of changing encoding. Subprocess messages larger than one JSONL line use the `tywrap-frame/1` framing protocol. See [Transport framing](https://bbopen.github.io/tywrap/transport-framing.md) for its size limits, negotiation history, and backend support. ## Feature detection The bridge `meta` response reports `scipyAvailable`, `torchAvailable`, and `sklearnAvailable`. Each field says whether the matching Python package is importable in that bridge environment. ## Runtime return validation Generated module functions and static methods validate decoded results before their promises resolve. A result that does not match its declared return type throws `BridgeValidationError` with the declared type, received shape, and call site. Decoded scientific values retain their envelope provenance. Return validation recognizes all six markers: `ndarray`, `dataframe`, `series`, `scipy.sparse`, `torch.tensor`, and `sklearn.estimator`. It checks the marker and, when present in the return contract, dimensions and dtype without walking Arrow data. Returns declared as `unknown`, `void`, or `Any` do not receive a runtime check. --- # Transport capability matrix Every tywrap transport exposes a static, transport-level capability descriptor via `Transport.capabilities()`. The same descriptor is surfaced by `RpcClient.capabilities()` (it delegates to the held transport). It tells callers what the wire channel can carry and how it frames messages **without** a network round-trip. This descriptor is deliberately separate from the bridge `meta` report (`BridgeInfo`, fetched via `RpcClient.getBridgeInfo()`): - The **transport descriptor** is authoritative for transport-level flags: what bytes the channel moves and how it frames them. It does not depend on lifecycle state, so it is safe to read before `init()` and after `dispose()`. - The **`BridgeInfo` meta report** is authoritative for the _Python environment_, including which optional libraries (`arrowAvailable`, `scipyAvailable`, `torchAvailable`, `sklearnAvailable`) happen to be importable in the running interpreter. To ask whether a transport can carry Arrow and whether Python has pyarrow, consult both: the transport descriptor for the channel, `BridgeInfo` for library availability. ## `TransportCapabilities` ```typescript interface TransportCapabilities { backend: 'subprocess' | 'http' | 'pyodide'; supportsArrow: boolean; supportsBinary: boolean; supportsChunking: boolean; supportsStreaming: boolean; maxFrameBytes: number; } ``` ## Matrix | Backend (transport) | `backend` | `supportsArrow` | `supportsBinary` | `supportsChunking` | `supportsStreaming` | `maxFrameBytes` | | ---------------------------- | ------------ | --------------- | ---------------- | -------------------------- | ------------------- | ---------------------------------- | | `SubprocessTransport` (Node) | `subprocess` | `true` | `true` | `true` (always-on framing) | `false` | Per-frame ceiling (default 100 MB) | | `HttpTransport` | `http` | `true` | `true` | `false` | `false` | `Number.POSITIVE_INFINITY` | | `PyodideTransport` (WASM) | `pyodide` | `false` | `true` | `false` | `false` | `Number.POSITIVE_INFINITY` | `SubprocessTransport.supportsChunking` is always `true`. The npm package includes the JS and Python framing peers together, so no runtime negotiation is needed. Chunking is subprocess-only. HTTP and Pyodide have no JSONL line ceiling. See [Transport framing](https://bbopen.github.io/tywrap/transport-framing.md). `PooledTransport` (the multi-process Node path) reports the capabilities of the worker transport it distributes across, which is `SubprocessTransport` in practice. Its `capabilities()` is a **static** descriptor read from an un-initialized probe worker built by the same factory. Each leased worker always uses `tywrap-frame/1`, and chunked traffic through a lease reassembles correctly. ## Notes per flag ### `supportsArrow` Whether the transport can carry Arrow-encoded payloads (binary IPC frames) on the wire. - **subprocess / http**: `true`. The channel can move Arrow bytes. Whether Arrow is actually _used_ for a given response still depends on the Python side (`BridgeInfo.arrowAvailable`) and the `TYWRAP_CODEC_FALLBACK` setting. Those are runtime/codec concerns, not transport-level ones. - **pyodide**: `false`. pyarrow is unavailable in WASM, so the Pyodide bootstrap forces JSON markers (`force_json_markers=True`) and reports `arrowAvailable: false`. The channel is JSON-only. ### `supportsBinary` Whether the transport can carry arbitrary binary data (e.g. Python `bytes`). It is `true` on all current backends. Binary rides through base64 `bytes` envelopes. ### `supportsChunking` and `supportsStreaming` `supportsChunking` is implemented for the **subprocess** backend as of **0.8.0**: it splits one logical message across multiple `tywrap-frame/1` frames so a payload can exceed the JSONL line ceiling. The capability is statically `true` for subprocess. HTTP and Pyodide stay `false`. See [Transport framing](https://bbopen.github.io/tywrap/transport-framing.md) for the wire format. `supportsStreaming` (incremental results for a single request) is `false` on every backend. It is not implemented as of 0.10.0. ### `maxFrameBytes` Maximum size, in bytes, of a single wire frame the transport itself will accept. `Number.POSITIVE_INFINITY` means the transport imposes no frame ceiling of its own. A higher layer, such as the codec's default 10 MB payload limit, may still cap the size). - **subprocess**: the JSONL line-length limit (`maxLineLength`, default `100 * 1024 * 1024` = 100 MB). A response line larger than this raises a protocol error. - **http**: `Number.POSITIVE_INFINITY`. The whole response body is read in one shot. The transport imposes no frame limit. - **pyodide**: `Number.POSITIVE_INFINITY`. Calls are in-memory string passing with no framing. ## Example ```typescript import { RpcClient } from 'tywrap/runtime'; // rpc holds a transport (e.g. via NodeBridge / PyodideBridge / HttpBridge). const caps = rpc.capabilities(); if (caps.supportsArrow) { // The channel can carry Arrow; pair with getBridgeInfo() to confirm pyarrow // is actually importable on the Python side before relying on Arrow encoding. const info = await rpc.getBridgeInfo(); const useArrow = info.arrowAvailable; } ``` --- # Chunked transport framing (`tywrap-frame/1`) Large Python results can exceed the single-line ceiling of the subprocess JSONL channel (`maxLineLength`, default 100 MB). `tywrap-frame/1` is the framing protocol that fragments one logical message across multiple wire frames and reassembles it on the other side, so a payload larger than one frame can still cross the boundary without a silent lossy fallback. This document is the spec. The wire contract and fragment/reassembly machinery are always enabled for the subprocess backend: the TS side fragments/reassembles in `src/runtime/subprocess-transport.ts` (with the pure codec in `src/runtime/frame-codec.ts`), and the Python side does the same in `runtime/python_bridge.py`. Chunking composes with the worker pool. Each leased worker uses the same packaged codec and a chunked request or response routed through a `PooledTransport` lease reassembles correctly (see [the capability matrix](https://bbopen.github.io/tywrap/transport-capabilities.md)). ## Scope: subprocess only `tywrap-frame/1` is **subprocess-only** as of 0.10.0. The subprocess transport is the only backend with a real frame ceiling, the JSONL line-length limit. The other backends stay single-frame and keep `supportsChunking: false`: - **HTTP** has no line ceiling, but still buffers a whole response via `response.text()`; chunking there is a later optimization, not a correctness requirement. - **Pyodide** is in-memory (`maxFrameBytes: Number.POSITIVE_INFINITY`, JSON-only). There is no wire to fragment. `supportsStreaming` stays `false` on every backend as of 0.10.0. ## Layering Chunking lives in the transport I/O layer, **below** `RpcClient`. `RpcClient` keeps its one-logical-string-in / one-logical-string-out contract; it never sees frames. Both sides fragment and reassemble: - TypeScript: `src/runtime/subprocess-transport.ts`. - Python: the read/write loop in `runtime/python_bridge.py`. `runtime/tywrap_bridge_core.py` (dispatch + serializers) stays oblivious. It produces and consumes complete logical JSON messages. The logical RPC protocol stays `tywrap/1`. `tywrap-frame/1` is a separate framing protocol below it. The npm package includes both sides, so version-skew negotiation would add states without providing compatibility. ## Frame envelope A frame envelope is distinct from the logical `ProtocolMessage` / `ProtocolResponse`. It carries a slice of the bytes of one **complete** logical JSON message: ```json { "__tywrap_frame__": "chunk", "frameProtocol": "tywrap-frame/1", "stream": "request", "id": 42, "seq": 0, "total": 8, "totalBytes": 7340032, "encoding": "utf8-slice", "data": "..." } ``` | Field | Meaning | | ------------------ | ------------------------------------------------------------------------------- | | `__tywrap_frame__` | `"chunk"` (data frame) or `"error"` (framing-layer error that ends the stream). | | `frameProtocol` | Must equal `tywrap-frame/1` (`FRAME_PROTOCOL_ID`). Any other value is rejected. | | `stream` | `"request"` (JS to Python) or `"response"` (Python to JS). | | `id` | RPC correlation id, shared with the logical `ProtocolMessage.id`. | | `seq` | Zero-based frame index within the stream. | | `total` | Total frame count for the stream, repeated on every frame. | | `totalBytes` | Byte length of the complete reassembled message, repeated on every frame. | | `encoding` | Per-frame payload encoding: `"utf8-slice"` (chosen) or `"utf8-base64"`. | | `data` | This frame's slice of the logical message, encoded per `encoding`. | The TypeScript type is `ChunkFrame` in `src/runtime/transport.ts`; the constants are `FRAME_PROTOCOL_ID` and `FRAME_PROTOCOL_VERSION` (the version derived from the trailing number, the same pattern as `TYWRAP_PROTOCOL_VERSION`). ## Encoding: `utf8-slice` (chosen) **Decision (plan decision #6): `tywrap-frame/1` uses `encoding: "utf8-slice"`.** The chunked payload is the bytes of a complete, valid-UTF-8 JSON message. Two candidates were considered: - **`utf8-base64`:** base64-encode the UTF-8 bytes of each chunk. Safe for arbitrary byte split points, but inflates the wire by ~33% and forces a full base64 decode plus a separate bytes buffer per frame. Holding the string, UTF-8 bytes, and base64 form at once multiplies memory use. - **`utf8-slice`** (chosen). Because the payload is already valid UTF-8, split it on **UTF-8 codepoint boundaries** and embed the raw string slice directly. Reassembly is plain concatenation of the slices; the result is the original JSON string verbatim. No base64, no ~33% inflation, no extra decode pass, and the memory ceiling stays close to `payload × (small constant)`. **Rationale.** `utf8-slice` is strictly cheaper on both wire size and memory for the one payload shape we ever chunk (a JSON message), and the only thing `utf8-base64` buys, tolerance for splitting mid-codepoint, is unnecessary when the sender controls the split points and splits on codepoint boundaries by construction. The codepoint-boundary rule is a sender obligation: a frame's `data` MUST NOT split a multi-byte UTF-8 sequence, so the receiver can decode each frame as valid UTF-8 and concatenate without re-aligning bytes across frame edges. `utf8-base64` remains a defined alternative in the wire schema for future use (e.g. a non-text payload), but tywrap does not emit it as of 0.10.0. > Implementation note for later workstreams: `total` and the per-frame split > points are computed over the message's UTF-8 **byte** length against > `maxFrameBytes` (the frame ceiling is a byte limit), but each frame boundary > is snapped back to the nearest codepoint boundary at or before the byte limit > so no multi-byte sequence is split. `totalBytes` is the exact UTF-8 byte > length of the full reassembled message. ## Correlation and reassembly - **Correlation** reuses the existing RPC `id`. A receiver groups inbound frames by `(stream, id)`. - `seq` is zero-based and dense: a complete stream has exactly one frame for each `seq` in `[0, total)`. - `total` and `totalBytes` are repeated on every frame and MUST be identical across all frames of a stream. A mismatch is a framing error. After the receiver has all `total` frames for an `id`, it validates **before** decoding: 1. `frameProtocol === FRAME_PROTOCOL_ID` on every frame. 2. No duplicate `seq`. The full `[0, total)` range is present exactly once. 3. The number of frames equals `total`. 4. The concatenated payload's UTF-8 byte length equals `totalBytes` exactly. 5. The concatenated payload decodes as strict UTF-8. Only when all five hold does the framing layer hand the reassembled string to the existing JSON/codec path and resolve the pending `id`. Any failure rejects the pending `id` and marks the subprocess for restart (the stdout stream can no longer be trusted to be frame-aligned). There is no silent single-frame fallback: a payload that requires chunking against a bridge that cannot chunk fails explicitly. ### Resource bounds Chunking removes the per-line stdout ceiling, so the reassembler enforces its own bounds, so a buggy or oversized peer cannot grow memory without limit: - **Logical payload cap.** The response reassembler is constructed with `maxReassemblyBytes` (default 10 MiB, matching the codec's `maxPayloadBytes`; `NodeBridge` sets it from the configured `codec.maxPayloadBytes`). A stream whose declared `totalBytes` or accumulated bytes exceeds the cap fails loud (`FRAME_PAYLOAD_TOO_LARGE`) on the first offending frame rather than buffering the whole payload. To carry a payload larger than 10 MiB you raise the codec cap. NodeBridge moves both together. The frame ceiling only controls wire fragmentation and is not a second logical payload limit. (Requests are already bounded by the codec's `encodeRequest` cap on the sending side, so the Python request reassembler relies on the post-reassembly `TYWRAP_REQUEST_MAX_BYTES` check.) - **Concurrent streams** are capped (`FRAME_TOO_MANY_STREAMS`) and the timed-out id discard set is FIFO-bounded, so neither grows without limit over a long-lived process. - **Stream direction** is enforced: the response reassembler rejects `request` frames and vice-versa. ## Always-on framing `SubprocessTransport` always accepts and emits `tywrap-frame/1`. It passes its configured `maxLineLength` to the packaged Python bridge as a private process argument, and both sides use that value as the per-frame data ceiling. Small messages remain one JSONL line. Messages over the ceiling are framed. ### Meta validation The same pass relaxes two over-strict `meta` checks so the honest per-backend identities validate: - `bridge` accepts the full `BridgeBackend` union (`python-subprocess` | `pyodide` | `http`) instead of hardcoding `python-subprocess`. All backends speak the identical `tywrap/1` protocol; this lets the Pyodide and HTTP facades route `getBridgeInfo()` through the same validator. (Today the Python subprocess server reports `python-subprocess`, the Pyodide bootstrap reports `pyodide`; HTTP uses the subprocess server and so reports `python-subprocess`. Honest per-backend HTTP reporting is a possible follow-up, but accepting the union now is safe and backward compatible.) - `pid` accepts a positive integer or `null`. Subprocess reports a real OS pid. In-WASM Pyodide (and HTTP) have no local process and report `null`. ## Backpressure The TS side needs a per-logical-request write mutex so one request's frames are written contiguously to stdin (reuse the existing `writeToStdin` drain primitive). Interleaving two requests' frames would corrupt both streams. Python writes and flushes one frame at a time. The OS pipe provides backpressure. ## Timeout, abort, and late-frame discard One timeout spans the whole logical exchange: request-write + execution + response-read + reassembly. On timeout or abort: - The JS side stops writing further request frames, rejects the logical `send`, and marks the `id` as timed out. - **Late response frames must be tracked and discarded.** A multi-frame response may still be arriving when the timeout fires. The current one-shot "consume a single timed-out id" logic is insufficient for streams: leftover frames for a timed-out `id` would otherwise desync the stdout reader and corrupt the next response. The reassembler keeps a discard set of timed-out ids and drops every frame whose `id` is in that set until the stream for that `id` completes (its `total`-th frame arrives) or the subprocess is restarted. ## Errors mid-stream - **Ordinary Python exceptions** stay normal: they serialize to a (possibly chunked) `ProtocolResponse.error` and flow through the standard path. - **Framing failures** emit `__tywrap_frame__: "error"` where the sender can still write (so the receiver learns the stream is aborted rather than waiting for frames that will never come). - **Unrecoverable framing corruption:** a malformed frame, a duplicate `seq`, a `totalBytes` mismatch, an unknown `frameProtocol`, or non-frame stdout pollution rejects the pending `id` and marks the subprocess for restart, because the stdout stream can no longer be trusted to be frame-aligned. ## See also - [Transport capability matrix](https://bbopen.github.io/tywrap/transport-capabilities.md) for backend framing support. - [Environment variables](https://bbopen.github.io/tywrap/reference/env-vars.md) for logical payload size guards. --- # API Reference This page covers the public programmatic API exported by `tywrap`. ## Core API ### `defineConfig(config)` Type-safe helper for `tywrap.config.ts`. ```ts import { defineConfig } from 'tywrap'; export default defineConfig({ pythonModules: { math: { typeHints: 'strict' }, }, }); ``` ### `resolveConfig(options?)` Loads a config file and merges it with defaults plus overrides. ```ts import { resolveConfig } from 'tywrap'; const config = await resolveConfig({ configFile: './tywrap.config.ts', overrides: { debug: true }, }); ``` ### `generate(options, runOptions?)` Generates wrapper files from config. ```ts import { generate } from 'tywrap'; await generate({ pythonModules: { math: { typeHints: 'strict' }, }, output: { dir: './generated', format: 'esm', declaration: false, sourceMap: false, }, }); ``` `runOptions.check` switches generation into compare-only mode, the same behavior used by `tywrap generate --check`. When `output.declaration` is enabled, `generate()` writes matching `.generated.d.ts` files from the same generic-aware pass as the runtime wrapper code. Simple `TypeVar` and callable `ParamSpec` declarations are preserved when tywrap can represent them safely. Unresolved return annotations now emit `Promise` and a located warning. This includes `TypeVar` and `ParamSpec` results without a supported value conversion. Input generics remain in the declaration. Earlier versions could emit `Promise` without validating the decoded value as `T`; callers must narrow the result or use a supported concrete overload. `generate()` returns: ```ts interface GenerateFailure { module: string; code: 'ir-unavailable'; message: string; } interface GenerateResult { written: string[]; warnings: string[]; failures: GenerateFailure[]; outOfDate?: string[]; } ``` `failures` is the structured fatal-generation channel. Compatibility warnings may still be present in `warnings`, but callers should use `failures` to detect modules that could not produce IR. ### `tywrap(options?)` Creates the lower-level mapper and generator objects for advanced use. ```ts import { tywrap } from 'tywrap'; const instance = await tywrap({ types: { presets: ['stdlib'] }, }); ``` ## Runtime Registry Generated wrappers call into a shared runtime bridge. ```ts import { setRuntimeBridge, clearRuntimeBridge } from 'tywrap/runtime'; import { NodeBridge } from 'tywrap/node'; const bridge = new NodeBridge({ pythonPath: 'python3' }); setRuntimeBridge(bridge); // later clearRuntimeBridge(); ``` ## Runtime Bridges tywrap does not create bridges from config for you. Your application constructs one explicitly. ### `NodeBridge` `NodeBridge` runs Python in a subprocess and is the default bridge for Node.js, Bun, and local Deno. ```ts import { NodeBridge } from 'tywrap/node'; const bridge = new NodeBridge({ pythonPath: 'python3', virtualEnv: './venv', timeoutMs: 30000, }); const result = await bridge.call('math', 'sqrt', [16]); const info = await bridge.getBridgeInfo({ refresh: true }); ``` | Option | Default | Notes | | ------------------------- | --------------- | --------------------------------------------------------------------------------------- | | `pythonPath` | auto-detect | Python executable | | `scriptPath` | built-in bridge | Custom `python_bridge.py` | | `virtualEnv` | not set | Virtual environment root | | `cwd` | `process.cwd()` | Subprocess working directory | | `timeoutMs` | `30000` | Per-call timeout | | `queueTimeoutMs` | `30000` | Wait time when the worker pool is saturated | | `minProcesses` | `1` | Minimum worker count | | `maxProcesses` | `1` | Maximum worker count | | `maxConcurrentPerProcess` | `1` | Concurrent requests per serial Python worker; use more worker processes for concurrency | | `inheritProcessEnv` | `false` | Pass full parent env through | | `env` | `{}` | Extra subprocess env vars | | `codec` | not set | `CodecOptions` for validation and byte handling | | `warmupCommands` | `[]` | Per-worker startup calls | Deprecated compatibility fields still exist on the interface: `maxIdleTime`, `maxRequestsPerProcess`, `enableJsonFallback`, and `maxLineLength`. ### `PyodideBridge` `PyodideBridge` runs Python in the browser through WebAssembly. ```ts import { PyodideBridge } from 'tywrap/pyodide'; const bridge = new PyodideBridge({ indexURL: 'https://cdn.jsdelivr.net/pyodide/v0.28.0/full/', packages: ['numpy'], timeoutMs: 30000, }); ``` | Option | Default | Notes | | ----------- | ----------- | ---------------------------- | | `indexURL` | Pyodide CDN | Package index URL | | `packages` | `[]` | Packages to load during init | | `timeoutMs` | `30000` | Default operation timeout | | `codec` | not set | `CodecOptions` | ### `HttpBridge` `HttpBridge` sends protocol messages over HTTP POST to a compatible server. ```ts import { HttpBridge } from 'tywrap/http'; const bridge = new HttpBridge({ baseURL: 'https://api.example.com/python', timeoutMs: 10000, headers: { Authorization: 'Bearer token' }, }); ``` | Option | Default | Notes | | ----------- | -------- | ----------------------- | | `baseURL` | required | Server endpoint URL | | `headers` | `{}` | Extra request headers | | `timeoutMs` | `30000` | Default request timeout | | `codec` | not set | `CodecOptions` | ## HTTP Server Contract `HttpBridge` sends the same JSON protocol used by the other transports. The request body is a serialized `ProtocolMessage`, and the server must reply with a serialized `ProtocolResponse`. Request shape: ```json { "id": 1, "protocol": "tywrap/1", "method": "call", "params": { "module": "math", "functionName": "sqrt", "args": [16], "kwargs": {} } } ``` Success response: ```json { "id": 1, "protocol": "tywrap/1", "result": 4 } ``` Error response: ```json { "id": 1, "protocol": "tywrap/1", "error": { "type": "ValueError", "message": "math domain error", "traceback": "..." } } ``` ## Codec Utilities ```ts import { autoRegisterArrowDecoder, registerArrowDecoder, clearArrowDecoder, decodeValue, decodeValueAsync, } from 'tywrap'; ``` - `autoRegisterArrowDecoder()` tries to load `apache-arrow` and register a decoder. - `registerArrowDecoder(fn)` installs a custom Arrow decoder. - `clearArrowDecoder()` removes the current Arrow decoder. - `decodeValue()` and `decodeValueAsync()` decode runtime envelopes such as ndarray, sparse matrix, torch tensor, and sklearn estimator payloads. ## Error Types ```ts import { BridgeError, BridgeCodecError, BridgeProtocolError, BridgeTimeoutError, BridgeDisposedError, BridgeExecutionError, } from 'tywrap'; ``` ## Dev Helpers ```ts import { createBridgeReloader, startNodeWatchSession } from 'tywrap/dev'; ``` - `startNodeWatchSession(...)` is the Node-only development hot reload helper for wrapper regeneration plus bridge swap. - `startNodeWatchSession(...)` passes the resolved config for that reload cycle into `createBridge(config)`. - `startNodeWatchSession(...)` watches local package trees by attaching one watcher per discovered directory, then refreshing that tree when directories are added, removed, or renamed. - `startNodeWatchSession(...)` keeps the last known good generated output and bridge live if a reload hits structured generation failures. - `createBridgeReloader(...)` is the manual reload primitive for cases like Pyodide. - HTTP server reload remains external to tywrap. ## Key Types ```ts interface TywrapOptions { pythonModules: Record; pythonImportPath?: string[]; output: OutputConfig; runtime: RuntimeConfig; performance: PerformanceConfig; types?: TypeMappingConfig; debug?: boolean; } ``` For the full exported type surface, see `src/index.ts`. --- # Architecture tywrap has a generation pipeline and a runtime pipeline. The generation pipeline turns Python declarations into TypeScript wrappers. The runtime pipeline executes a generated call and returns a decoded result that must match the declared return schema. ## Pipeline | Stage | Owners | Invariant | | --- | --- | --- | | Python analysis | `tywrap_ir/tywrap_ir/__main__.py`, `tywrap_ir/tywrap_ir/` | The extractor emits JSON typed IR for the requested module. | | IR contract | `src/core/ir-contract.ts`, `src/tywrap.ts` | Pure validation checks the complete IR structure and schema version `0.4.0`; generation checks the requested module. | | Callable compilation | `src/core/callable-compiler.ts`, `src/contracts/value-contract.ts` | Resolved contracts describe arguments, overloads, conversion, validation, capabilities and diagnostics without I/O. | | Wrapper generation | `src/core/generator.ts`, `src/core/emit-call.ts` | Declarations, call wrappers and return validators consume the resolved callable contracts. | | Runtime transport | `src/runtime/node.ts`, `src/runtime/subprocess-transport.ts`, `src/runtime/frame-codec.ts`, `runtime/python_bridge.py` | Node subprocess requests use JSONL and use `tywrap-frame/1` when a message exceeds one line. | | Python dispatch and encoding | `runtime/tywrap_bridge_core.py`, `runtime/safe_codec.py` | The bridge validates protocol input, then dispatches the requested call and returns JSON-safe values or versioned envelopes. | | JavaScript decoding | `src/utils/codec.ts`, `src/runtime/bridge-codec.ts` | Envelope shape and payload domains are checked before a decoded value reaches application code. | | Return validation | `src/runtime/validators.ts` | A decoded result matches the generated `ReturnSchema`, including scientific provenance when declared. | ## IR and generated contracts `generate()` in `src/tywrap.ts` owns Python discovery, IR caching and file writes. When it needs fresh IR, it invokes `python -m tywrap_ir --module `. The pure IR validator checks the complete returned structure with located errors. `compileContract()` resolves that IR using explicit conversion rules and runtime capabilities. It does not access the filesystem, Python interpreter or runtime registry. The generator emits wrappers and declarations from those resolved contracts. `generate()` writes `.generated.ts` and a stable `.contract.json`. The saved contract contains canonical Python IR, not the resolved callable model. It omits extractor metadata and sorts object keys. `contractInput` changes the source of IR. Instead of starting Python, generation reads a saved contract and applies the same complete IR validation. This supports reproducible generation. A version mismatch is a hard failure: update the matching package pair and regenerate the contract. The callable contract connects a result's logical type, decoded representation and validator. A precise return declaration requires a supported conversion and validation path. Unresolved results become `unknown` with diagnostics while retaining available partial runtime checks. The generator emits `createReturnValidator()` calls from these contracts. Return schemas cover primitives, containers, definitions and scientific markers. Each wrapper calls the active bridge and validates the decoded result before its promise resolves. See [callable contracts](https://bbopen.github.io/tywrap/maintainers/callable-contracts.md) for compiler boundaries and [value contracts](https://bbopen.github.io/tywrap/maintainers/value-contracts.md) for conversion policy. ## Runtime bridges `NodeBridge` starts `runtime/python_bridge.py` through the subprocess transport. Normal requests and responses are JSONL. When a request or response crosses the line ceiling, `src/runtime/frame-codec.ts` splits it into `tywrap-frame/1` frames. Reassembly remains bounded by the codec payload limit. `PyodideBridge` runs the shared Python bridge core in WebAssembly. Its generated bootstrap source lives in `src/runtime/pyodide-bootstrap-core.generated.ts`; do not edit it directly. Regenerate it from `runtime/tywrap_bridge_core.py` with the repository script. Pyodide uses JSON codec envelopes because pyarrow is not available in WASM. The bridge awaits Python call results under the event loop owned by each backend. See [coroutine calls](https://bbopen.github.io/tywrap/maintainers/coroutines.md) for timeout and disposal rules. `HttpBridge` sends the same RPC and codec concepts to a remote Python service. The transport owns HTTP concerns while the codec and return validator preserve the client-side invariants. ## Codec envelopes `runtime/tywrap_bridge_core.py` recognizes scientific values and replaces them with envelopes marked by `__tywrap__`, `codecVersion: 1`, and an `encoding`. The marker set is `ndarray`, `dataframe`, `series`, `scipy.sparse`, `torch.tensor`, and `sklearn.estimator`. `src/utils/codec.ts` is the matching decoder and validates each envelope before decoding it. The Python producer selects Arrow for scientific arrays and pandas tabular values whenever pyarrow is importable, and the JavaScript side never falls back on its own: an Arrow envelope arriving without an Arrow decoder throws with an installation hint. Choosing JSON is an explicit Python-side decision through `TYWRAP_CODEC_FALLBACK=json`, with narrower, documented domains. Pandas JSON values require an unnamed zero-based `RangeIndex`, and unsupported dtypes fail with a conversion recipe. Nested arrays and plain objects can contain envelopes. The producer permits a depth of 900 and at most 1,000,000 visited containers. The decoder permits a depth of 2048 and the same container count. The budget excludes primitive leaves. A decoded envelope is terminal, except for the required nested ndarray inside a torch envelope. ## Runtime return validation `src/runtime/validators.ts` validates the value after decoding. A mismatch throws `BridgeValidationError` with the call site, declared type, and received shape. Scientific values retain provenance from their envelope. Scalar scientific results use per-call provenance because a JavaScript number cannot hold metadata. The validator checks the marker and, when the schema declares them, dimensions and dtype without walking Arrow payload data. Explicit `unknown`, `void`, and `Any` schemas perform no check. A result whose declaration became `unknown` can still retain partial checks from its original annotation. ## One call end to end 1. A generated call such as `math.sqrt(16)` builds a bridge request and its generated `ReturnSchema` validator. 2. `NodeBridge` sends the request over JSONL, or `tywrap-frame/1` frames when it is too large for one line. 3. `runtime/python_bridge.py` reads the request and calls `dispatch_request()` in `runtime/tywrap_bridge_core.py`. 4. The core imports the requested module, dispatches `sqrt`, and serializes its result. A scientific result becomes a version-1 envelope; a plain number remains JSON. 5. The subprocess transport returns the response. `BridgeCodec` and `decodeEnvelopeAsync()` validate and decode its value. 6. The generated return validator accepts the decoded number and resolves the promise. A mismatched value throws `BridgeValidationError` instead. Read [Scientific Codec Envelopes](https://bbopen.github.io/tywrap/codec-envelopes) before changing an envelope. It defines the supported JSON domains and validation behavior. --- # Callable compilation contracts `tywrap_ir` emits IR 0.4. `validateIrContract()` checks required fields in unknown JSON. Each diagnostic has a JSON path. Validation stops after 100,000 array entries. `compileContract()` is an internal pure step. It accepts validated IR, exported-name selection, value conversions, capabilities, and a generator. It maps callable types from the validated IR. The selection input cannot change those types. It returns the resolved module, callable records, diagnostics, and TypeScript and declaration content. The compiler does not read files. It does not start Python. It does not read a cache. It does not write files. It does not access a runtime registry. `generate()` owns Python discovery, cache reads, output comparison, and writes. It validates IR and maps annotations, then calls `compileContract()` before emission. Offline contract files omit extractor metadata for stable bytes. The reader restores an empty metadata object. ## Resolved callable data The compiler records each parameter and result, including overloads. Each record has a Python type, direction, source path, and conversion result. One resolved module model serves declarations, call wrappers, and return validators. Each value conversion uses revision 2 of [value-contracts.v2.json](https://bbopen.github.io/tywrap/maintainers/value-contracts.v2.json). Do not infer a wire rule from the TypeScript type. | Case | Result | | --------------------- | ------------------------------------------------------------------------------------- | | `int` | JSON number, JavaScript number, safe-integer constraint | | `bytes` | Existing base64 envelope, decoded as `Uint8Array` | | nested `dict[str, T]` | JSON record with the child conversion at every value path | | `str \| int` | Selected union alternative with one child contract per option | | `tuple[int, str]` | Exact JSON array with a contract for each index | | float16 ndarray | Existing Arrow or JSON-fallback envelope, decoded as a numeric scalar or nested array | | float16 Torch tensor | Outer `torch.float16` envelope with a nested `float16` ndarray | The compiler requires an explicit float16 dtype. An untyped ndarray or Torch tensor stays unresolved. For an evaluated PyTorch annotation, use `torch.HalfTensor`. The compiler accepts this exact class and leaves other Torch tensor classes unresolved. NumPy can spell `NDArray[np.float16]` as `numpy.ndarray[tuple[Any, ...], numpy.dtype[numpy.float16]]`. The compiler reads the dtype argument, not the shape wildcard. Only a supported scientific result clears its matching external-annotation IR warning. Other external annotations remain warnings. The codec records scalar ndarray dtype and rank for one call. A generated return validator accepts a scalar only with matching proof. Nested proof follows its parent and array index or record key. Array index `0` and `"0"` refer to the same child. Equal numbers from separate calls do not share proof. Array and tensor objects keep their identity-based marker proof. The compiler keeps Arrow int64 behavior outside this prototype. Existing Arrow int64 values can decode as `bigint`. Selected local non-generic `TypedDict` classes use revision-2 record contracts when every field has a supported conversion. Nested records and selected simple aliases compose through the same resolver. Foreign and recursive names remain unresolved. ## Diagnostics and fallback Known unsupported cases create error diagnostics. The compiler changes the affected generated type to `unknown`. `generate()` returns each diagnostic as a warning, so `--fail-on-warn` rejects strict builds. - The default compiler capability covers coroutine execution in the in-repo Node and Pyodide bridges. A caller can mark this capability unavailable when it uses an older or custom bridge. The compiler then emits `unknown` and an error diagnostic for async results. - Dataclasses require an adapter owned by #339. The default capability stays unavailable. An explicit output record conversion can use the adapter capability when enabled. - Non-string record keys require an explicit conversion. Unresolved annotations create warning diagnostics. Their result type is `unknown`, including an output that contains `object` or a type variable. Python `object` can hold a primitive value. A type variable has no validated conversion to the decoded result. Neither annotation justifies a precise TypeScript return. The compiler keeps an existing best-effort return check when it widens a declaration. For example, an unresolved `list[object]` still rejects a non-array result, and a `DataFrame` annotation still requires a decoded marker. `Literal` returns keep their exact-value validator, but stay `unknown` until a literal value contract exists. Unresolved literal overloads cannot select a precise return at runtime. IR overload signatures stay separate from the implementation signature. The extractor collects type variables from overload annotations. Generated declarations retain each input-to-result relation. The hidden TypeScript implementation accepts `unknown` values so it can cover every declared overload. At runtime, the wrapper selects the first supported overload that matches bound arguments. This follows TypeScript declaration order. The compiler warns when overload inputs may overlap, including an omitted optional argument. Python 3.10 cannot expose registered overloads through `typing.get_overloads`; it keeps the implementation signature and emits an IR warning. ## Deferred work Revision 2 does not describe references or recursive aliases. The compiler reports these cases as unresolved. A later value-contract revision can give them a supported conversion. The client-binding work may consume the resolved callable model later. It must remain opt-in. It must not add files to default generation or mutate the global runtime registry. --- # Menagerie Discipline The menagerie is the executable truth table for generation and codec behavior. Its catalogue is `test/menagerie/manifest.ts`. It contains rows rather than a claim that a broad type family works. ## A row A runtime catalogue row describes one Python call under one codec path. It has an `id`, fixture, call, expected result or error, status, and optional package requirements. A row can select Arrow or JSON fallback. The test runs the call through `NodeBridge`. It does not test a serializer in isolation. The generation gate also generates and typechecks the tier-one fixture modules. Optional-library tests add cases that require installed scientific packages. ## Statuses | Status | Meaning | Obligation | | --- | --- | --- | | `EXPECTED_OK` | The call preserves the stated result. | Assert the exact value or declared structural expectation. | | `KNOWN_LIE` | The current transport resolves but loses stated semantics. | Record the observed result, and an `expectedFix` when a fix direction exists. Do not describe the behavior as lossless. | | `LOUD_FAIL` | The call must reject at the bridge boundary. | Supply an error expectation. The test checks that error rows and this status stay aligned. | `KNOWN_LIE` is documentation of a measured limitation, not a passing result to silently improve. `LOUD_FAIL` is a contract that a value outside the supported domain fails with an error that states the cause. ## Add or change a row 1. Add the fixture function in `test/menagerie/fixtures/` when needed. 2. Add one catalogue row in `test/menagerie/manifest.ts` with the codec, status, current behavior, and expectation. 3. Declare every optional library in `requires` and any feature condition in `featureProbe`. 4. Run `npm run test:menagerie` with the pinned dependencies below. 5. Update generated snapshots when the generation gate intentionally changes. When behavior changes, flip the rows in the same PR. A fix that turns a lossy result into a preserved result changes `KNOWN_LIE` to `EXPECTED_OK`; a newly supported representation needs its own row. Do not leave the catalogue describing yesterday's behavior. ## Pinned scientific job GitHub Actions runs `optional-scientific-menagerie` on Python 3.11 and Node 22. It installs `tywrap_ir`, then pins NumPy 2.3.5, pandas 3.0.2, pyarrow 24.0.0, SciPy 1.16.3, scikit-learn 1.8.0, and CPU-only torch 2.10.0. The pins make a truth-table change reviewable rather than a moving dependency result. Run the same gate locally in a fresh virtual environment: ```bash python3.11 -m venv .venv-menagerie .venv-menagerie/bin/python -m pip install -e tywrap_ir .venv-menagerie/bin/python -m pip install numpy==2.3.5 pandas==3.0.2 pyarrow==24.0.0 scipy==1.16.3 scikit-learn==1.8.0 .venv-menagerie/bin/python -m pip install torch==2.10.0 --index-url https://download.pytorch.org/whl/cpu TYWRAP_CODEC_PYTHON=.venv-menagerie/bin/python npm run test:menagerie ``` The test skips a row only when its declared optional dependency or feature probe is unavailable. The pinned CI job provides the expected scientific surface for review. --- # Examples Copy-pasteable examples for using tywrap with the supported runtimes. ## Generate Wrappers Create `tywrap.config.ts`: ```ts import { defineConfig } from 'tywrap'; export default defineConfig({ pythonModules: { math: { typeHints: 'strict' }, collections: { typeHints: 'strict' }, }, output: { dir: './generated', format: 'esm', declaration: true, sourceMap: true, }, }); ``` Generate: ```sh npx tywrap generate ``` ## Node.js Runtime ```ts import { NodeBridge } from 'tywrap/node'; import { setRuntimeBridge } from 'tywrap/runtime'; setRuntimeBridge(new NodeBridge({ pythonPath: 'python3' })); import { sqrt } from './generated/math.generated.js'; console.log(await sqrt(16)); ``` ## Browser Runtime (Pyodide) ```ts import { PyodideBridge } from 'tywrap/pyodide'; import { setRuntimeBridge } from 'tywrap/runtime'; setRuntimeBridge( new PyodideBridge({ indexURL: 'https://cdn.jsdelivr.net/pyodide/v0.28.0/full/', }) ); import { sin, pi } from './generated/math.generated.js'; console.log(await sin(pi / 4)); ``` ## Value-returning APIs v0.10 generated wrappers do not keep live Python class instances. Expose an operation as a value-returning module function instead: ```ts import { median } from './generated/statistics.generated.js'; console.log(await median([1, 2, 2])); // 2 ``` ## More Docs - [Getting started](https://bbopen.github.io/tywrap/guide/getting-started) - [Configuration](https://bbopen.github.io/tywrap/guide/configuration) - [Node runtime](https://bbopen.github.io/tywrap/guide/runtimes/node) - [Browser runtime](https://bbopen.github.io/tywrap/guide/runtimes/browser) - [Troubleshooting](https://bbopen.github.io/tywrap/troubleshooting/) - [Type mapping](https://bbopen.github.io/tywrap/reference/type-mapping) --- # Troubleshooting Guide Common issues and solutions when using tywrap. This guide helps you diagnose and fix problems quickly. ## Diagnostics ### Health Check Script Create a `tywrap-health.js` file to quickly diagnose your setup: ```javascript #!/usr/bin/env node import { execSync } from 'child_process'; import { existsSync } from 'fs'; console.log('tywrap Health Check\n'); // Check Node.js version const nodeVersion = process.version; console.log(`Node.js: ${nodeVersion}`); // Check Python installation try { const pythonVersion = execSync('python3 --version', { encoding: 'utf8' }); console.log(`Python: ${pythonVersion.trim()}`); } catch { console.log('Python: Not found or not accessible'); } // Check tywrap installation try { const tywrapVersion = execSync('npx tywrap --version', { encoding: 'utf8' }); console.log(`tywrap: ${tywrapVersion.trim()}`); } catch { console.log('tywrap: Not installed or not working'); } // Check configuration const configCandidates = [ 'tywrap.config.ts', 'tywrap.config.mts', 'tywrap.config.js', 'tywrap.config.mjs', 'tywrap.config.cjs', 'tywrap.config.json', ]; const configFile = configCandidates.find((name) => existsSync(name)); console.log( configFile ? `Configuration: Found ${configFile}` : 'Configuration: No config file found' ); console.log('\nRun this script with: node tywrap-health.js'); ``` ```bash # Run health check node tywrap-health.js ``` ## Common Error Categories ### Installation Issues - [Module not found errors](#module-not-found) - [Python path issues](#python-path-issues) - [Permission problems](#permission-problems) ### Configuration Issues - [Invalid configuration](#invalid-configuration) - [Module resolution failures](#module-resolution) - [Type generation errors](#type-generation-errors) ### Runtime Issues - [Process timeout errors](#process-timeouts) - [Memory issues](#memory-issues) - [Import errors](#import-errors) ### Build Issues - [Generation failures](#generation-failures) - [TypeScript compilation errors](#typescript-errors) - [Build tool integration](#build-tool-issues) --- ## Installation Issues ### Module Not Found **Error**: `Error: Cannot find module 'tywrap'` **Solutions**: ```bash # Verify installation npm list tywrap # Reinstall if missing npm install tywrap # Clear cache if corrupted npm cache clean --force npm install tywrap # Check in different package manager yarn list tywrap pnpm list tywrap ``` ### Python Path Issues **Error**: `Error: spawn python3 ENOENT` or `Python not found` **Solutions**: ```bash # Check Python installation which python3 python3 --version # Common Python locations /usr/bin/python3 /usr/local/bin/python3 /opt/homebrew/bin/python3 ~/.pyenv/shims/python3 # Update configuration { "runtime": { "node": { "pythonPath": "/usr/local/bin/python3" } } } # macOS with Homebrew brew install python3 # Ubuntu/Debian sudo apt-get update && sudo apt-get install python3 # Windows # Download from python.org or use Windows Store ``` ### Permission Problems **Error**: `Error: spawn EACCES` or `Permission denied` **Solutions**: ```bash # Check executable permissions ls -la $(which python3) # Fix permissions chmod +x /usr/local/bin/python3 # Check directory permissions ls -la ~/.npm ls -la node_modules # Fix npm permissions (if needed) npm config set prefix ~/.npm-global export PATH=~/.npm-global/bin:$PATH ``` --- ## Configuration Issues ### Invalid Configuration **Error**: `Invalid configuration` or schema validation errors **Solution**: ```typescript // Use TypeScript config for validation // tywrap.config.ts import { defineConfig } from 'tywrap'; export default defineConfig({ pythonModules: { numpy: { typeHints: 'strict' as const } }, output: { dir: './generated', format: 'esm' as const } }); ``` ```bash # Validate configuration npx tywrap validate-config ``` ### Module Resolution **Error**: `ModuleNotFoundError: No module named 'xyz'` **Solutions**: ```bash # Check Python module installation python3 -c "import sys; print(sys.path)" python3 -c "import numpy; print(numpy.__file__)" # Install missing module pip3 install numpy # Check virtual environment source venv/bin/activate pip list # Update configuration with correct path { "runtime": { "node": { "pythonPath": "./venv/bin/python", "virtualEnv": "./venv" } } } ``` ### Type Generation Errors **Error**: type analysis failures or incorrect types generated **Solutions**: ```json // Relax type checking for problematic modules { "pythonModules": { "problematic_module": { "typeHints": "loose", "functions": ["specific_function"], // Limit scope "classes": [] // Skip classes if problematic } } } ``` ```bash # Enable generator diagnostics npx tywrap generate --debug ``` --- ## Runtime Issues ### Process Timeouts **Error**: `Python call timed out` or hanging operations **Solutions**: ```typescript import { NodeBridge } from 'tywrap/node'; const bridge = new NodeBridge({ pythonPath: 'python3', timeoutMs: 60000, env: { OMP_NUM_THREADS: '1', MKL_NUM_THREADS: '1', }, }); ``` ```typescript // Implement timeout handling async function withTimeout(promise: Promise, ms: number): Promise { const timeout = new Promise((_, reject) => { setTimeout(() => reject(new Error(`Timeout after ${ms}ms`)), ms); }); return Promise.race([promise, timeout]); } // Usage try { const result = await withTimeout(math.complicated_calculation(), 10000); } catch (error) { console.error('Operation timed out:', error); } ``` ### Protocol Errors **Error**: `Protocol error from Python bridge` **Cause**: the Python process printed to stdout or returned malformed JSON. **Solutions**: - Remove `print()` statements or logging to stdout in Python code. - Ensure any debugging output goes to stderr. - Verify custom bridge scripts follow the JSON line protocol. ### Memory Issues **Error**: out of memory errors or process crashes **Solutions** Large payloads and highly parallel native libraries are the usual causes. Tune the bridge and the host process separately: ```typescript import { NodeBridge } from 'tywrap/node'; const bridge = new NodeBridge({ pythonPath: 'python3', env: { OMP_NUM_THREADS: '1', MKL_NUM_THREADS: '1', }, }); ``` ```bash # Increase the Node.js heap when your app process is the bottleneck node --max-old-space-size=4096 your-app.js # Cap bridge payload size so one call cannot blow up memory usage export TYWRAP_CODEC_MAX_BYTES=10485760 export TYWRAP_REQUEST_MAX_BYTES=1048576 # Check system memory free -h # Linux top # General ``` ### Import Errors **Error**: import failures in generated TypeScript code **Solutions**: ```typescript // Check generated code structure // generated/module.generated.ts should exist // Verify import paths import { function_name } from './generated/module.generated.js'; // ^^^ Note .js extension // For TypeScript module resolution issues { "compilerOptions": { "moduleResolution": "node", "esModuleInterop": true, "allowSyntheticDefaultImports": true } } ``` --- ## Build Issues ### Generation Failures **Error**: `Failed to generate wrappers` or empty output **Solutions**: ```bash # Clean and regenerate rm -rf generated/ rm -rf .tywrap/ npx tywrap generate # Enable generator diagnostics npx tywrap generate --debug # Check Python module structure python3 -c " import inspect import your_module print(dir(your_module)) for name, obj in inspect.getmembers(your_module): print(f'{name}: {type(obj)}') " ``` ### TypeScript Errors **Error**: TypeScript compilation errors in generated code **Solutions**: ```bash # Check TypeScript version compatibility npx tsc --version npm install typescript@latest # Update tsconfig.json { "compilerOptions": { "target": "ES2022", "module": "ESNext", "moduleResolution": "bundler", "allowImportingTsExtensions": false, "skipLibCheck": true } } # Regenerate with correct output format { "output": { "format": "esm", "declaration": true } } ``` ### Build Tool Issues **Error**: bundler integration problems in Vite, Webpack, or another tool **Vite fixes** ```typescript // vite.config.ts export default defineConfig({ optimizeDeps: { exclude: ['tywrap'] }, build: { commonjsOptions: { include: [/tywrap/, /node_modules/] } } }); ``` **Webpack Solutions**: ```javascript // webpack.config.js module.exports = { resolve: { fallback: { "child_process": false, "fs": false, "path": require.resolve("path-browserify") } } }; ``` --- ## Platform-Specific Issues ### macOS Issues ```bash # Xcode Command Line Tools required xcode-select --install # Homebrew Python issues brew install python3 brew link python3 # Fix PATH issues echo 'export PATH="/opt/homebrew/bin:$PATH"' >> ~/.zshrc ``` ### Windows Issues ```powershell # Install Python from Microsoft Store or python.org # Add Python to PATH # PowerShell execution policy Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser # Use forward slashes in configuration { "runtime": { "node": { "pythonPath": "C:/Python39/python.exe" } } } ``` ### Linux Issues ```bash # Install Python development headers sudo apt-get install python3-dev python3-pip # Fix library linking issues sudo apt-get install build-essential # SELinux issues (if applicable) setsebool -P httpd_exec_mem 1 ``` ### Docker Issues ```dockerfile # Dockerfile fixes FROM node:18-slim # Install Python RUN apt-get update && apt-get install -y \ python3 \ python3-pip \ python3-dev \ && rm -rf /var/lib/apt/lists/* # Keep Python subprocess output predictable ENV PYTHONDONTWRITEBYTECODE=1 ENV PYTHONUNBUFFERED=1 ``` If Python is not discoverable on `PATH` in your image, pass `pythonPath: '/usr/bin/python3'` when constructing `NodeBridge`. --- ## Debug Mode and Logging ### Enable Debug Mode ```bash # Generator diagnostics npx tywrap generate --debug # Runtime bridge diagnostics TYWRAP_LOG_LEVEL=DEBUG TYWRAP_LOG_JSON=1 node app.js 2>tywrap.log # Extra Node.js warning detail node --trace-warnings app.js ``` ### Bridge Diagnostics ```typescript import { NodeBridge } from 'tywrap/node'; const bridge = new NodeBridge({ pythonPath: 'python3' }); const info = await bridge.getBridgeInfo({ refresh: true }); console.error(info); ``` --- ## Performance Debugging ### Profile Generation Time ```bash # Time the generation process time npx tywrap generate # Profile Node.js execution node --prof your-script.js node --prof-process isolate-*.log > profile.txt ``` ### Monitor Runtime Performance ```typescript // Monitor Python call performance const originalCall = bridge.call; bridge.call = async function(module, fn, args) { const start = Date.now(); try { const result = await originalCall.call(this, module, fn, args); console.log(`${module}.${fn} took ${Date.now() - start}ms`); return result; } catch (error) { console.error(`${module}.${fn} failed after ${Date.now() - start}ms`); throw error; } }; ``` --- ## Getting Help ### Information to Include When reporting issues, please include: ```bash # System information node --version python3 --version npm list tywrap # Configuration cat tywrap.config.ts # Error logs npx tywrap generate 2>&1 | tee error.log # Environment env | grep -E "(TYWRAP|PYTHON|NODE)" ``` ### Create Minimal Reproduction ```bash # Create minimal test case mkdir tywrap-issue cd tywrap-issue npm init -y npm install tywrap # Create minimal config cat <<'EOF' > tywrap.config.ts import { defineConfig } from 'tywrap'; export default defineConfig({ pythonModules: { math: { typeHints: 'strict' } }, output: { dir: './generated', format: 'esm', declaration: false, sourceMap: false } }); EOF # Test generation npx tywrap generate ``` ### Community Resources - GitHub Issues: [report bugs](https://github.com/bbopen/tywrap/issues) - Discussions: [ask questions](https://github.com/bbopen/tywrap/discussions) - Discord: [real-time help](https://discord.gg/tywrap) - Stack Overflow: tag your questions with `tywrap` --- ## Advanced Debugging ### Python Bridge Debug ```python # Add to custom Python bridge import logging logging.basicConfig(level=logging.DEBUG) import sys import traceback def debug_request(request): print(f"DEBUG: Processing request: {request}", file=sys.stderr) try: # Process request result = handle_request(request) print(f"DEBUG: Request successful: {result}", file=sys.stderr) return result except Exception as e: print(f"DEBUG: Request failed: {e}", file=sys.stderr) print(f"DEBUG: Traceback: {traceback.format_exc()}", file=sys.stderr) raise ``` ### TypeScript Debug ```typescript // Enable TypeScript compiler debugging { "compilerOptions": { "traceResolution": true, "listFiles": true, "extendedDiagnostics": true } } // Run TypeScript compiler directly npx tsc --noEmit --traceResolution ``` This troubleshooting guide covers the most common issues. For runtime-specific detail, see: - [Node.js Runtime](https://bbopen.github.io/tywrap/guide/runtimes/node) - [Browser Runtime (Pyodide)](https://bbopen.github.io/tywrap/guide/runtimes/browser) - [Build Tool Issues](#build-tool-issues)