Reflow Documentation

Welcome to the Reflow documentation! Reflow is a powerful, actor-based workflow execution engine built in Rust that supports multi-language scripting and cross-platform deployment.

What is Reflow?

Reflow is a modular workflow engine that uses the actor model for concurrent, message-passing execution. It supports:

• Zeal IDE Integration: Real-time event streaming to Zeal via ZIP protocol (WebSocket + HTTP traces)
• 6,700+ API Actors: Pre-generated actors for 88 API services (Slack, GitHub, Stripe, etc.)
• Actor-Based Architecture: Isolated, concurrent actors with message passing
• Graph-Based Workflows: Visual workflow representation with history/undo
• Unified Rust Runtime Crate: reflow_rt is the crates.io-facing API surface for graph, actor, network, and component crates
• Real-Time Observability: EventBridge pipeline forwarding execution events to TraceCollector and ZipSession
• Media Processing: Image, audio, video, and optional graph-driven ML pipelines
• REST API + WebSocket: HTTP and WebSocket interfaces for headless workflow execution
• Cross-Platform: Native Rust execution + WebAssembly for browsers

Documentation Structure

Getting Started

Quick start guide, installation, and basic concepts

Runtime Crate

The public Rust runtime API surface, feature flags, and graph/component re-exports

Architecture

System architecture, actor model, execution engine, and event pipeline

Zeal Integration

ZIP session, template registration, real-time event streaming to Zeal IDE

REST API

HTTP and WebSocket API for direct workflow execution

Core API

Detailed API documentation for actors, messaging, and graphs

Components

Standard component library: flow control, transforms, logic, media, optional ML, and 6,700+ API actors

Observability

EventBridge, TraceCollector, ZIP event translation, and trace sessions

Deployment

Deployment options and operational considerations

Reference

Complete API reference and configuration options

Quick Links

• Installation Guide
• Runtime Crate
• First Workflow Tutorial
• Component Library
• API Service Actors
• Zeal IDE Integration
• REST API Reference

Community and Support

• GitHub Issues: Report bugs and request features
• Discussions: Community Q&A and announcements
• Contributing: See CONTRIBUTING.md for development guidelines

License

This project is dual-licensed under MIT or Apache-2.0 - see the LICENSE files for details.

Getting Started with Reflow

Welcome to Reflow! This guide will help you get up and running with the actor-based workflow engine.

What You'll Learn

This getting started guide covers:

1. Installation - Setting up Reflow on your system
2. Basic Concepts - Understanding actors, messages, and workflows
3. Development Setup - Setting up your development environment
4. First Workflow - Creating your first workflow

Quick Overview

Reflow is an actor-based workflow engine that allows you to:

• Create actors that process data and communicate via messages
• Connect actors into workflows that define data flow and processing logic
• Execute workflows with multi-language support (JavaScript/Deno, Python, WASM)
• Deploy workflows natively or in WebAssembly environments

Prerequisites

Before getting started with Reflow, you should have:

• Rust (1.85 or later) - for building and running Reflow
• Basic understanding of concurrent programming concepts
• Familiarity with at least one of: JavaScript, Python, or Rust

Architecture at a Glance

┌─────────────┐    ┌─────────────┐    ┌─────────────┐
│   Actor A   │───▶│   Actor B   │───▶│   Actor C   │
│ (JavaScript)│    │  (Python)   │    │    (Rust)   │
└─────────────┘    └─────────────┘    └─────────────┘
       │                  │                  │
       ▼                  ▼                  ▼
┌─────────────────────────────────────────────────────┐
│              Message Bus & Routing                  │
└─────────────────────────────────────────────────────┘

Key Concepts

• Actor: An isolated unit of computation that processes messages
• Message: Data passed between actors
• Port: Input/output connections on actors
• Workflow: A graph of connected actors
• Runtime: The execution environment (Deno, Python, etc.)

Next Steps

1. Start with Installation to set up Reflow
2. Read Basic Concepts to understand the fundamentals
3. Follow the First Workflow tutorial
4. Explore the Examples for more complex use cases

Getting Help

• Check the Troubleshooting Guide
• Browse the API Reference
• Look at Examples for working code
• Open an issue on GitHub for bugs or feature requests

Ready to start? Let's install Reflow!

Installation

This guide covers installing and setting up Reflow on your system. Pick a path based on the language you're integrating from.

Quickest path: a language SDK

Most users start with one of the language SDKs. They wrap the Rust runtime in idiomatic shapes and ship pre-built native binaries for darwin / linux / windows — no Rust toolchain required to use them.

The SDK chapters walk through each one with a hello-world example. Optional actor packs (loadPack(...)) bring heavier palettes — GPU, ML, browser automation, ~6,700 SaaS API actors — into any SDK install at runtime.

Building from source (Rust)

The rest of this page covers building the Rust runtime from source — what you'll want if you're embedding Reflow in your own native host, contributing to the runtime itself, or rebuilding libreflow_rt_capi for a platform we don't ship pre-built.

Prerequisites

Before installing Reflow, ensure you have:

Required

• Rust 1.85 or later
• Git for cloning the repository

Optional (for scripting support)

• Deno 1.30+ for JavaScript/TypeScript actors
• Python 3.8+ for Python actors
• Docker for isolated Python execution

Installation Methods

Method 1: Use Reflow as a Rust Library

For application code, depend on the unified runtime crate:

[dependencies]
reflow_rt = "0.1"

Method 2: Build from Source

1. Clone the repository:

   git clone https://github.com/offbit-ai/reflow.git
   cd reflow
   

2. Build the project:

   cargo build --release
   

3. Run examples or package crates locally:

   cargo test -p reflow_rt
   cargo package -p reflow_rt --list
   

Method 3: Use Lower-Level Crates Directly

reflow_rt is the recommended user-facing entry point. Lower-level crates remain available when a project needs a narrower dependency surface:

[dependencies]
reflow_graph = "0.1"
reflow_actor = "0.1"
reflow_network = "0.1"
reflow_components = { version = "0.1", default-features = false }

Feature Flags

reflow_rt keeps optional component families out of the default install path:

[dependencies]
reflow_rt = { version = "0.1", features = ["gpu", "media", "ml"] }

Available Features

Runtime Dependencies

JavaScript/TypeScript (Deno)

Install Deno:

# macOS/Linux
curl -fsSL https://deno.land/x/install/install.sh | sh

# Windows (PowerShell)
iwr https://deno.land/x/install/install.ps1 -useb | iex

# Using package managers
brew install deno          # macOS
scoop install deno         # Windows
snap install deno          # Linux

Python Support

Install Python 3.8+:

# macOS
brew install python

# Ubuntu/Debian
sudo apt update
sudo apt install python3 python3-pip

# Windows
# Download from https://python.org

For Docker-based Python execution:

# Install Docker
# macOS/Windows: Docker Desktop
# Linux: docker.io package
sudo apt install docker.io  # Ubuntu/Debian

Verification

Verify your installation:

# Check the user-facing runtime crate
cargo check -p reflow_rt

# Verify the crate package contents
cargo package -p reflow_rt --list

Platform-Specific Notes

macOS

• Use Homebrew for easy dependency management
• Xcode Command Line Tools required for Rust compilation

Linux

• Ensure build-essential is installed
• Some distributions may need pkg-config and libssl-dev

# Ubuntu/Debian
sudo apt install build-essential pkg-config libssl-dev

# CentOS/RHEL
sudo yum groupinstall "Development Tools"
sudo yum install openssl-devel

Windows

• Use Windows Subsystem for Linux (WSL) for best experience
• Visual Studio Build Tools required for Rust compilation
• Consider using scoop or chocolatey for dependency management

Configuration

Environment Variables

Set these environment variables for optimal performance:

# Enable shared Python environment (optional)
export USE_SHARED_ENV=true

# Set Python path (if needed)
export PYTHON_PATH=/usr/bin/python3

# Configure Deno permissions (optional)
export DENO_PERMISSIONS="--allow-all"

Config File

Create a reflow.toml configuration file:

[runtime]
default_engine = "deno"
enable_networking = true
enable_filesystem = true

[deno]
allow_all = false
allow_net = true
allow_read = true

[python]
use_docker = false
shared_environment = true

[performance]
thread_pool_size = 8
max_memory_mb = 1024

Next Steps

Now that Reflow is installed:

1. Learn the basics: Read Basic Concepts
2. Set up development: Follow Development Setup
3. Create your first workflow: Try First Workflow
4. Explore examples: Check out the Examples

Troubleshooting

Common Issues

Rust compilation errors:

# Update Rust to latest version
rustup update

Deno not found:

# Add Deno to PATH
export PATH="$HOME/.deno/bin:$PATH"

Python import errors:

# Install required Python packages
pip install numpy pandas  # or other dependencies

Permission denied errors:

# Fix file permissions
chmod +x reflow

For more troubleshooting, see the Troubleshooting Guide.

Basic Concepts

This guide introduces the fundamental concepts of Reflow's actor-based workflow engine.

Core Concepts

Actors

Actors are the building blocks of Reflow workflows. Each actor is an isolated unit of computation that:

• Processes incoming messages
• Maintains its own state
• Communicates only through message passing
• Runs concurrently with other actors

#![allow(unused)]
fn main() {
// Example: Simple actor that doubles numbers (using actor macro)
use std::collections::HashMap;
use reflow_network::{
    actor::ActorContext,
    message::Message,
};
use actor_macro::actor;

#[actor(
    DoublerActor,
    inports::<100>(number),
    outports::<50>(result)
)]
async fn doubler_actor(context: ActorContext) -> Result<HashMap<String, Message>, anyhow::Error> {
    let payload = context.get_payload();
    
    if let Some(Message::Integer(n)) = payload.get("number") {
        Ok([
            ("result".to_owned(), Message::integer(n * 2))
        ].into())
    } else {
        Err(anyhow::anyhow!("Expected integer input"))
    }
}

// Alternative: Manual implementation
use reflow_network::actor::{Actor, ActorBehavior, Port, ActorLoad};
use parking_lot::Mutex;
use std::sync::Arc;

pub struct ManualDoublerActor {
    inports: Port,
    outports: Port,
    load: Arc<Mutex<ActorLoad>>,
}

impl ManualDoublerActor {
    pub fn new() -> Self {
        Self {
            inports: flume::unbounded(),
            outports: flume::unbounded(),
            load: Arc::new(Mutex::new(ActorLoad::new(0))),
        }
    }
}

impl Actor for ManualDoublerActor {
    fn get_behavior(&self) -> ActorBehavior {
        Box::new(|context: ActorContext| {
            Box::pin(async move {
                let payload = context.get_payload();
                if let Some(Message::Integer(n)) = payload.get("number") {
                    Ok([
                        ("result".to_owned(), Message::Integer(n * 2))
                    ].into())
                } else {
                    Err(anyhow::anyhow!("Expected integer input"))
                }
            })
        })
    }
    
    fn get_inports(&self) -> Port { self.inports.clone() }
    fn get_outports(&self) -> Port { self.outports.clone() }
    fn load_count(&self) -> Arc<Mutex<ActorLoad>> { self.load.clone() }
    
    fn create_process(&self) -> std::pin::Pin<Box<dyn std::future::Future<Output = ()> + 'static + Send>> {
        // Process creation implementation...
        todo!("See creating-actors.md for complete implementation")
    }
}
}

Messages

Messages are the data that flows between actors. Reflow supports various message types:

#![allow(unused)]
fn main() {
pub enum Message {
    String(String),
    Integer(i64),
    Float(f64),
    Boolean(bool),
    Array(Vec<Message>),
    Object(HashMap<String, Message>),
    Binary(Vec<u8>),
    Null,
    Error(String),
}
}

Ports

Ports are the communication channels between actors:

• Input ports (inports): Receive messages from other actors
• Output ports (outports): Send messages to other actors

┌─────────────┐
│    Actor    │
│  ┌───────┐  │
│  │ Logic │  │
│  └───────┘  │
│             │
│ in1 ──────→ │ ──────→ out1
│ in2 ──────→ │ ──────→ out2
└─────────────┘

Workflows (Graphs)

Workflows are directed graphs of connected actors that define:

• Data flow between actors
• Processing logic and transformations
• Execution order and dependencies

┌─────────┐    ┌─────────┐    ┌─────────┐
│ Source  │───▶│Transform│───▶│  Sink   │
│ Actor   │    │ Actor   │    │ Actor   │
└─────────┘    └─────────┘    └─────────┘

Actor State

Each actor can maintain its own state that persists between message processing:

#![allow(unused)]
fn main() {
// Example: Counter actor with state (using actor macro)
use reflow_network::actor::MemoryState;

#[actor(
    CounterActor,
    state(MemoryState),
    inports::<100>(increment),
    outports::<50>(count)
)]
async fn counter_actor(context: ActorContext) -> Result<HashMap<String, Message>, anyhow::Error> {
    let payload = context.get_payload();
    let state = context.get_state();
    
    let mut state_guard = state.lock();
    let memory_state = state_guard
        .as_mut_any()
        .downcast_mut::<MemoryState>()
        .expect("Expected MemoryState");
    
    // Initialize state if needed
    if !memory_state.contains_key("count") {
        memory_state.insert("count", serde_json::json!(0));
    }
    
    // Get current count
    let current_count = memory_state.get("count")
        .and_then(|v| v.as_i64())
        .unwrap_or(0);
    
    // Increment by 1 or by specified amount
    let increment_by = if let Some(Message::Integer(amount)) = payload.get("increment") {
        *amount
    } else {
        1 // Default increment
    };
    
    let new_count = current_count + increment_by;
    
    // Update state
    memory_state.insert("count", serde_json::json!(new_count));
    
    Ok([
        ("count".to_owned(), Message::Integer(new_count))
    ].into())
}
}

Actor Types

Native Actors (Rust)

Built directly in Rust for maximum performance:

#![allow(unused)]
fn main() {
struct ProcessorActor {
    // Implementation in Rust
}
}

Script Actors

Execute scripts in various languages:

JavaScript/TypeScript (Deno)

// JavaScript actor function
function process(inputs, context) {
    const data = inputs.data;
    return { result: data.toUpperCase() };
}

Python

# Python actor script
import numpy as np

inputs = Context.get_inputs()
data = np.array(inputs["data"])
__return_value = data.sum()

WebAssembly

#![allow(unused)]
fn main() {
// WASM actor using Reflow Plugin SDK (reflow_wasm_actor)
use reflow_wasm_actor::*;
use std::collections::HashMap;

// Define plugin metadata
fn metadata() -> PluginMetadata {
    PluginMetadata {
        component: "ProcessorActor".to_string(),
        description: "Processes input data".to_string(),
        inports: vec![
            port_def!("input", "Input data", "String", required),
        ],
        outports: vec![
            port_def!("output", "Processed output", "String"),
        ],
        config_schema: None,
    }
}

// Implement actor behavior
fn process_actor(context: ActorContext) -> Result<ActorResult, Box<dyn std::error::Error>> {
    let mut outputs = HashMap::new();
    
    // Get input and process it
    if let Some(Message::String(input)) = context.payload.get("input") {
        let processed = format!("Processed: {}", input);
        outputs.insert("output".to_string(), Message::String(processed));
    }
    
    Ok(ActorResult {
        outputs,
        state: None, // No state changes
    })
}

// Register the plugin
actor_plugin!(
    metadata: metadata(),
    process: process_actor
);
}

Message Passing Patterns

Point-to-Point

One actor sends to one specific actor:

Actor A ───────▶ Actor B

Broadcast

One actor sends to multiple actors:

        ┌─────▶ Actor B
Actor A ┤
        └─────▶ Actor C

Collect/Merge

Multiple actors send to one actor:

Actor A ┐
        ├─────▶ Actor C
Actor B ┘

Concurrency Model

Actor Isolation

• Each actor runs in its own execution context
• No shared memory between actors
• Thread-safe by design

Message Processing

• Actors process messages asynchronously
• Messages are queued for processing
• Backpressure handling prevents overflow

Parallelism

• Multiple actors can run simultaneously
• Work is distributed across available CPU cores
• Network can span multiple machines

Error Handling

Actor-Level Errors

#![allow(unused)]
fn main() {
// Errors are returned as Error messages
Err(anyhow::anyhow!("Processing failed"))
}

Network-Level Errors

#![allow(unused)]
fn main() {
// Error propagation through the network
HashMap::from([
    ("error".to_string(), Message::Error("Network timeout".to_string()))
])
}

Recovery Patterns

• Dead letter queues for failed messages
• Circuit breakers for failing actors
• Supervisor actors for monitoring

Lifecycle Management

Actor Creation

#![allow(unused)]
fn main() {
let actor = MyActor::new(config);
let process = actor.create_process();
tokio::spawn(process);
}

Actor Termination

#![allow(unused)]
fn main() {
// Graceful shutdown
drop(inports); // Closes input channels
// Actor completes current message and exits
}

State Persistence

#![allow(unused)]
fn main() {
// State can be persisted and restored
let state = actor.get_state();
// Serialize state for persistence
}

Advanced Concepts

Hot Code Reloading

• Script actors can be updated without stopping the workflow
• State preservation during updates

Multi-tenancy

• Isolated workspaces for different users/projects
• Resource quotas and permissions

Distributed Execution

• Actors can run on different machines
• Network-transparent message passing

Best Practices

Actor Design

• Keep actors small and focused
• Avoid blocking operations in actor logic
• Use async/await for I/O operations

Message Design

• Use typed messages when possible
• Keep messages small and serializable
• Include error context in error messages

Workflow Design

• Design for failure (circuit breakers, timeouts)
• Monitor actor performance and health
• Use appropriate parallelism levels

Next Steps

Now that you understand the basic concepts:

1. Set up development: Development Setup
2. Create your first workflow: First Workflow
3. Learn about specific actors: Actor API
4. Explore scripting: JavaScript Runtime

Further Reading

• Actor Model Theory
• Message Passing Details
• Performance Considerations

Development Setup

This guide helps you set up a development environment for building workflows with Reflow.

Development Environment

Recommended Tools

• IDE: Visual Studio Code or RustRover
• Version Control: Git
• Package Manager: Cargo for Rust dependencies
• Terminal: Modern terminal with good Unicode support

VS Code Extensions

For the best development experience with VS Code:

{
  "recommendations": [
    "rust-lang.rust-analyzer",
    "vadimcn.vscode-lldb",
    "serayuzgur.crates",
    "tamasfe.even-better-toml",
    "ms-vscode.vscode-json"
  ]
}

Project Structure

Creating a New Reflow Project

# Create a new Rust project
cargo new my-reflow-app
cd my-reflow-app

# Add Reflow dependencies
cargo add reflow_rt
cargo add tokio --features rt-multi-thread,macros
cargo add serde_json anyhow

Recommended Project Structure

my-reflow-app/
├── Cargo.toml
├── src/
│   ├── main.rs
│   ├── actors/
│   │   ├── mod.rs
│   │   └── custom_actor.rs
│   ├── workflows/
│   │   ├── mod.rs
│   │   └── data_pipeline.rs
│   └── scripts/
│       ├── process.js
│       └── transform.py
├── config/
│   └── reflow.toml
├── tests/
│   └── integration_tests.rs
└── examples/
    └── basic_workflow.rs

Cargo.toml Configuration

[package]
name = "my-reflow-app"
version = "0.1.0"
edition = "2021"

[dependencies]
reflow_rt = "0.1"
tokio = { version = "1", features = ["rt-multi-thread", "macros"] }
serde_json = "1"
anyhow = "1"

[dev-dependencies]
tokio-test = "0.4"

[[example]]
name = "basic_workflow"
path = "examples/basic_workflow.rs"

Development Workflow

1. Setting Up the Main Application

Create src/main.rs:

use reflow_rt::prelude::*;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let mut graph = Graph::new("development", false, None);
    graph.add_node("tap", "tpl_passthrough", None);

    let network = Network::with_graph(NetworkConfig::default(), &graph);
    let _ = network;

    Ok(())
}

2. Creating Custom Actors

Create src/actors/mod.rs:

#![allow(unused)]
fn main() {
pub mod custom_actor;

pub use custom_actor::CustomActor;
}

Create src/actors/custom_actor.rs:

#![allow(unused)]
fn main() {
use reflow_rt::actor_runtime::{Actor, ActorBehavior, ActorContext, Port};
use reflow_rt::actor_runtime::message::Message;
use std::collections::HashMap;

pub struct CustomActor {
    inports: Port,
    outports: Port,
}

impl CustomActor {
    pub fn new() -> Self {
        Self {
            inports: flume::unbounded(),
            outports: flume::unbounded(),
        }
    }
}

impl Actor for CustomActor {
    fn get_behavior(&self) -> ActorBehavior {
        Box::new(|context: ActorContext| {
            Box::pin(async move {
                let payload = context.get_payload();
                
                // Your processing logic here
                let result = HashMap::from([
                    ("output".to_string(), Message::String("processed".to_string()))
                ]);
                
                Ok(result)
            })
        })
    }
    
    fn get_inports(&self) -> Port {
        self.inports.clone()
    }
    
    fn get_outports(&self) -> Port {
        self.outports.clone()
    }
    
    fn create_process(&self) -> std::pin::Pin<Box<dyn std::future::Future<Output = ()> + 'static + Send>> {
        // Default implementation from trait
        todo!("Implement process creation")
    }
}
}

3. Organizing Workflows

Create src/workflows/mod.rs:

#![allow(unused)]
fn main() {
pub mod data_pipeline;

pub use data_pipeline::create_data_pipeline;
}

Create src/workflows/data_pipeline.rs:

#![allow(unused)]
fn main() {
use reflow_network::network::Network;
use crate::actors::CustomActor;

pub async fn create_data_pipeline() -> Result<Network, Box<dyn std::error::Error>> {
    let mut network = Network::new();
    
    // Create actors
    let source_actor = CustomActor::new();
    let transform_actor = CustomActor::new();
    let sink_actor = CustomActor::new();
    
    // Add actors to network
    network.add_actor("source", Box::new(source_actor)).await?;
    network.add_actor("transform", Box::new(transform_actor)).await?;
    network.add_actor("sink", Box::new(sink_actor)).await?;
    
    // Connect actors
    network.connect("source", "output", "transform", "input").await?;
    network.connect("transform", "output", "sink", "input").await?;
    
    Ok(network)
}
}

Testing Setup

Unit Tests

Create src/actors/custom_actor.rs with tests:

#![allow(unused)]
fn main() {
#[cfg(test)]
mod tests {
    use super::*;
    use tokio_test;
    
    #[tokio::test]
    async fn test_custom_actor() {
        let actor = CustomActor::new();
        let behavior = actor.get_behavior();
        
        // Create test context
        let payload = HashMap::from([
            ("input".to_string(), Message::String("test".to_string()))
        ]);
        
        // Test behavior
        // Note: You'll need to create proper ActorContext for testing
        let result = behavior(/* test context */).await;
        assert!(result.is_ok());
    }
}
}

Integration Tests

Create tests/integration_tests.rs:

#![allow(unused)]
fn main() {
use my_reflow_app::workflows::create_data_pipeline;

#[tokio::test]
async fn test_data_pipeline() {
    let network = create_data_pipeline().await.unwrap();
    
    // Test the complete workflow
    // Send test data and verify output
}
}

Configuration Management

Environment Configuration

Create config/reflow.toml:

[development]
log_level = "debug"
thread_pool_size = 4

[production]
log_level = "info"
thread_pool_size = 8

[scripting]
deno_permissions = ["--allow-net", "--allow-read"]
python_interpreter = "python3"

[networking]
bind_address = "127.0.0.1:8080"
enable_metrics = true

Loading Configuration

#![allow(unused)]
fn main() {
use serde::{Deserialize, Serialize};
use std::fs;

#[derive(Debug, Deserialize, Serialize)]
struct Config {
    development: Option<EnvConfig>,
    production: Option<EnvConfig>,
    scripting: Option<ScriptingConfig>,
    networking: Option<NetworkingConfig>,
}

#[derive(Debug, Deserialize, Serialize)]
struct EnvConfig {
    log_level: String,
    thread_pool_size: usize,
}

fn load_config() -> Result<Config, Box<dyn std::error::Error>> {
    let config_str = fs::read_to_string("config/reflow.toml")?;
    let config: Config = toml::from_str(&config_str)?;
    Ok(config)
}
}

Development Scripts

Makefile

Create a Makefile for common tasks:

.PHONY: build test run clean docs

build:
	cargo build

test:
	cargo test

run:
	cargo run

clean:
	cargo clean

docs:
	cargo doc --open

check:
	cargo check
	cargo clippy -- -D warnings
	cargo fmt -- --check

dev:
	cargo watch -x run

install-tools:
	cargo install cargo-watch
	cargo install cargo-expand

Development Commands

# Development workflow
make build          # Build the project
make test           # Run tests
make check          # Run linting and formatting checks
make dev            # Run with auto-reload on changes

# Documentation
make docs           # Generate and open documentation
cargo doc --document-private-items  # Include private items

Debugging

Logging Setup

#![allow(unused)]
fn main() {
use tracing::{info, warn, error, debug};
use tracing_subscriber;

// In main.rs
fn init_logging() {
    tracing_subscriber::fmt()
        .with_max_level(tracing::Level::DEBUG)
        .init();
}

// In your actors
debug!("Processing message: {:?}", message);
info!("Actor started successfully");
warn!("High memory usage detected");
error!("Failed to process message: {}", error);
}

Debug Configuration

Add to Cargo.toml:

[profile.dev]
debug = true
debug-assertions = true
overflow-checks = true

[dependencies]
tracing = "0.1"
tracing-subscriber = "0.3"

Using Debugger

For VS Code, create .vscode/launch.json:

{
    "version": "0.2.0",
    "configurations": [
        {
            "type": "lldb",
            "request": "launch",
            "name": "Debug Reflow App",
            "cargo": {
                "args": ["build", "--bin=my-reflow-app"],
                "filter": {
                    "name": "my-reflow-app",
                    "kind": "bin"
                }
            },
            "args": [],
            "cwd": "${workspaceFolder}"
        }
    ]
}

Performance Profiling

Basic Profiling

#![allow(unused)]
fn main() {
use std::time::Instant;

// Time critical sections
let start = Instant::now();
// ... your code
let duration = start.elapsed();
println!("Time elapsed: {:?}", duration);
}

Advanced Profiling Tools

# Install profiling tools
cargo install cargo-profiler
cargo install flamegraph

# Generate flame graphs
cargo flamegraph --bin my-reflow-app

# Memory profiling with valgrind (Linux)
cargo build --release
valgrind --tool=massif ./target/release/my-reflow-app

Continuous Integration

GitHub Actions

Create .github/workflows/ci.yml:

name: CI

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - uses: actions-rs/toolchain@v1
      with:
        toolchain: stable
    - name: Build
      run: cargo build --verbose
    - name: Run tests
      run: cargo test --verbose
    - name: Check formatting
      run: cargo fmt -- --check
    - name: Run clippy
      run: cargo clippy -- -D warnings

Next Steps

Now that your development environment is set up:

1. Create your first workflow: First Workflow
2. Learn about actors: Creating Actors
3. Explore scripting: Deno Runtime
4. See examples: Examples

Resources

• Rust Book
• Tokio Tutorial
• Reflow Examples
• API Reference

Your First Workflow

This tutorial will guide you through creating and running your first Reflow workflow using the actual implementation patterns. We'll build a simple data processing pipeline that demonstrates the core concepts.

Overview

We'll create a workflow that:

1. Processes input numbers (Sum Actor)
2. Squares the result (Square Actor)
3. Validates the output (Assert Actor)

┌─────────┐    ┌─────────┐    ┌─────────┐
│   Sum   │───▶│ Square  │───▶│ Assert  │
│ Actor   │    │ Actor   │    │ Actor   │
└─────────┘    └─────────┘    └─────────┘

Prerequisites

Before starting, make sure you have:

• Completed the Installation guide
• Set up your Development Environment
• Understanding of Basic Concepts

Step 1: Create a New Project

# Create a new Rust project
cargo new hello-reflow
cd hello-reflow

# Add Reflow dependencies
cargo add reflow_network
cargo add actor_macro
cargo add tokio --features full
cargo add serde --features derive
cargo add serde_json anyhow
cargo add parking_lot

Your Cargo.toml should look like this:

[package]
name = "hello-reflow"
version = "0.1.0"
edition = "2021"

[dependencies]
reflow_network = { path = "../path/to/reflow/crates/reflow_network" }
actor_macro = { path = "../path/to/reflow/crates/actor_macro" }
tokio = { version = "1.0", features = ["full"] }
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
anyhow = "1.0"
parking_lot = "0.12"

Step 2: Create Your First Actors

Create src/main.rs with the correct actor patterns:

use std::collections::HashMap;
use reflow_network::{
    actor::{ActorContext, MemoryState},
    network::{Network, NetworkConfig},
    connector::{ConnectionPoint, Connector, InitialPacket},
    message::Message,
};
use actor_macro::actor;

// Sum Actor - adds two input numbers
#[actor(
    SumActor,
    inports::<100>(A, B),
    outports::<100>(Out),
    await_all_inports
)]
async fn sum_actor(context: ActorContext) -> Result<HashMap<String, Message>, anyhow::Error> {
    let payload = context.get_payload();

    let a_msg = payload.get("A").expect("expected to get data from port A");
    let b_msg = payload.get("B").expect("expected to get data from port B");

    let a = match a_msg {
        Message::Integer(value) => *value,
        _ => 0,
    };

    let b = match b_msg {
        Message::Integer(value) => *value,
        _ => 0,
    };

    let result = a + b;
    println!("Sum Actor: {} + {} = {}", a, b, result);

    Ok([("Out".to_owned(), Message::integer(result))].into())
}

// Square Actor - squares the input number
#[actor(
    SquareActor,
    inports::<100>(In),
    outports::<50>(Out)
)]
async fn square_actor(context: ActorContext) -> Result<HashMap<String, Message>, anyhow::Error> {
    let payload = context.get_payload();
    let message = payload.get("In").expect("expected input");
    
    let input = match message {
        Message::Integer(value) => *value,
        _ => 0,
    };

    let result = input * input;
    println!("Square Actor: {} squared = {}", input, result);

    Ok([("Out".to_owned(), Message::Integer(result))].into())
}

// Print Actor - displays the final result
#[actor(
    PrintActor,
    inports::<100>(Value),
    outports::<50>(Done)
)]
async fn print_actor(context: ActorContext) -> Result<HashMap<String, Message>, anyhow::Error> {
    let payload = context.get_payload();
    let message = payload.get("Value").expect("expected value");
    
    match message {
        Message::Integer(value) => {
            println!("🎉 Final Result: {}", value);
        },
        _ => {
            println!("📄 Final Result: {:?}", message);
        }
    }

    Ok([("Done".to_owned(), Message::Boolean(true))].into())
}

#[tokio::main]
async fn main() -> Result<(), anyhow::Error> {
    println!("🚀 Starting Hello Reflow workflow...");
    
    // Create network with default configuration
    let mut network = Network::new(NetworkConfig::default());

    // Register actor types
    network.register_actor("sum_process", SumActor::new())?;
    network.register_actor("square_process", SquareActor::new())?;
    network.register_actor("print_process", PrintActor::new())?;

    // Add actor instances (nodes)
    network.add_node("sum", "sum_process")?;
    network.add_node("square", "square_process")?;
    network.add_node("print", "print_process")?;

    // Connect the workflow: sum -> square -> print
    network.add_connection(Connector {
        from: ConnectionPoint {
            actor: "sum".to_owned(),
            port: "Out".to_owned(),
            ..Default::default()
        },
        to: ConnectionPoint {
            actor: "square".to_owned(),
            port: "In".to_owned(),
            ..Default::default()
        },
    });

    network.add_connection(Connector {
        from: ConnectionPoint {
            actor: "square".to_owned(),
            port: "Out".to_owned(),
            ..Default::default()
        },
        to: ConnectionPoint {
            actor: "print".to_owned(),
            port: "Value".to_owned(),
            ..Default::default()
        },
    });

    // Add initial data to start the workflow
    network.add_initial(InitialPacket {
        to: ConnectionPoint {
            actor: "sum".to_owned(),
            port: "A".to_owned(),
            initial_data: Some(Message::Integer(5)),
        },
    });

    network.add_initial(InitialPacket {
        to: ConnectionPoint {
            actor: "sum".to_owned(),
            port: "B".to_owned(),
            initial_data: Some(Message::Integer(3)),
        },
    });

    // Start the network
    network.start().await?;

    // Give the workflow time to complete
    tokio::time::sleep(tokio::time::Duration::from_secs(2)).await;

    println!("✅ Workflow completed!");
    
    Ok(())
}

Step 3: Run the Workflow

cargo run

You should see output like:

🚀 Starting Hello Reflow workflow...
Sum Actor: 5 + 3 = 8
Square Actor: 8 squared = 64
🎉 Final Result: 64
✅ Workflow completed!

Step 4: Understanding the Code

Actor Macro Usage

The #[actor] macro simplifies actor creation:

#![allow(unused)]
fn main() {
#[actor(
    SumActor,                    // Generated struct name
    inports::<100>(A, B),        // Input ports with capacity
    outports::<100>(Out),        // Output ports with capacity
    await_all_inports            // Wait for all inputs before processing
)]
async fn sum_actor(context: ActorContext) -> Result<HashMap<String, Message>, anyhow::Error>
}

Function Signature

All actor functions must have this exact signature:

• async fn - Asynchronous function
• context: ActorContext - Single parameter containing payload, state, config
• Result<HashMap<String, Message>, anyhow::Error> - Return type

Network API Pattern

1. Register actor types: network.register_actor("name", ActorStruct::new())
2. Add node instances: network.add_node("instance_id", "actor_type")
3. Connect with Connector structs
4. Initialize with InitialPacket structs

Step 5: Add State Management

Let's create a stateful actor that counts operations:

#![allow(unused)]
fn main() {
// Counter Actor - keeps track of how many values it has processed
#[actor(
    CounterActor,
    state(MemoryState),
    inports::<100>(Value),
    outports::<50>(Count, Total)
)]
async fn counter_actor(context: ActorContext) -> Result<HashMap<String, Message>, anyhow::Error> {
    let payload = context.get_payload();
    let state = context.get_state();
    let input = payload.get("Value").expect("expected value");
    
    let value = match input {
        Message::Integer(n) => *n,
        _ => 0,
    };

    // Update state
    let (count, total) = {
        let mut state_guard = state.lock();
        let memory_state = state_guard
            .as_mut_any()
            .downcast_mut::<MemoryState>()
            .expect("Expected MemoryState");
        
        // Get current count and total
        let current_count = memory_state
            .get("count")
            .and_then(|v| v.as_i64())
            .unwrap_or(0);
        
        let current_total = memory_state
            .get("total")
            .and_then(|v| v.as_i64())
            .unwrap_or(0);
        
        // Update values
        let new_count = current_count + 1;
        let new_total = current_total + value;
        
        memory_state.insert("count", serde_json::json!(new_count));
        memory_state.insert("total", serde_json::json!(new_total));
        
        (new_count, new_total)
    };

    println!("Counter Actor: processed {} values, total sum: {}", count, total);

    Ok([
        ("Count".to_owned(), Message::Integer(count)),
        ("Total".to_owned(), Message::Integer(total)),
    ].into())
}
}

Step 6: Multiple Input Example

Create an actor that waits for multiple inputs:

#![allow(unused)]
fn main() {
// Multiply Actor - multiplies two inputs
#[actor(
    MultiplyActor,
    inports::<100>(X, Y),
    outports::<50>(Result),
    await_all_inports  // This makes it wait for both X and Y
)]
async fn multiply_actor(context: ActorContext) -> Result<HashMap<String, Message>, anyhow::Error> {
    let payload = context.get_payload();

    let x = match payload.get("X").expect("expected X") {
        Message::Integer(value) => *value,
        _ => 1,
    };

    let y = match payload.get("Y").expect("expected Y") {
        Message::Integer(value) => *value,
        _ => 1,
    };

    let result = x * y;
    println!("Multiply Actor: {} × {} = {}", x, y, result);

    Ok([("Result".to_owned(), Message::Integer(result))].into())
}
}

Step 7: Complex Workflow Example

Here's a more complex workflow that demonstrates multiple patterns:

#[tokio::main]
async fn main() -> Result<(), anyhow::Error> {
    println!("🚀 Starting Complex Reflow workflow...");
    
    let mut network = Network::new(NetworkConfig::default());

    // Register all actor types
    network.register_actor("sum_process", SumActor::new())?;
    network.register_actor("multiply_process", MultiplyActor::new())?;
    network.register_actor("counter_process", CounterActor::new())?;
    network.register_actor("print_process", PrintActor::new())?;

    // Create network topology
    network.add_node("sum1", "sum_process")?;
    network.add_node("multiply1", "multiply_process")?;
    network.add_node("counter1", "counter_process")?;
    network.add_node("print1", "print_process")?;

    // Connect workflow
    network.add_connection(Connector {
        from: ConnectionPoint {
            actor: "sum1".to_owned(),
            port: "Out".to_owned(),
            ..Default::default()
        },
        to: ConnectionPoint {
            actor: "multiply1".to_owned(),
            port: "X".to_owned(),
            ..Default::default()
        },
    });

    network.add_connection(Connector {
        from: ConnectionPoint {
            actor: "multiply1".to_owned(),
            port: "Result".to_owned(),
            ..Default::default()
        },
        to: ConnectionPoint {
            actor: "counter1".to_owned(),
            port: "Value".to_owned(),
            ..Default::default()
        },
    });

    network.add_connection(Connector {
        from: ConnectionPoint {
            actor: "counter1".to_owned(),
            port: "Total".to_owned(),
            ..Default::default()
        },
        to: ConnectionPoint {
            actor: "print1".to_owned(),
            port: "Value".to_owned(),
            ..Default::default()
        },
    });

    // Initial data
    network.add_initial(InitialPacket {
        to: ConnectionPoint {
            actor: "sum1".to_owned(),
            port: "A".to_owned(),
            initial_data: Some(Message::Integer(10)),
        },
    });

    network.add_initial(InitialPacket {
        to: ConnectionPoint {
            actor: "sum1".to_owned(),
            port: "B".to_owned(),
            initial_data: Some(Message::Integer(5)),
        },
    });

    network.add_initial(InitialPacket {
        to: ConnectionPoint {
            actor: "multiply1".to_owned(),
            port: "Y".to_owned(),
            initial_data: Some(Message::Integer(3)),
        },
    });

    // Start the network
    network.start().await?;
    
    tokio::time::sleep(tokio::time::Duration::from_secs(2)).await;
    
    println!("✅ Complex workflow completed!");
    
    Ok(())
}

Expected output:

🚀 Starting Complex Reflow workflow...
Sum Actor: 10 + 5 = 15
Multiply Actor: 15 × 3 = 45
Counter Actor: processed 1 values, total sum: 45
🎉 Final Result: 45
✅ Complex workflow completed!

Key Concepts Demonstrated

Actor Macro Features

• Port Definitions: inports::<capacity>(Port1, Port2)
• State Management: state(MemoryState) for stateful actors
• Input Synchronization: await_all_inports waits for all inputs

Network Configuration

• Registration: Register actor types before use
• Instantiation: Create specific instances with unique IDs
• Connection: Use structured Connector objects
• Initialization: Send initial data with InitialPacket

Message Flow

• Messages flow through typed ports
• Actors process inputs and produce outputs
• State is maintained per actor instance

Error Handling

Actors can return errors that will be logged:

#![allow(unused)]
fn main() {
#[actor(
    ValidatorActor,
    inports::<100>(Input),
    outports::<50>(Valid, Invalid)
)]
async fn validator_actor(context: ActorContext) -> Result<HashMap<String, Message>, anyhow::Error> {
    let payload = context.get_payload();
    let input = payload.get("Input").expect("expected input");
    
    match input {
        Message::Integer(n) if *n > 0 => {
            Ok([("Valid".to_owned(), input.clone())].into())
        },
        Message::Integer(n) if *n <= 0 => {
            Ok([("Invalid".to_owned(), input.clone())].into())
        },
        _ => {
            Err(anyhow::anyhow!("Expected integer input, got {:?}", input))
        }
    }
}
}

Next Steps

Now that you understand the basic patterns:

1. Learn more actor patterns: Creating Actors
2. Explore message types: Message Passing
3. Add scripting: JavaScript Integration
4. Use pre-built components: Standard Library
5. See more examples: Examples

Troubleshooting

Common Issues

Compilation errors with actor macro: Make sure actor_macro is in your dependencies

Port connection errors: Verify port names match exactly between connections

Runtime panics: Check that initial data types match what actors expect

Deadlocks: Ensure await_all_inports actors receive all required inputs

For more help, see the Troubleshooting Guide.

Complete Example Code

The complete working examples are available in the examples directory.

Language SDKs

Reflow's runtime is Rust, but its native shape is a language-agnostic actor model — graphs of typed-port nodes connected by message edges. Each first-party SDK is a thin native binding plus an idiomatic surface in that language. They all link to (or embed) the same reflow_rt_capi C ABI, so behavior is identical across languages.

At a glance

What's the same across languages

Each SDK exposes the same five core types with idiomatic naming:

• Message — typed payload (Flow / Boolean / Integer / Float / String / Object / Array / Bytes / Stream).
• Graph — declarative DAG: add nodes, connect ports, register groups, expose subgraph ports.
• Actor — either a registered template id (the bundled catalog) or a callback-driven actor authored in the host language.
• Network — the executor that runs a graph; takes initial packets, emits a runtime event stream.
• EventStream — async event tap for tracing / observability.

Plus a pack loader that lets any SDK install ship optional actor palettes (GPU renderers, ML, browser automation, ~6,700 SaaS API actors) at runtime via loadPack(path).

What's slightly different

• Concurrency model. Node uses ThreadSafeFunction callbacks; Python uses GIL-aware shims; Go uses cgo callbacks; JVM uses JNIEnv::attach_current_thread; C++ pushes raw C ABI threading concerns to the user. The SDK READMEs cover the per-language gotchas.
• JSON conversion. Each SDK auto-converts where idiomatic (Map/dict/map[string]any/Map<String,Object>/std::string). C++ returns std::string and lets you pick the JSON parser.
• Async patterns. Node returns Promises, Python exposes async-friendly entry points, Go uses channels, JVM has Kotlin suspend/Flow adapters, C++ uses callbacks.

Picking an SDK

• Browser / Electron / VS Code extension → Node.
• ML pipelines, data engineering, CLI tooling → Python.
• Backend services, CLI binaries, embedded gateways → Go.
• Android, JVM-based platforms (Kafka, Spark, Flink), desktop with Compose → JVM.
• Embedded, native engines, plugins for existing C++ apps → C++.
• Roll your own runtime → use reflow_rt_capi directly; every SDK above is built on it.

Versioning

The SDKs ship independently but track the same runtime semantics. Tag schemes:

Pack ABI versions are toolchain-locked — pair a pack-v* release with the SDK release built from the same workspace revision.

Node.js SDK

@offbit-ai/reflow — Node 18+, prebuilt addons for darwin / linux / windows.

Install

npm install @offbit-ai/reflow

npm resolves the right per-platform .node addon via optionalDependencies; no compilation step on the user side.

Hello world

import {
  Graph, Network, Actor, Message,
} from "@offbit-ai/reflow";

class Doubler extends Actor {
  static component = "doubler";
  static inports = ["in"];
  static outports = ["out"];

  run(ctx) {
    const n = ctx.inputs.in?.data ?? 0;
    ctx.done({ out: Message.integer(Number(n) * 2) });
  }
}

const net = new Network();
net.registerActor("tpl_doubler", new Doubler());
net.addNode("a", "tpl_doubler");
net.addInitial("a", "in", { type: "Integer", data: 21 });
net.start();

for await (const ev of net.events()) {
  console.log(ev);
  if (ev._type === "ActorCompleted") break;
}

Authoring graphs

const g = new Graph("demo");
g.addNode("a", "tpl_x");
g.addNode("b", "tpl_y");
g.addConnection("a", "out", "b", "in");
g.addGroup("pipe", ["a", "b"], { caption: "pipeline" });
g.renameNode("a", "alpha");
console.log(g.groups());           // [{ id: "pipe", nodes: ["alpha","b"], … }]
console.log(g.connections());      // …

The full graph API (T1 mutators + T2 queries — renames, groups, port lifecycle, metadata setters, queries) is mirrored in the SDK; see the SDK README for the complete surface.

Bundled component catalog

Every install ships reflow_components's av-core slice — ~270 templates covering animation, flow control, math, vector / 2D graphics, asset DB, scene graph, HTTP integration, stream ops, DSP, procedural generation. Heavier palettes (GPU, ML, browser, video, window events, ~6,700 API actors) install as .rflpack bundles.

import { templateActor, templateList, loadPack } from "@offbit-ai/reflow";

console.log(templateList().filter(id => id.startsWith("tpl_math_")));

// Plug in the ML pack at runtime — adds 12 more template ids.
loadPack("./reflow.pack.ml-0.2.0.rflpack");
const inferenceActor = templateActor("tpl_ml_run_inference");

Subgraphs

import { SubgraphBuilder } from "@offbit-ai/reflow";
const sub = new SubgraphBuilder(graphExportJson);
sub.registerActor("my_custom", new MyCustom());
sub.fillFromCatalog();   // resolve remaining components from bundled + loaded packs
net.registerActor("tpl_sub", sub.build());

Streams

const stream = net.createStream({ bufferSize: 64, contentType: "image/jpeg" });
producer.emit({ stream });        // Flow + chunked frames
for await (const frame of stream) console.log(frame.kind, frame.data?.length);

See also

• Full Node SDK README — complete API + threading notes.
• Authoring Actors — runtime semantics shared by every SDK.
• Packs — extending the catalog at runtime.

Python SDK

offbit-reflow — Python 3.9+, abi3 wheels for darwin / linux / windows.

Install

pip install offbit-reflow

PyPI ships pre-built wheels (one per platform, abi3-py39 so 3.9–3.13+ work from the same wheel). No build dependencies on the user side.

Hello world

import offbit_reflow as reflow

class Doubler(reflow.Actor):
    component = "doubler"
    inports = ["in"]
    outports = ["out"]

    async def run(self, ctx):
        n = ctx.inputs.get("in", {}).get("data", 0)
        ctx.emit("out", reflow.Message.integer(int(n) * 2))

net = reflow.Network()
net.register_actor("tpl_doubler", Doubler())
net.add_node("a", "tpl_doubler")
net.add_initial("a", "in", {"type": "Integer", "data": 21})
net.start()

for ev in net.events():
    print(ev)
    if ev["_type"] == "ActorCompleted":
        break

Authoring graphs

g = reflow.Graph("demo")
g.add_node("a", "tpl_x")
g.add_node("b", "tpl_y")
g.add_connection("a", "out", "b", "in")
g.add_group("pipe", ["a", "b"], {"caption": "pipeline"})
g.rename_node("a", "alpha")
print(g.groups())
print(g.connections())

Full graph API: rename / port lifecycle / metadata setters / group CRUD / queries — same surface across every SDK. See sdk/python/README.md for the complete signature list.

Bundled component catalog

The wheel ships ~270 pure-Rust + av-core templates. Heavier palettes (GPU, ML, browser, video, window events, ~6,700 API actors) install as .rflpack bundles.

print([t for t in reflow.template_list() if t.startswith("tpl_math_")])

reflow.load_pack("./reflow.pack.ml-0.2.0.rflpack")
infer = reflow.template_actor("tpl_ml_run_inference")

Subgraphs

sub = reflow.SubgraphBuilder(graph_export_json)
sub.register_actor("my_custom", MyCustom())
sub.fill_from_catalog()
net.register_actor("tpl_sub", sub.build())

Streams

stream = net.create_stream(buffer_size=64, content_type="image/jpeg")
producer.emit({"stream": stream})
for frame in stream:
    print(frame["kind"], len(frame.get("data", b"")))

See also

• Full Python SDK README — complete API + GIL notes.
• Authoring Actors.
• Packs.

Go SDK

github.com/offbit-ai/reflow/sdk/go — Go 1.21+, links the runtime via cgo.

Unlike the Node / Python / JVM SDKs, the Go module doesn't bundle the native runtime. You install the Go module from source, then drop the matching libreflow_rt_capi.{so,dylib,dll} next to it.

Install

go get github.com/offbit-ai/reflow/sdk/go@v0.2.1

# After go get, run the bundled installer to fetch the matching native lib:
cd "$(go env GOMODCACHE)/github.com/offbit-ai/reflow/sdk/go@v0.2.1"
./scripts/install_lib.sh v0.2.1

install_lib.sh downloads the per-triple tarball from the sdk/go/v* GitHub Release and unpacks it into lib/<goos>_<goarch>/ and include/, where cgo finds it at compile time.

For repo-local development (you've cloned the monorepo and want to test against your local Rust changes), use scripts/link_dev_lib.sh instead — it symlinks target/<profile>/libreflow_rt_capi.* into the same sdk/go/{lib,include}/ layout.

Hello world

package main

import (
    "fmt"
    "time"
    reflow "github.com/offbit-ai/reflow/sdk/go"
)

type Doubler struct{ reflow.BaseActor }

func newDoubler() *Doubler {
    return &Doubler{BaseActor: reflow.BaseActor{
        ComponentName: "doubler",
        InportsList:   []string{"in"},
        OutportsList:  []string{"out"},
    }}
}

func (d *Doubler) Run(ctx *reflow.ActorContext) error {
    in := ctx.Input("in")
    if in == nil { return nil }
    n, _ := in.AsInteger()
    return ctx.Emit("out", reflow.MessageInteger(n*2))
}

func main() {
    net := reflow.NewNetwork()
    defer net.Close()
    _ = net.RegisterActor("tpl_doubler", newDoubler())
    _ = net.AddNode("a", "tpl_doubler", nil)
    _ = net.AddInitial("a", "in", map[string]any{"type": "Integer", "data": 21}, nil)
    _ = net.Start()
    time.Sleep(200 * time.Millisecond)

    fmt.Println("done")
}

Authoring graphs

g := reflow.NewGraph("demo", false)
defer g.Close()
_ = g.AddNode("a", "tpl_x", nil)
_ = g.AddNode("b", "tpl_y", nil)
_ = g.AddConnection("a", "out", "b", "in", nil)
_ = g.AddGroup("pipe", []string{"a", "b"}, map[string]any{"caption": "pipeline"})
_ = g.RenameNode("a", "alpha")

groupsJSON, _ := g.GroupsJSON()  // []byte; parse with encoding/json

The full graph API is mirrored — see sdk/go/README.md for the complete method list. Read-side methods all return []byte (JSON) so callers pick their own decoder.

Bundled component catalog

ids, _ := reflow.TemplateList()
actor, _ := reflow.TemplateActor("tpl_http_request")
_ = net.RegisterActor("tpl_http_request", actor)

Packs

templates, _ := reflow.LoadPack("./reflow.pack.ml-0.2.0.rflpack")
fmt.Println("loaded:", templates)
infer, _ := reflow.TemplateActor("tpl_ml_run_inference")

See Packs for the full pack workflow.

Subgraphs

b, _ := reflow.NewSubgraphBuilder(exportJSON)
_ = b.RegisterGoActor("my_custom", NewCustom())
_ = b.FillFromCatalog()
sg, _ := b.Build()
_ = net.RegisterActor("tpl_sub", sg)

Streams

s := reflow.NewStream(reflow.StreamOptions{BufferSize: 64, ContentType: "image/jpeg"})
producer.Emit("stream", reflow.MessageStream(s))
reader := s.Reader()
for {
    frame, err := reader.Recv()
    if err != nil { break }
    fmt.Println(frame.Kind, len(frame.Data))
}

See also

• Full Go SDK README — install troubleshooting, cgo notes.
• Authoring Actors.
• Packs.

JVM SDK (Java + Kotlin)

ai.offbit:reflow — JDK 17+, single fat jar with native libs bundled as classpath resources for darwin / linux / windows.

Install

// Gradle (Kotlin DSL)
dependencies {
    implementation("ai.offbit:reflow:0.2.2")
}

<!-- Maven -->
<dependency>
    <groupId>ai.offbit</groupId>
    <artifactId>reflow</artifactId>
    <version>0.2.2</version>
</dependency>

A JNI loader inside the jar detects the host triple at startup, extracts the matching libreflow_rt_jvm.{dylib,so,dll} to java.io.tmpdir, and System.loads it. No separate native-lib install step.

Hello world (Kotlin DSL)

import ai.offbit.reflow.*

network {
    val doubler = actor {
        component = "doubler"
        inports = listOf("in")
        outports = listOf("out")
        onRun { ctx ->
            val n = parseIntegerInput(ctx.inputsJson(), "in")
            ctx.emit("out", Message.integer(n * 2))
            ctx.done()
        }
    }
    registerActor("tpl_doubler", doubler)
    addNode("d", "tpl_doubler")
    addInitial("d", "in", """{"type":"Integer","data":21}""")
    start()
}

Hello world (Java)

import ai.offbit.reflow.*;

class Doubler implements Actor {
    @Override public String component() { return "doubler"; }
    @Override public List<String> inports() { return List.of("in"); }
    @Override public List<String> outports() { return List.of("out"); }
    @Override public void run(ActorCallContext ctx) {
        long n = parseInteger(ctx.inputsJson(), "in");
        ctx.emit("out", Message.integer(n * 2));
        ctx.done();
    }
}

try (var net = new Network()) {
    net.registerActor("tpl_doubler", new Doubler());
    net.addNode("d", "tpl_doubler");
    net.addInitial("d", "in", "{\"type\":\"Integer\",\"data\":21}");
    net.start();
}

Authoring graphs

val g = Graph("demo")
g.addNode("a", "tpl_x")
 .addNode("b", "tpl_y")
 .addConnection("a", "out", "b", "in")
 .addGroup("pipe", "[\"a\",\"b\"]", "{\"caption\":\"pipeline\"}")
 .renameNode("a", "alpha")

println(g.groupsJson())
println(g.connectionsJson())

The full graph API is mirrored on Graph — see sdk/jvm/README.md for the complete method list. Query methods (*_json) return String; pair with Jackson, Moshi, or kotlinx.serialization to parse.

Bundled component catalog

long httpActor = Templates.templateActor("tpl_http_request");
net.registerActor("tpl_http_request", httpActor);
String allTemplates = Templates.templateListJson();

Packs

import ai.offbit.reflow.Packs

Packs.loadPack("./reflow.pack.ml-0.2.0.rflpack")
val infer = Templates.templateActor("tpl_ml_run_inference")
println(Packs.listPacks())

See Packs for the full pack workflow.

Subgraphs

SubgraphBuilder(graphExportJson).use { sub ->
    sub.registerActor("my_custom", MyCustom())
    sub.fillFromCatalog()
    val sg = sub.build()
    net.registerActor("tpl_sub", sg)
}

Streams

val stream = net.createStream(bufferSize = 64, contentType = "image/jpeg")
producer.emit(Message.stream(stream))
stream.reader().use { reader ->
    while (true) {
        val frame = reader.recv() ?: break
        println("${frame.kind} ${frame.data.size}")
    }
}

API reference

• Maven Central artifact page
• javadoc.io — Dokka-rendered docs with the SDK README as the landing page.
• Full JVM SDK README.

See also

• Authoring Actors.
• Packs.

C++ SDK

Header-only C++17 RAII wrapper over libreflow_rt_capi. Lives at sdk/cpp/ in the repo. No package manager today — pull the header directly + link the runtime library.

Requirements

• C++17 compiler (clang ≥ 9, gcc ≥ 9, MSVC 2019+)
• libreflow_rt_capi.{so,dylib,dll} — install via either the sdk/go/v* GitHub Release tarballs (pre-built per triple) or by building from the monorepo (cargo build -p reflow_rt_capi --release).

Install

# Drop the header tree under your third_party/.
git clone --branch sdk/go/v0.2.1 --depth 1 \
    https://github.com/offbit-ai/reflow third_party/reflow

# Grab the matching native lib for your platform.
VER=0.2.1
TRIPLE=aarch64-apple-darwin
curl -LO https://github.com/offbit-ai/reflow/releases/download/sdk/go/v$VER/reflow-rt-capi-$TRIPLE-v$VER.tar.gz
sudo tar -xzf reflow-rt-capi-$TRIPLE-v$VER.tar.gz -C /usr/local

CMake:

add_subdirectory(third_party/reflow/sdk/cpp)
target_link_libraries(myapp PRIVATE reflow::cpp)

If find_library(reflow_rt_capi) doesn't pick up the runtime automatically:

set(REFLOW_RT_CAPI_LIB "/usr/local/lib/libreflow_rt_capi.dylib")
add_subdirectory(third_party/reflow/sdk/cpp)

Hello world

#include <reflow/reflow.hpp>
#include <atomic>
#include <chrono>
#include <thread>

int main() {
    reflow::Network net;

    auto doubler = reflow::Actor::from_callback(
        "doubler", {"in"}, {"out"},
        [](reflow::Context& ctx) {
            auto in = ctx.input_json("in");
            if (!in) return;
            auto pos = in->find("\"data\":");
            if (pos == std::string::npos) return;
            int64_t n = std::stoll(in->substr(pos + 7));
            ctx.emit("out", reflow::Message::integer(n * 2));
        });

    std::atomic<int64_t> got{0};
    auto collector = reflow::Actor::from_callback(
        "collector", {"in"}, {},
        [&](reflow::Context& ctx) {
            if (auto m = ctx.take_input("in")) {
                auto j = m->as_json();
                auto pos = j.find("\"data\":");
                if (pos != std::string::npos) got = std::stoll(j.substr(pos + 7));
            }
        });

    net.register_actor("tpl_doubler", std::move(doubler));
    net.register_actor("tpl_collector", std::move(collector));
    net.add_node("a", "tpl_doubler");
    net.add_node("b", "tpl_collector");
    net.add_connection("a", "out", "b", "in");
    net.add_initial("a", "in", R"({"type":"Integer","data":21})");
    net.start();

    std::this_thread::sleep_for(std::chrono::milliseconds(500));
    std::cout << "got: " << got.load() << "\n";  // → 42
    net.shutdown();
}

Authoring graphs

reflow::Graph g("demo");
g.add_node("a", "tpl_x");
g.add_node("b", "tpl_y");
g.add_connection("a", "out", "b", "in");
g.add_group("pipe", R"(["a","b"])", R"({"caption":"pipeline"})");
g.rename_node("a", "alpha");

if (auto node = g.get_node_json("alpha")) std::cout << *node << "\n";
std::cout << g.groups_json() << "\n";
std::cout << g.connections_json() << "\n";

std::optional<std::string> for nullable returns (get_node_json, get_connection_json); plain std::string everywhere else. Pick your own JSON parser — nlohmann/json, simdjson, RapidJSON, etc.

Error handling

Every C ABI call is checked. Non-OK status throws reflow::Error, which carries the original rfl_status and the runtime's last error string:

try {
    net.add_initial("missing-actor", "in", R"({"type":"Flow"})");
} catch (const reflow::Error& e) {
    std::cerr << e.what() << " status=" << e.status() << "\n";
}

Packs

auto templates = reflow::pack::load("./reflow.pack.ml-0.2.0.rflpack");
auto manifest  = reflow::pack::inspect_json("./reflow.pack.ml-0.2.0.rflpack");
auto loaded    = reflow::pack::list_json();

See Packs for the full pack workflow.

Subgraphs

reflow::SubgraphBuilder sub(graph_export_json);
sub.register_actor("my_custom", std::move(custom_actor));
sub.fill_from_catalog();
auto sg = sub.build();
net.register_actor("tpl_sub", std::move(sg));

ABI lockstep

The C++ wrapper has no version of its own — it tracks the libreflow_rt_capi it links against. Pull the header from the same tag as the runtime tarball you install. reflow::pack::abi_version() returns the runtime's pack-ABI hash so you can sanity-check at startup.

See also

• Full C++ SDK README — full CMake config + linkage notes.
• sdk/cpp/examples/quickstart.cpp — the worked end-to-end example.
• Authoring Actors.
• Packs.

Actor Packs

A .rflpack is a multi-platform native plugin bundle that publishes additional actor templates into a Reflow runtime at load time. Every SDK can load packs the same way, via a single loadPack(path) call.

The default SDK install is intentionally lightweight — ~270 templates covering data, animation, math, scene, HTTP, and basic media. Heavier palettes (GPU renderers, ML inference, browser automation, ~6,700 SaaS API actors) ship as packs so you only pay for what you use.

Why packs

Without packs, the only way to extend the SDK catalog is to recompile the runtime with new Cargo features and rebuild every per-platform native artifact. Packs make actor catalogs a runtime concern:

• Smaller default install. No GPU drivers, no LiteRT, no Chromium dependencies in the base SDK.
• Modular delivery. Ship third-party actor palettes as drop-in .rflpack files.
• Faster iteration. New actors → new pack → users loadPack(...) without bumping the SDK.
• Same lifecycle as bundled actors. templateActor("tpl_pack_owned_id") and graph references work identically once a pack is loaded.

Bundle format

A .rflpack is a zip archive:

mypack-0.2.0.rflpack
├── manifest.json
└── lib/
    ├── aarch64-apple-darwin/libmypack.dylib
    ├── x86_64-apple-darwin/libmypack.dylib
    ├── x86_64-unknown-linux-gnu/libmypack.so
    ├── aarch64-unknown-linux-gnu/libmypack.so
    └── x86_64-pc-windows-msvc/mypack.dll

manifest.json declares the pack name, version, supported triples, advertised template ids, and the runtime ABI version it was built against:

{
  "manifest_version": 1,
  "name": "offbit.ml",
  "version": "0.2.0",
  "reflow_pack_abi_version": 1380148208,
  "entrypoint": "reflow_pack_register",
  "targets": {
    "aarch64-apple-darwin":     { "file": "lib/aarch64-apple-darwin/libmypack.dylib" },
    "x86_64-pc-windows-msvc":   { "file": "lib/x86_64-pc-windows-msvc/mypack.dll" }
  },
  "templates": ["tpl_ml_run_inference", "tpl_cv_image_to_tensor", "..."]
}

The host validates the manifest, picks the dylib for the current triple, dlopens it, and asks the pack to register its templates with the runtime. Templates published by a pack are first-class — they participate in templateList(), subgraph resolution, network registration, and so on, identical to bundled templates.

First-party packs

Six are published on every pack-vX.Y.Z GitHub Release:

Per-pack template inventories live in sdk/packs/README.md.

Loading a pack

Every SDK exposes the same four entry points, named idiomatically:

loadPack is idempotent — repeated calls with the same pack name are no-ops and return the previously published template ids.

import { loadPack, templateList, templateActor } from "@offbit-ai/reflow";

const ids = loadPack("./reflow.pack.ml-0.2.0.rflpack");
console.log(ids);                                           // ["tpl_ml_run_inference", …]
console.log(templateList().filter(t => t.startsWith("tpl_ml")));
const infer = templateActor("tpl_ml_run_inference");        // resolves through the pack

Either ship the .rflpack alongside your application binary or download it on first run from a GitHub Release:

VER=0.2.0
curl -LO https://github.com/offbit-ai/reflow/releases/download/pack-v$VER/reflow.pack.ml-$VER.rflpack

ABI lockstep

The pack handshake is toolchain-locked. Every pack stamps a reflow_pack_abi_version value at build time, computed from fnv1a(rustc_verbose_version || PACK_ABI_REVISION). The host loader refuses to dlopen a pack whose ABI doesn't match the SDK release the user installed.

Practically:

• A pack-v0.2.0 release is paired with node-v0.2.0 / python-v0.2.1 / sdk/jvm/v0.2.0 / sdk/go/v0.2.0. CI builds them from the same workspace revision with the same rustc.
• Mixing pack and SDK versions (e.g. pack-v0.2.0 with node-v0.3.0) errors at load with pack ABI X != host ABI Y.
• Third-party packs follow the same rule: the pack author rebuilds when the runtime upgrades. We document reflow_pack_loader::REFLOW_PACK_ABI_VERSION so authors can lock against a specific runtime.

This is the trade-off chosen for v1: Arc<dyn Actor> over the C ABI requires layout agreement, which means same rustc + same reflow_actor crate. A callback-only ABI that survives toolchain mismatches is a future option (see reflow_pack_loader).

See also

• Authoring a pack — write and ship your own.
• sdk/packs/README.md — first-party pack catalog with template inventories.

Authoring a Pack

A pack is a Rust cdylib crate using reflow_pack_sdk. The SDK provides:

• The #[reflow_pack] attribute macro that emits the C ABI entrypoints.
• A safe PackHost API for registering template factories.
• Re-exports of Actor, Message, ActorContext, etc.

You write Rust actors as you would for any Reflow runtime, then ship the resulting .rflpack from a CI workflow.

Skeleton

# my_pack/Cargo.toml
[package]
name = "my_pack"
version = "0.1.0"
edition = "2024"

[lib]
crate-type = ["cdylib"]

[dependencies]
reflow_pack_sdk = { version = "0.2.0", path = "../reflow/crates/reflow_pack_sdk" }
anyhow = "1"
flume = "0.11"
parking_lot = "0.12"

#![allow(unused)]
fn main() {
// my_pack/src/lib.rs
use reflow_pack_sdk::{
    reflow_pack, Actor, ActorBehavior, ActorContext, ActorLoad,
    ActorState, MemoryState, Message, PackHost, Port,
};

use parking_lot::Mutex;
use std::collections::HashMap;
use std::future::Future;
use std::pin::Pin;
use std::sync::Arc;

struct EchoActor {
    inports: Port,
    outports: Port,
    load: Arc<ActorLoad>,
}

impl EchoActor {
    fn new() -> Self {
        Self {
            inports: flume::bounded(16),
            outports: flume::bounded(16),
            load: Arc::new(ActorLoad::new(0)),
        }
    }
}

impl Actor for EchoActor {
    fn get_behavior(&self) -> ActorBehavior {
        Box::new(|ctx: ActorContext| -> Pin<
            Box<
                dyn Future<Output = Result<HashMap<String, Message>, anyhow::Error>>
                    + Send + 'static,
            >,
        > {
            Box::pin(async move {
                let payload = ctx.get_payload().clone();
                let input = payload.get("input").cloned().unwrap_or(Message::Flow);
                let mut out = HashMap::new();
                out.insert("output".to_string(), input);
                Ok(out)
            })
        })
    }
    fn get_inports(&self)  -> Port { self.inports.clone() }
    fn get_outports(&self) -> Port { self.outports.clone() }
    fn inport_names(&self)  -> Vec<String> { vec!["input".into()] }
    fn outport_names(&self) -> Vec<String> { vec!["output".into()] }
    fn create_state(&self)  -> Arc<Mutex<dyn ActorState>> {
        Arc::new(Mutex::new(MemoryState::default()))
    }
    fn load_count(&self) -> Arc<ActorLoad> { Arc::clone(&self.load) }
    fn create_instance(&self) -> Arc<dyn Actor> { Arc::new(EchoActor::new()) }
}

#[reflow_pack]
fn register(host: &mut PackHost) {
    host.register("my.pack.echo", || Arc::new(EchoActor::new()));
}
}

The #[reflow_pack] macro expands register into the two C symbols the loader looks up: reflow_pack_abi_version (returns the host ABI hash) and reflow_pack_register (calls back into the host vtable to register every template factory).

Reflow.pack.toml

A small companion file that drives the reflow-pack CLI, which assembles the multi-platform .rflpack zip:

[pack]
name = "my.pack.echo"
version = "0.1.0"
description = "Echo actor that copies input → output"
authors = ["Your Name"]
license = "MIT"
templates = ["my.pack.echo"]

# Paths are relative to this file. CI populates one entry per built triple.
[targets.files]
aarch64-apple-darwin = "../../target/release/libmy_pack.dylib"

Build

# 1. Build the cdylib for each triple you want to ship.
cargo build --release -p my_pack --target aarch64-apple-darwin
cargo build --release -p my_pack --target x86_64-unknown-linux-gnu
# … plus any others you target

# 2. Build the packaging CLI.
cargo build --release -p reflow_pack_cli

# 3. Read the host ABI version. The pack must be stamped with the
#    same number, so build the CLI with the SAME rustc as the runtime
#    your users will load this pack into.
target/release/reflow-pack abi
# abi_version = 1380148208
# host_triple = aarch64-apple-darwin

# 4. Bundle.
REFLOW_PACK_ABI_VERSION=1380148208 target/release/reflow-pack build \
    --manifest my_pack/Reflow.pack.toml \
    --out-dir target/packs

# 5. Inspect.
target/release/reflow-pack inspect target/packs/my.pack.echo-0.1.0.rflpack

CI: ship a multi-platform pack

.github/workflows/publish-packs.yml is a concrete example. The pattern:

1. Matrix-build the cdylib on all five supported runners (mac aarch64 / mac x86_64 / linux x86_64 / linux aarch64 / windows x86_64).
2. Upload the per-triple cdylib as a GitHub Actions artifact.
3. Assembly job downloads every artifact, generates a Reflow.pack.toml pointing at all of them, runs reflow-pack build to zip into a single .rflpack, attaches to a Release.

Pin dtolnay/rust-toolchain@stable everywhere so every cdylib in the bundle shares one ABI hash.

Distribution options

• GitHub Release attachment — easiest; users curl the .rflpack.
• Internal artifact registry — anything that serves files works (Artifactory, Nexus, S3, GCS).
• Bundled with your application — drop the .rflpack next to your binary and loadPack(__dirname + "/x.rflpack") at startup.
• Inside an npm / PyPI package — ship the .rflpack as a data file; your install script runs loadPack on first import.

ABI lockstep — what to communicate to users

A pack tied to runtime ABI version X only loads into SDK releases built against the same X. Make it easy on consumers:

• Tag your pack with the same vX.Y.Z as the SDK release it targets.
• Document the supported SDK versions in your README.
• If you maintain multiple SDK targets (e.g. one for node-v0.2.0 and one for node-v0.3.0), publish two .rflpack files and label them clearly.

When the runtime bumps PACK_ABI_REVISION (vtable shape changes), all packs need to be rebuilt. Watch the reflow_pack_loader/build.rs constant for changes.

Troubleshooting

• "pack ABI X != host ABI Y" — the pack was built with a different rustc or a different PACK_ABI_REVISION. Rebuild from the same toolchain as the host.
• "pack has no build for triple T" — the manifest doesn't include the user's platform. Add the missing triple to [targets.files] and re-bundle.
• "unable to execute patch / no NASM" — Linux cross-compile environments may need apt install patch nasm gcc-aarch64-linux-gnu g++-aarch64-linux-gnu. See publish-packs.yml for the working CI matrix.

See also

• Pack overview — what packs are, how to load them.
• crates/reflow_pack_sdk — author-facing crate with the macro + safe types.
• crates/reflow_pack_loader — runtime loader (read this if you're embedding Reflow in a custom host).
• sdk/packs/ — first-party pack source as worked examples.

Architecture Overview

This document provides a high-level overview of Reflow's architecture, covering its core components, design principles, and system interactions.

System Architecture

Reflow follows a modular, actor-based architecture designed for scalability, reliability, and multi-language support.

graph TB
    subgraph "reflow_server"
        REST[REST API + WebSocket]
        ENG[Execution Engine]
        EB[EventBridge]
        TC[TraceCollector]
        ZIP[ZipSession]
        ZC[Zeal Converter]
    end

    subgraph "reflow_components"
        FC[Flow Control]
        TR[Transforms]
        INT[Integration]
        LG[Logic]
        MD[Media]
        API[API Actors x6,697]
    end

    subgraph "reflow_network"
        NET[Network]
        GR[Graph]
        MSG[Message System]
        CON[Connectors]
    end

    subgraph "External"
        ZEAL[Zeal IDE]
        CLIENT[HTTP / WS Clients]
    end

    CLIENT --> REST
    REST --> ENG
    ENG --> NET
    NET --> CON
    CON --> MSG

    ENG --> EB
    EB --> TC
    EB --> ZIP

    TC -->|HTTP traces| ZEAL
    ZIP -->|WebSocket events| ZEAL
    ZIP -->|Register templates| ZEAL

    ZC --> ENG

Core Components

1. Actor System (reflow_network)

The foundation of Reflow, implementing the actor model for concurrent computation:

• Actors: Isolated units of computation
• Messages: Immutable data passed between actors
• Ports: Communication channels (input/output)
• Network: Manages actor lifecycle and message routing

2. Script Runtime (reflow_script)

Multi-language execution environment supporting:

• Deno Runtime: JavaScript/TypeScript execution
• Python Engine: Python script execution (with optional Docker isolation)
• WebAssembly: WASM plugin system via Extism
• Script Context: Execution environment and state management

3. Component Library (reflow_components)

Pre-built, reusable workflow components organized by category:

• Flow Control: ConditionalBranchActor, SwitchCaseActor, LoopActor
• Transform: DataTransformActor, DataOperationsActor, inline JS evaluation via rquickjs
• Integration: HttpRequestActor
• Logic: RulesEngineActor
• Media: ImageInputActor, AudioInputActor, VideoInputActor
• API Actors (feature-gated): 6,697 pre-generated actors across 88 API services (Slack, GitHub, Stripe, etc.)

Script execution (JavaScript, Python, SQL) is handled externally by dynASB — this crate only contains native actors.

4. Execution Server (reflow_server)

The server wraps the engine and components into a deployable node:

• ExecutionEngine: Creates isolated Network per execution, translates NetworkEvents into EngineEvents
• EventBridge: Per-execution consumer task forwarding events to TraceCollector + ZipSession
• ZipSession: Outbound WebSocket connection to Zeal IDE for real-time event streaming and template registration
• TraceCollector: HTTP-based trace session submission to Zeal's TracesAPI with batched events
• REST API: Axum-based HTTP + WebSocket API for headless workflow execution
• Zeal Converter: Translates Zeal workflow format into Reflow graph format

5. Network Layer (reflow_network)

Handles execution and communication:

• Message Routing: Efficient message delivery via flume channels
• Graph Management: Workflow topology and execution
• Connection Management: Inter-actor connectivity via Connector + ConnectionPoint
• NetworkEvent Stream: ActorStarted, ActorCompleted, ActorFailed, MessageSent, NetworkIdle, NetworkShutdown

Design Principles

Actor Model

Reflow is built on the Actor Model of computation:

#![allow(unused)]
fn main() {
pub trait Actor {
    fn get_behavior(&self) -> ActorBehavior;
    fn get_inports(&self) -> Port;
    fn get_outports(&self) -> Port;
    fn create_process(&self) -> Pin<Box<dyn Future<Output = ()> + Send + 'static>>;
}
}

Key Properties:

• Isolation: No shared state between actors
• Concurrency: Actors run concurrently
• Message Passing: Communication via immutable messages
• Location Transparency: Actors can be local or remote

Immutable Messages

All communication uses immutable message types:

#![allow(unused)]
fn main() {
pub enum Message {
    String(String),
    Integer(i64),
    Float(f64),
    Boolean(bool),
    Array(Vec<Message>),
    Object(HashMap<String, Message>),
    Binary(Vec<u8>),
    Null,
    Error(String),
}
}

Async-First Design

Built on Rust's async/await system using Tokio:

• Non-blocking I/O operations
• Efficient resource utilization
• Scalable concurrent execution
• Backpressure handling

Execution Model

Actor Lifecycle

stateDiagram-v2
    [*] --> Created
    Created --> Initialized
    Initialized --> Running
    Running --> Processing
    Processing --> Running
    Running --> Stopping
    Stopping --> Stopped
    Stopped --> [*]
    
    Processing --> Error
    Error --> Running
    Error --> Stopping

1. Creation: Actor instance created with configuration
2. Initialization: Resources allocated, connections established
3. Running: Actor ready to process messages
4. Processing: Executing behavior function on incoming messages
5. Stopping: Graceful shutdown initiated
6. Stopped: All resources cleaned up

Message Flow

sequenceDiagram
    participant S as Source Actor
    participant M as Message Bus
    participant T as Target Actor
    
    S->>M: Send Message
    M->>M: Route Message
    M->>T: Deliver Message
    T->>T: Process Message
    T->>M: Send Response
    M->>S: Deliver Response

Graph Execution

Workflows are executed as directed acyclic graphs (DAGs):

• Topological Ordering: Ensures correct execution sequence
• Parallel Execution: Independent branches run concurrently
• Backpressure: Prevents resource exhaustion
• Error Propagation: Failures are handled gracefully

Runtime Architecture

Native Runtime (Rust)

Direct Rust implementation for maximum performance:

#![allow(unused)]
fn main() {
struct NativeActor {
    behavior: Box<dyn Fn(ActorContext) -> Pin<Box<dyn Future<Output = Result<HashMap<String, Message>, Error>> + Send>>>,
    // ... other fields
}
}

Script Runtimes

Deno Runtime

• Sandbox: Secure execution environment
• Permissions: Fine-grained access control
• TypeScript: Full TypeScript support
• NPM: Package ecosystem access

Python Runtime

• Isolation: Process-level or Docker isolation
• Libraries: Full Python ecosystem support
• Async: Async/await support
• Error Handling: Exception propagation

WebAssembly Runtime

• Portability: Cross-platform execution
• Security: Sandboxed execution
• Performance: Near-native speed
• Multi-language: Support for multiple source languages

Memory Management

Ownership Model

Follows Rust's ownership principles:

• Single Ownership: Each value has a single owner
• Borrowing: Temporary access without ownership transfer
• Lifetimes: Compile-time memory safety guarantees
• Reference Counting: Shared ownership where needed

Message Serialization

Efficient serialization for message passing:

#![allow(unused)]
fn main() {
// Compressed serialization for performance
let compressed = compress_message(&message)?;
let serialized = bitcode::serialize(&compressed)?;

// Network transmission
send_over_network(serialized).await?;

// Deserialization
let message = bitcode::deserialize(&received_data)?;
let decompressed = decompress_message(&message)?;
}

Networking Architecture

Local Communication

graph LR
    A1[Actor 1] --> C1[Channel]
    C1 --> A2[Actor 2]
    A2 --> C2[Channel]
    C2 --> A3[Actor 3]

Local Channels:

• Flume: High-performance async channels
• Zero-copy: Direct memory access where possible
• Backpressure: Flow control mechanisms

Distributed Communication

graph TB
    subgraph "Node 1"
        A1[Actor A]
        A2[Actor B]
    end
    
    subgraph "Network Layer"
        N1[Network Bridge]
        N2[Network Bridge]
    end
    
    subgraph "Node 2"
        A3[Actor C]
        A4[Actor D]
    end
    
    A1 --> N1
    N1 --> N2
    N2 --> A3

Network Features:

• WebSocket: Real-time communication
• Compression: Efficient data transfer
• Encryption: Secure communication
• Discovery: Automatic node discovery

Error Handling

Hierarchical Error Management

graph TD
    A[Actor Error] --> B[Network Error Handler]
    B --> C[Workflow Error Handler]
    C --> D[Application Error Handler]
    
    B --> E[Circuit Breaker]
    C --> F[Retry Logic]
    D --> G[Dead Letter Queue]

Error Strategies:

• Isolation: Errors don't affect other actors
• Propagation: Structured error reporting
• Recovery: Automatic retry and fallback
• Monitoring: Error tracking and alerting

Security Model

Sandboxing

Each runtime environment provides isolation:

• Deno: V8 isolates with permission system
• Python: Process isolation or containerization
• WASM: Memory-safe execution environment
• Native: Rust's memory safety guarantees

Permission System

Fine-grained access control:

#![allow(unused)]
fn main() {
pub struct Permissions {
    pub file_system: FileSystemPermissions,
    pub network: NetworkPermissions,
    pub environment: EnvironmentPermissions,
}
}

Performance Characteristics

Throughput

• Message Rate: >1M messages/second (local)
• Latency: <1ms (local), <10ms (network)
• Memory: ~1KB per actor overhead
• CPU: Scales with core count

Scalability

• Horizontal: Distribute across machines
• Vertical: Utilize all CPU cores
• Elastic: Dynamic resource allocation
• Backpressure: Graceful degradation under load

Configuration

Runtime Configuration

[actor_system]
thread_pool_size = 8
max_actors_per_node = 10000
message_buffer_size = 1000

[networking]
bind_address = "0.0.0.0:8080"
compression_enabled = true
encryption_enabled = true

[runtimes.deno]
permissions = ["--allow-net", "--allow-read"]
memory_limit = "512MB"

[runtimes.python]
use_docker = false
shared_environment = true

Observability Pipeline

Reflow's observability is built on an event pipeline that connects the execution engine to Zeal IDE:

sequenceDiagram
    participant N as Network
    participant E as ExecutionEngine
    participant EB as EventBridge
    participant TC as TraceCollector
    participant ZS as ZipSession
    participant Z as Zeal IDE

    N->>E: NetworkEvent (ActorCompleted, MessageSent, etc.)
    E->>E: Translate to EngineEvent (add duration, output_size)
    E->>EB: Send via flume channel
    EB->>TC: Forward event
    EB->>ZS: Forward event
    TC->>Z: Submit trace events (HTTP batch)
    ZS->>Z: Emit ZIP event (WebSocket)

EventBridge

One bridge task is spawned per execution. It drains the engine's event channel and forwards to both consumers:

• TraceCollector: Buffers events (batch size 50), submits them as TraceEvents via Zeal's TracesAPI over HTTP
• ZipSession: Translates EngineEvents to ZipExecutionEvents and pushes them over WebSocket in real-time

EngineEvent Types

The engine translates low-level NetworkEvents into rich events with timing and size metadata:

See Observability Architecture and Event Types for details.

Extension Points

Custom Actors

#![allow(unused)]
fn main() {
impl Actor for CustomActor {
    fn get_behavior(&self) -> ActorBehavior {
        Box::new(|context| {
            Box::pin(async move {
                // Custom processing logic
                Ok(HashMap::new())
            })
        })
    }
}
}

Custom Runtimes

#![allow(unused)]
fn main() {
#[async_trait]
impl ScriptEngine for CustomEngine {
    async fn init(&mut self, config: &ScriptConfig) -> Result<()>;
    async fn call(&mut self, context: &ScriptContext) -> Result<HashMap<String, Message>>;
    async fn cleanup(&mut self) -> Result<()>;
}
}

Next Steps

For detailed information on specific components:

• Actor Model - Deep dive into actor implementation
• Message Passing - Message system details
• Graph System - Workflow graph management
• Zeal IDE Integration - ZIP session, template registration, WebSocket events
• REST API - HTTP and WebSocket API reference
• Component Library - Native actors and API service actors
• Observability Architecture - EventBridge, TraceCollector, ZipSession

Actor Model

This document provides an in-depth look at how Reflow implements the Actor Model of computation.

Introduction

The Actor Model is a mathematical model of concurrent computation that treats "actors" as the universal primitives of concurrent computation. In Reflow, actors are isolated computational units that communicate exclusively through message passing.

Core Principles

1. Everything is an Actor

In Reflow's actor system:

• Data processing units are actors
• Message routers are actors
• Database connections are actors
• Network services are actors

2. Actors Communicate via Messages

#![allow(unused)]
fn main() {
// Messages are immutable and serializable
pub enum Message {
    String(String),
    Integer(i64),
    Float(f64),
    Boolean(bool),
    Array(Vec<Message>),
    Object(HashMap<String, Message>),
    Binary(Vec<u8>),
    Null,
    Error(String),
}
}

3. Actors Have Private State

#![allow(unused)]
fn main() {
pub trait ActorState: Send + Sync + 'static {
    fn as_any(&self) -> &dyn Any;
    fn as_mut_any(&mut self) -> &mut dyn Any;
}

#[derive(Default, Debug, Clone)]
pub struct MemoryState(pub HashMap<String, Value>);
}

4. Actors Process Messages Sequentially

Each actor processes one message at a time, ensuring thread safety without locks.

Actor Implementation

Actor Trait

#![allow(unused)]
fn main() {
pub trait Actor: Send + Sync + 'static {
    /// Defines how the actor processes messages
    fn get_behavior(&self) -> ActorBehavior;
    
    /// Access to input ports
    fn get_inports(&self) -> Port;
    
    /// Access to output ports
    fn get_outports(&self) -> Port;
    
    /// Create the actor's execution process
    fn create_process(&self) -> Pin<Box<dyn Future<Output = ()> + 'static + Send>>;
    
    /// Load counting for backpressure (optional)
    fn load_count(&self) -> Arc<Mutex<ActorLoad>> {
        Arc::new(Mutex::new(ActorLoad::new(0)))
    }
}
}

Actor Behavior

The behavior function defines how an actor responds to messages:

#![allow(unused)]
fn main() {
pub type ActorBehavior = Box<
    dyn Fn(ActorContext) -> Pin<Box<dyn Future<Output = Result<HashMap<String, Message>, anyhow::Error>> + Send + 'static>>
        + Send + Sync + 'static,
>;
}

Actor Context

The context provides access to the actor's environment:

#![allow(unused)]
fn main() {
pub struct ActorContext {
    pub payload: ActorPayload,
    pub outports: Port,
    pub state: Arc<Mutex<dyn ActorState>>,
    pub config: HashMap<String, Value>,
    load: Arc<Mutex<ActorLoad>>,
}

impl ActorContext {
    pub fn get_state(&self) -> Arc<Mutex<dyn ActorState>>;
    pub fn get_config(&self) -> &HashMap<String, Value>;
    pub fn get_payload(&self) -> &ActorPayload;
    pub fn get_outports(&self) -> Port;
    pub fn done(&self);
}
}

Actor Types

1. Native Actors

Written directly in Rust for maximum performance:

#![allow(unused)]
fn main() {
use reflow_network::actor::{Actor, ActorBehavior, ActorContext, Port, MemoryState};
use reflow_network::message::Message;
use std::collections::HashMap;

pub struct FilterActor {
    threshold: f64,
    inports: Port,
    outports: Port,
}

impl FilterActor {
    pub fn new(threshold: f64) -> Self {
        Self {
            threshold,
            inports: flume::unbounded(),
            outports: flume::unbounded(),
        }
    }
}

impl Actor for FilterActor {
    fn get_behavior(&self) -> ActorBehavior {
        let threshold = self.threshold;
        
        Box::new(move |context: ActorContext| {
            Box::pin(async move {
                let payload = context.get_payload();
                let mut results = HashMap::new();
                
                if let Some(Message::Float(value)) = payload.get("input") {
                    if *value > threshold {
                        results.insert("output".to_string(), Message::Float(*value));
                    }
                }
                
                Ok(results)
            })
        })
    }
    
    fn get_inports(&self) -> Port { self.inports.clone() }
    fn get_outports(&self) -> Port { self.outports.clone() }
    
    fn create_process(&self) -> Pin<Box<dyn Future<Output = ()> + 'static + Send>> {
        // Implementation details...
        todo!()
    }
}
}

2. Script Actors

Execute scripts in various languages:

#![allow(unused)]
fn main() {
use reflow_script::{ScriptActor, ScriptConfig, ScriptRuntime, ScriptEnvironment};

// JavaScript Actor
let js_config = ScriptConfig {
    environment: ScriptEnvironment::SYSTEM,
    runtime: ScriptRuntime::JavaScript,
    source: include_bytes!("script.js").to_vec(),
    entry_point: "process".to_string(),
    packages: None,
};

let js_actor = ScriptActor::new(js_config);
}

3. Component Actors

Pre-built components from the library:

#![allow(unused)]
fn main() {
use reflow_components::flow_control::ConditionalActor;
use reflow_components::data_operations::MapActor;

let conditional = ConditionalActor::new(|msg| {
    if let Message::Integer(n) = msg {
        *n > 0
    } else {
        false
    }
});

let mapper = MapActor::new(|msg| {
    if let Message::Integer(n) = msg {
        Message::Integer(n * 2)
    } else {
        msg.clone()
    }
});
}

Message Passing Semantics

Asynchronous Messaging

Messages are sent asynchronously without blocking:

#![allow(unused)]
fn main() {
// Send message without waiting
outport.send_async(message).await?;

// Receive message when available
let message = inport.recv_async().await?;
}

Message Ordering

• Messages between the same pair of actors maintain order
• No global ordering guarantees across different actor pairs
• Use synchronization actors for coordination when needed

Message Delivery

• At-most-once: Messages may be lost but never duplicated
• Best-effort: System attempts delivery but doesn't guarantee it
• Backpressure: Slow consumers cause senders to block

Actor Lifecycle Management

Creation and Initialization

#![allow(unused)]
fn main() {
// Create actor
let actor = MyActor::new(config);

// Initialize ports and state
let inports = actor.get_inports();
let outports = actor.get_outports();

// Start actor process
let process = actor.create_process();
tokio::spawn(process);
}

Message Processing Loop

#![allow(unused)]
fn main() {
pub fn create_process(&self) -> Pin<Box<dyn Future<Output = ()> + 'static + Send>> {
    let inports = self.get_inports();
    let behavior = self.get_behavior();
    let state = Arc::new(Mutex::new(MemoryState::default()));
    let outports = self.get_outports();
    
    Box::pin(async move {
        while let Ok(payload) = inports.1.recv_async().await {
            let context = ActorContext::new(
                payload,
                outports.clone(),
                state.clone(),
                HashMap::new(),
                Arc::new(Mutex::new(ActorLoad::new(0))),
            );
            
            match behavior(context).await {
                Ok(result) => {
                    if !result.is_empty() {
                        let _ = outports.0.send_async(result).await;
                    }
                },
                Err(e) => {
                    let error_msg = HashMap::from([
                        ("error".to_string(), Message::Error(e.to_string()))
                    ]);
                    let _ = outports.0.send_async(error_msg).await;
                }
            }
        }
    })
}
}

Termination

Actors terminate when:

• Input ports are closed (no more messages)
• Explicit shutdown signal
• Unrecoverable error occurs

State Management

Actor State Types

#![allow(unused)]
fn main() {
// Simple memory state
let state = MemoryState::default();

// Custom state implementation
struct CounterState {
    count: AtomicU64,
}

impl ActorState for CounterState {
    fn as_any(&self) -> &dyn Any { self }
    fn as_mut_any(&mut self) -> &mut dyn Any { self }
}
}

State Persistence

#![allow(unused)]
fn main() {
// Access state in behavior
fn get_behavior(&self) -> ActorBehavior {
    Box::new(|context: ActorContext| {
        Box::pin(async move {
            let state = context.get_state();
            let mut state_guard = state.lock();
            
            // Read/modify state
            if let Some(memory_state) = state_guard.as_mut_any().downcast_mut::<MemoryState>() {
                memory_state.insert("counter", serde_json::json!(42));
            }
            
            Ok(HashMap::new())
        })
    })
}
}

Error Handling

Actor-Level Errors

#![allow(unused)]
fn main() {
// Return error from behavior
Err(anyhow::anyhow!("Processing failed: {}", reason))

// Handle errors in message processing
match behavior(context).await {
    Ok(result) => send_result(result).await,
    Err(e) => send_error(e).await,
}
}

Error Propagation

#![allow(unused)]
fn main() {
// Error message format
let error_message = HashMap::from([
    ("error".to_string(), Message::Error("Database connection failed".to_string())),
    ("code".to_string(), Message::Integer(500)),
    ("timestamp".to_string(), Message::String(Utc::now().to_rfc3339())),
]);
}

Supervision Strategies

#![allow(unused)]
fn main() {
// Supervisor actor monitors children
struct SupervisorActor {
    children: Vec<ActorRef>,
    restart_policy: RestartPolicy,
}

enum RestartPolicy {
    OneForOne,    // Restart only failed actor
    OneForAll,    // Restart all actors
    RestForOne,   // Restart failed and subsequent actors
}
}

Concurrency and Parallelism

Actor Isolation

• Each actor runs in isolation
• No shared mutable state
• Communication only via messages
• Thread-safe by design

Parallel Execution

#![allow(unused)]
fn main() {
// Multiple actors can run simultaneously
tokio::spawn(actor1.create_process());
tokio::spawn(actor2.create_process());
tokio::spawn(actor3.create_process());

// Actors on different CPU cores
let rt = tokio::runtime::Builder::new_multi_thread()
    .worker_threads(num_cpus::get())
    .build()?;
}

Load Balancing

#![allow(unused)]
fn main() {
// Round-robin message distribution
struct LoadBalancerActor {
    workers: Vec<Port>,
    current: AtomicUsize,
}

impl LoadBalancerActor {
    fn next_worker(&self) -> &Port {
        let index = self.current.fetch_add(1, Ordering::Relaxed) % self.workers.len();
        &self.workers[index]
    }
}
}

Performance Considerations

Memory Usage

• Actors have minimal overhead (~1KB per actor)
• Messages are reference-counted when possible
• State is lazily allocated

Message Throughput

• Local messages: >1M messages/second
• Network messages: 10K-100K messages/second
• Batch processing for high throughput

Backpressure Handling

#![allow(unused)]
fn main() {
// Check actor load before sending
let load = actor.load_count();
if load.lock().get() > MAX_LOAD {
    // Apply backpressure
    tokio::time::sleep(Duration::from_millis(10)).await;
}
}

Advanced Patterns

Actor Pooling

#![allow(unused)]
fn main() {
struct ActorPool<T: Actor> {
    actors: Vec<T>,
    distributor: LoadBalancerActor,
}

impl<T: Actor> ActorPool<T> {
    pub fn new(size: usize, factory: impl Fn() -> T) -> Self {
        let actors: Vec<T> = (0..size).map(|_| factory()).collect();
        // ... setup distributor
    }
}
}

Hot Swapping

#![allow(unused)]
fn main() {
// Replace actor behavior without stopping
actor.update_behavior(new_behavior).await?;

// Migrate state to new actor version
let old_state = old_actor.get_state();
new_actor.set_state(old_state).await?;
}

Circuit Breaker

#![allow(unused)]
fn main() {
struct CircuitBreakerActor {
    target: ActorRef,
    failure_count: AtomicU32,
    state: AtomicU8, // Open, Closed, HalfOpen
}

impl CircuitBreakerActor {
    fn should_allow_request(&self) -> bool {
        match self.state.load(Ordering::Relaxed) {
            0 => true,  // Closed
            1 => false, // Open
            2 => true,  // HalfOpen
            _ => false,
        }
    }
}
}

Testing Actors

Unit Testing

#![allow(unused)]
fn main() {
#[tokio::test]
async fn test_filter_actor() {
    let actor = FilterActor::new(5.0);
    let behavior = actor.get_behavior();
    
    // Create test context
    let payload = HashMap::from([
        ("input".to_string(), Message::Float(10.0))
    ]);
    
    let context = create_test_context(payload);
    let result = behavior(context).await.unwrap();
    
    assert!(result.contains_key("output"));
}
}

Integration Testing

#![allow(unused)]
fn main() {
#[tokio::test]
async fn test_actor_pipeline() {
    let source = SourceActor::new();
    let filter = FilterActor::new(5.0);
    let sink = SinkActor::new();
    
    // Connect actors
    connect_actors(&source, &filter).await;
    connect_actors(&filter, &sink).await;
    
    // Start pipeline
    let handles = vec![
        tokio::spawn(source.create_process()),
        tokio::spawn(filter.create_process()),
        tokio::spawn(sink.create_process()),
    ];
    
    // Test data flow
    // ... assertions
}
}

Best Practices

Actor Design

1. Keep actors small and focused - Single responsibility principle
2. Avoid blocking operations - Use async/await for I/O
3. Handle errors gracefully - Don't let actors crash
4. Design for failure - Expect message loss and actor failures

Message Design

1. Keep messages immutable - Never modify messages after sending
2. Use appropriate message sizes - Balance between batching and latency
3. Include context - Messages should carry enough information
4. Handle malformed messages - Validate input gracefully

State Management

1. Minimize state - Less state means fewer bugs
2. Make state serializable - Enable persistence and distribution
3. Avoid shared state - Each actor owns its state
4. Design for recovery - State should be reconstructible

Next Steps

• Message Passing - Detailed message system
• Graph System - Workflow composition
• Creating Actors - Practical guide
• Performance Optimization - Tuning guidelines

Message Passing

This document details Reflow's message passing system, which is the primary communication mechanism between actors.

Message Types

Reflow uses a strongly-typed message system with built-in serialization support:

#![allow(unused)]
fn main() {
#[derive(Clone, Debug, Serialize, Deserialize, Encode, Decode, PartialEq)]
pub enum Message {
    Flow,
    Event(EncodableValue),
    Boolean(bool),
    Integer(i64),
    Float(f64),
    String(Arc<String>),
    Object(Arc<EncodableValue>),
    Array(Arc<Vec<EncodableValue>>),
    Stream(Arc<Vec<u8>>),
    Encoded(Arc<Vec<u8>>),
    Optional(Option<Arc<EncodableValue>>),
    Any(Arc<EncodableValue>),
    Error(Arc<String>),
}
}

EncodableValue

Reflow uses EncodableValue as a wrapper for complex data types:

#![allow(unused)]
fn main() {
#[derive(Clone, Debug, Serialize, Deserialize, Encode, Decode, PartialEq, Eq)]
pub struct EncodableValue {
    pub(crate) data: Vec<u8>,
}

impl EncodableValue {
    pub fn new<T: Encode>(value: &T) -> Self {
        Self {
            data: bitcode::encode(value),
        }
    }

    pub fn decode<'a, T: Decode<'a>>(&'a self) -> Option<T> {
        bitcode::decode(&self.data).ok()
    }
}
}

Message Conversion

#![allow(unused)]
fn main() {
use serde_json::Value;

// From JSON values
let msg = Message::from(serde_json::json!(42));

// To JSON values  
let json: Value = message.into();

// Type checking
if let Message::Integer(n) = message {
    println!("Number: {}", n);
}

// Working with EncodableValue - modern approach
let data = serde_json::json!({"key": "value"});
let encodable = EncodableValue::from(data);
let object_msg = Message::object(encodable);

// Create arrays with EncodableValue - modern approach
let array_items = vec![
    EncodableValue::from(serde_json::json!("hello")),
    EncodableValue::from(serde_json::json!(42)),
];
let array_msg = Message::array(array_items);

// Alternative: using helper methods for simple values
let string_msg = Message::string("hello world".to_string());
let int_msg = Message::integer(42);
let bool_msg = Message::boolean(true);
let float_msg = Message::float(3.14);
let error_msg = Message::error("Something went wrong".to_string());
}

Communication Channels

Ports

Ports are the communication endpoints for actors:

#![allow(unused)]
fn main() {
pub type Port = (
    flume::Sender<HashMap<String, Message>>,
    flume::Receiver<HashMap<String, Message>>,
);

// Actor payload format
pub type ActorPayload = HashMap<String, Message>;
}

Channel Properties

• Asynchronous: Non-blocking send/receive operations
• Bounded: Configurable buffer sizes for backpressure
• Multi-producer, Single-consumer: Multiple senders, one receiver per port
• Type-safe: Compile-time message type checking

Message Flow Patterns

Point-to-Point

Direct communication between two actors:

#![allow(unused)]
fn main() {
// Actor A sends to Actor B - using helper method
let message = HashMap::from([
    ("data".to_string(), Message::string("hello".to_string()))
]);
sender.send_async(message).await?;
}

Broadcast

One actor sends to multiple receivers:

#![allow(unused)]
fn main() {
// Using actor macro for broadcast
#[actor(
    BroadcastActor,
    inports::<100>(input),
    outports::<50>(output1, output2, output3)
)]
async fn broadcast_actor(context: ActorContext) -> Result<HashMap<String, Message>, anyhow::Error> {
    let payload = context.get_payload();
    
    if let Some(input_msg) = payload.get("input") {
        // Broadcast to all output ports
        Ok([
            ("output1".to_owned(), input_msg.clone()),
            ("output2".to_owned(), input_msg.clone()),
            ("output3".to_owned(), input_msg.clone()),
        ].into())
    } else {
        Err(anyhow::anyhow!("No input to broadcast"))
    }
}

// Manual implementation for dynamic outputs
struct ManualBroadcastActor {
    inports: Port,
    outports: Port,
    outputs: Vec<flume::Sender<HashMap<String, Message>>>,
    load: Arc<Mutex<ActorLoad>>,
}

impl ManualBroadcastActor {
    async fn broadcast(&self, message: HashMap<String, Message>) -> Result<(), anyhow::Error> {
        for output in &self.outputs {
            output.send_async(message.clone()).await?;
        }
        Ok(())
    }
}
}

Fan-In (Merge)

Multiple actors send to one receiver:

#![allow(unused)]
fn main() {
struct MergeActor {
    inputs: Vec<flume::Receiver<HashMap<String, Message>>>,
    output: flume::Sender<HashMap<String, Message>>,
}

impl MergeActor {
    async fn merge_loop(&self) {
        use futures::stream::{FuturesUnordered, StreamExt};
        
        let mut streams: FuturesUnordered<_> = self.inputs
            .iter()
            .map(|rx| rx.recv_async())
            .collect();
            
        while let Some(result) = streams.next().await {
            if let Ok(message) = result {
                let _ = self.output.send_async(message).await;
            }
        }
    }
}
}

Serialization and Transport

Local Serialization

For local communication, messages use efficient in-memory representation:

#![allow(unused)]
fn main() {
// Zero-copy for simple types
let msg = Message::Integer(42); // No allocation

// Reference counting for complex types
let complex = Message::Object(data); // Rc<HashMap<String, Message>>
}

Network Serialization

For distributed communication:

#![allow(unused)]
fn main() {
use bitcode;
use flate2::Compression;

// Compress and serialize
let compressed = compress_message(&message, Compression::default())?;
let bytes = bitcode::serialize(&compressed)?;

// Send over network
network_send(bytes).await?;

// Receive and deserialize
let received = network_receive().await?;
let message = bitcode::deserialize(&received)?;
let decompressed = decompress_message(&message)?;
}

Message Routing

Router Actor

#![allow(unused)]
fn main() {
pub struct RouterActor {
    routes: HashMap<String, flume::Sender<HashMap<String, Message>>>,
    default_route: Option<flume::Sender<HashMap<String, Message>>>,
}

impl RouterActor {
    pub fn route_message(&self, key: &str, message: HashMap<String, Message>) -> Result<()> {
        if let Some(sender) = self.routes.get(key) {
            sender.try_send(message)?;
        } else if let Some(default) = &self.default_route {
            default.try_send(message)?;
        }
        Ok(())
    }
}
}

Content-Based Routing

#![allow(unused)]
fn main() {
impl RouterActor {
    fn route_by_content(&self, message: &HashMap<String, Message>) -> Option<&str> {
        // Route based on message content
        if let Some(Message::String(msg_type)) = message.get("type") {
            match msg_type.as_str() {
                "user_event" => Some("user_handler"),
                "system_event" => Some("system_handler"),
                "error" => Some("error_handler"),
                _ => None,
            }
        } else {
            None
        }
    }
}
}

Error Handling

Error Message Format

#![allow(unused)]
fn main() {
// Standard error message structure
let error_msg = HashMap::from([
    ("error".to_string(), Message::error("Processing failed".to_string())),
    ("code".to_string(), Message::integer(500)),
    ("source".to_string(), Message::string("database_actor".to_string())),
    ("timestamp".to_string(), Message::string(Utc::now().to_rfc3339())),
    ("details".to_string(), Message::object(error_details)),
]);
}

Dead Letter Queue

#![allow(unused)]
fn main() {
pub struct DeadLetterQueue {
    storage: Arc<Mutex<Vec<(String, HashMap<String, Message>)>>>,
    max_size: usize,
}

impl DeadLetterQueue {
    pub async fn store_failed_message(
        &self, 
        reason: String, 
        message: HashMap<String, Message>
    ) {
        let mut storage = self.storage.lock();
        if storage.len() >= self.max_size {
            storage.remove(0); // Remove oldest
        }
        storage.push((reason, message));
    }
}
}

Backpressure Management

Flow Control

#![allow(unused)]
fn main() {
pub struct FlowControlActor {
    input: flume::Receiver<HashMap<String, Message>>,
    output: flume::Sender<HashMap<String, Message>>,
    buffer_size: usize,
    current_load: Arc<AtomicUsize>,
}

impl FlowControlActor {
    async fn process_with_backpressure(&self) {
        while let Ok(message) = self.input.recv_async().await {
            // Check current load
            let load = self.current_load.load(Ordering::Relaxed);
            
            if load > self.buffer_size {
                // Apply backpressure - slow down
                tokio::time::sleep(Duration::from_millis(10)).await;
            }
            
            self.current_load.fetch_add(1, Ordering::Relaxed);
            
            // Process message
            if let Err(_) = self.output.try_send(message) {
                // Output buffer full, apply backpressure
                tokio::time::sleep(Duration::from_millis(1)).await;
            }
            
            self.current_load.fetch_sub(1, Ordering::Relaxed);
        }
    }
}
}

Message Ordering

Ordered Delivery

#![allow(unused)]
fn main() {
pub struct OrderedDeliveryActor {
    sequence_number: AtomicU64,
    expected_sequence: AtomicU64,
    buffer: Arc<Mutex<BTreeMap<u64, HashMap<String, Message>>>>,
}

impl OrderedDeliveryActor {
    fn add_sequence_number(&self, mut message: HashMap<String, Message>) -> HashMap<String, Message> {
        let seq = self.sequence_number.fetch_add(1, Ordering::Relaxed);
        message.insert("sequence".to_string(), Message::Integer(seq as i64));
        message
    }
    
    async fn deliver_in_order(&self, message: HashMap<String, Message>) {
        if let Some(Message::Integer(seq)) = message.get("sequence") {
            let seq = *seq as u64;
            let expected = self.expected_sequence.load(Ordering::Relaxed);
            
            if seq == expected {
                // Deliver immediately
                self.deliver_message(message).await;
                self.expected_sequence.fetch_add(1, Ordering::Relaxed);
                
                // Check buffer for next messages
                self.deliver_buffered_messages().await;
            } else {
                // Buffer out-of-order message
                self.buffer.lock().insert(seq, message);
            }
        }
    }
}
}

Performance Optimization

Message Batching

#![allow(unused)]
fn main() {
use crate::message::{Message, EncodableValue};

pub struct BatchingActor {
    batch_size: usize,
    batch_timeout: Duration,
    current_batch: Vec<HashMap<String, Message>>,
    input: flume::Receiver<HashMap<String, Message>>,
    output: flume::Sender<HashMap<String, Message>>,
}

impl BatchingActor {
    async fn process_with_batching(&mut self) {
        let mut interval = tokio::time::interval(self.batch_timeout);
        
        loop {
            tokio::select! {
                // Receive new message
                Ok(message) = self.input.recv_async() => {
                    self.current_batch.push(message);
                    
                    if self.current_batch.len() >= self.batch_size {
                        self.flush_batch().await;
                    }
                }
                
                // Timeout - flush partial batch
                _ = interval.tick() => {
                    if !self.current_batch.is_empty() {
                        self.flush_batch().await;
                    }
                }
            }
        }
    }
    
    async fn flush_batch(&mut self) {
        if !self.current_batch.is_empty() {
            // Convert to EncodableValue for proper serialization
            let batch_items: Vec<EncodableValue> = self.current_batch
                .drain(..)
                .map(|msg| EncodableValue::from(serde_json::to_value(msg).unwrap()))
                .collect();
            
            let batch = Message::Array(batch_items);
            
            let batch_message = HashMap::from([
                ("batch".to_string(), batch)
            ]);
            
            let _ = self.output.send_async(batch_message).await;
        }
    }
}
}

Zero-Copy Optimization

#![allow(unused)]
fn main() {
use bytes::Bytes;

// Use Bytes for zero-copy binary data
let data = Bytes::from(vec![1, 2, 3, 4]);
let message = Message::Binary(data.to_vec());

// Reference counting for large objects
use std::sync::Arc;

struct LargeData {
    content: Vec<u8>,
}

let large_data = Arc::new(LargeData { content: vec![0; 1000000] });
// Pass Arc around instead of cloning large data
}

Message Validation

Schema Validation

#![allow(unused)]
fn main() {
use serde_json::Value;

pub struct MessageValidator {
    schemas: HashMap<String, Value>, // JSON Schema
}

impl MessageValidator {
    pub fn validate_message(
        &self, 
        message_type: &str, 
        message: &HashMap<String, Message>
    ) -> Result<(), ValidationError> {
        if let Some(schema) = self.schemas.get(message_type) {
            let json_value: Value = message.clone().into();
            validate_json_schema(&json_value, schema)?;
        }
        Ok(())
    }
}
}

Type Safety

#![allow(unused)]
fn main() {
// Type-safe message builders
pub struct UserEventBuilder {
    user_id: Option<String>,
    event_type: Option<String>,
    timestamp: Option<String>,
}

impl UserEventBuilder {
    pub fn user_id(mut self, id: String) -> Self {
        self.user_id = Some(id);
        self
    }
    
    pub fn event_type(mut self, event_type: String) -> Self {
        self.event_type = Some(event_type);
        self
    }
    
    pub fn build(self) -> Result<HashMap<String, Message>, BuildError> {
        let user_id = self.user_id.ok_or(BuildError::MissingUserId)?;
        let event_type = self.event_type.ok_or(BuildError::MissingEventType)?;
        
        Ok(HashMap::from([
            ("user_id".to_string(), Message::string(user_id)),
            ("event_type".to_string(), Message::string(event_type)),
            ("timestamp".to_string(), Message::string(Utc::now().to_rfc3339())),
        ]))
    }
}
}

Testing Message Passing

Mock Channels

#![allow(unused)]
fn main() {
pub struct MockChannel {
    sent_messages: Arc<Mutex<Vec<HashMap<String, Message>>>>,
    responses: Arc<Mutex<VecDeque<HashMap<String, Message>>>>,
}

impl MockChannel {
    pub fn new() -> Self {
        Self {
            sent_messages: Arc::new(Mutex::new(Vec::new())),
            responses: Arc::new(Mutex::new(VecDeque::new())),
        }
    }
    
    pub fn expect_message(&self, message: HashMap<String, Message>) {
        self.responses.lock().push_back(message);
    }
    
    pub fn verify_sent(&self, expected: &HashMap<String, Message>) -> bool {
        self.sent_messages.lock().contains(expected)
    }
}
}

Integration Testing

#![allow(unused)]
fn main() {
#[tokio::test]
async fn test_message_pipeline() {
    let (tx1, rx1) = flume::unbounded();
    let (tx2, rx2) = flume::unbounded();
    
    // Create test actors
    let source = TestSourceActor::new(tx1);
    let processor = TestProcessorActor::new(rx1, tx2);
    let sink = TestSinkActor::new(rx2);
    
    // Start actors
    tokio::spawn(source.run());
    tokio::spawn(processor.run());
    tokio::spawn(sink.run());
    
    // Test message flow
    let test_message = HashMap::from([
        ("data".to_string(), Message::string("test".to_string()))
    ]);
    
    source.send(test_message.clone()).await;
    
    // Verify message received
    let received = sink.receive_next().await;
    assert_eq!(received.get("data"), test_message.get("data"));
}
}

Best Practices

Message Design

1. Keep messages immutable - Never modify after creation
2. Use appropriate granularity - Not too fine, not too coarse
3. Include enough context - Messages should be self-contained
4. Design for evolution - Use versioned message formats

Performance

1. Batch when possible - Reduce overhead
2. Use appropriate data types - Binary for large data
3. Implement backpressure - Prevent resource exhaustion
4. Monitor message rates - Track performance metrics

Error Handling

1. Use structured errors - Include error codes and context
2. Implement dead letter queues - Don't lose failed messages
3. Design for retry - Make operations idempotent
4. Log message failures - Enable debugging

Next Steps

• Graph System - Workflow composition
• Multi-Language Support - Script integration
• Performance Considerations - Optimization
• Creating Actors - Practical implementation

Graph System Architecture

Reflow's graph system provides a comprehensive flow-based programming (FBP) foundation for building visual workflow editors, data processing pipelines, and complex computational graphs. The system supports real-time validation, automatic layout, performance analysis, and both native Rust and WebAssembly implementations.

Core Concepts

Graph Structure

A Reflow graph consists of:

• Nodes: Processing units that represent actors or components
• Connections: Data flow paths between node ports
• Ports: Input/output endpoints with typed interfaces
• Initial Information Packets (IIPs): Static data injected into the graph
• Groups: Logical collections of related nodes

#![allow(unused)]
fn main() {
use reflow_network::graph::{Graph, GraphNode, GraphConnection, GraphEdge, PortType};
use std::collections::HashMap;

// Create a new graph
let mut graph = Graph::new("MyWorkflow", false, None);

// Add nodes
graph.add_node("source", "DataSource", None);
graph.add_node("processor", "DataProcessor", None);
graph.add_node("sink", "DataSink", None);

// Connect nodes
graph.add_connection("source", "output", "processor", "input", None);
graph.add_connection("processor", "output", "sink", "input", None);
}

Port Type System

Reflow uses a sophisticated type system to ensure data compatibility between connected nodes:

#![allow(unused)]
fn main() {
#[derive(Debug, Clone, PartialEq)]
pub enum PortType {
    Any,                              // Accepts any data type
    Flow,                            // Control flow signals
    Event,                           // Event-driven data
    Boolean,                         // Boolean values
    Integer,                         // Integer numbers
    Float,                          // Floating-point numbers
    String,                         // Text data
    Object(String),                 // Structured objects with schema
    Array(Box<PortType>),          // Arrays of typed elements
    Stream,                        // Streaming data
    Encoded,                       // Binary encoded data
    Option(Box<PortType>),         // Optional values
}
}

Type Compatibility

The system automatically validates type compatibility when connections are made:

#![allow(unused)]
fn main() {
// These connections are valid
graph.add_connection("int_source", "out", "float_sink", "in", None); // Integer → Float
graph.add_connection("any_source", "out", "string_sink", "in", None); // Any → String
graph.add_connection("data", "out", "stream", "in", None);            // Any → Stream

// This would be invalid and rejected
// graph.add_connection("string_source", "out", "int_sink", "in", None); // String ↛ Integer
}

Graph Operations

Node Management

#![allow(unused)]
fn main() {
// Add node with metadata
let metadata = HashMap::from([
    ("x".to_string(), json!(100)),
    ("y".to_string(), json!(200)),
    ("description".to_string(), json!("Processes incoming data"))
]);
graph.add_node("processor", "DataProcessor", Some(metadata));

// Update node metadata
graph.set_node_metadata("processor", HashMap::from([
    ("color".to_string(), json!("#ff0000"))
]));

// Remove node (also removes all connections)
graph.remove_node("processor");
}

Connection Management

#![allow(unused)]
fn main() {
// Add connection with metadata
let conn_metadata = HashMap::from([
    ("weight".to_string(), json!(0.8)),
    ("priority".to_string(), json!("high"))
]);
graph.add_connection("source", "data", "sink", "input", Some(conn_metadata));

// Get connection details
if let Some(connection) = graph.get_connection("source", "data", "sink", "input") {
    println!("Connection: {:?}", connection);
}

// Remove specific connection
graph.remove_connection("source", "data", "sink", "input");
}

Initial Information Packets (IIPs)

IIPs allow you to inject static data into the graph at startup:

#![allow(unused)]
fn main() {
use serde_json::json;

// Add configuration data
graph.add_initial(
    json!({"database_url": "postgresql://localhost/mydb"}),
    "database_connector",
    "config",
    None
);

// Add initial data with index for array ports
graph.add_initial_index(
    json!("input_file.txt"),
    "file_reader",
    "filenames",
    0,
    None
);
}

Graph Ports

Expose internal node ports as graph-level interfaces:

#![allow(unused)]
fn main() {
// Add input port to graph
graph.add_inport(
    "data_input",           // External port name
    "processor",            // Internal node
    "input",               // Internal port
    PortType::Any,         // Port type
    None                   // Metadata
);

// Add output port to graph
graph.add_outport(
    "processed_data",      // External port name
    "processor",           // Internal node
    "output",              // Internal port
    PortType::Object("ProcessedData".to_string()),
    None
);
}

Graph Validation

Automatic Validation

The graph system performs continuous validation:

#![allow(unused)]
fn main() {
// Validate entire graph
let validation_result = graph.validate_flow()?;

if !validation_result.cycles.is_empty() {
    println!("Cycles detected: {:?}", validation_result.cycles);
}

if !validation_result.orphaned_nodes.is_empty() {
    println!("Orphaned nodes: {:?}", validation_result.orphaned_nodes);
}

for mismatch in validation_result.port_mismatches {
    println!("Port mismatch: {}", mismatch);
}
}

Cycle Detection

Advanced cycle detection with path tracking:

#![allow(unused)]
fn main() {
// Detect first cycle
if let Some(cycle) = graph.detect_cycles() {
    println!("Cycle found: {:?}", cycle);
}

// Comprehensive cycle analysis
let cycle_analysis = graph.analyze_cycles();
println!("Total cycles: {}", cycle_analysis.total_cycles);
println!("Nodes in cycles: {:?}", cycle_analysis.nodes_in_cycles);
}

Performance Analysis

Parallelism Detection

Identify opportunities for parallel execution:

#![allow(unused)]
fn main() {
let parallelism = graph.analyze_parallelism();

// Parallel branches that can execute simultaneously
for branch in parallelism.parallel_branches {
    println!("Parallel branch: {:?}", branch.nodes);
}

// Pipeline stages for sequential execution
for stage in parallelism.pipeline_stages {
    println!("Stage {}: {:?}", stage.level, stage.nodes);
}
}

Bottleneck Analysis

Find performance bottlenecks:

#![allow(unused)]
fn main() {
let bottlenecks = graph.detect_bottlenecks();

for bottleneck in bottlenecks {
    match bottleneck {
        Bottleneck::HighDegree(node) => {
            println!("High-degree bottleneck at node: {}", node);
        }
        Bottleneck::SequentialChain(chain) => {
            println!("Sequential chain that could be parallelized: {:?}", chain);
        }
    }
}
}

Resource Analysis

Estimate execution requirements:

#![allow(unused)]
fn main() {
let analysis = graph.analyze_for_runtime();

println!("Estimated execution time: {:.2}s", analysis.estimated_execution_time);
println!("Resource requirements: {:?}", analysis.resource_requirements);

for suggestion in analysis.optimization_suggestions {
    match suggestion {
        OptimizationSuggestion::ParallelizableChain { nodes } => {
            println!("Consider parallelizing: {:?}", nodes);
        }
        OptimizationSuggestion::RedundantNode { node, reason } => {
            println!("Redundant node {}: {}", node, reason);
        }
        OptimizationSuggestion::ResourceBottleneck { resource, severity } => {
            println!("Resource bottleneck in {}: {:.1}%", resource, severity * 100.0);
        }
        OptimizationSuggestion::DataTypeOptimization { from, to, suggestion } => {
            println!("Optimize {} → {}: {}", from, to, suggestion);
        }
    }
}
}

Graph Layout

Automatic Layout

The system provides intelligent automatic layout:

#![allow(unused)]
fn main() {
// Calculate optimal positions
let positions = graph.calculate_layout();

for (node_id, position) in positions {
    println!("Node {}: x={:.1}, y={:.1}", node_id, position.x, position.y);
}

// Apply layout to graph metadata
graph.auto_layout()?;
}

Manual Positioning

Set custom node positions:

#![allow(unused)]
fn main() {
// Set specific position
graph.set_node_position("processor", 150.0, 100.0)?;

// Set position with custom dimensions and anchor
let metadata = HashMap::from([
    ("position".to_string(), json!({"x": 200, "y": 150})),
    ("dimensions".to_string(), json!({
        "width": 120,
        "height": 80,
        "anchor": {"x": 0.5, "y": 0.5}  // Center anchor
    }))
]);
graph.set_node_metadata("custom_node", metadata);
}

Event System

Real-time Updates

Subscribe to graph changes:

#![allow(unused)]
fn main() {
use reflow_network::graph::GraphEvents;

// Graph creates event channel automatically
let (sender, receiver) = graph.event_channel;

// Listen for events
while let Ok(event) = receiver.recv() {
    match event {
        GraphEvents::AddNode(node_data) => {
            println!("Node added: {:?}", node_data);
        }
        GraphEvents::AddConnection(conn_data) => {
            println!("Connection added: {:?}", conn_data);
        }
        GraphEvents::RemoveNode(node_data) => {
            println!("Node removed: {:?}", node_data);
        }
        // ... handle other events
        _ => {}
    }
}
}

Event Types

Complete list of graph events:

• AddNode / RemoveNode / RenameNode / ChangeNode
• AddConnection / RemoveConnection / ChangeConnection
• AddInitial / RemoveInitial
• AddGroup / RemoveGroup / RenameGroup / ChangeGroup
• AddInport / RemoveInport / RenameInport / ChangeInport
• AddOutport / RemoveOutport / RenameOutport / ChangeOutport
• ChangeProperties
• StartTransaction / EndTransaction / Transaction

Serialization

Export Format

Graphs can be serialized to JSON for storage and interchange:

#![allow(unused)]
fn main() {
// Export to JSON-compatible format
let export = graph.export();
let json_string = serde_json::to_string_pretty(&export)?;

// Load from JSON
let loaded_graph = Graph::load(export, Some(metadata));
}

Export Structure

{
  "caseSensitive": false,
  "properties": {
    "name": "MyWorkflow",
    "description": "A sample workflow"
  },
  "processes": {
    "source": {
      "id": "source",
      "component": "DataSource",
      "metadata": {"x": 0, "y": 0}
    }
  },
  "connections": [
    {
      "from": {"nodeId": "source", "portId": "output"},
      "to": {"nodeId": "sink", "portId": "input"},
      "metadata": {}
    }
  ],
  "inports": {},
  "outports": {},
  "groups": []
}

WebAssembly Support

Browser Integration

The graph system compiles to WebAssembly for browser usage:

import { Graph } from 'reflow-network';

// Create graph in browser
const graph = new Graph("WebWorkflow", false, {});

// Add nodes and connections
graph.addNode("input", "InputNode", {x: 0, y: 0});
graph.addNode("output", "OutputNode", {x: 200, y: 0});
graph.addConnection("input", "out", "output", "in", {});

// Subscribe to events
graph.subscribe((event) => {
    console.log("Graph event:", event);
});

// Export for persistence
const exported = graph.toJSON();
localStorage.setItem('workflow', JSON.stringify(exported));

TypeScript Support

Full TypeScript definitions are generated:

interface GraphNode {
    id: string;
    component: string;
    metadata?: Map<string, any>;
}

interface GraphConnection {
    from: GraphEdge;
    to: GraphEdge;
    metadata?: Map<string, any>;
    data?: any;
}

type PortType = 
  | { type: "flow" }
  | { type: "event" }
  | { type: "boolean" }
  | { type: "integer" }
  | { type: "float" }
  | { type: "string" }
  | { type: "object", value: string }
  | { type: "array", value: PortType }
  | { type: "stream" }
  | { type: "encoded" }
  | { type: "any" }
  | { type: "option", value: PortType };

Graph History

Undo/Redo System

Track changes for undo/redo functionality:

#![allow(unused)]
fn main() {
// Create graph with history tracking
let (mut graph, mut history) = Graph::with_history();

// Make changes
graph.add_node("test", "TestNode", None);
graph.add_connection("test", "out", "sink", "in", None);

// Undo last change
if let Some(event) = history.undo() {
    // Apply inverse operation
    history.apply_inverse(&mut graph, event)?;
}

// Redo change
if let Some(event) = history.redo() {
    // Reapply operation
    history.apply_event(&mut graph, event)?;
}
}

Advanced Features

Subgraph Analysis

Extract and analyze subgraphs:

#![allow(unused)]
fn main() {
// Get reachable subgraph from a node
if let Some(subgraph) = graph.get_reachable_subgraph("start_node") {
    let analysis = graph.analyze_subgraph(&subgraph);

    println!("Subgraph nodes: {}", analysis.node_count);
    println!("Max depth: {}", analysis.max_depth);
    println!("Is cyclic: {}", analysis.is_cyclic);
    println!("Branching factor: {:.2}", analysis.branching_factor);
}
}

Graph Traversal

Efficient traversal algorithms:

#![allow(unused)]
fn main() {
// Depth-first traversal
graph.traverse_depth_first("start_node", |node| {
    println!("Visiting node: {}", node.id);
})?;

// Breadth-first traversal
graph.traverse_breadth_first("start_node", |node| {
    println!("Processing: {} ({})", node.id, node.component);
})?;
}

Node Groups

Organize nodes into logical groups:

#![allow(unused)]
fn main() {
// Create group
graph.add_group("data_processing", vec!["filter".to_string(), "transform".to_string()], None);

// Add node to existing group
graph.add_to_group("data_processing", "validator");

// Remove from group
graph.remove_from_group("data_processing", "validator");
}

Best Practices

Performance Optimization

1. Use indexed operations: The graph uses internal indices for O(1) lookups
2. Batch modifications: Group related changes to minimize event overhead
3. Validate incrementally: Use targeted validation for better performance
4. Cache analysis results: Store expensive analysis results when graph is stable

Memory Management

1. Clean up connections: Always remove connections before removing nodes
2. Limit history size: Use with_history_and_limit() for bounded memory usage
3. Dispose of event listeners: Unsubscribe from events when no longer needed

Error Handling

1. Check return values: Most operations return Result types
2. Validate before execution: Use validation methods before running workflows
3. Handle cycles gracefully: Implement cycle detection in your workflow runtime
4. Monitor resource usage: Track memory and CPU usage for large graphs

Integration Examples

Visual Editor Integration

#![allow(unused)]
fn main() {
// In a visual editor, sync UI with graph events
graph.subscribe(|event| {
    match event {
        GraphEvents::AddNode(data) => ui.add_node_widget(data),
        GraphEvents::RemoveNode(data) => ui.remove_node_widget(data.id),
        GraphEvents::AddConnection(data) => ui.draw_connection(data),
        _ => {}
    }
});
}

Workflow Execution

#![allow(unused)]
fn main() {
// Convert graph to executable network
let network = Network::from_graph(&graph)?;

// Execute with runtime
let runtime = Runtime::new();
runtime.execute(network).await?;
}

Next Steps

• Creating Graphs - Detailed API guide
• Graph Analysis - Validation and performance analysis
• Layout System - Positioning and visualization
• SubgraphActor - Hierarchical graph composition via reflow_network
• Multi-Graph Composition - Workspace discovery and graph composition
• Distributed Networks - Cross-network execution and PeerMesh
• Building Visual Editors - Complete tutorial

Distributed Networks

Reflow's distributed network system enables bi-directional communication between separate Reflow instances, allowing you to build scalable, multi-node workflows while maintaining the familiar actor-based programming model.

Overview

The distributed network architecture extends Reflow's local actor model to support remote communication across network boundaries. This enables:

• Cross-Network Actor Communication: Actors in one Reflow instance can send messages to actors in remote instances
• Network-Transparent Operation: Remote actors appear as local actors in your workflows
• Bi-directional Message Flow: Full duplex communication between distributed nodes
• Automatic Discovery: Networks can discover and register with each other automatically
• Conflict Resolution: Smart handling of actor name conflicts across networks

Architecture Components

┌─────────────────────────────────────────────────────────────────────┐
│                    Distributed Reflow Network                      │
├─────────────────────────────────────────────────────────────────────┤
│  Instance A (Server)           │  Instance B (Client)               │
│ ┌─────────────────────────────┐ │ ┌─────────────────────────────────┐ │
│ │ Local Network               │ │ │ Local Network                   │ │
│ │ ├─ Actor A1 ─┐              │ │ │ ├─ Actor B1 ─┐                  │ │
│ │ ├─ Actor A2 ─┤              │ │ │ ├─ Actor B2 ─┤                  │ │
│ │ └─ Actor A3 ─┘              │ │ │ └─ Actor B3 ─┘                  │ │
│ └─────────────────────────────┘ │ └─────────────────────────────────┘ │
│            │                    │                    │                │
│ ┌─────────────────────────────┐ │ ┌─────────────────────────────────┐ │
│ │ Network Bridge              │◄─┤ │ Network Bridge                  │ │
│ │ ├─ Discovery Service        │ │ │ ├─ Discovery Service             │ │
│ │ ├─ Message Router           │ │ │ ├─ Message Router                │ │
│ │ ├─ Connection Manager       │ │ │ ├─ Connection Manager            │ │
│ │ └─ Remote Actor Proxy       │ │ │ └─ Remote Actor Proxy            │ │
│ └─────────────────────────────┘ │ └─────────────────────────────────┘ │
│            │                    │                    │                │
│ ┌─────────────────────────────┐ │ ┌─────────────────────────────────┐ │
│ │ Transport Layer             │◄─┤ │ Transport Layer                 │ │
│ │ ├─ WebSocket/TCP Server     │ │ │ ├─ WebSocket/TCP Client          │ │
│ │ ├─ Protocol Handler         │ │ │ ├─ Protocol Handler              │ │
│ │ └─ Serialization            │ │ │ └─ Serialization                 │ │
│ └─────────────────────────────┘ │ └─────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────┘

Core Components

1. DistributedNetwork: Main orchestrator that combines local networks with distributed communication
2. NetworkBridge: Handles all cross-network communication and actor registration
3. DiscoveryService: Automatic network discovery and registration
4. MessageRouter: Routes messages between local and remote actors
5. RemoteActorProxy: Local representatives of remote actors
6. TransportLayer: WebSocket/TCP communication infrastructure

Basic Setup

Creating a Distributed Network

#![allow(unused)]
fn main() {
use reflow_network::distributed_network::{DistributedNetwork, DistributedConfig};
use reflow_network::network::NetworkConfig;

// Configure the distributed network
let config = DistributedConfig {
    network_id: "main_workflow_engine".to_string(),
    instance_id: "server_001".to_string(),
    bind_address: "0.0.0.0".to_string(),
    bind_port: 8080,
    discovery_endpoints: vec![
        "http://discovery.example.com:3000".to_string()
    ],
    auth_token: Some("secure_token".to_string()),
    max_connections: 100,
    heartbeat_interval_ms: 30000,
    local_network_config: NetworkConfig::default(),
};

// Create and start the distributed network
let mut distributed_network = DistributedNetwork::new(config).await?;
distributed_network.start().await?;
}

Registering Local Actors

#![allow(unused)]
fn main() {
use your_actors::DataProcessorActor;

// Register actors that will be available to remote networks
distributed_network.register_local_actor(
    "data_processor",
    DataProcessorActor::new(),
    Some(HashMap::from([
        ("capability".to_string(), serde_json::Value::String("data_processing".to_string())),
        ("version".to_string(), serde_json::Value::String("1.0.0".to_string())),
    ]))
)?;
}

Connecting to Remote Networks

#![allow(unused)]
fn main() {
// Connect to another network
distributed_network.connect_to_network("192.168.1.100:8080").await?;

// Register a remote actor for local use
distributed_network.register_remote_actor(
    "remote_validator",      // Remote actor ID
    "validation_network"     // Remote network ID
).await?;
}

Actor Communication Patterns

Direct Remote Messaging

#![allow(unused)]
fn main() {
use reflow_network::message::Message;

// Send message to remote actor
distributed_network.send_to_remote_actor(
    "validation_network",    // Target network
    "remote_validator",      // Target actor
    "input",                 // Target port
    Message::String("validate this data".to_string().into())
).await?;
}

Workflow Integration

Remote actors integrate seamlessly into local workflows:

#![allow(unused)]
fn main() {
// Get local network handle
let local_network = distributed_network.get_local_network();
let mut network = local_network.write();

// Add local actor
network.add_node("local_collector", "data_collector", None)?;

// Add remote actor (appears as local)
network.add_node("remote_processor", "remote_validator@validation_network", None)?;

// Connect them in a workflow
network.add_connection(Connector {
    from: ConnectionPoint {
        actor: "local_collector".to_string(),
        port: "output".to_string(),
        ..Default::default()
    },
    to: ConnectionPoint {
        actor: "remote_processor".to_string(),
        port: "input".to_string(),
        ..Default::default()
    },
})?;
}

Network Discovery

Automatic Discovery

The discovery service can automatically find and register remote networks:

#![allow(unused)]
fn main() {
// Enable automatic discovery
let config = DistributedConfig {
    // ... other config
    discovery_endpoints: vec![
        "http://service-discovery.local:3000".to_string(),
        "http://backup-discovery.local:3000".to_string(),
    ],
    // ...
};
}

Manual Network Registration

#![allow(unused)]
fn main() {
// Manually connect to specific networks
let networks_to_connect = vec![
    "analytics.company.com:8080",
    "ml-pipeline.company.com:8080",
    "data-warehouse.company.com:8080",
];

for endpoint in networks_to_connect {
    match distributed_network.connect_to_network(endpoint).await {
        Ok(_) => println!("Connected to {}", endpoint),
        Err(e) => eprintln!("Failed to connect to {}: {}", endpoint, e),
    }
}
}

Conflict Resolution

When multiple networks have actors with the same name, Reflow provides several resolution strategies:

Automatic Aliasing

#![allow(unused)]
fn main() {
// Register remote actor with automatic conflict resolution
let alias = distributed_network.register_remote_actor_with_strategy(
    "data_processor",                    // Remote actor name (conflicts with local)
    "analytics_network",                 // Remote network
    ConflictResolutionStrategy::AutoAlias // Strategy
).await?;

println!("Remote actor available as: {}", alias);
// Output: "Remote actor available as: analytics_network_data_processor"
}

Manual Aliasing

#![allow(unused)]
fn main() {
// Provide custom aliases for clarity
distributed_network.register_remote_actor_with_strategy(
    "validator",
    "security_network",
    ConflictResolutionStrategy::ManualAlias("security_validator".to_string())
).await?;
}

Security Considerations

Authentication

#![allow(unused)]
fn main() {
let config = DistributedConfig {
    // Use authentication tokens
    auth_token: Some("your_secure_token_here".to_string()),
    // ... other config
};
}

Network Isolation

#![allow(unused)]
fn main() {
// Restrict which networks can connect
let config = DistributedConfig {
    // Only allow specific discovery endpoints
    discovery_endpoints: vec![
        "https://trusted-discovery.company.com:3000".to_string()
    ],
    max_connections: 10, // Limit concurrent connections
    // ... other config
};
}

Monitoring and Health Checks

Connection Status

#![allow(unused)]
fn main() {
// Check network health
let bridge_status = distributed_network.get_bridge_status().await?;
println!("Connected networks: {}", bridge_status.connected_networks.len());

for (network_id, status) in &bridge_status.connected_networks {
    println!("  {}: {:?}", network_id, status);
}
}

Heartbeat Monitoring

#![allow(unused)]
fn main() {
let config = DistributedConfig {
    heartbeat_interval_ms: 15000, // 15 second heartbeats
    // ... other config
};
}

Error Handling

Connection Failures

#![allow(unused)]
fn main() {
use reflow_network::distributed_network::DistributedError;

match distributed_network.connect_to_network("unreachable:8080").await {
    Ok(_) => println!("Connected successfully"),
    Err(DistributedError::ConnectionTimeout) => {
        eprintln!("Connection timed out - network may be down");
    },
    Err(DistributedError::AuthenticationFailed) => {
        eprintln!("Authentication failed - check token");
    },
    Err(e) => eprintln!("Other error: {}", e),
}
}

Message Delivery Failures

#![allow(unused)]
fn main() {
// Messages automatically retry with backoff
match distributed_network.send_to_remote_actor(
    "target_network", "target_actor", "input", message
).await {
    Ok(_) => println!("Message sent successfully"),
    Err(e) => {
        eprintln!("Failed to send message: {}", e);
        // Message will be retried automatically
    }
}
}

Performance Considerations

Connection Pooling

#![allow(unused)]
fn main() {
let config = DistributedConfig {
    max_connections: 50, // Adjust based on load
    // ... other config
};
}

Message Batching

Messages are automatically batched for efficiency, but you can tune batching behavior:

#![allow(unused)]
fn main() {
// Large messages are automatically compressed
let large_data = Message::Object(/* large JSON object */);
distributed_network.send_to_remote_actor(
    "target_network", "target_actor", "bulk_input", large_data
).await?;
}

Best Practices

Network Design

1. Use Descriptive Network IDs: Choose meaningful names like analytics_cluster instead of network1
2. Plan for Conflicts: Use descriptive actor names to minimize naming conflicts
3. Group Related Services: Co-locate related actors in the same network for efficiency
4. Design for Failure: Always handle network partitions and connection failures gracefully

Actor Organization

#![allow(unused)]
fn main() {
// Good: Descriptive, specific names
distributed_network.register_local_actor("customer_data_validator", validator, None)?;
distributed_network.register_local_actor("payment_processor", processor, None)?;

// Avoid: Generic names likely to conflict
// distributed_network.register_local_actor("validator", validator, None)?;
// distributed_network.register_local_actor("processor", processor, None)?;
}

Resource Management

#![allow(unused)]
fn main() {
// Always clean up connections
struct DistributedWorkflow {
    network: DistributedNetwork,
}

impl Drop for DistributedWorkflow {
    fn drop(&mut self) {
        // Gracefully shutdown connections
        if let Err(e) = tokio::task::block_in_place(|| {
            tokio::runtime::Handle::current().block_on(self.network.shutdown())
        }) {
            eprintln!("Error during cleanup: {}", e);
        }
    }
}
}

Troubleshooting

Common Issues

1. Connection Refused: Check firewall settings and ensure target network is running
2. Authentication Failed: Verify auth tokens match between networks
3. Actor Not Found: Ensure remote actor is registered and network is connected
4. Message Timeouts: Check network latency and increase timeout values if needed

Debug Logging

Enable detailed logging for troubleshooting:

#![allow(unused)]
fn main() {
use tracing_subscriber;

// Enable debug logging
tracing_subscriber::fmt()
    .with_max_level(tracing::Level::DEBUG)
    .init();
}

Health Check Endpoint

Networks automatically expose health endpoints:

# Check network health
curl http://your-network:8080/health

# Get network status
curl http://your-network:8080/status

PeerMesh — Server-Side Distributed Orchestration

When running as a Reflow server node connected to Zeal, the PeerMesh manages peer-to-peer connections for distributed execution. It creates one DistributedNetwork per execution and responds to orchestration commands from Zeal.

Architecture

#![allow(unused)]
fn main() {
pub struct PeerMesh {
    networks: RwLock<HashMap<String, DistributedNetwork>>,
    node_id: String,
    bind_address: String,
    base_port: u16,
}
}

The PeerMesh:

• Creates a DistributedNetwork per execution, each binding on an incrementing port
• Responds to subgraph.assign commands from Zeal (via ZipSession) to take ownership of subgraph execution
• Responds to peer.connect commands to establish peer-to-peer links between nodes
• Tears down per-execution networks on completion

Integration with Zeal

When Zeal orchestrates a distributed workflow:

1. Zeal sends subgraph.assign to each Reflow node via the ZIP WebSocket
2. The PeerMesh creates a DistributedNetwork for the assigned execution
3. Zeal sends peer.connect to establish connections between nodes
4. The PeerMesh calls connect_peer() to link networks via WebSocket bridges
5. Remote actors are registered as RemoteActorProxy instances in the local network
6. On execution completion, teardown_execution() cleans up the distributed network

#![allow(unused)]
fn main() {
// PeerMesh responds to Zeal commands
peer_mesh.connect_peer(execution_id, peer_address).await?;
peer_mesh.register_remote_actor(execution_id, actor_id, network_id).await?;
peer_mesh.teardown_execution(execution_id).await;
}

Distributed Composition Planning

For workflows spanning multiple Reflow nodes, the DistributedComposition system plans execution across network boundaries:

#![allow(unused)]
fn main() {
pub struct DistributedGraphComposition {
    pub local_sources: Vec<GraphSource>,
    pub remote_sources: Vec<RemoteGraphConfig>,
    pub local_connections: Vec<CompositionConnection>,
    pub distributed_connections: Vec<DistributedConnection>,
    pub execution_targets: HashMap<String, String>,  // graph → network_id
}
}

The DistributedNamespaceResolver maps processes to their home networks using qualified names like {network_id}/{namespace}/{process}:

#![allow(unused)]
fn main() {
let mut resolver = DistributedNamespaceResolver::new();
resolver.register_local_graph("data_pipeline", &graph)?;
resolver.register_remote_graph("ml_pipeline", "ml_node_1", &remote_graph)?;

// Find edges that cross network boundaries
let cross_edges = resolver.find_cross_network_connections(&connections)?;
}

The planner produces a DistributedCompositionPlan with:

• local_composition — the graph to execute on this node
• proxy_actors — ProxyActorSpec entries for actors that proxy to remote networks
• cross_network_edges — connections requiring proxy bridges
• remote_executions — graphs delegated to other nodes

Next Steps

• Multi-Graph Composition - Workspace discovery and graph composition
• SubgraphActor - Hierarchical graph composition
• Graph System - Core graph operations
• Remote Actors - Detailed remote actor API
• Discovery & Registration - Network discovery details
• Zeal IDE Integration - ZIP session and orchestration commands
• Distributed Workflow Tutorial - Step-by-step example

SubgraphActor

The SubgraphActor wraps an entire inner Network as a single Actor, allowing a graph to be embedded inside another graph as a composable unit. This is the foundation of hierarchical workflow composition in reflow_network.

Architecture

graph TB
    subgraph "Parent Network"
        EXT_IN[External Inport] --> SA[SubgraphActor]
        SA --> EXT_OUT[External Outport]
    end

    subgraph "SubgraphActor (inner Network)"
        IN_A[Actor A] --> IN_B[Actor B]
        IN_B --> OB[OutportBridge]
    end

    EXT_IN -.->|inport_map| IN_A
    OB -.->|external_sender| EXT_OUT

The parent network treats the SubgraphActor as an opaque actor with external inports and outports. Internally, it manages a full Network with its own actors, connectors, and message routing.

SubgraphActor Struct

#![allow(unused)]
fn main() {
pub struct SubgraphActor {
    inner_network: Arc<Mutex<Network>>,
    inport_map: HashMap<String, (String, String)>,   // ext_port → (actor_id, port_name)
    outport_map: HashMap<String, (String, String)>,   // ext_port → (actor_id, port_name)
    inports: Port,
    outports: Port,
    load: Arc<ActorLoad>,
    shutdown_tx: Arc<tokio::sync::watch::Sender<bool>>,
    shutdown_rx: tokio::sync::watch::Receiver<bool>,
}
}

• inner_network — the wrapped Network containing all inner actors and connections
• inport_map / outport_map — maps external port names to (inner_actor_id, inner_port_name) tuples
• inports / outports — the external ports exposed to the parent network
• shutdown_tx / shutdown_rx — tokio watch channel for graceful termination

Constructing from GraphExport

The primary constructor from_graph_export() builds a SubgraphActor from a serialized graph:

1. Creates a new inner Network
2. Registers all actors from the actors map
3. Adds nodes and internal connections from the graph export
4. Inport mapping — maps each external inport to (inner_node_id, inner_port_name) for direct message injection
5. Outport bridging — for each external outport, creates an OutportBridge actor inside the inner network, connected to the source actor's port via a standard Connector

#![allow(unused)]
fn main() {
let actors: HashMap<String, Arc<dyn Actor>> = /* component instances */;
let subgraph = SubgraphActor::from_graph_export(graph_export, actors)?;
// subgraph implements Actor — add it to a parent Network as any other actor
}

OutportBridge

The OutportBridge is a lightweight internal actor that forwards messages from an inner actor's outport to the SubgraphActor's external outport channel:

#![allow(unused)]
fn main() {
struct OutportBridge {
    external_sender: flume::Sender<Packet>,  // → SubgraphActor outport
    external_port_name: String,
    inner_port_name: String,
    inports: Port,
    outports: Port,
    load: Arc<ActorLoad>,
}
}

Bridge actors are registered inside the inner network with generated IDs (__outport_bridge_{ext_port}) and connected to source actors via standard Connectors. The bridge's create_process() receives packets on its inport, extracts the message by inner_port_name, wraps it with external_port_name, and sends it via external_sender.

Boundary Mapping

Inbound flow — The create_process() loop receives packets on external inports, looks up the target (actor_id, port) in inport_map, and calls network.send_to_actor() to inject the message into the inner network.

Outbound flow — Inner actors send messages through normal Connectors to OutportBridge actors. Each bridge extracts the message and sends it via external_sender, making it appear on the SubgraphActor's external outport.

Actor Trait Implementation

The SubgraphActor implements Actor:

• get_behavior() — returns a no-op closure; all routing is handled in create_process()
• get_inports() / get_outports() — return the external port pairs
• create_process() — starts the inner network, then runs an inbound routing loop using tokio::select! with the shutdown signal
• shutdown() — signals the routing loop to stop and shuts down the inner network

Lifecycle

1. create_process() starts the inner network (initializing all inner actors, connectors, and OutportBridge actors) and begins the inbound routing loop
2. The inner network runs its actors concurrently
3. shutdown() sends true on the watch channel, breaking the routing loop, and calls network.shutdown()

Next Steps

• Multi-Graph Composition - Workspace discovery and distributed composition
• Distributed Networks - Cross-network execution with PeerMesh
• Actor Model - Actor trait and port system

Multi-Graph Composition

Reflow's multi-graph composition system enables automatic discovery and intelligent composition of multiple graph files into unified, executable workflows. This system transforms complex multi-graph projects into seamlessly integrated workflows through workspace discovery and intelligent stitching.

Overview

The multi-graph composition architecture provides:

• Automatic Workspace Discovery: Recursively finds all *.graph.json and *.graph.yaml files in your project
• Folder-Based Namespacing: Uses directory structure as natural namespaces for organization
• Smart Auto-Connections: Automatically detects compatible interfaces between graphs
• Dependency Resolution: Resolves inter-graph dependencies and execution ordering
• One-Command Composition: Single command transforms entire workspace into executable workflow
• Interface Analysis: Analyzes graph interfaces for compatibility and suggests connections

Architecture Components

┌─────────────────────────────────────────────────────────────────────┐
│                    Workspace Discovery System                      │
├─────────────────────────────────────────────────────────────────────┤
│ File Discovery Layer                                               │
│ ┌─────────────────────────────────────────────────────────────────┐ │
│ │ *.graph.json    │ *.graph.yaml    │ Pattern Matching │ Filters  │ │
│ │ (data_flow)     │ (ml_pipeline)   │ (glob patterns)  │ (exclude)│ │
│ │ - 3 processes   │ - 5 processes   │ - depth limits   │ - test/  │ │
│ └─────────────────────────────────────────────────────────────────┘ │
├─────────────────────────────────────────────────────────────────────┤
│ Namespace & Analysis Layer                                          │
│ ┌─────────────────────────────────────────────────────────────────┐ │
│ │ Namespace Mgr   │ Interface Analysis │ Dependency Res │ Auto-Connect │
│ │ • Folder-based  │ • Exposed ports    │ • Graph deps   │ • Port match │
│ │ • Conflict res  │ • Required ports   │ • Order deps   │ • Confidence │ │
│ │ • Custom rules  │ • Compatibility   │ • Validation   │ • Heuristics │ │
│ └─────────────────────────────────────────────────────────────────┘ │
├─────────────────────────────────────────────────────────────────────┤
│ Unified Network Instance                                            │
│ ┌─────────────────────────────────────────────────────────────────┐ │
│ │ data/           │ ml/            │ monitoring/    │ shared/      │ │
│ │ ├─ ingestion/   │ ├─ training/   │ ├─ metrics     │ ├─ logging  │ │
│ │ │  └─ collector │ │  └─ trainer   │ ├─ alerts     │ ├─ auth     │ │
│ │ └─ processing/  │ └─ inference/  │ └─ dashboard   │ └─ config   │ │
│ │    └─ transformer│   └─ predictor│                │              │ │
│ └─────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────┘

Core Components

1. WorkspaceDiscovery: Discovers and loads all graph files in a workspace
2. GraphLoader: Loads and validates individual graph files
3. GraphComposer: Orchestrates composition of multiple graphs
4. NamespaceManager: Manages namespaces and resolves conflicts
5. DependencyResolver: Analyzes and resolves graph dependencies
6. InterfaceAnalyzer: Detects compatible interfaces for auto-connections

Workspace Structure

Multi-graph workspaces organize graph files using directory structure as namespaces:

workspace/
├── data/
│   ├── ingestion/
│   │   ├── api_collector.graph.json      → namespace: data/ingestion
│   │   └── file_reader.graph.yaml        → namespace: data/ingestion
│   ├── processing/
│   │   ├── cleaner.graph.json            → namespace: data/processing
│   │   ├── transformer.graph.json        → namespace: data/processing
│   │   └── validator.graph.yaml          → namespace: data/processing
│   └── storage/
│       ├── database_writer.graph.json    → namespace: data/storage
│       └── cache_manager.graph.yaml      → namespace: data/storage
├── ml/
│   ├── training/
│   │   ├── model_trainer.graph.json      → namespace: ml/training
│   │   └── feature_engineer.graph.yaml   → namespace: ml/training
│   ├── inference/
│   │   ├── predictor.graph.json          → namespace: ml/inference
│   │   └── batch_scorer.graph.json       → namespace: ml/inference
│   └── evaluation/
│       └── model_evaluator.graph.yaml    → namespace: ml/evaluation
├── monitoring/
│   ├── metrics.graph.json                → namespace: monitoring
│   ├── alerts.graph.yaml                 → namespace: monitoring
│   └── dashboard.graph.json              → namespace: monitoring
└── shared/
    ├── logging.graph.yaml                 → namespace: shared
    ├── auth.graph.json                    → namespace: shared
    └── config.graph.json                  → namespace: shared

Basic Usage

Simple Workspace Discovery

#![allow(unused)]
fn main() {
use reflow_network::multi_graph::workspace::{WorkspaceDiscovery, WorkspaceConfig};

// Configure workspace discovery
let config = WorkspaceConfig {
    root_path: PathBuf::from("./my_workspace"),
    graph_patterns: vec![
        "**/*.graph.json".to_string(),
        "**/*.graph.yaml".to_string(),
    ],
    excluded_paths: vec![
        "**/node_modules/**".to_string(),
        "**/target/**".to_string(),
        "**/test/**".to_string(),
    ],
    max_depth: Some(8),
    namespace_strategy: NamespaceStrategy::FolderStructure,
    ..WorkspaceConfig::default()
};

// Discover all graphs in workspace
let discovery = WorkspaceDiscovery::new(config);
let workspace = discovery.discover_workspace().await?;

println!("🎉 Discovered {} graphs across {} namespaces", 
    workspace.graphs.len(), 
    workspace.namespaces.len()
);
}

Automatic Composition

#![allow(unused)]
fn main() {
use reflow_network::multi_graph::{GraphComposer, GraphComposition};

// Create composer and auto-compose workspace
let mut composer = GraphComposer::new();
let composition = GraphComposition::from_workspace(workspace)?;

// Compose into single executable graph
let unified_graph = composer.compose_graphs(composition).await?;

// The unified graph can now be executed as a single workflow
let mut network = Network::new(NetworkConfig::default());
let graph = Graph::load(unified_graph, None);
// Use the composed graph...
}

Graph Dependencies and Interfaces

Declaring Dependencies

Graphs can declare explicit dependencies on other graphs:

{
  "caseSensitive": false,
  "properties": {
    "name": "ml_trainer",
    "namespace": "ml/training",
    "version": "1.0.0"
  },
  "processes": {
    "feature_engineer": {
      "component": "FeatureEngineerActor",
      "metadata": {}
    },
    "model_trainer": {
      "component": "ModelTrainerActor",
      "metadata": {}
    }
  },
  "connections": [
    {
      "from": { "nodeId": "feature_engineer", "portId": "Output" },
      "to": { "nodeId": "model_trainer", "portId": "Input" }
    }
  ],
  "inports": {
    "training_data": {
      "nodeId": "feature_engineer",
      "portId": "Input"
    }
  },
  "outports": {
    "trained_model": {
      "nodeId": "model_trainer",
      "portId": "Output"
    }
  },
  
  "graphDependencies": [
    {
      "graphName": "data_transformer",
      "namespace": "data/processing",
      "versionConstraint": ">=1.0.0",
      "required": true,
      "description": "Requires clean data from transformer"
    }
  ],
  "externalConnections": [
    {
      "connectionId": "transformer_to_trainer",
      "targetGraph": "data_transformer",
      "targetNamespace": "data/processing",
      "fromProcess": "normalizer",
      "fromPort": "Output",
      "toProcess": "feature_engineer",
      "toPort": "Input",
      "description": "Use cleaned data for training"
    }
  ],
  "providedInterfaces": {
    "trained_model_output": {
      "interfaceId": "trained_model_output",
      "processName": "model_trainer",
      "portName": "Output",
      "dataType": "TrainedModel",
      "description": "Trained ML model"
    }
  },
  "requiredInterfaces": {
    "clean_data_input": {
      "interfaceId": "clean_data_input",
      "processName": "feature_engineer",
      "portName": "Input",
      "dataType": "CleanedDataRecord",
      "description": "Clean data from processing pipeline",
      "required": true
    }
  }
}

Interface-Based Connections

Graphs can connect via defined interfaces rather than direct process connections:

#![allow(unused)]
fn main() {
use reflow_network::multi_graph::GraphConnectionBuilder;

// Build connections between discovered graphs
let mut connection_builder = GraphConnectionBuilder::new(workspace);

// Connect using interfaces (recommended)
connection_builder
    .connect_interface(
        "data_transformer",     // Source graph
        "clean_data_output",    // Source interface
        "ml_trainer",           // Target graph
        "clean_data_input"      // Target interface
    )?
    .connect_interface(
        "ml_trainer",
        "trained_model_output",
        "ml_predictor",
        "model_input"
    )?;

let connections = connection_builder.build();
}

Namespace Management

Folder-Based Namespacing

By default, directory structure becomes namespace hierarchy:

#![allow(unused)]
fn main() {
// File: data/processing/transformer.graph.json
// Namespace: "data/processing"
// Qualified name: "data/processing/transformer"

// File: ml/training/trainer.graph.json  
// Namespace: "ml/training"
// Qualified name: "ml/training/trainer"
}

Custom Namespace Strategies

#![allow(unused)]
fn main() {
use reflow_network::multi_graph::NamespaceStrategy;

// Semantic-based namespacing
let config = WorkspaceConfig {
    namespace_strategy: NamespaceStrategy::custom(
        "semantic_based", 
        Some(serde_json::json!({
            "keywords": {
                "ml": ["model", "train", "predict", "feature"],
                "data": ["ingest", "collect", "process", "clean"],
                "monitoring": ["metric", "alert", "dashboard", "log"]
            }
        }))
    )?,
    // ... other config
};
}

Conflict Resolution

When graphs have conflicting names, the system provides several resolution strategies:

#![allow(unused)]
fn main() {
use reflow_network::multi_graph::NamespaceConflictPolicy;

let namespace_manager = GraphNamespaceManager::new(NamespaceConflictPolicy::AutoResolve);

// Automatic resolution generates unique names:
// "data_processor" -> "data_processor" (first)
// "data_processor" -> "data_processor_1" (second)
// "data_processor" -> "data_processor_2" (third)
}

Advanced Features

Workspace Configuration

# workspace.config.yaml
workspace:
  root_path: "./my_project"
  
  graph_patterns:
    - "**/*.graph.json"
    - "**/*.graph.yaml"
  
  excluded_paths:
    - "**/node_modules/**"
    - "**/target/**" 
    - "**/.git/**"
    - "**/test/**"
  
  max_depth: 8
  
  namespace_strategy:
    type: "folder_structure"
  
  auto_connect: true
  dependency_resolution: "automatic"

composer:
  enable_auto_connections: true
  connection_confidence_threshold: 0.75
  validate_before_compose: true
  output_path: "./workspace.composed.graph.json"

Auto-Connection Discovery

The system can automatically suggest connections between compatible graph interfaces:

#![allow(unused)]
fn main() {
use reflow_network::multi_graph::InterfaceAnalyzer;

let analyzer = InterfaceAnalyzer::new();
let suggestions = analyzer.analyze_workspace(&workspace).await?;

for suggestion in suggestions.auto_connections {
    println!("🔗 Suggested connection: {} -> {}",
        suggestion.from_interface, suggestion.to_interface);
    println!("   Confidence: {:.2}", suggestion.confidence);
    println!("   Reason: {}", suggestion.reasoning);
}
}

Dependency Resolution

#![allow(unused)]
fn main() {
use reflow_network::multi_graph::DependencyResolver;

let resolver = DependencyResolver::new();
let ordered_graphs = resolver.resolve_dependencies(&workspace.graphs)?;

println!("📊 Dependency Resolution Order:");
for (i, graph) in ordered_graphs.iter().enumerate() {
    println!("  {}. {}", i + 1, graph.get_name());
}
}

Programmatic API

Workspace Discovery API

#![allow(unused)]
fn main() {
// Programmatic workspace discovery
let mut discovery = WorkspaceDiscovery::new(config);

// Custom filtering
discovery.add_filter(|path: &Path| -> bool {
    // Only include graphs with "production" in the name
    path.to_string_lossy().contains("production")
});

// Custom namespace generation
discovery.set_namespace_generator(|path: &Path| -> String {
    // Custom logic for namespace generation
    if path.to_string_lossy().contains("critical") {
        format!("critical/{}", path.parent().unwrap().file_name().unwrap().to_string_lossy())
    } else {
        path.parent().unwrap().to_string_lossy().to_string()
    }
});

let workspace = discovery.discover_workspace().await?;
}

Dynamic Graph Loading

#![allow(unused)]
fn main() {
use reflow_network::multi_graph::GraphSource;

// Load graphs from different sources
let sources = vec![
    GraphSource::JsonFile("./graphs/processor.graph.json".to_string()),
    GraphSource::NetworkApi("http://config-server/graphs/ml_model".to_string()),
    GraphSource::JsonContent(json_string),
];

let loader = GraphLoader::new();
let graphs = loader.load_multiple_graphs(sources).await?;
}

Custom Graph Composition

#![allow(unused)]
fn main() {
// Custom composition logic
let composition = GraphComposition {
    sources: workspace.graph_sources(),
    connections: vec![
        CompositionConnection {
            from: CompositionEndpoint {
                process: "data/processing/cleaner".to_string(),
                port: "Output".to_string(),
                index: None,
            },
            to: CompositionEndpoint {
                process: "ml/training/trainer".to_string(),
                port: "Input".to_string(),
                index: None,
            },
            metadata: Some(HashMap::from([
                ("priority".to_string(), serde_json::Value::String("high".to_string())),
            ])),
        }
    ],
    shared_resources: vec![
        SharedResource {
            name: "logger".to_string(),
            component: "LoggerActor".to_string(),
            metadata: Some(HashMap::from([
                ("level".to_string(), serde_json::Value::String("info".to_string())),
            ])),
        }
    ],
    properties: HashMap::from([
        ("name".to_string(), serde_json::Value::String("workspace_composition".to_string())),
        ("version".to_string(), serde_json::Value::String("1.0.0".to_string())),
    ]),
    case_sensitive: Some(false),
    metadata: None,
};
}

Command Line Interface

Discovery Commands

# Discover all graphs in workspace
reflow workspace discover --path ./my_project

# Output discovery results
reflow workspace discover --path ./my_project --output workspace.json

# Analyze workspace structure and dependencies
reflow workspace analyze --path ./my_project --output analysis.json

# List discovered graphs and namespaces
reflow workspace list --path ./my_project --format table

Composition Commands

# Auto-compose with high confidence connections
reflow workspace compose \
    --path ./my_project \
    --auto-connect \
    --confidence-threshold 0.8 \
    --validate \
    --output workspace.composed.graph.json

# Use configuration file
reflow workspace compose --config workspace.config.yaml

# Validate workspace before composition
reflow workspace validate --path ./my_project

Export Commands

# Export enhanced graph schemas
reflow workspace export --path ./my_project --enhanced --output enhanced_graphs/

# Generate workspace documentation
reflow workspace docs --path ./my_project --output docs/

Best Practices

Graph Organization

1. Use Descriptive Namespaces: Organize graphs logically by function, not just technology
2. Define Clear Interfaces: Use provided/required interfaces for loose coupling
3. Minimize Dependencies: Reduce inter-graph dependencies for flexibility
4. Version Your Graphs: Include version information for dependency management

Directory Structure

my_project/
├── core/                    # Core business logic graphs
│   ├── user_management/
│   ├── order_processing/
│   └── payment_handling/
├── integrations/            # External system integrations
│   ├── crm_sync/
│   ├── analytics_export/
│   └── notification_service/
├── pipelines/              # Data processing pipelines
│   ├── etl/
│   ├── ml_training/
│   └── reporting/
└── utilities/              # Shared utility graphs
    ├── logging/
    ├── monitoring/
    └── configuration/

Interface Design

{
  "providedInterfaces": {
    "user_data": {
      "interfaceId": "user_data",
      "processName": "user_processor",
      "portName": "Output",
      "dataType": "UserRecord",
      "description": "Processed user data with validation",
      "required": false,
      "metadata": {
        "schema_version": "1.2.0",
        "format": "json",
        "compression": "none"
      }
    }
  },
  "requiredInterfaces": {
    "raw_user_input": {
      "interfaceId": "raw_user_input",
      "processName": "user_processor", 
      "portName": "Input",
      "dataType": "RawUserData",
      "description": "Raw user data for processing",
      "required": true,
      "metadata": {
        "max_size": "10MB",
        "format": "json"
      }
    }
  }
}

Error Handling

Discovery Errors

#![allow(unused)]
fn main() {
use reflow_network::multi_graph::DiscoveryError;

match discovery.discover_workspace().await {
    Ok(workspace) => {
        // Process workspace
    },
    Err(DiscoveryError::GlobError(e)) => {
        eprintln!("Pattern matching error: {}", e);
    },
    Err(DiscoveryError::LoadError(path, e)) => {
        eprintln!("Failed to load {}: {}", path.display(), e);
    },
    Err(DiscoveryError::ValidationError(e)) => {
        eprintln!("Graph validation failed: {}", e);
    },
    Err(e) => eprintln!("Discovery error: {}", e),
}
}

Composition Errors

#![allow(unused)]
fn main() {
use reflow_network::multi_graph::CompositionError;

match composer.compose_graphs(composition).await {
    Ok(graph) => {
        // Use composed graph
    },
    Err(CompositionError::DependencyError(e)) => {
        eprintln!("Dependency resolution failed: {}", e);
    },
    Err(CompositionError::NamespaceError(e)) => {
        eprintln!("Namespace conflict: {}", e);
    },
    Err(e) => eprintln!("Composition error: {}", e),
}
}

Performance Considerations

Large Workspaces

#![allow(unused)]
fn main() {
// Optimize for large workspaces
let config = WorkspaceConfig {
    max_depth: Some(6),  // Limit directory traversal depth
    excluded_paths: vec![
        "**/node_modules/**".to_string(),
        "**/target/**".to_string(),
        "**/.git/**".to_string(),
        "**/build/**".to_string(),
        "**/dist/**".to_string(),
    ],
    // ... other config
};
}

Parallel Loading

#![allow(unused)]
fn main() {
// Discovery automatically parallelizes graph loading
let workspace = discovery.discover_workspace().await?;
// Graphs are loaded concurrently for better performance
}

Caching

#![allow(unused)]
fn main() {
// Enable caching for repeated discoveries
let config = WorkspaceConfig {
    cache_discoveries: true,
    cache_ttl_seconds: Some(300), // 5 minutes
    // ... other config
};
}

Distributed Composition

When a multi-graph workspace spans multiple Reflow nodes, the DistributedGraphComposition system extends local composition with cross-network awareness.

DistributedGraphComposition

#![allow(unused)]
fn main() {
pub struct DistributedGraphComposition {
    pub local_sources: Vec<GraphSource>,
    pub remote_sources: Vec<RemoteGraphConfig>,
    pub local_connections: Vec<CompositionConnection>,
    pub distributed_connections: Vec<DistributedConnection>,
    pub properties: HashMap<String, Value>,
    pub execution_targets: HashMap<String, String>,  // graph → network_id
}
}

Remote sources describe graphs fetched from other networks:

#![allow(unused)]
fn main() {
pub struct RemoteGraphConfig {
    pub network_id: String,
    pub graph_name: String,
    pub execution_target: Option<String>,  // where the graph executes
}
}

Distributed Connections

Cross-network connections use DistributedEndpoints with an optional network_id (None = local):

#![allow(unused)]
fn main() {
pub struct DistributedConnection {
    pub from: DistributedEndpoint,
    pub to: DistributedEndpoint,
    pub metadata: Option<HashMap<String, Value>>,
}

pub struct DistributedEndpoint {
    pub network_id: Option<String>,  // None = local
    pub process: String,             // "namespace/process"
    pub port: String,
    pub index: Option<usize>,
}
}

Namespace Resolution

The DistributedNamespaceResolver maps every process to its home network using qualified names {network_id}/{namespace}/{process}:

#![allow(unused)]
fn main() {
let mut resolver = DistributedNamespaceResolver::new("local");
resolver.register_local_graph("data_pipeline", &graph)?;
resolver.register_remote_graph("gpu_cluster", "ml", &remote_graph)?;

// Detect connections that cross network boundaries
let cross_edges = resolver.find_cross_network_connections(&connections)?;
}

Each CrossNetworkEdge records from_network, to_network, and the port details, plus a proxy_actor_name() method that generates a name like "ml/trainer@gpu_cluster".

Composition Planning

plan_distributed_composition() produces a DistributedCompositionPlan:

#![allow(unused)]
fn main() {
pub struct DistributedCompositionPlan {
    pub local_composition: GraphComposition,
    pub proxy_actors: Vec<ProxyActorSpec>,
    pub cross_network_edges: Vec<CrossNetworkEdge>,
    pub remote_executions: HashMap<String, String>,
}
}

The planner:

1. Identifies cross-network edges from distributed connections
2. Creates ProxyActorSpec entries for each unique remote target
3. Rewrites local connections to route through proxy actors
4. Builds the local composition with proxy-aware wiring
5. Tracks which graphs are delegated to remote nodes

Each proxy spec describes a local stand-in for a remote actor:

#![allow(unused)]
fn main() {
pub struct ProxyActorSpec {
    pub proxy_name: String,         // e.g., "ml/trainer@gpu_cluster"
    pub remote_network_id: String,
    pub remote_actor_id: String,
}
}

At execution time, execute_distributed_plan() materializes proxy specs into RemoteActorProxy instances (30s forward timeout) that bridge messages through the NetworkBridge WebSocket layer.

Next Steps

• Workspace Discovery API - Detailed API documentation
• Distributed Networks - PeerMesh, NetworkBridge, and RemoteActorProxy
• Graph System - SubgraphActor and core graph operations
• Multi-Graph Workspace Tutorial - Step-by-step guide

dynASB — Lightweight Actor FaaS

Reflow integrates with dynASB as a lightweight Function-as-a-Service (FaaS) backend for remote script actor execution. Script actors are deployed as functions into dynASB microVMs and communicate with Reflow over WebSocket JSON-RPC.

Overview

sequenceDiagram
    participant R as Reflow Engine
    participant C as DynASBClient
    participant D as dynASB API
    participant VM as microVM
    participant WS as WebSocket

    R->>C: deploy(name, runtime, code)
    C->>D: POST /api/v1/functions
    D->>VM: Boot microVM
    C->>D: GET /api/v1/functions/{id} (poll)
    D-->>C: status: Ready

    R->>C: create_actor(func, metadata)
    C->>WS: Connect ws://{ws_url}/{function_id}
    C-->>R: WebSocketScriptActor

    R->>WS: JSON-RPC "process" call
    WS->>VM: Execute handler
    VM-->>WS: Result
    WS-->>R: Response

    R->>C: undeploy(function_id)
    C->>D: POST /api/v1/functions/{id}/undeploy
    D->>VM: Shutdown

Configuration

#![allow(unused)]
fn main() {
pub struct DynASBConfig {
    pub api_url: String,    // e.g., "http://localhost:8080"
    pub ws_url: String,     // e.g., "ws://localhost:8080/ws"
    pub redis_url: String,  // For actor state persistence
}
}

Deployment Lifecycle

1. Deploy

The DynASBClient deploys script code to dynASB via its REST API:

#![allow(unused)]
fn main() {
let client = DynASBClient::new(DynASBConfig {
    api_url: "http://localhost:8080".into(),
    ws_url: "ws://localhost:8080/ws".into(),
    redis_url: "redis://localhost:6379".into(),
});

let func = client.deploy(
    "my_transform",             // function name
    "javascript",               // runtime
    "export function handler(input) { ... }", // code
    "handler",                  // entry point
    None,                       // optional dependencies
    Some(30),                   // timeout_seconds
).await?;
}

This sends a POST /api/v1/functions request. dynASB boots a microVM for the function and returns a DynASBFunction handle:

#![allow(unused)]
fn main() {
pub struct DynASBFunction {
    pub function_id: String,
    pub name: String,
    pub runtime: String,
    pub status: DeploymentStatus,
    pub deployment_time_ms: u64,
    pub vm_id: Option<String>,
}
}

2. Health Check & Readiness

After deployment, the microVM needs time to boot. Poll until ready:

#![allow(unused)]
fn main() {
let status = client.wait_until_ready(
    &func.function_id,
    Duration::from_secs(60),   // timeout
    Duration::from_millis(500), // poll interval
).await?;
}

This calls GET /api/v1/functions/{id} in a loop, checking the DeploymentStatus:

Terminal states (Unhealthy, Stopped) cause wait_until_ready to return an error immediately.

3. Create Actor

Once ready, create a WebSocketScriptActor that communicates with the function over JSON-RPC 2.0:

#![allow(unused)]
fn main() {
let actor = client.create_actor(&func, script_metadata).await?;
// actor implements Actor — register it in the network
}

Under the hood this:

1. Constructs a WebSocket URL: {ws_url}/{function_id}
2. Creates a WebSocketRpcClient pointing to that URL
3. Wraps it in a WebSocketScriptActor with the script metadata and Redis URL for state persistence

The actor communicates with the microVM using JSON-RPC 2.0 "process" method calls — the same protocol used by all WebSocket script actors in Reflow.

4. Undeploy

When done, remove the function:

#![allow(unused)]
fn main() {
client.undeploy(&func.function_id).await?;
// or undeploy everything:
client.undeploy_all().await?;
}

This sends POST /api/v1/functions/{id}/undeploy to shut down the microVM.

5. Automatic Cleanup

The DynASBClient implements Drop — if any functions remain deployed when the client is dropped, a background task spawns to undeploy them all:

#![allow(unused)]
fn main() {
impl Drop for DynASBClient {
    fn drop(&mut self) {
        if !self.deployed.is_empty() {
            let ids: Vec<String> = self.deployed.keys().cloned().collect();
            let api_url = self.config.api_url.clone();
            let http = self.http.clone();
            tokio::spawn(async move {
                for id in ids {
                    let url = format!("{}/api/v1/functions/{}/undeploy", api_url, id);
                    let _ = http.post(&url).send().await;
                }
            });
        }
    }
}
}

Deployment Metadata

deployment_metadata() produces a metadata map for injection into GraphNode metadata, prefixed with dynasb.:

#![allow(unused)]
fn main() {
let meta = client.deployment_metadata(&func.function_id);
// Keys: dynasb.function_id, dynasb.name, dynasb.runtime,
//       dynasb.status, dynasb.vm_id, dynasb.deployment_time_ms
}

This allows the execution engine and observability pipeline to track which actors are running on dynASB microVMs.

Integration with Reflow

dynASB serves as the remote execution backend for script actors that need isolation or dedicated resources. The integration point is the WebSocketScriptActor, which is the same actor type used for all WebSocket-based script runtimes — dynASB simply provides the deployment and lifecycle management layer on top.

[Script Discovery] → [DynASBClient.deploy()] → [microVM boots]
                   → [DynASBClient.create_actor()] → [WebSocketScriptActor]
                   → [Network registers actor] → [JSON-RPC "process" calls]
                   → [DynASBClient.undeploy()] → [microVM shuts down]

Next Steps

• Distributed Networks - Cross-network execution with PeerMesh
• Architecture Overview - How dynASB fits into the system
• Standard Component Library - Native actor reference

AssetDB — Entity-Component-System Data Store

AssetDB is Reflow's persistent, queryable world state. It replaces in-memory actor state for anything that needs to be shared across systems, inspected by tools, or survive workflow restarts.

Design Principles

1. Entities are data, not actors. An entity's physics, camera, material — these are JSON components in the DB, not Rust structs.
2. Systems are DAG actors. The DAG wires which systems run, on which entities, in what order.
3. Any tool can query. Zeal editor, debug inspector, Python scripts, unit tests — all read/write the same DB. Not coupled to Reflow.
4. Explicit over magic. The DAG shows the flow. No hidden subscriptions (except opt-in :bind).

Entity-Component Model

An entity is a name prefix. A component is a type suffix. The ID convention is entity:component.

player:transform    → { "position": [0, 1, 0], "rotation": [0, 0, 0, 1] }
player:rigidbody    → { "bodyType": "dynamic", "mass": 80 }
player:collider     → { "shape": "capsule", "radius": 0.3, "height": 1.8 }
player:mesh         → <binary 24-byte stride>
player:material     → { "albedo": [0.8, 0.2, 0.1], "roughness": 0.5 }

sun:light           → { "type": "directional", "color": [1, 1, 0.9] }
main:camera         → { "mode": "thirdPerson", "target": "player", "fov": 60 }

API

Put / Get (entity-style)

#![allow(unused)]
fn main() {
let db = AssetDB::open("./game.db")?;    // FileBackend (native)
let db = AssetDB::in_memory()?;           // MemoryBackend (wasm/testing)

// Put binary data (mesh, texture)
db.put("snake:mesh", &mesh_bytes, json!({"stride": 24}))?;

// Put JSON data (transform, material, config)
db.put_json("player:transform", json!({"position": [0, 1, 0]}), json!({}))?;

// Get
let asset = db.get("snake:mesh")?;
let data: Vec<u8> = asset.data;
let meta: Value = asset.entry.metadata;

// Check existence
db.has("player:rigidbody");  // true/false
}

Component Access (ECS-style)

#![allow(unused)]
fn main() {
// Set component on entity
db.set_component_json("player", "transform", json!({...}), json!({}))?;

// Get component
let tf = db.get_component("player", "transform")?;

// List all components on an entity
db.components_of("player")?;           // → ["transform", "rigidbody", "mesh"]

// Find entities with specific components
db.entities_with(&["rigidbody", "transform"])?;  // → ["player", "enemy_1"]

// Entity snapshot (all components as JSON)
db.entity_snapshot("player")?;
// → { "transform": {...}, "rigidbody": {...}, "material": {...} }

// Spawn from template
db.spawn_from("crate_template", "crate_42")?;

// Destroy entity (removes all components)
db.destroy_entity("crate_42")?;
}

Tags

#![allow(unused)]
fn main() {
db.tag("sword:mesh", &["weapon", "melee"])?;

db.query_dsl(&json!({"tags": ["weapon"]}))?;              // has ANY tag
db.query_dsl(&json!({"tags": {"$all": ["weapon", "melee"]}}))?;  // has ALL tags
}

Query DSL

Queries describe the shape of what you're looking for. The config IS the query.

{ "type": "mesh", "tags": ["snake"], "$sort": "newest", "$limit": 5 }
{ "name": { "$contains": "body" }, "metadata.stride": 24 }
{ "type": { "$in": ["mesh", "texture"] }, "size": { "$gt": 10000 } }
{ "tags": { "$all": ["weapon", "melee"] } }

Operators: $gt, $gte, $lt, $lte, $in, $contains, $startsWith, $all, $between, $not, $exists

Control keys: $sort (newest/oldest/largest/smallest/name), $limit

Storage Backends

All backends implement the StorageBackend trait:

#![allow(unused)]
fn main() {
pub trait StorageBackend: Send + Sync {
    fn read_manifest(&self) -> Result<Vec<AssetEntry>>;
    fn write_manifest(&self, entries: &[AssetEntry]) -> Result<()>;
    fn read_blob(&self, hash: &str) -> Result<Vec<u8>>;
    fn write_blob(&self, hash: &str, data: &[u8]) -> Result<()>;
    fn blob_exists(&self, hash: &str) -> bool;
    fn delete_blob(&self, hash: &str) -> Result<()>;
}
}

Compression

Binary blobs are transparently LZ4-compressed (via lz4_flex, pure Rust, wasm-safe). Compression is selective:

• Mesh/animation blobs: compressed (~15-25% savings)
• JSON components: compressed (~60-80% savings)
• Textures/audio/video: stored raw (already compressed)
• Small blobs (<256 bytes): stored raw

get() always returns uncompressed data. Callers never see compression.

Content Addressing

Blobs are stored by content hash. Identical data → same hash → same blob. Re-importing the same asset costs zero additional storage. put() is an upsert — same entity ID overwrites, but if the content hasn't changed, no disk write occurs.

System Actors

Systems read components, process, write results back. The DAG determines execution order and which entities each system operates on.

Entity Selectors

Every system accepts explicit entity targeting:

{ "entity": "player" }                           // single entity
{ "entities": ["sun", "torch", "lamp"] }         // explicit list
{ "selector": { "tags": ["enemy"] } }            // query-based

The entity_id inport allows dynamic selection per-tick from the DAG.

Layout Sync — DOM / Layout Tree Integration

The layout sync system bridges AssetDB with the DOM (browser) or a native layout tree. The layout tree is the source of truth at initialization — AssetDB observes and drives it.

Flow

Startup:
  DOM / Layout Tree ──hydrate()──→ AssetDB

Each tick:
  Layout ──poll_events()──→ :triggers components
                               ↓
                        Systems process (behavior, tween, state machine)
                               ↓
  AssetDB ──sync()────────→ Layout (DOM updates)

Inline queries:
  @layout(entity:property) ──→ resolved directly from layout backend
                                (never writes to AssetDB)

LayoutBackend Trait

Pluggable backend for different platforms:

#![allow(unused)]
fn main() {
pub trait LayoutBackend: Send + Sync {
    fn hydrate(&self, db: &Arc<AssetDB>) -> Result<()>;
    fn sync(&self, db: &Arc<AssetDB>) -> Result<()>;
    fn poll_events(&self, db: &Arc<AssetDB>) -> Result<()>;
    fn query(&self, entity: &str, property: &str) -> Option<f64>;
    fn query_string(&self, entity: &str, property: &str) -> Option<String>;
    fn hit_test(&self, x: f64, y: f64) -> Option<String>;
    fn parent_of(&self, entity: &str) -> Option<String>;
    fn children_of(&self, entity: &str) -> Option<Vec<String>>;
}
}

Queryable Properties

@layout() Variables

Behavior rules and expressions can reference layout-computed values inline without writing to AssetDB:

{
  "entity": "hero",
  "component": "behavior",
  "data": {
    "rules": [{
      "name": "parallax",
      "target": "transform.position.y",
      "expr": "scroll * -200",
      "vars": {
        "scroll": "@layout(page:scrollProgress)"
      }
    }]
  }
}

The @layout(entity:property) prefix tells the variable resolver to query the layout backend directly. The value is resolved at evaluation time — ephemeral, never stored.

Two-Way Binding

Entities with a :bind component get automatic bi-directional sync. No explicit DAG wiring needed for the data flow itself.

Full Bind

{ "entity": "slider", "component": "bind", "data": true }

Syncs all standard properties: transform, style, value, scroll.

Selective Bind

{
  "entity": "input_field",
  "component": "bind",
  "data": {
    "value": true,
    "scroll": true,
    "transform": false,
    "style": false
  }
}

Only syncs the properties you opt into.

What Happens Each Tick

For bound entities, the LayoutSyncSystem automatically:

Traceability

All bind operations include metadata.source = "layout_pull" so you can trace them in Reflow's tracing system. The bound_changes outport emits every pull/push operation for the DAG inspector:

[
  {"entity": "slider", "direction": "pull", "property": "transform"},
  {"entity": "slider", "direction": "push"}
]

Bind vs Explicit Wiring

Both are visible in the DAG inspector. Bind is convenience — it doesn't bypass the system, it just saves you from wiring the obvious.

DOM Component Schemas

:dom — Element definition

{
  "tag": "button",
  "text": "Click me",
  "parent": "nav",
  "width": 120,
  "height": 40
}

:style — Visual properties

{
  "opacity": 1.0,
  "backgroundColor": "#007bff",
  "borderRadius": "8px",
  "transform": "scale(1.0)"
}

:transform — Position/rotation/scale

{
  "position": [100, 200, 0],
  "rotation": [0, 0, 0],
  "scale": [1, 1, 1]
}

:triggers — Events (consumed each tick)

["pointerEnter", "pointerDown", "scroll"]

Written by poll_events() or external input systems. Read and cleared by StateMachineSystem / BehaviorSystem.

:bind — Two-way sync toggle

true

or selective:

{ "transform": true, "value": true, "scroll": false }

Example: Animated Button

Set up an interactive button with hover/press states and tween animations, all driven by data:

#![allow(unused)]
fn main() {
let db = get_or_create_db("./app.db")?;

// Element
db.set_component_json("btn", "dom", json!({
    "tag": "button", "text": "Submit"
}), json!({}))?;

db.set_component_json("btn", "transform", json!({
    "position": [0, 0, 0], "scale": [1, 1, 1]
}), json!({}))?;

db.set_component_json("btn", "style", json!({
    "opacity": 1.0, "backgroundColor": "#007bff"
}), json!({}))?;

// State machine
db.put_json("btn:state_machine", json!({
    "current": "idle",
    "states": {
        "idle": { "onEnter": { "tween": "btn_scale_normal" } },
        "hover": { "onEnter": { "tween": "btn_scale_up" } },
        "pressed": { "onEnter": { "tween": "btn_scale_down" } }
    },
    "transitions": [
        { "from": "idle", "to": "hover", "trigger": "pointerEnter" },
        { "from": "hover", "to": "idle", "trigger": "pointerLeave" },
        { "from": "hover", "to": "pressed", "trigger": "pointerDown" },
        { "from": "pressed", "to": "hover", "trigger": "pointerUp" }
    ]
}), json!({}))?;

// Tweens
db.put_json("btn_scale_up:tween", json!({
    "target": "btn:transform.scale",
    "from": [1, 1, 1], "to": [1.05, 1.05, 1.05],
    "duration": 0.15, "easing": "easeOutCubic",
    "state": "paused"
}), json!({}))?;

db.put_json("btn_scale_down:tween", json!({
    "target": "btn:transform.scale",
    "from": [1.05, 1.05, 1.05], "to": [0.95, 0.95, 0.95],
    "duration": 0.1, "easing": "easeOutCubic",
    "state": "paused"
}), json!({}))?;

db.put_json("btn_scale_normal:tween", json!({
    "target": "btn:transform.scale",
    "from": [0.95, 0.95, 0.95], "to": [1, 1, 1],
    "duration": 0.2, "easing": "easeOutCubic",
    "state": "paused"
}), json!({}))?;

// Bind for auto sync
db.set_component_json("btn", "bind", json!(true), json!({}))?;
}

DAG:

IntervalTrigger(16ms) → LayoutSync(phase: "both", entity: "btn")
                      → StateMachineSystem(entity: "btn")
                      → TweenSystem(entity: "btn")

The button responds to hover/press with smooth scale animations. No Rust code for the interaction logic — it's all data in the AssetDB.

Observability Overview

Reflow provides a comprehensive observability framework that enables deep introspection into distributed actor networks. The observability system captures detailed execution traces, performance metrics, and data flow patterns across all components in your system.

Key Features

🔍 Comprehensive Event Tracing

• Actor Lifecycle: Track creation, startup, execution, completion, and failures
• Message Flow: Monitor all message passing between actors with detailed metadata
• Data Flow Tracing: NEW - Automatic tracing of data flow between connected actors
• State Changes: Capture state transitions with diff support for time-travel debugging
• Network Events: Monitor distributed network operations and health

📊 Real-time Monitoring

• Live Event Streaming: WebSocket-based real-time event notifications
• Performance Metrics: CPU usage, memory consumption, throughput measurements
• Custom Dashboards: Build monitoring interfaces using the WebSocket API
• Alerting: Set up custom alerts based on event patterns and thresholds

🗄️ Flexible Storage

• SQLite: Embedded database perfect for development and small deployments
• PostgreSQL: Production-ready backend with ACID guarantees and concurrent access
• Memory: High-performance in-memory storage for testing and temporary analysis

🌐 Distributed Tracing

• Cross-Network Visibility: Trace execution across multiple network instances
• Causality Tracking: Maintain event dependency chains across distributed components
• Span Integration: Compatible with OpenTelemetry and Jaeger for unified observability

Architecture Overview

graph TB
    subgraph "Client Applications"
        A1[Actor Network 1]
        A2[Actor Network 2] 
        A3[Actor Network N]
    end
    
    subgraph "Tracing Infrastructure"
        TC[TracingClient]
        WS[WebSocket Protocol]
        TS[Tracing Server]
    end
    
    subgraph "Storage Layer"
        SQLite[(SQLite)]
        Postgres[(PostgreSQL)]
        Memory[(Memory)]
    end
    
    subgraph "Analysis & Monitoring"
        RT[Real-time Dashboard]
        HQ[Historical Queries]
        AL[Alerting]
    end
    
    A1 -->|Events| TC
    A2 -->|Events| TC
    A3 -->|Events| TC
    TC -->|BatchedEvents| WS
    WS --> TS
    TS --> SQLite
    TS --> Postgres
    TS --> Memory
    TS -->|Live Events| RT
    TS -->|Query Results| HQ
    TS -->|Notifications| AL

Event Types

Core Actor Events

• ActorCreated: Actor instance creation with configuration
• ActorStarted: Actor begins execution
• ActorCompleted: Successful actor completion
• ActorFailed: Actor error with detailed error information

Communication Events

• MessageSent: Message transmission between actors
• MessageReceived: Message reception confirmation
• DataFlow: Automatic data flow tracing between connected actors
• PortConnected: Port connection establishment
• PortDisconnected: Port disconnection

System Events

• StateChanged: Actor state modifications with diffs
• NetworkEvent: Distributed network operations

Integration Patterns

Automatic Integration

The tracing framework integrates automatically with Reflow networks:

#![allow(unused)]
fn main() {
use reflow_network::{Network, NetworkConfig};
use reflow_network::tracing::TracingConfig;

// Enable tracing with minimal configuration
let tracing_config = TracingConfig {
    server_url: "ws://localhost:8080".to_string(),
    enabled: true,
    ..Default::default()
};

let network_config = NetworkConfig {
    tracing: tracing_config,
    ..Default::default()
};

let network = Network::new(network_config);
// All actor operations are now automatically traced!
}

Manual Event Recording

For custom events and detailed control:

#![allow(unused)]
fn main() {
use reflow_tracing_protocol::{TraceEvent, TracingIntegration};

// Record custom events
if let Some(tracing) = global_tracing() {
    tracing.trace_actor_created("custom_actor").await?;
    tracing.trace_data_flow(
        "source_actor", "output",
        "target_actor", "input",
        "CustomMessage", 1024
    ).await?;
}
}

Data Flow Tracing

The latest enhancement to the observability framework provides automatic data flow tracing:

Automatic Capture

• Zero Configuration: Works out-of-the-box with existing actor networks
• Connector Integration: Captures data flow at the connector level for accuracy
• Bidirectional Tracking: Traces both source and destination information
• Performance Metadata: Includes message size, type, and timing information

Rich Context

#![allow(unused)]
fn main() {
// Data flow events automatically include:
DataFlow {
    from_actor: "data_processor",
    from_port: "output",
    to_actor: "analytics_engine", 
    to_port: "input",
    message_type: "ProcessedData",
    size_bytes: 2048,
    timestamp: "2025-01-07T06:00:00Z",
    causality_chain: [...],
    performance_metrics: {...}
}
}

Use Cases

Development & Debugging

• Execution Visualization: See exactly how data flows through your system
• Performance Profiling: Identify bottlenecks and optimization opportunities
• Error Investigation: Trace error propagation through actor networks
• State Debugging: Time-travel debugging with state diffs

Production Monitoring

• Health Monitoring: Track system health and detect anomalies
• Performance Monitoring: Monitor throughput, latency, and resource usage
• Capacity Planning: Analyze usage patterns for scaling decisions
• Incident Response: Rapid diagnosis of production issues

Analytics & Optimization

• Usage Patterns: Understand how your system is actually used
• Performance Optimization: Data-driven optimization decisions
• Architecture Evolution: Make informed architectural changes
• Compliance: Maintain audit trails for regulatory requirements

Getting Started

1. Quick Start Guide - Get tracing running in 5 minutes
2. Architecture Deep Dive - Understand the technical details
3. Configuration Guide - Customize for your environment
4. Deployment Guide - Production deployment patterns

Next Steps

• Learn about event types and their uses
• Explore storage backend options
• Set up production monitoring
• Integrate with existing monitoring systems

Observability Quick Start

Get Reflow's observability framework running in under 5 minutes. This guide will walk you through setting up tracing for a simple actor network and viewing the results.

Prerequisites

• Rust 1.85 or later
• Basic familiarity with Reflow actors

Step 1: Start the Tracing Server

First, start the reflow_tracing server:

# From the project root
cd examples/tracing_integration
./scripts/start_server.sh

This starts the tracing server on ws://127.0.0.1:8080 with SQLite storage.

Step 2: Create a Simple Traced Network

Create a new Rust project or add to an existing one:

# Cargo.toml
[dependencies]
reflow_network = { path = "../../crates/reflow_network" }
reflow_actor = { path = "../../crates/reflow_actor" }
reflow_tracing_protocol = { path = "../../crates/reflow_tracing_protocol" }
tokio = { version = "1.0", features = ["full"] }
anyhow = "1.0"
tracing = "0.1"
tracing-subscriber = "0.3"

Create a simple actor network with tracing enabled:

// src/main.rs
use anyhow::Result;
use reflow_network::{Network, NetworkConfig};
use reflow_network::tracing::{TracingConfig, init_global_tracing};
use reflow_actor::{Actor, ActorBehavior, ActorContext, Port, ActorLoad, ActorConfig};
use std::collections::HashMap;
use std::sync::Arc;
use std::time::Duration;
use parking_lot::Mutex;

// Simple data processor actor
#[derive(Clone)]
struct DataProcessor {
    name: String,
}

impl DataProcessor {
    fn new(name: &str) -> Self {
        Self { name: name.to_string() }
    }
}

impl Actor for DataProcessor {
    fn get_behavior(&self) -> ActorBehavior {
        let name = self.name.clone();
        Box::new(move |context: ActorContext| {
            let actor_name = name.clone();
            Box::pin(async move {
                println!("🎬 {} processing messages", actor_name);
                
                let mut results = HashMap::new();
                for (port, message) in context.get_payload() {
                    println!("📨 {} received on {}: {:?}", actor_name, port, message);
                    
                    // Simulate processing
                    tokio::time::sleep(Duration::from_millis(100)).await;
                    
                    let output = reflow_actor::message::Message::string(
                        format!("Processed by {}: {:?}", actor_name, message)
                    );
                    results.insert("output".to_string(), output);
                }
                
                Ok(results)
            })
        })
    }

    fn get_inports(&self) -> Port { flume::unbounded() }
    fn get_outports(&self) -> Port { flume::unbounded() }
    fn load_count(&self) -> Arc<Mutex<ActorLoad>> { 
        Arc::new(Mutex::new(ActorLoad::new(0))) 
    }
    
    fn create_process(&self, _config: ActorConfig) -> std::pin::Pin<Box<dyn std::future::Future<Output = ()> + Send + 'static>> {
        Box::pin(async {})
    }
    
    fn shutdown(&self) {}
}

#[tokio::main]
async fn main() -> Result<()> {
    // Initialize logging
    tracing_subscriber::fmt::init();
    
    println!("🚀 Starting traced actor network");
    
    // Step 1: Configure tracing
    let tracing_config = TracingConfig {
        server_url: "ws://127.0.0.1:8080".to_string(),
        batch_size: 10,
        batch_timeout: Duration::from_millis(500),
        enable_compression: false,
        enabled: true,
        retry_config: reflow_network::tracing::RetryConfig {
            max_retries: 3,
            initial_delay: Duration::from_millis(100),
            max_delay: Duration::from_secs(5),
            backoff_multiplier: 2.0,
        },
    };
    
    // Step 2: Initialize global tracing
    init_global_tracing(tracing_config.clone())?;
    println!("✅ Global tracing initialized");
    
    // Step 3: Create network with tracing enabled
    let network_config = NetworkConfig {
        tracing: tracing_config,
        ..Default::default()
    };
    
    let mut network = Network::new(network_config);
    println!("✅ Network created with tracing");
    
    // Step 4: Register actors
    let processor1 = DataProcessor::new("processor1");
    let processor2 = DataProcessor::new("processor2");
    
    network.register_actor("processor", processor1)?;
    network.register_actor("formatter", processor2)?;
    
    // Step 5: Add nodes to network
    network.add_node("proc1", "processor", None)?;
    network.add_node("proc2", "formatter", None)?;
    
    // Step 6: Start the network (automatic tracing begins here)
    network.start()?;
    println!("✅ Network started - tracing active");
    
    // Step 7: Send some messages (these will be automatically traced)
    println!("📨 Sending test messages...");
    
    for i in 1..=3 {
        let message = reflow_actor::message::Message::string(
            format!("Test message {}", i)
        );
        network.send_to_actor("proc1", "input", message)?;
        tokio::time::sleep(Duration::from_millis(300)).await;
    }
    
    // Step 8: Execute actors directly for more detailed tracing
    let result = network.execute_actor(
        "proc2",
        HashMap::from([
            ("input".to_string(), reflow_actor::message::Message::string("Direct execution test".to_string()))
        ])
    ).await?;
    
    println!("✅ Direct execution result: {:?}", result);
    
    // Step 9: Let the system run to generate traces
    tokio::time::sleep(Duration::from_secs(2)).await;
    
    // Step 10: Manual tracing API demonstration
    if let Some(tracing) = reflow_network::tracing::global_tracing() {
        println!("🔍 Demonstrating manual tracing API...");
        
        tracing.trace_actor_created("manual_actor").await?;
        tracing.trace_message_sent(
            "manual_actor", 
            "output", 
            "ManualMessage", 
            256
        ).await?;
        
        println!("✅ Manual events recorded");
    }
    
    // Graceful shutdown
    println!("🛑 Shutting down...");
    network.shutdown();
    tokio::time::sleep(Duration::from_millis(500)).await;
    
    println!("🎉 Quick start complete! Check the tracing server for events.");
    println!("💡 Next: Run the monitoring client to see live events:");
    println!("   cargo run --bin monitoring_client");
    
    Ok(())
}

Step 3: Run Your Application

cargo run

You should see output like:

🚀 Starting traced actor network
✅ Global tracing initialized
✅ Network created with tracing
✅ Network started - tracing active
📨 Sending test messages...
🎬 processor1 processing messages
📨 processor1 received on input: String("Test message 1")
...
🎉 Quick start complete! Check the tracing server for events.

Step 4: View Live Events

In another terminal, run the monitoring client:

cd examples/tracing_integration
cargo run --bin monitoring_client

You'll see real-time trace events:

🔍 Monitoring live trace events...
📊 Connected to tracing server at ws://127.0.0.1:8080

[2025-01-07T06:00:00Z] ActorCreated: processor1
[2025-01-07T06:00:00Z] ActorCreated: processor2  
[2025-01-07T06:00:00Z] MessageSent: processor1 -> output (String, 256 bytes)
[2025-01-07T06:00:00Z] DataFlow: processor1:output -> processor2:input (String, 256 bytes)
[2025-01-07T06:00:01Z] ActorCompleted: processor1
...

What Just Happened?

Your simple actor network generated several types of trace events:

1. Actor Creation: When actors were instantiated
2. Message Sending: When messages were sent between actors
3. Data Flow: Automatic tracing of data flowing between connected actors
4. Actor Completion: When actors finished processing

All of this happened automatically - the tracing framework integrated seamlessly with your existing Reflow network.

Exploring the Data

SQLite Database

The trace data is stored in examples/tracing_integration/data/traces.db. You can explore it directly:

sqlite3 examples/tracing_integration/data/traces.db
.tables
SELECT * FROM trace_events LIMIT 5;

Query API

Use the monitoring client with query options:

# Get last 10 events
cargo run --bin monitoring_client -- --query --limit 10

# Filter by actor
cargo run --bin monitoring_client -- --actor-ids processor1

# Filter by event type
cargo run --bin monitoring_client -- --event-types ActorCreated,MessageSent

Next Steps

🎯 Learn More About Event Types

• Read about all event types and their uses
• Understand data flow tracing capabilities

⚙️ Customize Your Setup

• Configure different storage backends
• Explore configuration options

🚀 Production Deployment

• Set up production monitoring
• Learn about scaling and performance

🔧 Integration

• Build custom monitoring dashboards
• Integrate with existing monitoring systems

Troubleshooting

Connection Issues

If the client can't connect to the tracing server:

# Check if server is running
curl -I http://127.0.0.1:8080
# or
telnet 127.0.0.1 8080

No Events Appearing

• Ensure enabled: true in your TracingConfig
• Check that init_global_tracing() was called before network operations
• Verify the server URL is correct

Performance Impact

For production systems, consider:

• Increasing batch_size to reduce network overhead
• Enabling compression with enable_compression: true
• Using PostgreSQL backend for better concurrent performance

Get help in our troubleshooting guide or check the architecture documentation for deeper understanding.

Observability Architecture

Reflow's observability is built on an event pipeline that translates low-level network events into rich engine events and forwards them to Zeal IDE for visibility, tracing, and replay.

System Components

1. ExecutionEngine

The engine creates an isolated Network per workflow execution and translates NetworkEvents (from reflow_network) into enriched EngineEvents with timing, size, and connection metadata.

The engine maintains a HashMap<String, Instant> to track per-actor start times, computing duration_ms when actors complete.

2. EventBridge

The bridge connects the engine's per-execution event channel to two consumers:

#![allow(unused)]
fn main() {
pub struct EventBridge {
    trace_collector: Option<Arc<TraceCollector>>,
    zip_session: Option<Arc<ZipSession>>,
}
}

One bridge task is spawned per execution via bridge.attach(workflow_id, execution_id, event_rx). The task:

1. Begins a trace session via TraceCollector
2. Drains the flume::Receiver<EngineEvent> channel
3. Forwards each event to both TraceCollector and ZipSession
4. Tracks terminal state (success/failure based on Completed and Failed events)
5. Completes the trace session when the channel closes (sender dropped)

3. TraceCollector

Submits per-node execution data to Zeal's TracesAPI over HTTP. Manages trace session lifecycle:

#![allow(unused)]
fn main() {
pub struct TraceCollector {
    traces_api: tokio::sync::Mutex<TracesAPI>,
    sessions: tokio::sync::Mutex<HashMap<String, ActiveSession>>,
    batch_size: usize,  // default: 50
}
}

Each ActiveSession tracks:

• session_id — returned by Zeal on session creation
• start_time — for duration calculation
• nodes_completed / nodes_failed — aggregate counters
• total_data_processed — bytes processed across all nodes
• pending_events — buffered TraceEvents awaiting flush

Events are flushed when the buffer reaches batch_size (50) or when the session completes.

4. ZipSession

Manages the outbound connection to Zeal IDE:

• Template Registration: Registers all actor templates (native + API) on startup
• WebSocket Channel: Opens a tokio-tungstenite connection to /ws/zip
• Event Translation: Converts EngineEvents to ZipExecutionEvents using zeal-sdk helpers
• Event Emission: Pushes ZIP events as JSON text frames over WebSocket

#![allow(unused)]
fn main() {
pub struct ZipSession {
    config: ZipSessionConfig,
    client: ZealClient,
    ws: ZipWebSocket,       // Mutex<Option<WsSink>>
    engine: Arc<ExecutionEngine>,
    shutdown: Arc<Notify>,
}
}

Event Flow

sequenceDiagram
    participant N as Network
    participant E as ExecutionEngine
    participant EB as EventBridge
    participant TC as TraceCollector
    participant ZS as ZipSession
    participant Z as Zeal IDE

    N->>E: NetworkEvent (ActorStarted, ActorCompleted, MessageSent, NetworkIdle)
    E->>E: Translate to EngineEvent (enrich with duration_ms, output_size)
    E->>EB: flume channel
    par TraceCollector
        EB->>TC: process_event()
        TC->>TC: Buffer TraceEvent (batch_size=50)
        TC->>Z: POST /api/traces/sessions/{id}/events
    and ZipSession
        EB->>ZS: emit_engine_event()
        ZS->>ZS: translate_event() → ZipExecutionEvent
        ZS->>Z: WebSocket text frame (JSON)
    end

NetworkEvent → EngineEvent Translation

The engine's run_execution() loop translates each NetworkEvent into an EngineEvent:

The engine waits for NetworkIdle or NetworkShutdown before emitting the Completed event — it does not fire prematurely after network.start().

EngineEvent → ZIP Event Translation

The ZipSession::translate_event() method maps engine events to Zeal SDK types:

MessageSent and NetworkIdle have no ZIP mapping and are silently dropped.

EngineEvent → TraceEvent Translation

The TraceCollector::process_event() method maps engine events to zeal-sdk trace types:

Trace Session Lifecycle

stateDiagram-v2
    [*] --> Created: POST /api/traces/sessions
    Created --> Active: Events flowing
    Active --> Active: Buffer events (batch_size=50)
    Active --> Flushing: Buffer full
    Flushing --> Active: POST events batch
    Active --> Completing: Channel closed
    Completing --> Done: POST complete with SessionSummary
    Done --> [*]

The SessionSummary submitted on completion includes:

• total_nodes — nodes completed + failed
• successful_nodes / failed_nodes
• total_duration — wall clock ms
• total_data_processed — bytes across all nodes

Configuration

The observability pipeline is activated when zeal_url is set in ServerConfig:

#![allow(unused)]
fn main() {
let config = ServerConfig {
    zeal_url: Some("http://localhost:3000".to_string()),
    // ...
};
}

When zeal_url is None, no EventBridge is created and executions run without observability forwarding. The REST API still works for headless execution.

Graceful Degradation

• If the WebSocket connection fails during ZipSession::start(), a warning is logged but the session continues (traces still work via HTTP)
• If TraceCollector fails to begin a session, an error is logged but execution continues
• Individual event forwarding failures are logged at debug level and do not interrupt execution

Next Steps

• Event Types Reference - Detailed EngineEventType and ZIP event documentation
• Zeal IDE Integration - ZIP session, template registration
• Architecture Overview - System-level architecture

Event Types Reference

This reference covers the event types in Reflow's observability pipeline: low-level NetworkEvents from the actor runtime, enriched EngineEvents from the execution engine, and the ZIP events sent to Zeal IDE.

EngineEvent Structure

All engine events share a common structure:

#![allow(unused)]
fn main() {
pub struct EngineEvent {
    pub workflow_id: String,
    pub execution_id: String,
    pub event_type: EngineEventType,
    pub timestamp: u64,
    pub data: serde_json::Value,
}
}

EngineEventType

Started

Emitted when an execution begins.

#![allow(unused)]
fn main() {
EngineEventType::Started
}

ZIP mapping: ZipExecutionEvent::ExecutionStarted

NodeExecuting

Emitted when an actor begins processing. Generated from NetworkEvent::ActorStarted.

#![allow(unused)]
fn main() {
EngineEventType::NodeExecuting {
    node_id: String,     // Actor/node identifier
    component: String,   // Component type name
}
}

ZIP mapping: ZipExecutionEvent::NodeExecuting

ActorCompleted

Emitted when an actor finishes successfully. Generated from NetworkEvent::ActorCompleted. The engine computes duration_ms by comparing the ActorStarted timestamp stored in a HashMap<String, Instant>.

#![allow(unused)]
fn main() {
EngineEventType::ActorCompleted {
    actor_id: String,
    component: String,
    duration_ms: Option<u64>,          // Time from ActorStarted → ActorCompleted
    output_size: Option<u64>,          // Serialized output size in bytes
    output_connections: Vec<String>,   // IDs of outbound connections
}
}

ZIP mapping: ZipExecutionEvent::NodeCompleted with NodeCompletedOptions { duration, output_size }

Trace mapping: TraceEventType::Output with TraceData { size, data_type: "application/json", preview, duration }

ActorFailed

Emitted when an actor errors. Generated from NetworkEvent::ActorFailed.

#![allow(unused)]
fn main() {
EngineEventType::ActorFailed {
    actor_id: String,
    component: String,
    error: String,                     // Error message
    output_connections: Vec<String>,   // Outbound connections (for error routing)
}
}

ZIP mapping: ZipExecutionEvent::NodeFailed with NodeError { message, code, stack }

Trace mapping: TraceEventType::Error with TraceError { message }

MessageSent

Emitted when data flows between actors. Generated from NetworkEvent::MessageSent.

#![allow(unused)]
fn main() {
EngineEventType::MessageSent {
    from_node: String,
    from_port: String,
    to_node: String,
    to_port: String,
    size: usize,    // Serialized message size in bytes
}
}

ZIP mapping: None (silently dropped)

Trace mapping: TraceEventType::Output with TraceData { data_type: "message", size, preview: { to_node, to_port } }

NetworkIdle

Emitted when the network has no more messages to process. Used internally to trigger the Completed event.

#![allow(unused)]
fn main() {
EngineEventType::NetworkIdle
}

ZIP mapping: None

Completed

Emitted when the execution finishes. Generated after NetworkIdle or NetworkShutdown. Includes aggregate statistics.

#![allow(unused)]
fn main() {
EngineEventType::Completed {
    duration_ms: u64,       // Total execution wall-clock time
    nodes_executed: u32,    // Total actors that ran
    nodes_failed: u32,      // Actors that failed
}
}

ZIP mapping: ZipExecutionEvent::ExecutionCompleted with:

#![allow(unused)]
fn main() {
ExecutionCompletedOptions {
    summary: Some(ExecutionSummary {
        success_count: nodes_executed - nodes_failed,
        error_count: nodes_failed,
        warning_count: 0,
    }),
}
}

Failed

Emitted when the execution fails at the engine level (not an individual actor failure).

#![allow(unused)]
fn main() {
EngineEventType::Failed {
    error: String,
    duration_ms: Option<u64>,
}
}

ZIP mapping: ZipExecutionEvent::ExecutionFailed with:

#![allow(unused)]
fn main() {
ExecutionError { message, code: None, node_id: None }
ExecutionFailedOptions { duration }
}

NetworkEvent (Source Events)

These are the raw events from reflow_network that the engine translates:

TraceEvent (Zeal TracesAPI)

Events submitted to Zeal's TracesAPI via HTTP:

#![allow(unused)]
fn main() {
pub struct TraceEvent {
    pub timestamp: i64,
    pub node_id: String,
    pub port_id: Option<String>,
    pub event_type: TraceEventType,   // Input, Output, Error
    pub data: TraceData,
    pub duration: Option<Duration>,
    pub metadata: Option<Value>,
    pub error: Option<TraceError>,
}

pub struct TraceData {
    pub size: usize,
    pub data_type: String,
    pub preview: Option<Value>,
    pub full_data: Option<Value>,
}
}

TraceEventType

ZipExecutionEvent (Zeal WebSocket)

Events sent over the ZIP WebSocket to Zeal in real-time:

All ZIP events are created using zeal-sdk helper functions (create_execution_started_event, create_node_completed_event, etc.) and serialized as JSON text frames.

Event Lifecycle Example

For a workflow with two actors (A → B):

1. EngineEventType::Started
2. EngineEventType::NodeExecuting { node_id: "A", component: "tpl_http_request" }
3. EngineEventType::ActorCompleted { actor_id: "A", duration_ms: Some(150), output_size: Some(2048) }
4. EngineEventType::MessageSent { from: "A", to: "B", size: 2048 }
5. EngineEventType::NodeExecuting { node_id: "B", component: "tpl_data_transformer" }
6. EngineEventType::ActorCompleted { actor_id: "B", duration_ms: Some(10), output_size: Some(512) }
7. EngineEventType::NetworkIdle
8. EngineEventType::Completed { duration_ms: 165, nodes_executed: 2, nodes_failed: 0 }

This generates:

• 6 ZIP WebSocket events (Started, NodeExecuting x2, NodeCompleted x2, ExecutionCompleted)
• 6 TraceEvents (Input x2, Output x3 including MessageSent, Output for completion)
• 1 trace session begin + 1 trace session complete with summary

Next Steps

• Observability Architecture - Pipeline architecture and configuration
• Data Flow Tracing - Message-level data flow tracking
• Zeal IDE Integration - ZIP session and template registration

Data Flow Tracing

Data Flow Tracing is a core component of Reflow's observability framework, providing automatic and comprehensive tracking of data movement between actors in your network. This feature gives you unprecedented visibility into how information flows through your system.

Overview

Traditional actor monitoring focuses on individual actor behavior - creation, completion, and failures. Data Flow Tracing extends this by capturing the connections between actors, providing insights into:

• Message Routing: How messages travel through your actor network
• Data Lineage: Complete paths of data transformation
• Performance Bottlenecks: Where data flow slows down or gets congested
• System Dependencies: Which actors depend on which data sources

How Data Flow Tracing Works

Automatic Capture

Data Flow Tracing operates at the connector level, intercepting messages as they flow between actors:

#![allow(unused)]
fn main() {
// Automatic tracing in connector implementation
impl Connector {
    pub async fn send_message(&self, message: Message) -> Result<()> {
        // Send the actual message
        self.channel.send(message.clone()).await?;
        
        // Automatically record data flow event
        if let Some(tracing) = global_tracing() {
            tracing.trace_data_flow(
                &self.from_actor, &self.from_port,
                &self.to_actor, &self.to_port,
                message.type_name(), message.size_bytes()
            ).await?;
        }
        
        Ok(())
    }
}
}

This approach provides several advantages:

• Zero Configuration: Works immediately with existing actor networks
• Complete Coverage: Captures all message flows without missing any
• Accurate Timing: Records actual transmission times
• Minimal Overhead: Efficient implementation with batching

Event Structure

Data Flow events contain rich metadata about the message transfer:

#![allow(unused)]
fn main() {
pub struct DataFlowEvent {
    // Standard event fields
    event_id: EventId,
    timestamp: DateTime<Utc>,
    event_type: TraceEventType::DataFlow {
        to_actor: String,    // Destination actor
        to_port: String,     // Destination port
    },
    actor_id: String,        // Source actor (from_actor)
    
    // Data flow specific information
    data: TraceEventData {
        port: Some("output".to_string()),  // Source port
        message: Some(MessageSnapshot {
            message_type: "SensorReading".to_string(),
            size_bytes: 256,
            checksum: "sha256:abc123...",
            serialized_data: vec![], // Optional data capture
        }),
        performance_metrics: PerformanceMetrics {
            execution_time_ns: 1_500_000,  // 1.5ms transfer time
            queue_depth: 3,                // Destination queue depth
            throughput_msgs_per_sec: 1000.0,
            memory_usage_bytes: 512,       // Memory for message processing
            cpu_usage_percent: 2.5,
        },
        custom_attributes: HashMap::from([
            ("source_actor", json!("sensor_reader")),
            ("source_port", json!("data")),
            ("destination_actor", json!("data_processor")),
            ("destination_port", json!("input")),
            ("message_id", json!("msg_12345")),
            ("protocol", json!("memory_channel")),
            ("compression", json!("none")),
        ]),
        ..Default::default()
    },
}
}

Use Cases

1. Data Lineage Tracking

Track how data flows and transforms through your entire pipeline:

graph LR
    A[Sensor Reader] -->|SensorReading| B[Data Validator]
    B -->|ValidatedReading| C[Data Transformer]
    C -->|ProcessedData| D[Analytics Engine]
    D -->|Insights| E[Dashboard]
    
    style A fill:#e1f5fe
    style E fill:#f3e5f5

Query for complete data lineage:

#![allow(unused)]
fn main() {
// Find all data flow for a specific message
let query = TraceQuery {
    event_types: Some(vec![TraceEventType::DataFlow { 
        to_actor: "*".to_string(), 
        to_port: "*".to_string() 
    }]),
    custom_filter: Some("message_id = 'msg_12345'"),
    ..Default::default()
};

let lineage = tracing_client.query_traces(query).await?;
}

2. Performance Analysis

Identify bottlenecks in your data processing pipeline:

#![allow(unused)]
fn main() {
// Query for slow data transfers
let slow_transfers = TraceQuery {
    event_types: Some(vec![TraceEventType::DataFlow { 
        to_actor: "*".to_string(), 
        to_port: "*".to_string() 
    }]),
    performance_filter: Some("execution_time_ns > 10000000"), // > 10ms
    time_range: Some((Utc::now() - Duration::hours(1), Utc::now())),
    ..Default::default()
};
}

3. System Dependency Mapping

Understand which actors depend on which data sources:

-- Find most active data flows
SELECT 
    source_actor,
    destination_actor,
    COUNT(*) as message_count,
    AVG(execution_time_ns) as avg_transfer_time,
    SUM(size_bytes) as total_bytes
FROM data_flow_events 
WHERE timestamp > NOW() - INTERVAL '1 hour'
GROUP BY source_actor, destination_actor
ORDER BY message_count DESC;

4. Real-time Monitoring

Monitor data flow in real-time for operational awareness:

#![allow(unused)]
fn main() {
// Subscribe to data flow events for specific actors
let filters = SubscriptionFilters {
    actor_ids: Some(vec!["critical_processor".to_string()]),
    event_types: Some(vec![TraceEventType::DataFlow { 
        to_actor: "*".to_string(), 
        to_port: "*".to_string() 
    }]),
    ..Default::default()
};

tracing_client.subscribe(filters).await?;
}

Configuration

Enabling Data Flow Tracing

Data Flow Tracing is enabled automatically when you enable the observability framework:

#![allow(unused)]
fn main() {
let tracing_config = TracingConfig {
    server_url: "ws://localhost:8080".to_string(),
    enabled: true,                    // Enables all tracing including data flow
    batch_size: 50,                  // Batch size for data flow events
    batch_timeout: Duration::from_millis(1000),
    enable_compression: true,         // Recommended for data flow events
    ..Default::default()
};
}

Selective Tracing

For high-throughput systems, you might want to selectively trace certain data flows:

#![allow(unused)]
fn main() {
// Custom connector with selective tracing
impl SelectiveConnector {
    pub async fn send_message(&self, message: Message) -> Result<()> {
        self.channel.send(message.clone()).await?;
        
        // Only trace certain message types or conditions
        if should_trace_message(&message) {
            if let Some(tracing) = global_tracing() {
                tracing.trace_data_flow(
                    &self.from_actor, &self.from_port,
                    &self.to_actor, &self.to_port,
                    message.type_name(), message.size_bytes()
                ).await?;
            }
        }
        
        Ok(())
    }
}

fn should_trace_message(message: &Message) -> bool {
    // Trace based on message type, size, or other criteria
    match message.type_name() {
        "CriticalAlert" => true,        // Always trace alerts
        "DebugInfo" => false,           // Never trace debug info
        "DataUpdate" if message.size_bytes() > 1024 => true, // Large updates only
        _ => rand::random::<f64>() < 0.1, // Sample 10% of other messages
    }
}
}

Sampling Configuration

For extremely high-throughput scenarios, implement sampling:

#![allow(unused)]
fn main() {
pub struct DataFlowSampler {
    sample_rate: f64,      // 0.0 to 1.0
    always_trace: Vec<String>, // Actor names to always trace
    never_trace: Vec<String>,  // Actor names to never trace
}

impl DataFlowSampler {
    pub fn should_trace(&self, from_actor: &str, to_actor: &str) -> bool {
        if self.never_trace.contains(&from_actor.to_string()) ||
           self.never_trace.contains(&to_actor.to_string()) {
            return false;
        }
        
        if self.always_trace.contains(&from_actor.to_string()) ||
           self.always_trace.contains(&to_actor.to_string()) {
            return true;
        }
        
        rand::random::<f64>() < self.sample_rate
    }
}
}

Advanced Features

Message Content Capture

For debugging purposes, you can optionally capture message content:

#![allow(unused)]
fn main() {
let event = TraceEvent::data_flow_with_content(
    from_actor, from_port,
    to_actor, to_port,
    message_type, size_bytes,
    Some(message.serialize()?) // Optional content capture
);
}

⚠️ Security Warning: Be careful when capturing message content in production. Ensure no sensitive data is included.

Custom Metadata

Add custom metadata to data flow events:

#![allow(unused)]
fn main() {
// Enhanced data flow tracing with custom metadata
pub async fn trace_enhanced_data_flow(
    tracing: &TracingIntegration,
    from_actor: &str, from_port: &str,
    to_actor: &str, to_port: &str,
    message: &Message,
    custom_metadata: HashMap<String, serde_json::Value>
) -> Result<()> {
    let mut event = TraceEvent::data_flow(
        from_actor.to_string(), from_port.to_string(),
        to_actor.to_string(), to_port.to_string(),
        message.type_name(), message.size_bytes()
    );
    
    // Add custom metadata
    event.data.custom_attributes.extend(custom_metadata);
    
    // Add message-specific metadata
    event.data.custom_attributes.insert(
        "message_priority".to_string(), 
        json!(message.priority())
    );
    event.data.custom_attributes.insert(
        "message_correlation_id".to_string(), 
        json!(message.correlation_id())
    );
    
    tracing.record_event(event).await
}
}

Causality Tracking

Link data flow events to their triggering events:

#![allow(unused)]
fn main() {
pub async fn trace_causally_linked_data_flow(
    tracing: &TracingIntegration,
    triggering_event_id: EventId,
    from_actor: &str, from_port: &str,
    to_actor: &str, to_port: &str,
    message: &Message
) -> Result<()> {
    let mut event = TraceEvent::data_flow(
        from_actor.to_string(), from_port.to_string(),
        to_actor.to_string(), to_port.to_string(),
        message.type_name(), message.size_bytes()
    );
    
    // Link to triggering event
    event.causality.parent_event_id = Some(triggering_event_id);
    event.causality.dependency_chain.push(triggering_event_id);
    
    tracing.record_event(event).await
}
}

Performance Considerations

Overhead Analysis

Data Flow Tracing introduces minimal overhead:

• Memory: ~200 bytes per event
• CPU: ~0.1ms per event (including serialization)
• Network: Batched transmission reduces network calls
• Storage: ~1KB per event when stored

Optimization Strategies

1. Batching: Use larger batch sizes for high-throughput scenarios
2. Compression: Enable compression for network transmission
3. Sampling: Sample events rather than capturing every one
4. Filtering: Use selective tracing based on criticality
5. Async Processing: All tracing operations are non-blocking

Monitoring Performance Impact

Monitor the tracing system's own performance:

#![allow(unused)]
fn main() {
// Monitor tracing overhead
let tracing_metrics = global_tracing()
    .unwrap()
    .get_performance_metrics()
    .await?;

println!("Events per second: {}", tracing_metrics.events_per_second);
println!("Average latency: {}ms", tracing_metrics.avg_latency_ms);
println!("Memory usage: {}MB", tracing_metrics.memory_usage_mb);
}

Visualization and Analysis

Data Flow Diagrams

Generate visual representations of your data flow:

#![allow(unused)]
fn main() {
// Generate data flow graph for the last hour
let flow_data = tracing_client.query_data_flows(
    TraceQuery {
        time_range: Some((Utc::now() - Duration::hours(1), Utc::now())),
        ..Default::default()
    }
).await?;

let graph = DataFlowGraph::from_events(&flow_data);
graph.render_to_file("data_flow_diagram.svg")?;
}

Real-time Dashboard

Build real-time monitoring dashboards:

// WebSocket connection for real-time data flow monitoring
const ws = new WebSocket('ws://tracing-server:8080');

ws.onmessage = (event) => {
    const traceEvent = JSON.parse(event.data);
    if (traceEvent.event_type.DataFlow) {
        updateDataFlowVisualization(traceEvent);
    }
};

Troubleshooting

Common Issues

No Data Flow Events Appearing:

• Verify tracing is enabled: enabled: true
• Check that actors are connected via standard connectors
• Ensure global tracing is initialized before network operations

Too Many Events:

• Implement sampling: reduce sample_rate
• Use selective tracing for specific actors only
• Increase batch_size to reduce network overhead

Performance Impact:

• Enable compression: enable_compression: true
• Use PostgreSQL backend for better concurrent performance
• Consider async event processing

Debugging Data Flow Issues

Use data flow tracing to debug connectivity and performance issues:

#![allow(unused)]
fn main() {
// Debug missing data flows
let missing_flows = TraceQuery {
    actor_filter: Some("source_actor".to_string()),
    event_types: Some(vec![TraceEventType::MessageSent]),
    time_range: Some((start_time, end_time)),
    ..Default::default()
};

let sent_messages = tracing_client.query_traces(missing_flows).await?;

// Check if corresponding DataFlow events exist
for sent_event in sent_messages {
    let corresponding_flow = find_data_flow_for_message(&sent_event).await?;
    if corresponding_flow.is_none() {
        println!("Missing data flow for message: {:?}", sent_event);
    }
}
}

Best Practices

1. Start Simple: Begin with default settings and tune based on your needs
2. Monitor Overhead: Keep an eye on the performance impact of tracing
3. Use Sampling: For high-throughput systems, sample rather than trace everything
4. Secure Sensitive Data: Never trace sensitive message content
5. Regular Cleanup: Set up automatic cleanup of old trace data
6. Correlate Events: Use causality tracking to link related events
7. Custom Metadata: Add domain-specific metadata for better insights

Data Flow Tracing provides unprecedented visibility into your actor network's communication patterns. Use it to understand, debug, and optimize your distributed systems with confidence.

Configuration

Reflow's observability framework provides flexible configuration options to suit different deployment scenarios and performance requirements.

Basic Configuration

TracingConfig Structure

#![allow(unused)]
fn main() {
use reflow_network::tracing::TracingConfig;
use std::time::Duration;

let config = TracingConfig {
    server_url: "ws://localhost:8080".to_string(),
    batch_size: 50,
    batch_timeout: Duration::from_millis(1000),
    enable_compression: true,
    enabled: true,
    retry_config: RetryConfig {
        max_retries: 3,
        initial_delay: Duration::from_millis(500),
        max_delay: Duration::from_secs(5),
        backoff_multiplier: 2.0,
    },
};
}

Configuration Parameters

Retry Configuration

#![allow(unused)]
fn main() {
pub struct RetryConfig {
    pub max_retries: u32,           // Maximum retry attempts
    pub initial_delay: Duration,    // Initial delay before first retry
    pub max_delay: Duration,        // Maximum delay between retries
    pub backoff_multiplier: f64,    // Exponential backoff multiplier
}
}

Environment-Based Configuration

Using Environment Variables

# Basic tracing configuration
export REFLOW_TRACING_ENABLED=true
export REFLOW_TRACING_SERVER_URL="ws://tracing-server:8080"
export REFLOW_TRACING_BATCH_SIZE=100
export REFLOW_TRACING_BATCH_TIMEOUT_MS=2000

# Compression and retry settings
export REFLOW_TRACING_COMPRESSION=true
export REFLOW_TRACING_MAX_RETRIES=5
export REFLOW_TRACING_INITIAL_DELAY_MS=1000
export REFLOW_TRACING_MAX_DELAY_MS=30000

Configuration from Environment

#![allow(unused)]
fn main() {
use reflow_network::tracing::TracingConfig;

let config = TracingConfig::from_env().unwrap_or_default();
}

File-Based Configuration

TOML Configuration

# tracing.toml
[tracing]
enabled = true
server_url = "ws://localhost:8080"
batch_size = 50
batch_timeout_ms = 1000
enable_compression = true

[tracing.retry]
max_retries = 3
initial_delay_ms = 500
max_delay_ms = 5000
backoff_multiplier = 2.0

[tracing.filters]
# Optional: Configure event filtering
actor_patterns = ["sensor_*", "processor_*"]
exclude_actors = ["debug_*", "test_*"]
event_types = ["ActorCreated", "DataFlow", "ActorFailed"]

Loading from File

#![allow(unused)]
fn main() {
use reflow_network::tracing::TracingConfig;

let config = TracingConfig::from_file("tracing.toml")?;
}

Performance Tuning

High-Throughput Scenarios

For systems with high message throughput:

#![allow(unused)]
fn main() {
let config = TracingConfig {
    batch_size: 200,                // Larger batches
    batch_timeout: Duration::from_millis(5000), // Longer timeout
    enable_compression: true,       // Reduce network overhead
    retry_config: RetryConfig {
        max_retries: 5,            // More resilient
        initial_delay: Duration::from_millis(100),
        max_delay: Duration::from_secs(30),
        backoff_multiplier: 1.5,   // Gentler backoff
    },
    ..Default::default()
};
}

Low-Latency Requirements

For real-time monitoring needs:

#![allow(unused)]
fn main() {
let config = TracingConfig {
    batch_size: 1,                  // Send immediately
    batch_timeout: Duration::from_millis(10), // Very short timeout
    enable_compression: false,      // Reduce CPU overhead
    retry_config: RetryConfig {
        max_retries: 1,            // Fast failure
        initial_delay: Duration::from_millis(50),
        max_delay: Duration::from_millis(500),
        backoff_multiplier: 2.0,
    },
    ..Default::default()
};
}

Memory-Constrained Environments

For embedded or resource-limited deployments:

#![allow(unused)]
fn main() {
let config = TracingConfig {
    batch_size: 10,                 // Small batches
    batch_timeout: Duration::from_millis(500),
    enable_compression: true,       // Save memory in transit
    retry_config: RetryConfig {
        max_retries: 2,            // Limit retry overhead
        initial_delay: Duration::from_millis(1000),
        max_delay: Duration::from_secs(10),
        backoff_multiplier: 2.0,
    },
    ..Default::default()
};
}

Event Filtering

Actor-Based Filtering

#![allow(unused)]
fn main() {
use reflow_network::tracing::{TracingConfig, EventFilter};

let filter = EventFilter::new()
    .include_actors(&["critical_*", "payment_*"])
    .exclude_actors(&["debug_*", "test_*"])
    .include_event_types(&[
        TraceEventType::ActorCreated,
        TraceEventType::ActorFailed,
        TraceEventType::DataFlow { to_actor: "*".to_string(), to_port: "*".to_string() }
    ]);

let config = TracingConfig::default()
    .with_filter(filter);
}

Sampling Configuration

#![allow(unused)]
fn main() {
use reflow_network::tracing::SamplingStrategy;

// Sample 10% of all events
let config = TracingConfig::default()
    .with_sampling(SamplingStrategy::Percentage(10.0));

// Sample every 5th event
let config = TracingConfig::default()
    .with_sampling(SamplingStrategy::EveryNth(5));

// Adaptive sampling based on load
let config = TracingConfig::default()
    .with_sampling(SamplingStrategy::Adaptive {
        base_rate: 10.0,
        max_rate: 100.0,
        load_threshold: 0.8,
    });
}

Security Configuration

TLS/SSL Configuration

#![allow(unused)]
fn main() {
use reflow_network::tracing::{TracingConfig, TlsConfig};

let tls_config = TlsConfig {
    ca_cert_path: Some("ca-cert.pem".to_string()),
    client_cert_path: Some("client-cert.pem".to_string()),
    client_key_path: Some("client-key.pem".to_string()),
    verify_hostname: true,
};

let config = TracingConfig {
    server_url: "wss://secure-tracing-server:8443".to_string(),
    tls_config: Some(tls_config),
    ..Default::default()
};
}

Authentication

#![allow(unused)]
fn main() {
use reflow_network::tracing::AuthConfig;

let auth_config = AuthConfig::ApiKey {
    key: "your-api-key".to_string(),
    header: "X-API-Key".to_string(),
};

let config = TracingConfig {
    auth_config: Some(auth_config),
    ..Default::default()
};
}

Dynamic Configuration

Runtime Configuration Updates

#![allow(unused)]
fn main() {
use reflow_network::tracing;

// Get current global configuration
let current_config = tracing::get_global_config();

// Update configuration at runtime
let updated_config = current_config
    .with_batch_size(100)
    .with_compression(false);

tracing::update_global_config(updated_config)?;
}

Configuration Monitoring

#![allow(unused)]
fn main() {
use reflow_network::tracing::ConfigWatcher;

// Watch for configuration file changes
let watcher = ConfigWatcher::new("tracing.toml")?;
watcher.on_change(|new_config| {
    println!("Configuration updated: {:?}", new_config);
    tracing::update_global_config(new_config)
})?;
}

Validation and Testing

Configuration Validation

#![allow(unused)]
fn main() {
use reflow_network::tracing::TracingConfig;

let config = TracingConfig::default();

// Validate configuration
if let Err(e) = config.validate() {
    eprintln!("Invalid configuration: {}", e);
    return Err(e);
}

// Test connection
if config.test_connection().await.is_err() {
    eprintln!("Cannot connect to tracing server");
}
}

Configuration Examples

#![allow(unused)]
fn main() {
// Development configuration
let dev_config = TracingConfig {
    server_url: "ws://localhost:8080".to_string(),
    batch_size: 10,
    batch_timeout: Duration::from_millis(100),
    enable_compression: false,
    enabled: true,
    ..Default::default()
};

// Production configuration
let prod_config = TracingConfig {
    server_url: "wss://tracing.prod.company.com:443".to_string(),
    batch_size: 100,
    batch_timeout: Duration::from_millis(2000),
    enable_compression: true,
    enabled: true,
    tls_config: Some(TlsConfig::default()),
    auth_config: Some(AuthConfig::from_env()?),
    ..Default::default()
};

// Testing configuration (disabled)
let test_config = TracingConfig {
    enabled: false,
    ..Default::default()
};
}

Best Practices

1. Start Conservative: Begin with small batch sizes and short timeouts, then tune based on observed performance.

2. Monitor Overhead: Track the performance impact of tracing and adjust configuration accordingly.

3. Use Environment Variables: Make configuration environment-specific without code changes.

4. Enable Compression: For network-constrained environments, compression typically provides significant benefits.

5. Configure Retries: Set appropriate retry parameters based on your network reliability.

6. Filter Strategically: Use event filtering to reduce overhead while maintaining necessary observability.

7. Secure Connections: Always use TLS in production environments.

8. Test Configuration: Validate configuration in development and staging environments.

9. Document Settings: Maintain clear documentation of configuration choices and their rationale.

10. Version Configuration: Track configuration changes alongside code changes.

Storage Backends

Reflow's observability framework supports multiple storage backends to accommodate different operational requirements, from development and testing to large-scale production deployments.

Overview

The tracing system provides a pluggable storage architecture that allows you to choose the most appropriate backend for your needs:

• Memory Storage: Fast, ephemeral storage for development and testing
• SQLite Storage: Lightweight, embedded database for small to medium deployments
• PostgreSQL Storage: Robust, scalable database for production environments
• ClickHouse Storage: High-performance analytical database for massive scale
• Custom Storage: Implement your own storage adapter

Memory Storage

When to Use

• Development and testing environments
• Temporary trace analysis
• Systems with limited persistence requirements
• Quick prototyping and debugging

Configuration

#![allow(unused)]
fn main() {
use reflow_tracing::storage::MemoryStorage;

let storage = MemoryStorage::new();
}

Features

• Ultra-fast: No disk I/O overhead
• Zero configuration: Works out of the box
• Bounded capacity: Configurable memory limits
• Automatic cleanup: LRU eviction when capacity is reached

#![allow(unused)]
fn main() {
use reflow_tracing::storage::{MemoryStorage, MemoryConfig};

let config = MemoryConfig {
    max_traces: 10_000,
    max_events_per_trace: 1_000,
    max_memory_mb: 256,
    eviction_policy: EvictionPolicy::LRU,
};

let storage = MemoryStorage::with_config(config);
}

Limitations

• No persistence: Data lost on restart
• Memory bound: Limited by available RAM
• Single process: No sharing between instances
• No complex queries: Basic filtering only

SQLite Storage

When to Use

• Small to medium production deployments
• Single-node applications
• Applications requiring persistence without database administration
• Development environments with persistence needs

Configuration

#![allow(unused)]
fn main() {
use reflow_tracing::storage::SqliteStorage;

let storage = SqliteStorage::new("traces.db").await?;
}

Features

• Persistent: Data survives restarts
• ACID transactions: Data integrity guarantees
• Full SQL support: Complex queries and analysis
• Embedded: No separate database server required
• Backup friendly: Single file for easy backups

#![allow(unused)]
fn main() {
use reflow_tracing::storage::{SqliteStorage, SqliteConfig};

let config = SqliteConfig {
    database_path: "traces.db".to_string(),
    journal_mode: JournalMode::WAL,
    synchronous: SynchronousMode::Normal,
    cache_size_mb: 64,
    busy_timeout_ms: 5000,
    max_connections: 10,
};

let storage = SqliteStorage::with_config(config).await?;
}

Performance Tuning

#![allow(unused)]
fn main() {
// Optimize for write performance
let fast_config = SqliteConfig {
    journal_mode: JournalMode::WAL,      // Write-Ahead Logging
    synchronous: SynchronousMode::Normal, // Balanced durability/speed
    cache_size_mb: 128,                  // Larger cache
    busy_timeout_ms: 10000,              // Handle contention
    ..Default::default()
};

// Optimize for read performance
let read_config = SqliteConfig {
    cache_size_mb: 256,                  // Very large cache
    temp_store: TempStore::Memory,       // In-memory temp tables
    mmap_size_mb: 512,                   // Memory-mapped I/O
    ..Default::default()
};
}

Limitations

• Single writer: Write concurrency limited
• File size: Large databases can become unwieldy
• Network access: No remote access without additional tools

PostgreSQL Storage

When to Use

• Production environments with multiple instances
• High-concurrency applications
• Applications requiring advanced SQL features
• Distributed systems
• Long-term data retention requirements

Configuration

#![allow(unused)]
fn main() {
use reflow_tracing::storage::PostgresStorage;

let storage = PostgresStorage::new("postgresql://user:pass@localhost/traces").await?;
}

Features

• High concurrency: Excellent multi-client performance
• ACID compliance: Strong consistency guarantees
• Advanced SQL: Window functions, CTEs, advanced analytics
• JSON support: Native support for trace event JSON
• Partitioning: Time-based table partitioning
• Replication: Built-in streaming replication

#![allow(unused)]
fn main() {
use reflow_tracing::storage::{PostgresStorage, PostgresConfig};

let config = PostgresConfig {
    connection_url: "postgresql://user:pass@localhost/traces".to_string(),
    max_connections: 20,
    min_connections: 5,
    connection_timeout_ms: 5000,
    idle_timeout_ms: 600000,
    max_lifetime_ms: 1800000,
    schema_name: "tracing".to_string(),
    enable_partitioning: true,
    partition_interval: PartitionInterval::Daily,
};

let storage = PostgresStorage::with_config(config).await?;
}

Schema Setup

-- Create dedicated schema
CREATE SCHEMA IF NOT EXISTS tracing;

-- Create partitioned tables
CREATE TABLE tracing.traces (
    trace_id UUID PRIMARY KEY,
    flow_id VARCHAR(255) NOT NULL,
    execution_id UUID NOT NULL,
    start_time TIMESTAMPTZ NOT NULL,
    end_time TIMESTAMPTZ,
    status VARCHAR(50) NOT NULL,
    metadata JSONB,
    created_at TIMESTAMPTZ DEFAULT NOW()
) PARTITION BY RANGE (start_time);

CREATE TABLE tracing.events (
    event_id UUID PRIMARY KEY,
    trace_id UUID NOT NULL REFERENCES tracing.traces(trace_id),
    timestamp TIMESTAMPTZ NOT NULL,
    event_type VARCHAR(100) NOT NULL,
    actor_id VARCHAR(255) NOT NULL,
    data JSONB NOT NULL,
    created_at TIMESTAMPTZ DEFAULT NOW()
) PARTITION BY RANGE (timestamp);

-- Create indexes for performance
CREATE INDEX idx_traces_flow_id ON tracing.traces(flow_id);
CREATE INDEX idx_traces_start_time ON tracing.traces(start_time);
CREATE INDEX idx_events_trace_id ON tracing.events(trace_id);
CREATE INDEX idx_events_timestamp ON tracing.events(timestamp);
CREATE INDEX idx_events_actor_id ON tracing.events(actor_id);
CREATE INDEX idx_events_type ON tracing.events(event_type);

-- GIN index for JSON queries
CREATE INDEX idx_events_data_gin ON tracing.events USING GIN(data);

Partitioning Management

#![allow(unused)]
fn main() {
// Automatic partition management
let config = PostgresConfig {
    enable_partitioning: true,
    partition_interval: PartitionInterval::Daily,
    partition_retention_days: 30,
    auto_create_partitions: true,
    ..Default::default()
};
}

Performance Optimization

-- Optimize PostgreSQL configuration
ALTER SYSTEM SET shared_buffers = '256MB';
ALTER SYSTEM SET effective_cache_size = '1GB';
ALTER SYSTEM SET maintenance_work_mem = '64MB';
ALTER SYSTEM SET checkpoint_completion_target = 0.9;
ALTER SYSTEM SET wal_buffers = '16MB';
ALTER SYSTEM SET default_statistics_target = 100;
SELECT pg_reload_conf();

ClickHouse Storage

When to Use

• Very high-volume trace data (millions of events per second)
• Analytical workloads and reporting
• Time-series analysis
• Long-term data retention with compression
• Real-time dashboards and monitoring

Configuration

#![allow(unused)]
fn main() {
use reflow_tracing::storage::ClickHouseStorage;

let storage = ClickHouseStorage::new("http://localhost:8123").await?;
}

Features

• Columnar storage: Excellent compression and analytical performance
• Distributed architecture: Horizontal scaling
• Real-time ingestion: Handle massive write loads
• Advanced analytics: Built-in analytical functions
• Time-series optimized: Purpose-built for time-ordered data

#![allow(unused)]
fn main() {
use reflow_tracing::storage::{ClickHouseStorage, ClickHouseConfig};

let config = ClickHouseConfig {
    url: "http://clickhouse:8123".to_string(),
    database: "tracing".to_string(),
    cluster: Some("cluster".to_string()),
    username: Some("default".to_string()),
    password: None,
    compression: CompressionMethod::LZ4,
    batch_size: 10000,
    flush_interval_ms: 5000,
    max_memory_usage: 1_000_000_000, // 1GB
    max_execution_time_ms: 300_000,   // 5 minutes
};

let storage = ClickHouseStorage::with_config(config).await?;
}

Schema Design

-- Optimized ClickHouse schema
CREATE TABLE tracing.events_local ON CLUSTER cluster (
    timestamp DateTime64(3),
    trace_id UUID,
    event_id UUID,
    flow_id String,
    execution_id UUID,
    event_type LowCardinality(String),
    actor_id String,
    port String,
    message_type String,
    message_size UInt32,
    execution_time_ns UInt64,
    memory_usage UInt64,
    cpu_usage Float32,
    data String -- JSON as string for flexibility
) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{cluster}/{shard}/events', '{replica}')
PARTITION BY toYYYYMM(timestamp)
ORDER BY (timestamp, trace_id, event_id)
SETTINGS index_granularity = 8192;

-- Distributed table
CREATE TABLE tracing.events ON CLUSTER cluster AS tracing.events_local
ENGINE = Distributed(cluster, tracing, events_local, rand());

-- Materialized views for aggregations
CREATE MATERIALIZED VIEW tracing.event_metrics
ENGINE = SummingMergeTree()
PARTITION BY toYYYYMM(timestamp)
ORDER BY (timestamp, actor_id, event_type)
AS SELECT
    toStartOfMinute(timestamp) as timestamp,
    actor_id,
    event_type,
    count() as event_count,
    avg(execution_time_ns) as avg_execution_time,
    max(execution_time_ns) as max_execution_time,
    sum(message_size) as total_bytes
FROM tracing.events_local
GROUP BY timestamp, actor_id, event_type;

Performance Tuning

<!-- ClickHouse configuration -->
<yandex>
    <profiles>
        <default>
            <max_memory_usage>10000000000</max_memory_usage>
            <use_uncompressed_cache>1</use_uncompressed_cache>
            <load_balancing>random</load_balancing>
        </default>
    </profiles>
    
    <users>
        <default>
            <profile>default</profile>
            <networks incl="networks" replace="replace">
                <ip>::/0</ip>
            </networks>
        </default>
    </users>
</yandex>

Custom Storage Implementation

Storage Trait

#![allow(unused)]
fn main() {
use async_trait::async_trait;
use reflow_tracing::storage::{StorageBackend, StorageError};

#[async_trait]
pub trait StorageBackend: Send + Sync {
    async fn store_trace(&self, trace: FlowTrace) -> Result<(), StorageError>;
    async fn get_trace(&self, trace_id: TraceId) -> Result<Option<FlowTrace>, StorageError>;
    async fn query_traces(&self, query: TraceQuery) -> Result<Vec<FlowTrace>, StorageError>;
    async fn store_event(&self, trace_id: TraceId, event: TraceEvent) -> Result<(), StorageError>;
    async fn get_events(&self, trace_id: TraceId) -> Result<Vec<TraceEvent>, StorageError>;
    async fn health_check(&self) -> Result<(), StorageError>;
}
}

Example: Redis Storage

#![allow(unused)]
fn main() {
use redis::{Client, Connection};
use reflow_tracing::storage::{StorageBackend, StorageError};

pub struct RedisStorage {
    client: Client,
}

impl RedisStorage {
    pub fn new(url: &str) -> Result<Self, StorageError> {
        let client = Client::open(url)?;
        Ok(Self { client })
    }
}

#[async_trait]
impl StorageBackend for RedisStorage {
    async fn store_trace(&self, trace: FlowTrace) -> Result<(), StorageError> {
        let mut conn = self.client.get_connection()?;
        let key = format!("trace:{}", trace.trace_id);
        let value = serde_json::to_string(&trace)?;
        
        redis::cmd("SET")
            .arg(&key)
            .arg(&value)
            .arg("EX")
            .arg(3600) // 1 hour TTL
            .query(&mut conn)?;
            
        Ok(())
    }
    
    async fn get_trace(&self, trace_id: TraceId) -> Result<Option<FlowTrace>, StorageError> {
        let mut conn = self.client.get_connection()?;
        let key = format!("trace:{}", trace_id);
        
        let value: Option<String> = redis::cmd("GET")
            .arg(&key)
            .query(&mut conn)?;
            
        match value {
            Some(json) => Ok(Some(serde_json::from_str(&json)?)),
            None => Ok(None),
        }
    }
    
    // Implement other methods...
}
}

Storage Selection Guide

Decision Matrix

Recommendations

Development/Testing:

#![allow(unused)]
fn main() {
// Quick start with memory storage
let storage = MemoryStorage::new();
}

Small Production:

#![allow(unused)]
fn main() {
// SQLite for simple deployments
let storage = SqliteStorage::new("traces.db").await?;
}

Medium Production:

#![allow(unused)]
fn main() {
// PostgreSQL for robust applications
let storage = PostgresStorage::new("postgresql://...").await?;
}

Large Scale/Analytics:

#![allow(unused)]
fn main() {
// ClickHouse for high-volume scenarios
let storage = ClickHouseStorage::new("http://clickhouse:8123").await?;
}

Migration Between Backends

Export/Import Tool

#![allow(unused)]
fn main() {
use reflow_tracing::migration::StorageMigrator;

// Migrate from SQLite to PostgreSQL
let migrator = StorageMigrator::new(
    SqliteStorage::new("traces.db").await?,
    PostgresStorage::new("postgresql://...").await?
);

migrator.migrate_all_traces().await?;
}

Backup and Restore

#![allow(unused)]
fn main() {
// Backup to file
let backup_path = "traces_backup.json";
storage.export_to_file(backup_path).await?;

// Restore from file
storage.import_from_file(backup_path).await?;
}

Monitoring Storage Performance

Metrics Collection

#![allow(unused)]
fn main() {
use reflow_tracing::storage::StorageMetrics;

let metrics = storage.get_metrics().await?;
println!("Storage performance:");
println!("  Write latency: {}ms", metrics.avg_write_latency_ms);
println!("  Read latency: {}ms", metrics.avg_read_latency_ms);
println!("  Storage size: {}MB", metrics.storage_size_mb);
println!("  Query performance: {}ms", metrics.avg_query_latency_ms);
}

Health Monitoring

#![allow(unused)]
fn main() {
// Regular health checks
tokio::spawn(async move {
    loop {
        match storage.health_check().await {
            Ok(_) => println!("Storage healthy"),
            Err(e) => eprintln!("Storage unhealthy: {}", e),
        }
        tokio::time::sleep(Duration::from_secs(30)).await;
    }
});
}

Best Practices

1. Choose Appropriate Backend: Match storage backend to your scale and requirements
2. Plan for Growth: Start simple but design for scale
3. Monitor Performance: Track storage metrics and query performance
4. Regular Backups: Implement automated backup strategies
5. Partition Large Tables: Use time-based partitioning for better performance
6. Index Strategically: Create indexes for common query patterns
7. Manage Retention: Implement data retention policies to control growth
8. Test Disaster Recovery: Regularly test backup and restore procedures
9. Optimize Queries: Use EXPLAIN to understand and optimize query performance
10. Monitor Resources: Keep an eye on disk space, memory, and CPU usage

Production Deployment

This guide covers deploying Reflow's observability framework in production environments, including scalability considerations, security best practices, and operational procedures.

Architecture Overview

Production Architecture

graph TB
    App1[Reflow App 1] --> LB[Load Balancer]
    App2[Reflow App 2] --> LB
    App3[Reflow App N] --> LB
    
    LB --> TS1[Tracing Server 1]
    LB --> TS2[Tracing Server 2]
    
    TS1 --> DB[(PostgreSQL Primary)]
    TS2 --> DB
    
    DB --> Replica[(PostgreSQL Replica)]
    
    TS1 --> Cache[(Redis Cache)]
    TS2 --> Cache
    
    Grafana[Grafana] --> DB
    Grafana --> Cache
    
    Monitor[Monitoring] --> TS1
    Monitor --> TS2
    Monitor --> DB

Component Responsibilities

• Reflow Applications: Generate trace events
• Load Balancer: Distribute connections across tracing servers
• Tracing Servers: Receive, process, and store trace data
• PostgreSQL: Primary data storage with replication
• Redis: Caching and real-time data
• Grafana: Visualization and dashboards
• Monitoring: Health checks and alerting

Infrastructure Requirements

Minimum Production Setup

Tracing Server:

• CPU: 2 cores
• Memory: 4GB RAM
• Storage: 50GB SSD
• Network: 1Gbps

Database (PostgreSQL):

• CPU: 4 cores
• Memory: 8GB RAM
• Storage: 200GB SSD (for data) + 100GB (for WAL)
• Network: 1Gbps

Cache (Redis):

• CPU: 2 cores
• Memory: 4GB RAM
• Storage: 20GB SSD
• Network: 1Gbps

High-Scale Production Setup

Tracing Server Cluster:

• 3+ instances
• CPU: 8 cores each
• Memory: 16GB RAM each
• Storage: 100GB SSD each
• Network: 10Gbps

Database Cluster:

• Primary + 2 replicas
• CPU: 16 cores each
• Memory: 64GB RAM each
• Storage: 1TB NVMe SSD each
• Network: 10Gbps

Cache Cluster:

• 3 instance Redis cluster
• CPU: 4 cores each
• Memory: 16GB RAM each
• Storage: 50GB SSD each
• Network: 10Gbps

Container Deployment

Docker Compose

# docker-compose.prod.yml
version: '3.8'

services:
  tracing-server:
    image: reflow/tracing-server:latest
    deploy:
      replicas: 3
      resources:
        limits:
          cpus: '4'
          memory: 8G
        reservations:
          cpus: '2'
          memory: 4G
    environment:
      - RUST_LOG=info
      - TRACING_DATABASE_URL=postgresql://user:pass@postgres:5432/tracing
      - TRACING_REDIS_URL=redis://redis:6379
      - TRACING_BIND_ADDRESS=0.0.0.0:8080
      - TRACING_MAX_CONNECTIONS=1000
    ports:
      - "8080:8080"
    networks:
      - tracing-network
    depends_on:
      - postgres
      - redis
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
      interval: 30s
      timeout: 10s
      retries: 3

  postgres:
    image: postgres:15
    environment:
      - POSTGRES_DB=tracing
      - POSTGRES_USER=tracing_user
      - POSTGRES_PASSWORD_FILE=/run/secrets/postgres_password
      - POSTGRES_INITDB_ARGS=--auth-host=scram-sha-256
    volumes:
      - postgres_data:/var/lib/postgresql/data
      - ./init.sql:/docker-entrypoint-initdb.d/init.sql
    ports:
      - "5432:5432"
    networks:
      - tracing-network
    secrets:
      - postgres_password
    command: postgres -c shared_preload_libraries=pg_stat_statements
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U tracing_user -d tracing"]
      interval: 30s
      timeout: 5s
      retries: 5

  redis:
    image: redis:7-alpine
    volumes:
      - redis_data:/data
    ports:
      - "6379:6379"
    networks:
      - tracing-network
    command: redis-server --appendonly yes --maxmemory 2gb --maxmemory-policy allkeys-lru
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 30s
      timeout: 5s
      retries: 3

  nginx:
    image: nginx:alpine
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf
    ports:
      - "80:80"
      - "443:443"
    networks:
      - tracing-network
    depends_on:
      - tracing-server

volumes:
  postgres_data:
  redis_data:

networks:
  tracing-network:
    driver: overlay

secrets:
  postgres_password:
    external: true

Kubernetes Deployment

# tracing-server-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: tracing-server
  labels:
    app: tracing-server
spec:
  replicas: 3
  selector:
    matchLabels:
      app: tracing-server
  template:
    metadata:
      labels:
        app: tracing-server
    spec:
      containers:
      - name: tracing-server
        image: reflow/tracing-server:latest
        ports:
        - containerPort: 8080
        env:
        - name: RUST_LOG
          value: "info"
        - name: TRACING_DATABASE_URL
          valueFrom:
            secretKeyRef:
              name: tracing-secrets
              key: database-url
        - name: TRACING_REDIS_URL
          value: "redis://redis:6379"
        resources:
          requests:
            memory: "2Gi"
            cpu: "1000m"
          limits:
            memory: "4Gi"
            cpu: "2000m"
        livenessProbe:
          httpGet:
            path: /health
            port: 8080
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /ready
            port: 8080
          initialDelaySeconds: 5
          periodSeconds: 5

---
apiVersion: v1
kind: Service
metadata:
  name: tracing-server
spec:
  selector:
    app: tracing-server
  ports:
  - protocol: TCP
    port: 8080
    targetPort: 8080
  type: ClusterIP

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: tracing-server-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
spec:
  tls:
  - hosts:
    - tracing.yourdomain.com
    secretName: tracing-tls
  rules:
  - host: tracing.yourdomain.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: tracing-server
            port:
              number: 8080

PostgreSQL Configuration

# postgres-deployment.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: postgres
spec:
  serviceName: postgres
  replicas: 1
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
      - name: postgres
        image: postgres:15
        env:
        - name: POSTGRES_DB
          value: tracing
        - name: POSTGRES_USER
          value: tracing_user
        - name: POSTGRES_PASSWORD
          valueFrom:
            secretKeyRef:
              name: postgres-secrets
              key: password
        ports:
        - containerPort: 5432
        volumeMounts:
        - name: postgres-storage
          mountPath: /var/lib/postgresql/data
        - name: postgres-config
          mountPath: /etc/postgresql/postgresql.conf
          subPath: postgresql.conf
        resources:
          requests:
            memory: "4Gi"
            cpu: "2000m"
          limits:
            memory: "8Gi"
            cpu: "4000m"
      volumes:
      - name: postgres-config
        configMap:
          name: postgres-config
  volumeClaimTemplates:
  - metadata:
      name: postgres-storage
    spec:
      accessModes: ["ReadWriteOnce"]
      resources:
        requests:
          storage: 200Gi
      storageClassName: fast-ssd

Configuration Management

Environment-Specific Configuration

# config/production.toml
[server]
bind_address = "0.0.0.0:8080"
max_connections = 1000
worker_threads = 8
keep_alive_timeout = 30

[database]
url = "postgresql://user:pass@postgres-cluster:5432/tracing"
max_connections = 20
min_connections = 5
connection_timeout = 5000
statement_timeout = 30000

[redis]
url = "redis://redis-cluster:6379"
pool_size = 10
connection_timeout = 3000

[tracing]
batch_size = 100
batch_timeout_ms = 2000
max_event_size = 1048576  # 1MB
compression = true

[logging]
level = "info"
format = "json"
target = "stdout"

[metrics]
enabled = true
bind_address = "0.0.0.0:9090"

Secret Management

# Kubernetes secrets
kubectl create secret generic tracing-secrets \
  --from-literal=database-url="postgresql://user:pass@postgres:5432/tracing" \
  --from-literal=redis-url="redis://redis:6379" \
  --from-literal=jwt-secret="your-jwt-secret"

kubectl create secret generic postgres-secrets \
  --from-literal=password="secure-postgres-password"

# Docker secrets
echo "secure-postgres-password" | docker secret create postgres_password -

Security Configuration

TLS/SSL Setup

# nginx.conf
events {
    worker_connections 1024;
}

http {
    upstream tracing_backend {
        server tracing-server:8080;
        keepalive 32;
    }

    server {
        listen 80;
        return 301 https://$server_name$request_uri;
    }

    server {
        listen 443 ssl http2;
        server_name tracing.yourdomain.com;

        ssl_certificate /etc/ssl/certs/tracing.crt;
        ssl_certificate_key /etc/ssl/private/tracing.key;
        ssl_protocols TLSv1.2 TLSv1.3;
        ssl_ciphers ECDHE-RSA-AES256-GCM-SHA512:DHE-RSA-AES256-GCM-SHA512;

        location / {
            proxy_pass http://tracing_backend;
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "upgrade";
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
        }
    }
}

Authentication Configuration

#![allow(unused)]
fn main() {
// Server configuration with authentication
use reflow_tracing::auth::{AuthConfig, JwtAuth};

let auth_config = AuthConfig {
    jwt_secret: env::var("JWT_SECRET")?,
    token_expiry: Duration::from_hours(24),
    issuer: "reflow-tracing".to_string(),
    audience: "reflow-clients".to_string(),
};

let server_config = ServerConfig {
    auth: Some(auth_config),
    require_auth: true,
    ..Default::default()
};
}

Network Security

# Network policies
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: tracing-network-policy
spec:
  podSelector:
    matchLabels:
      app: tracing-server
  policyTypes:
  - Ingress
  - Egress
  ingress:
  - from:
    - podSelector:
        matchLabels:
          app: reflow-client
    ports:
    - protocol: TCP
      port: 8080
  egress:
  - to:
    - podSelector:
        matchLabels:
          app: postgres
    ports:
    - protocol: TCP
      port: 5432
  - to:
    - podSelector:
        matchLabels:
          app: redis
    ports:
    - protocol: TCP
      port: 6379

Monitoring and Observability

Prometheus Metrics

# prometheus-config.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: prometheus-config
data:
  prometheus.yml: |
    global:
      scrape_interval: 15s
    
    scrape_configs:
    - job_name: 'tracing-server'
      static_configs:
      - targets: ['tracing-server:9090']
      metrics_path: /metrics
      scrape_interval: 10s
    
    - job_name: 'postgres'
      static_configs:
      - targets: ['postgres-exporter:9187']
    
    - job_name: 'redis'
      static_configs:
      - targets: ['redis-exporter:9121']

Health Checks

#![allow(unused)]
fn main() {
// Health check endpoints
use warp::Filter;

let health = warp::path("health")
    .and(warp::get())
    .map(|| {
        // Check database connectivity
        // Check Redis connectivity
        // Check disk space
        warp::reply::json(&json!({
            "status": "healthy",
            "timestamp": Utc::now(),
            "checks": {
                "database": "ok",
                "redis": "ok",
                "disk_space": "ok"
            }
        }))
    });

let ready = warp::path("ready")
    .and(warp::get())
    .map(|| {
        // Check if server is ready to accept traffic
        warp::reply::json(&json!({
            "status": "ready",
            "timestamp": Utc::now()
        }))
    });
}

Alerting Rules

# alerting-rules.yaml
groups:
- name: tracing-server
  rules:
  - alert: TracingServerDown
    expr: up{job="tracing-server"} == 0
    for: 5m
    labels:
      severity: critical
    annotations:
      summary: "Tracing server is down"
      description: "Tracing server {{ $labels.instance }} has been down for more than 5 minutes"

  - alert: HighLatency
    expr: tracing_request_duration_seconds{quantile="0.95"} > 0.5
    for: 10m
    labels:
      severity: warning
    annotations:
      summary: "High latency detected"
      description: "95th percentile latency is {{ $value }}s"

  - alert: HighErrorRate
    expr: rate(tracing_requests_total{status="error"}[5m]) > 0.1
    for: 5m
    labels:
      severity: critical
    annotations:
      summary: "High error rate"
      description: "Error rate is {{ $value }} requests/second"

  - alert: DatabaseConnectionsHigh
    expr: pg_stat_activity_count > 80
    for: 5m
    labels:
      severity: warning
    annotations:
      summary: "High number of database connections"
      description: "{{ $value }} active connections to PostgreSQL"

Performance Tuning

PostgreSQL Optimization

-- postgresql.conf optimizations
shared_buffers = 2GB                    # 25% of RAM
effective_cache_size = 6GB              # 75% of RAM
maintenance_work_mem = 512MB
work_mem = 16MB
max_connections = 200
wal_buffers = 16MB
checkpoint_completion_target = 0.9
random_page_cost = 1.1                  # For SSDs
effective_io_concurrency = 200          # For SSDs

-- Enable query logging for optimization
log_min_duration_statement = 1000       # Log queries > 1s
log_checkpoints = on
log_connections = on
log_disconnections = on
log_lock_waits = on

Redis Optimization

# redis.conf
maxmemory 4gb
maxmemory-policy allkeys-lru
save 900 1
save 300 10
save 60 10000
tcp-keepalive 300
tcp-backlog 511

Application Tuning

#![allow(unused)]
fn main() {
// Server configuration for high performance
let config = ServerConfig {
    worker_threads: num_cpus::get(),
    max_connections: 1000,
    connection_pool_size: 20,
    batch_size: 200,
    batch_timeout: Duration::from_millis(5000),
    compression: true,
    buffer_size: 65536,
    ..Default::default()
};
}

Backup and Recovery

Database Backup

#!/bin/bash
# backup.sh
BACKUP_DIR="/backups"
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
DB_NAME="tracing"

# Create backup
pg_dump -h postgres -U tracing_user -d $DB_NAME | gzip > "$BACKUP_DIR/tracing_$TIMESTAMP.sql.gz"

# Upload to S3
aws s3 cp "$BACKUP_DIR/tracing_$TIMESTAMP.sql.gz" s3://your-backup-bucket/database/

# Cleanup old backups (keep 30 days)
find $BACKUP_DIR -name "tracing_*.sql.gz" -mtime +30 -delete

Automated Backup with CronJob

# backup-cronjob.yaml
apiVersion: batch/v1
kind: CronJob
metadata:
  name: postgres-backup
spec:
  schedule: "0 2 * * *"  # Daily at 2 AM
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: postgres-backup
            image: postgres:15
            env:
            - name: PGPASSWORD
              valueFrom:
                secretKeyRef:
                  name: postgres-secrets
                  key: password
            command:
            - /bin/bash
            - -c
            - |
              pg_dump -h postgres -U tracing_user tracing | gzip > /backup/tracing_$(date +%Y%m%d_%H%M%S).sql.gz
              # Upload to cloud storage
              aws s3 cp /backup/tracing_*.sql.gz s3://backup-bucket/
            volumeMounts:
            - name: backup-storage
              mountPath: /backup
          volumes:
          - name: backup-storage
            persistentVolumeClaim:
              claimName: backup-pvc
          restartPolicy: OnFailure

Disaster Recovery

#!/bin/bash
# restore.sh
BACKUP_FILE=$1

if [ -z "$BACKUP_FILE" ]; then
    echo "Usage: $0 <backup_file>"
    exit 1
fi

# Download backup from S3
aws s3 cp "s3://your-backup-bucket/database/$BACKUP_FILE" ./

# Restore database
gunzip -c "$BACKUP_FILE" | psql -h postgres -U tracing_user -d tracing

echo "Database restored from $BACKUP_FILE"

Scaling Strategies

Horizontal Scaling

# hpa.yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: tracing-server-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: tracing-server
  minReplicas: 3
  maxReplicas: 20
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70
  - type: Resource
    resource:
      name: memory
      target:
        type: Utilization
        averageUtilization: 80

Database Scaling

-- Read replicas configuration
-- Primary server
ALTER SYSTEM SET wal_level = replica;
ALTER SYSTEM SET max_wal_senders = 3;
ALTER SYSTEM SET max_replication_slots = 3;
SELECT pg_reload_conf();

-- Create replication slot
SELECT pg_create_physical_replication_slot('replica_1');

-- Replica server setup
standby_mode = 'on'
primary_conninfo = 'host=postgres-primary port=5432 user=replicator'

Maintenance Procedures

Rolling Updates

#!/bin/bash
# rolling-update.sh
kubectl set image deployment/tracing-server tracing-server=reflow/tracing-server:v2.0.0
kubectl rollout status deployment/tracing-server
kubectl rollout history deployment/tracing-server

Database Maintenance

-- Regular maintenance tasks
VACUUM ANALYZE tracing.events;
VACUUM ANALYZE tracing.traces;
REINDEX INDEX CONCURRENTLY idx_events_timestamp;

-- Partition maintenance
SELECT create_monthly_partitions('tracing.events', '2024-01-01'::date);
SELECT drop_old_partitions('tracing.events', interval '90 days');

Log Rotation

# fluent-bit-config.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: fluent-bit-config
data:
  fluent-bit.conf: |
    [INPUT]
        Name tail
        Path /var/log/containers/*tracing-server*.log
        Parser docker
        Tag tracing.*
    
    [OUTPUT]
        Name es
        Match tracing.*
        Host elasticsearch.logging.svc.cluster.local
        Port 9200
        Index tracing-logs
        Type _doc

Troubleshooting

Common Issues

High Memory Usage:

# Check memory usage
kubectl top pods
kubectl describe pod tracing-server-xxx

# Adjust memory limits
kubectl patch deployment tracing-server -p '{"spec":{"template":{"spec":{"containers":[{"name":"tracing-server","resources":{"limits":{"memory":"8Gi"}}}]}}}}'

Database Connection Issues:

-- Check active connections
SELECT count(*) FROM pg_stat_activity;

-- Kill long-running queries
SELECT pg_terminate_backend(pid) FROM pg_stat_activity WHERE state = 'active' AND query_start < NOW() - INTERVAL '10 minutes';

Performance Issues:

# Check metrics
curl http://tracing-server:9090/metrics | grep -E "latency|throughput|errors"

# Scale up
kubectl scale deployment tracing-server --replicas=10

This production deployment guide provides a comprehensive foundation for running Reflow's observability framework at scale with proper security, monitoring, and operational procedures.

Creating Actors

This guide covers how to create custom actors in Reflow using the correct implementation patterns. Learn everything from basic actors to advanced patterns with state management and error handling.

Creating Actors: Two Approaches

Reflow provides two ways to create actors:

1. Actor Macro (Recommended): Use the #[actor] macro for simple, declarative actor creation
2. Manual Implementation: Implement the Actor trait directly for maximum control

Using the Actor Macro

The #[actor] macro is the recommended way to create actors. It generates all the necessary boilerplate code including the Actor trait implementation, port management, and process creation.

Basic Actor

#![allow(unused)]
fn main() {
use std::collections::HashMap;
use reflow_network::{
    actor::ActorContext,
    message::Message,
};
use actor_macro::actor;

#[actor(
    HelloActor,
    inports::<100>(input),
    outports::<50>(output)
)]
async fn hello_actor(context: ActorContext) -> Result<HashMap<String, Message>, anyhow::Error> {
    let payload = context.get_payload();
    
    if let Some(Message::String(text)) = payload.get("input") {
        let response = format!("Hello, {}!", text);
        
        Ok([
            ("output".to_owned(), Message::string(response))
        ].into())
    } else {
        Err(anyhow::anyhow!("Expected string input"))
    }
}
}

Actor with Multiple Inputs

#![allow(unused)]
fn main() {
#[actor(
    GreeterActor,
    inports::<100>(name, age),
    outports::<50>(greeting),
    await_all_inports  // Wait for both inputs before processing
)]
async fn greeter_actor(context: ActorContext) -> Result<HashMap<String, Message>, anyhow::Error> {
    let payload = context.get_payload();
    
    let name = match payload.get("name").expect("expected name") {
        Message::String(s) => s,
        _ => return Err(anyhow::anyhow!("Name must be a string")),
    };
    
    let age = match payload.get("age").expect("expected age") {
        Message::Integer(n) => *n,
        _ => return Err(anyhow::anyhow!("Age must be an integer")),
    };
    
    let greeting = format!("Hello {}, you are {} years old!", name, age);
    
    Ok([
        ("greeting".to_owned(), Message::string(greeting))
    ].into())
}
}

Stateful Actor

#![allow(unused)]
fn main() {
use reflow_network::actor::MemoryState;

#[actor(
    CounterActor,
    state(MemoryState),
    inports::<100>(increment, reset),
    outports::<50>(count, total)
)]
async fn counter_actor(context: ActorContext) -> Result<HashMap<String, Message>, anyhow::Error> {
    let payload = context.get_payload();
    let state = context.get_state();
    
    let mut state_guard = state.lock();
    let memory_state = state_guard
        .as_mut_any()
        .downcast_mut::<MemoryState>()
        .expect("Expected MemoryState");
    
    // Initialize state if needed
    if !memory_state.contains_key("count") {
        memory_state.insert("count", serde_json::json!(0));
        memory_state.insert("total", serde_json::json!(0));
    }
    
    let current_count = memory_state.get("count")
        .and_then(|v| v.as_i64())
        .unwrap_or(0);
    
    let current_total = memory_state.get("total")
        .and_then(|v| v.as_i64())
        .unwrap_or(0);
    
    let (new_count, new_total) = if payload.contains_key("reset") {
        // Reset counter
        (0, current_total)
    } else if let Some(Message::Integer(amount)) = payload.get("increment") {
        // Increment by specific amount
        let new_count = current_count + amount;
        (new_count, current_total + amount)
    } else {
        // Default increment by 1
        let new_count = current_count + 1;
        (new_count, current_total + 1)
    };
    
    // Update state
    memory_state.insert("count", serde_json::json!(new_count));
    memory_state.insert("total", serde_json::json!(new_total));
    
    println!("Counter: {} (total: {})", new_count, new_total);
    
    Ok([
        ("count".to_owned(), Message::Integer(new_count)),
        ("total".to_owned(), Message::Integer(new_total)),
    ].into())
}
}

Actor Macro Parameters

Port Definitions

#![allow(unused)]
fn main() {
// Basic ports (unbounded channels)
inports(A, B, C)
outports(X, Y)

// Ports with capacity (bounded channels)
inports::<100>(A, B)      // Input ports with capacity 100
outports::<50>(X, Y)      // Output ports with capacity 50
}

State Management

#![allow(unused)]
fn main() {
// Use built-in MemoryState
state(MemoryState)

// Custom state types can also be used
// (must implement ActorState trait)
}

Input Synchronization

#![allow(unused)]
fn main() {
// Process inputs as they arrive (default)
#[actor(MyActor, inports(A, B), outports(C))]

// Wait for ALL inputs before processing
#[actor(MyActor, inports(A, B), outports(C), await_all_inports)]
}

Practical Examples

Data Processing Pipeline

#![allow(unused)]
fn main() {
// Sum Actor - adds numbers from multiple sources
#[actor(
    SumActor,
    inports::<100>(numbers),
    outports::<50>(sum, count)
)]
async fn sum_actor(context: ActorContext) -> Result<HashMap<String, Message>, anyhow::Error> {
    let payload = context.get_payload();
    
    if let Some(Message::Array(numbers)) = payload.get("numbers") {
        let mut sum = 0i64;
        let mut count = 0usize;
        
        for num in numbers {
            if let Message::Integer(n) = num {
                sum += n;
                count += 1;
            }
        }
        
        println!("Sum Actor: {} numbers, sum = {}", count, sum);
        
        Ok([
            ("sum".to_owned(), Message::Integer(sum)),
            ("count".to_owned(), Message::Integer(count as i64)),
        ].into())
    } else {
        Err(anyhow::anyhow!("Expected array of numbers"))
    }
}

// Filter Actor - filters values based on condition
#[actor(
    FilterActor,
    inports::<100>(values, threshold),
    outports::<50>(passed, failed),
    await_all_inports
)]
async fn filter_actor(context: ActorContext) -> Result<HashMap<String, Message>, anyhow::Error> {
    let payload = context.get_payload();
    
    let threshold = match payload.get("threshold").expect("expected threshold") {
        Message::Integer(t) => *t,
        _ => return Err(anyhow::anyhow!("Threshold must be integer")),
    };
    
    if let Some(Message::Array(values)) = payload.get("values") {
        let mut passed = Vec::new();
        let mut failed = Vec::new();
        
        for value in values {
            if let Message::Integer(n) = value {
                if *n >= threshold {
                    passed.push(value.clone());
                } else {
                    failed.push(value.clone());
                }
            }
        }
        
        println!("Filter Actor: {} passed, {} failed (threshold: {})", 
                passed.len(), failed.len(), threshold);
        
        Ok([
            ("passed".to_owned(), Message::Array(passed)),
            ("failed".to_owned(), Message::Array(failed)),
        ].into())
    } else {
        Err(anyhow::anyhow!("Expected array of values"))
    }
}
}

HTTP Client Actor

#![allow(unused)]
fn main() {
use reqwest;

#[actor(
    HttpClientActor,
    inports::<50>(request),
    outports::<25>(response, error)
)]
async fn http_client_actor(context: ActorContext) -> Result<HashMap<String, Message>, anyhow::Error> {
    let payload = context.get_payload();
    
    // Parse request
    let request = match payload.get("request") {
        Some(Message::Object(obj)) => obj,
        _ => return Err(anyhow::anyhow!("Expected request object")),
    };
    
    let url = match request.get("url") {
        Some(Message::String(s)) => s,
        _ => return Err(anyhow::anyhow!("Missing URL in request")),
    };
    
    let method = request.get("method")
        .and_then(|m| if let Message::String(s) = m { Some(s.as_str()) } else { None })
        .unwrap_or("GET");
    
    // Make HTTP request
    let client = reqwest::Client::new();
    
    let result = match method {
        "GET" => {
            match client.get(url).send().await {
                Ok(response) => {
                    let status = response.status().as_u16();
                    let text = response.text().await.unwrap_or_default();
                    
                    let response_obj = [
                        ("status".to_owned(), Message::Integer(status as i64)),
                        ("body".to_owned(), Message::String(text)),
                        ("url".to_owned(), Message::String(url.clone())),
                    ].into();
                    
                    [("response".to_owned(), Message::Object(response_obj))].into()
                },
                Err(e) => {
                    let error_obj = [
                        ("message".to_owned(), Message::String(e.to_string())),
                        ("url".to_owned(), Message::String(url.clone())),
                    ].into();
                    
                    [("error".to_owned(), Message::Object(error_obj))].into()
                }
            }
        },
        "POST" => {
            let body = request.get("body")
                .and_then(|b| if let Message::String(s) = b { Some(s) } else { None })
                .unwrap_or("");
            
            match client.post(url).body(body.to_string()).send().await {
                Ok(response) => {
                    let status = response.status().as_u16();
                    let text = response.text().await.unwrap_or_default();
                    
                    let response_obj = [
                        ("status".to_owned(), Message::Integer(status as i64)),
                        ("body".to_owned(), Message::String(text)),
                        ("url".to_owned(), Message::String(url.clone())),
                    ].into();
                    
                    [("response".to_owned(), Message::Object(response_obj))].into()
                },
                Err(e) => {
                    let error_obj = [
                        ("message".to_owned(), Message::String(e.to_string())),
                        ("url".to_owned(), Message::String(url.clone())),
                    ].into();
                    
                    [("error".to_owned(), Message::Object(error_obj))].into()
                }
            }
        },
        _ => {
            let error_obj = [
                ("message".to_owned(), Message::String(format!("Unsupported method: {}", method))),
            ].into();
            
            [("error".to_owned(), Message::Object(error_obj))].into()
        }
    };
    
    Ok(result)
}
}

Batch Processing Actor

#![allow(unused)]
fn main() {
#[actor(
    BatchActor,
    state(MemoryState),
    inports::<200>(item, flush),
    outports::<50>(batch, count)
)]
async fn batch_actor(context: ActorContext) -> Result<HashMap<String, Message>, anyhow::Error> {
    let payload = context.get_payload();
    let state = context.get_state();
    
    let mut state_guard = state.lock();
    let memory_state = state_guard
        .as_mut_any()
        .downcast_mut::<MemoryState>()
        .expect("Expected MemoryState");
    
    // Initialize batch storage
    if !memory_state.contains_key("batch") {
        memory_state.insert("batch", serde_json::json!([]));
        memory_state.insert("batch_size", serde_json::json!(10)); // Configurable batch size
    }
    
    let batch_size = memory_state.get("batch_size")
        .and_then(|v| v.as_u64())
        .unwrap_or(10) as usize;
    
    let mut current_batch: Vec<serde_json::Value> = memory_state.get("batch")
        .and_then(|v| v.as_array())
        .cloned()
        .unwrap_or_default();
    
    // Handle flush command
    if payload.contains_key("flush") {
        if !current_batch.is_empty() {
            let batch_messages: Vec<Message> = current_batch
                .into_iter()
                .map(|v| Message::from(v))
                .collect();
            
            // Clear batch
            memory_state.insert("batch", serde_json::json!([]));
            
            let count = batch_messages.len();
            println!("Batch Actor: Flushing {} items", count);
            
            return Ok([
                ("batch".to_owned(), Message::Array(batch_messages)),
                ("count".to_owned(), Message::Integer(count as i64)),
            ].into());
        } else {
            return Ok(HashMap::new()); // No items to flush
        }
    }
    
    // Handle new item
    if let Some(item) = payload.get("item") {
        current_batch.push(serde_json::json!(item));
        
        // Check if batch is full
        if current_batch.len() >= batch_size {
            let batch_messages: Vec<Message> = current_batch
                .iter()
                .map(|v| Message::from(v.clone()))
                .collect();
            
            // Clear batch
            memory_state.insert("batch", serde_json::json!([]));
            
            let count = batch_messages.len();
            println!("Batch Actor: Full batch of {} items", count);
            
            Ok([
                ("batch".to_owned(), Message::Array(batch_messages)),
                ("count".to_owned(), Message::Integer(count as i64)),
            ].into())
        } else {
            // Update batch state
            memory_state.insert("batch", serde_json::json!(current_batch));
            
            // Return empty result (batch not ready yet)
            Ok(HashMap::new())
        }
    } else {
        Err(anyhow::anyhow!("Expected item or flush command"))
    }
}
}

Error Handling Patterns

Graceful Error Handling

#![allow(unused)]
fn main() {
#[actor(
    ValidatorActor,
    inports::<100>(data),
    outports::<50>(valid, invalid, error)
)]
async fn validator_actor(context: ActorContext) -> Result<HashMap<String, Message>, anyhow::Error> {
    let payload = context.get_payload();
    
    match payload.get("data") {
        Some(Message::Integer(n)) if *n > 0 => {
            println!("Validator: Valid number {}", n);
            Ok([("valid".to_owned(), Message::Integer(*n))].into())
        },
        Some(Message::Integer(n)) => {
            println!("Validator: Invalid number {} (must be positive)", n);
            Ok([("invalid".to_owned(), Message::Integer(*n))].into())
        },
        Some(other) => {
            let error_msg = format!("Expected integer, got {:?}", other);
            println!("Validator: {}", error_msg);
            Ok([("error".to_owned(), Message::Error(error_msg))].into())
        },
        None => {
            Err(anyhow::anyhow!("Missing data field"))
        }
    }
}

// Retry Actor - implements retry logic with exponential backoff
#[actor(
    RetryActor,
    state(MemoryState),
    inports::<50>(task),
    outports::<25>(success, failure)
)]
async fn retry_actor(context: ActorContext) -> Result<HashMap<String, Message>, anyhow::Error> {
    let payload = context.get_payload();
    let state = context.get_state();
    
    let task = payload.get("task")
        .ok_or_else(|| anyhow::anyhow!("Missing task"))?;
    
    let max_retries = 3;
    let base_delay_ms = 100;
    
    for attempt in 1..=max_retries {
        match simulate_task_processing(task).await {
            Ok(result) => {
                println!("Retry Actor: Task succeeded on attempt {}", attempt);
                return Ok([("success".to_owned(), result)].into());
            },
            Err(e) => {
                if attempt < max_retries {
                    let delay = base_delay_ms * (2_u64.pow(attempt - 1));
                    println!("Retry Actor: Attempt {} failed, retrying in {}ms: {}", 
                            attempt, delay, e);
                    
                    tokio::time::sleep(tokio::time::Duration::from_millis(delay)).await;
                } else {
                    println!("Retry Actor: All {} attempts failed: {}", max_retries, e);
                    return Ok([
                        ("failure".to_owned(), Message::Error(format!("Failed after {} attempts: {}", max_retries, e)))
                    ].into());
                }
            }
        }
    }
    
    unreachable!()
}

async fn simulate_task_processing(task: &Message) -> Result<Message, anyhow::Error> {
    // Simulate processing that might fail
    use rand::Rng;
    let mut rng = rand::thread_rng();
    
    if rng.gen_bool(0.7) { // 70% success rate
        Ok(Message::String(format!("Processed: {:?}", task)))
    } else {
        Err(anyhow::anyhow!("Simulated task failure"))
    }
}
}

Using Actors in Networks

Registration and Instantiation

use reflow_network::{Network, NetworkConfig};
use reflow_network::connector::{Connector, ConnectionPoint, InitialPacket};

#[tokio::main]
async fn main() -> Result<(), anyhow::Error> {
    let mut network = Network::new(NetworkConfig::default());
    
    // Register actor types
    network.register_actor("hello_process", HelloActor::new())?;
    network.register_actor("counter_process", CounterActor::new())?;
    network.register_actor("validator_process", ValidatorActor::new())?;
    
    // Create actor instances
    network.add_node("hello1", "hello_process")?;
    network.add_node("counter1", "counter_process")?;
    network.add_node("validator1", "validator_process")?;
    
    // Connect actors
    network.add_connection(Connector {
        from: ConnectionPoint {
            actor: "hello1".to_owned(),
            port: "output".to_owned(),
            ..Default::default()
        },
        to: ConnectionPoint {
            actor: "validator1".to_owned(),
            port: "data".to_owned(),
            ..Default::default()
        },
    });
    
    // Send initial data
    network.add_initial(InitialPacket {
        to: ConnectionPoint {
            actor: "hello1".to_owned(),
            port: "input".to_owned(),
            initial_data: Some(Message::String("World".to_owned())),
        },
    });
    
    // Start the network
    network.start().await?;
    
    // Wait for processing
    tokio::time::sleep(tokio::time::Duration::from_secs(2)).await;
    
    Ok(())
}

Testing Actors

Unit Testing Actor Functions

#![allow(unused)]
fn main() {
#[cfg(test)]
mod tests {
    use super::*;
    use reflow_network::actor::{ActorContext, MemoryState, ActorLoad};
    use std::sync::Arc;
    use parking_lot::Mutex;
    
    fn create_test_context(payload: HashMap<String, Message>) -> ActorContext {
        let (tx, _rx) = flume::unbounded();
        let state: Arc<Mutex<dyn reflow_network::actor::ActorState>> = 
            Arc::new(Mutex::new(MemoryState::default()));
        
        ActorContext::new(
            payload,
            (tx, _rx),
            state,
            HashMap::new(),
            Arc::new(Mutex::new(ActorLoad::new(0))),
        )
    }
    
    #[tokio::test]
    async fn test_hello_actor() {
        let payload = HashMap::from([
            ("input".to_string(), Message::String("Test".to_string()))
        ]);
        
        let context = create_test_context(payload);
        let result = hello_actor(context).await.unwrap();
        
        assert_eq!(
            result.get("output"),
            Some(&Message::String("Hello, Test!".to_string()))
        );
    }
    
    #[tokio::test]
    async fn test_counter_actor_increment() {
        let payload = HashMap::from([
            ("increment".to_string(), Message::Integer(5))
        ]);
        
        let context = create_test_context(payload);
        let result = counter_actor(context).await.unwrap();
        
        assert_eq!(result.get("count"), Some(&Message::Integer(5)));
        assert_eq!(result.get("total"), Some(&Message::Integer(5)));
    }
    
    #[tokio::test]
    async fn test_greeter_actor() {
        let payload = HashMap::from([
            ("name".to_string(), Message::String("Alice".to_string())),
            ("age".to_string(), Message::Integer(30)),
        ]);
        
        let context = create_test_context(payload);
        let result = greeter_actor(context).await.unwrap();
        
        assert_eq!(
            result.get("greeting"),
            Some(&Message::String("Hello Alice, you are 30 years old!".to_string()))
        );
    }
    
    #[tokio::test]
    async fn test_validator_actor_valid() {
        let payload = HashMap::from([
            ("data".to_string(), Message::Integer(42))
        ]);
        
        let context = create_test_context(payload);
        let result = validator_actor(context).await.unwrap();
        
        assert_eq!(result.get("valid"), Some(&Message::Integer(42)));
        assert!(!result.contains_key("invalid"));
        assert!(!result.contains_key("error"));
    }
    
    #[tokio::test]
    async fn test_validator_actor_invalid() {
        let payload = HashMap::from([
            ("data".to_string(), Message::Integer(-5))
        ]);
        
        let context = create_test_context(payload);
        let result = validator_actor(context).await.unwrap();
        
        assert_eq!(result.get("invalid"), Some(&Message::Integer(-5)));
        assert!(!result.contains_key("valid"));
    }
}
}

Best Practices

Actor Design Guidelines

1. Single Responsibility: Each actor should have one clear purpose
2. Idempotent Processing: Handle duplicate messages gracefully
3. Error Propagation: Use both Result returns and error output ports
4. State Minimal: Keep state minimal and well-defined
5. Port Naming: Use descriptive port names

Performance Tips

#![allow(unused)]
fn main() {
// Use appropriate channel capacities
inports::<1000>(high_volume_input)   // High throughput
inports::<10>(low_volume_input)      // Low throughput

// Batch processing for efficiency
#[actor(
    EfficientProcessor,
    inports::<500>(batch),
    outports::<100>(results)
)]
async fn efficient_processor(context: ActorContext) -> Result<HashMap<String, Message>, anyhow::Error> {
    let payload = context.get_payload();
    
    if let Some(Message::Array(items)) = payload.get("batch") {
        // Process items in parallel
        use futures::stream::{self, StreamExt};
        
        let results: Vec<Message> = stream::iter(items.iter())
            .map(|item| async move {
                process_single_item(item).await
            })
            .buffer_unordered(10) // Process 10 items concurrently
            .collect()
            .await;
        
        Ok([("results".to_owned(), Message::Array(results))].into())
    } else {
        Err(anyhow::anyhow!("Expected batch of items"))
    }
}

async fn process_single_item(item: &Message) -> Message {
    // Simulate processing
    tokio::time::sleep(tokio::time::Duration::from_millis(10)).await;
    item.clone()
}
}

Memory Management

#![allow(unused)]
fn main() {
// Avoid cloning large data when possible
#[actor(
    MemoryEfficientActor,
    inports::<100>(data),
    outports::<50>(processed)
)]
async fn memory_efficient_actor(context: ActorContext) -> Result<HashMap<String, Message>, anyhow::Error> {
    let payload = context.get_payload();
    
    // Process data in-place when possible
    if let Some(Message::Array(items)) = payload.get("data") {
        let count = items.len();
        
        // Instead of cloning all items, just extract what we need
        let summary = Message::Object([
            ("count".to_owned(), Message::Integer(count as i64)),
            ("first_item".to_owned(), items.first().cloned().unwrap_or(Message::None)),
            ("last_item".to_owned(), items.last().cloned().unwrap_or(Message::None)),
        ].into());
        
        Ok([("processed".to_owned(), summary)].into())
    } else {
        Err(anyhow::anyhow!("Expected array data"))
    }
}
}

Manual Actor Implementation

For maximum control or when the macro limitations are insufficient, you can implement the Actor trait manually. This approach gives you complete control over the actor's behavior and lifecycle.

Basic Manual Actor

#![allow(unused)]
fn main() {
use reflow_network::actor::{Actor, ActorBehavior, ActorContext, Port, MemoryState, ActorLoad};
use reflow_network::message::Message;
use std::collections::HashMap;
use std::sync::Arc;
use parking_lot::Mutex;
use std::pin::Pin;
use std::future::Future;

pub struct ManualActor {
    inports: Port,
    outports: Port,
    name: String,
    load: Arc<Mutex<ActorLoad>>,
}

impl ManualActor {
    pub fn new(name: String) -> Self {
        Self {
            inports: flume::unbounded(),
            outports: flume::unbounded(),
            name,
            load: Arc::new(Mutex::new(ActorLoad::new(0))),
        }
    }
}

impl Actor for ManualActor {
    fn get_behavior(&self) -> ActorBehavior {
        let name = self.name.clone();
        
        Box::new(move |context: ActorContext| {
            let name = name.clone();
            
            Box::pin(async move {
                let payload = context.get_payload();
                
                if let Some(Message::String(text)) = payload.get("input") {
                    let response = format!("{}: Processing '{}'", name, text);
                    println!("{}", response);
                    
                    Ok([
                        ("output".to_owned(), Message::String(response))
                    ].into())
                } else {
                    Err(anyhow::anyhow!("Expected string input"))
                }
            })
        })
    }
    
    fn get_inports(&self) -> Port {
        self.inports.clone()
    }
    
    fn get_outports(&self) -> Port {
        self.outports.clone()
    }
    
    fn load_count(&self) -> Arc<Mutex<ActorLoad>> {
        self.load.clone()
    }
    
    fn create_process(&self) -> Pin<Box<dyn Future<Output = ()> + 'static + Send>> {
        let inports = self.get_inports();
        let behavior = self.get_behavior();
        let outports = self.get_outports();
        let state: Arc<Mutex<dyn reflow_network::actor::ActorState>> = 
            Arc::new(Mutex::new(MemoryState::default()));
        let load_count = self.load_count();
        
        Box::pin(async move {
            use futures::stream::StreamExt;
            
            loop {
                if let Some(payload) = inports.1.stream().next().await {
                    // Increment load count
                    {
                        let mut load = load_count.lock();
                        load.inc();
                    }
                    
                    let context = ActorContext::new(
                        payload,
                        outports.clone(),
                        state.clone(),
                        HashMap::new(),
                        load_count.clone(),
                    );
                    
                    match behavior(context).await {
                        Ok(result) => {
                            if !result.is_empty() {
                                let _ = outports.0.send(result);
                            }
                        },
                        Err(e) => {
                            eprintln!("Error in actor behavior: {:?}", e);
                        }
                    }
                    
                    // Decrement load count
                    {
                        let mut load = load_count.lock();
                        load.dec();
                    }
                }
            }
        })
    }
}
}

Stateful Manual Actor

#![allow(unused)]
fn main() {
use serde::{Deserialize, Serialize};

#[derive(Debug, Clone, Serialize, Deserialize, Default)]
pub struct CustomState {
    pub counter: i64,
    pub last_message: String,
    pub timestamps: Vec<i64>,
}

impl reflow_network::actor::ActorState for CustomState {
    fn as_mut_any(&mut self) -> &mut dyn std::any::Any {
        self
    }
    
    fn as_any(&self) -> &dyn std::any::Any {
        self
    }
}

pub struct StatefulManualActor {
    inports: Port,
    outports: Port,
    initial_state: CustomState,
    load: Arc<Mutex<ActorLoad>>,
}

impl StatefulManualActor {
    pub fn new(initial_state: CustomState) -> Self {
        Self {
            inports: flume::unbounded(),
            outports: flume::unbounded(),
            initial_state,
            load: Arc::new(Mutex::new(ActorLoad::new(0))),
        }
    }
}

impl Actor for StatefulManualActor {
    fn get_behavior(&self) -> ActorBehavior {
        Box::new(|context: ActorContext| {
            Box::pin(async move {
                let payload = context.get_payload();
                let state = context.get_state();
                
                let mut state_guard = state.lock();
                let custom_state = state_guard
                    .as_mut_any()
                    .downcast_mut::<CustomState>()
                    .expect("Expected CustomState");
                
                // Update counter
                custom_state.counter += 1;
                
                // Record timestamp
                let now = chrono::Utc::now().timestamp_millis();
                custom_state.timestamps.push(now);
                
                // Keep only last 10 timestamps
                if custom_state.timestamps.len() > 10 {
                    custom_state.timestamps.remove(0);
                }
                
                // Process message
                if let Some(Message::String(text)) = payload.get("message") {
                    custom_state.last_message = text.clone();
                    
                    let response = format!(
                        "Processed message #{}: '{}' (last 5 timestamps: {:?})",
                        custom_state.counter,
                        text,
                        custom_state.timestamps.iter().rev().take(5).collect::<Vec<_>>()
                    );
                    
                    Ok([
                        ("response".to_owned(), Message::String(response)),
                        ("counter".to_owned(), Message::Integer(custom_state.counter)),
                    ].into())
                } else {
                    Err(anyhow::anyhow!("Expected message field"))
                }
            })
        })
    }
    
    fn get_inports(&self) -> Port {
        self.inports.clone()
    }
    
    fn get_outports(&self) -> Port {
        self.outports.clone()
    }
    
    fn load_count(&self) -> Arc<Mutex<ActorLoad>> {
        self.load.clone()
    }
    
    fn create_process(&self) -> Pin<Box<dyn Future<Output = ()> + 'static + Send>> {
        let inports = self.get_inports();
        let behavior = self.get_behavior();
        let outports = self.get_outports();
        let state: Arc<Mutex<dyn reflow_network::actor::ActorState>> = 
            Arc::new(Mutex::new(self.initial_state.clone()));
        let load_count = self.load_count();
        
        Box::pin(async move {
            use futures::stream::StreamExt;
            
            loop {
                if let Some(payload) = inports.1.stream().next().await {
                    // Increment load count
                    {
                        let mut load = load_count.lock();
                        load.inc();
                    }
                    
                    let context = ActorContext::new(
                        payload,
                        outports.clone(),
                        state.clone(),
                        HashMap::new(),
                        load_count.clone(),
                    );
                    
                    match behavior(context).await {
                        Ok(result) => {
                            if !result.is_empty() {
                                let _ = outports.0.send(result);
                            }
                        },
                        Err(e) => {
                            eprintln!("Error in stateful actor behavior: {:?}", e);
                        }
                    }
                    
                    // Decrement load count
                    {
                        let mut load = load_count.lock();
                        load.dec();
                    }
                }
            }
        })
    }
}
}

Multi-Input Manual Actor

#![allow(unused)]
fn main() {
pub struct MultiInputActor {
    inports: Port,
    outports: Port,
    await_all_inputs: bool,
    input_ports: Vec<String>,
    load: Arc<Mutex<ActorLoad>>,
}

impl MultiInputActor {
    pub fn new(input_ports: Vec<String>, await_all_inputs: bool) -> Self {
        Self {
            inports: flume::bounded(100),
            outports: flume::bounded(50),
            await_all_inputs,
            input_ports,
            load: Arc::new(Mutex::new(ActorLoad::new(0))),
        }
    }
}

impl Actor for MultiInputActor {
    fn get_behavior(&self) -> ActorBehavior {
        Box::new(|context: ActorContext| {
            Box::pin(async move {
                let payload = context.get_payload();
                
                // Collect all available data
                let mut results = HashMap::new();
                let mut total_value = 0i64;
                let mut value_count = 0;
                
                for (port, message) in &payload {
                    if let Message::Integer(value) = message {
                        total_value += value;
                        value_count += 1;
                        
                        results.insert(
                            format!("processed_{}", port), 
                            Message::Integer(value * 2)
                        );
                    }
                }
                
                if value_count > 0 {
                    results.insert("sum".to_owned(), Message::Integer(total_value));
                    results.insert("average".to_owned(), Message::Integer(total_value / value_count));
                    results.insert("count".to_owned(), Message::Integer(value_count));
                }
                
                println!("MultiInput Actor: processed {} values, sum = {}", value_count, total_value);
                
                Ok(results)
            })
        })
    }
    
    fn get_inports(&self) -> Port {
        self.inports.clone()
    }
    
    fn get_outports(&self) -> Port {
        self.outports.clone()
    }
    
    fn load_count(&self) -> Arc<Mutex<ActorLoad>> {
        self.load.clone()
    }
    
    fn create_process(&self) -> Pin<Box<dyn Future<Output = ()> + 'static + Send>> {
        let inports = self.get_inports();
        let behavior = self.get_behavior();
        let outports = self.get_outports();
        let state: Arc<Mutex<dyn reflow_network::actor::ActorState>> = 
            Arc::new(Mutex::new(MemoryState::default()));
        let load_count = self.load_count();
        let await_all_inputs = self.await_all_inputs;
        let input_ports_count = self.input_ports.len();
        
        Box::pin(async move {
            use futures::stream::StreamExt;
            let mut all_inputs: HashMap<String, Message> = HashMap::new();
            
            loop {
                if let Some(packet) = inports.1.stream().next().await {
                    // Increment load count
                    {
                        let mut load = load_count.lock();
                        load.inc();
                    }
                    
                    if await_all_inputs {
                        // Accumulate inputs until we have all expected ports
                        all_inputs.extend(packet);
                        
                        if all_inputs.len() >= input_ports_count {
                            let context = ActorContext::new(
                                all_inputs.clone(),
                                outports.clone(),
                                state.clone(),
                                HashMap::new(),
                                load_count.clone(),
                            );
                            
                            match behavior(context).await {
                                Ok(result) => {
                                    if !result.is_empty() {
                                        let _ = outports.0.send(result);
                                    }
                                },
                                Err(e) => {
                                    eprintln!("Error in multi-input actor behavior: {:?}", e);
                                }
                            }
                            
                            all_inputs.clear();
                        } else {
                            // Continue without decrementing load count
                            {
                                let mut load = load_count.lock();
                                load.dec();
                            }
                            continue;
                        }
                    } else {
                        // Process immediately
                        let context = ActorContext::new(
                            packet,
                            outports.clone(),
                            state.clone(),
                            HashMap::new(),
                            load_count.clone(),
                        );
                        
                        match behavior(context).await {
                            Ok(result) => {
                                if !result.is_empty() {
                                    let _ = outports.0.send(result);
                                }
                            },
                            Err(e) => {
                                eprintln!("Error in multi-input actor behavior: {:?}", e);
                            }
                        }
                    }
                    
                    // Decrement load count
                    {
                        let mut load = load_count.lock();
                        load.dec();
                    }
                }
            }
        })
    }
}
}

When to Use Manual Implementation

Use manual implementation when:

1. Complex State Requirements: You need custom state types or complex state initialization
2. Custom Port Logic: You need dynamic port creation or complex routing logic
3. Advanced Error Handling: You need sophisticated error recovery or circuit breaker patterns
4. Performance Optimization: You need fine-grained control over message processing
5. Integration Requirements: You need to integrate with external systems in specific ways

Example: Circuit Breaker Actor

#![allow(unused)]
fn main() {
#[derive(Debug, Clone)]
enum CircuitState {
    Closed,   // Normal operation
    Open,     // Failing, rejecting requests
    HalfOpen, // Testing if service recovered
}

pub struct CircuitBreakerActor {
    inports: Port,
    outports: Port,
    failure_threshold: u32,
    timeout_ms: u64,
    load: Arc<Mutex<ActorLoad>>,
}

impl CircuitBreakerActor {
    pub fn new(failure_threshold: u32, timeout_ms: u64) -> Self {
        Self {
            inports: flume::unbounded(),
            outports: flume::unbounded(),
            failure_threshold,
            timeout_ms,
            load: Arc::new(Mutex::new(ActorLoad::new(0))),
        }
    }
}

impl Actor for CircuitBreakerActor {
    fn get_behavior(&self) -> ActorBehavior {
        let failure_threshold = self.failure_threshold;
        let timeout_ms = self.timeout_ms;
        
        Box::new(move |context: ActorContext| {
            Box::pin(async move {
                let payload = context.get_payload();
                let state = context.get_state();
                
                let mut state_guard = state.lock();
                let memory_state = state_guard
                    .as_mut_any()
                    .downcast_mut::<MemoryState>()
                    .expect("Expected MemoryState");
                
                // Initialize circuit breaker state
                if !memory_state.contains_key("circuit_state") {
                    memory_state.insert("circuit_state", serde_json::json!("Closed"));
                    memory_state.insert("failure_count", serde_json::json!(0));
                    memory_state.insert("last_failure_time", serde_json::json!(0));
                }
                
                let circuit_state_str = memory_state.get("circuit_state")
                    .and_then(|v| v.as_str())
                    .unwrap_or("Closed");
                
                let failure_count = memory_state.get("failure_count")
                    .and_then(|v| v.as_u64())
                    .unwrap_or(0) as u32;
                
                let last_failure_time = memory_state.get("last_failure_time")
                    .and_then(|v| v.as_i64())
                    .unwrap_or(0);
                
                let circuit_state = match circuit_state_str {
                    "Open" => CircuitState::Open,
                    "HalfOpen" => CircuitState::HalfOpen,
                    _ => CircuitState::Closed,
                };
                
                let now = chrono::Utc::now().timestamp_millis();
                
                match circuit_state {
                    CircuitState::Open => {
                        // Check if timeout has passed
                        if now - last_failure_time > timeout_ms as i64 {
                            memory_state.insert("circuit_state", serde_json::json!("HalfOpen"));
                            println!("Circuit breaker: Transitioning to HalfOpen");
                        } else {
                            return Ok([
                                ("rejected".to_owned(), 
                                 Message::Error("Circuit breaker is OPEN".to_string()))
                            ].into());
                        }
                    },
                    CircuitState::HalfOpen => {
                        // Process one request to test
                    },
                    CircuitState::Closed => {
                        // Normal operation
                    }
                }
                
                // Simulate processing the request
                if let Some(request) = payload.get("request") {
                    // Simulate success/failure (in real implementation, you'd call actual service)
                    let success = payload.get("simulate_success")
                        .and_then(|v| if let Message::Boolean(b) = v { Some(*b) } else { None })
                        .unwrap_or(true);
                    
                    if success {
                        // Success - reset failure count if in HalfOpen
                        if matches!(circuit_state, CircuitState::HalfOpen) {
                            memory_state.insert("circuit_state", serde_json::json!("Closed"));
                            memory_state.insert("failure_count", serde_json::json!(0));
                            println!("Circuit breaker: Transitioning to Closed");
                        }
                        
                        Ok([
                            ("success".to_owned(), Message::String("Request processed".to_string()))
                        ].into())
                    } else {
                        // Failure
                        let new_failure_count = failure_count + 1;
                        memory_state.insert("failure_count", serde_json::json!(new_failure_count));
                        memory_state.insert("last_failure_time", serde_json::json!(now));
                        
                        if new_failure_count >= failure_threshold {
                            memory_state.insert("circuit_state", serde_json::json!("Open"));
                            println!("Circuit breaker: Transitioning to Open");
                        }
                        
                        Ok([
                            ("failure".to_owned(), 
                             Message::Error(format!("Request failed (failures: {})", new_failure_count)))
                        ].into())
                    }
                } else {
                    Err(anyhow::anyhow!("Missing request"))
                }
            })
        })
    }
    
    fn get_inports(&self) -> Port { self.inports.clone() }
    fn get_outports(&self) -> Port { self.outports.clone() }
    fn load_count(&self) -> Arc<Mutex<ActorLoad>> { self.load.clone() }
    
    fn create_process(&self) -> Pin<Box<dyn Future<Output = ()> + 'static + Send>> {
        let inports = self.get_inports();
        let behavior = self.get_behavior();
        let outports = self.get_outports();
        let state: Arc<Mutex<dyn reflow_network::actor::ActorState>> = 
            Arc::new(Mutex::new(MemoryState::default()));
        let load_count = self.load_count();
        
        Box::pin(async move {
            use futures::stream::StreamExt;
            
            loop {
                if let Some(payload) = inports.1.stream().next().await {
                    {
                        let mut load = load_count.lock();
                        load.inc();
                    }
                    
                    let context = ActorContext::new(
                        payload,
                        outports.clone(),
                        state.clone(),
                        HashMap::new(),
                        load_count.clone(),
                    );
                    
                    match behavior(context).await {
                        Ok(result) => {
                            if !result.is_empty() {
                                let _ = outports.0.send(result);
                            }
                        },
                        Err(e) => {
                            eprintln!("Error in circuit breaker: {:?}", e);
                        }
                    }
                    
                    {
                        let mut load = load_count.lock();
                        load.dec();
                    }
                }
            }
        })
    }
}
}

Testing Manual Actors

#![allow(unused)]
fn main() {
#[cfg(test)]
mod manual_actor_tests {
    use super::*;
    
    #[tokio::test]
    async fn test_manual_actor() {
        let actor = ManualActor::new("TestActor".to_string());
        let behavior = actor.get_behavior();
        
        let payload = HashMap::from([
            ("input".to_string(), Message::String("test".to_string()))
        ]);
        
        let (tx, _rx) = flume::unbounded();
        let state: Arc<Mutex<dyn reflow_network::actor::ActorState>> = 
            Arc::new(Mutex::new(MemoryState::default()));
        
        let context = ActorContext::new(
            payload,
            (tx, _rx),
            state,
            HashMap::new(),
            Arc::new(Mutex::new(ActorLoad::new(0))),
        );
        
        let result = behavior(context).await.unwrap();
        
        assert!(result.contains_key("output"));
        if let Some(Message::String(output)) = result.get("output") {
            assert!(output.contains("TestActor"));
            assert!(output.contains("test"));
        }
    }
    
    #[tokio::test]
    async fn test_stateful_manual_actor() {
        let initial_state = CustomState {
            counter: 0,
            last_message: String::new(),
            timestamps: Vec::new(),
        };
        
        let actor = StatefulManualActor::new(initial_state);
        let behavior = actor.get_behavior();
        
        let payload = HashMap::from([
            ("message".to_string(), Message::String("hello".to_string()))
        ]);
        
        let (tx, _rx) = flume::unbounded();
        let state: Arc<Mutex<dyn reflow_network::actor::ActorState>> = 
            Arc::new(Mutex::new(CustomState::default()));
        
        let context = ActorContext::new(
            payload,
            (tx, _rx),
            state,
            HashMap::new(),
            Arc::new(Mutex::new(ActorLoad::new(0))),
        );
        
        let result = behavior(context).await.unwrap();
        
        assert_eq!(result.get("counter"), Some(&Message::Integer(1)));
        assert!(result.contains_key("response"));
    }
}
}

Choosing Between Macro and Manual Implementation

Use Actor Macro When:

• Simple, stateless processing
• Standard input/output patterns
• Rapid prototyping
• Most common use cases

Use Manual Implementation When:

• Complex state management
• Custom error handling strategies
• Performance-critical applications
• Integration with external systems
• Advanced patterns (circuit breakers, rate limiting, etc.)

Next Steps

• State Management: Advanced State Patterns
• Network Integration: Building Workflows
• Performance: Actor Optimization
• Examples: Real-World Examples

ActorConfig System

The ActorConfig system provides a unified configuration framework for all actors in Reflow, enabling dynamic configuration, runtime parameter adjustment, and consistent actor behavior across different deployment environments.

Overview

ActorConfig replaces the previous ad-hoc configuration approach with a structured, type-safe system that supports:

• Type-Safe Configuration: Strongly typed configuration parameters with validation
• Dynamic Updates: Runtime configuration changes without actor restart
• Environment Variables: Automatic environment variable injection
• JSON/YAML Support: Flexible configuration file formats
• Validation & Defaults: Built-in validation with sensible defaults
• Metadata Integration: Rich metadata for configuration documentation

Basic Usage

Simple Actor Configuration

#![allow(unused)]
fn main() {
use reflow_network::actor::{Actor, ActorConfig, ActorContext};
use std::collections::HashMap;

#[derive(Debug)]
struct ProcessorActor {
    config: ActorConfig,
}

impl ProcessorActor {
    fn new() -> Self {
        Self {
            config: ActorConfig::default(),
        }
    }
}

impl Actor for ProcessorActor {
    fn create_process(&self, config: ActorConfig) -> Pin<Box<dyn Future<Output = ()> + Send + 'static>> {
        // Extract configuration values
        let batch_size = config.get_number("batch_size").unwrap_or(10.0) as usize;
        let timeout_ms = config.get_number("timeout_ms").unwrap_or(5000.0) as u64;
        let enable_retry = config.get_boolean("enable_retry").unwrap_or(true);
        let processor_name = config.get_string("name").unwrap_or("default_processor".to_string().into());
        
        Box::pin(async move {
            println!("Processor {} starting with batch_size={}, timeout={}ms, retry={}", 
                processor_name, batch_size, timeout_ms, enable_retry);
            
            // Actor implementation using configuration...
        })
    }
    
    // ... other Actor trait methods
}
}

Configuration from JSON

{
  "name": "data_processor",
  "batch_size": 50,
  "timeout_ms": 10000,
  "enable_retry": true,
  "processing_mode": "parallel",
  "max_retries": 3,
  "retry_delay_ms": 1000
}

#![allow(unused)]
fn main() {
// Load configuration from JSON
let config_json = r#"
{
  "name": "data_processor",
  "batch_size": 50,
  "timeout_ms": 10000,
  "enable_retry": true,
  "processing_mode": "parallel"
}
"#;

let config = ActorConfig::from_json(config_json)?;
let actor = ProcessorActor::new();

// Use configuration when creating actor process
let process = actor.create_process(config);
tokio::spawn(process);
}

Configuration Sources

Environment Variables

ActorConfig automatically reads from environment variables with configurable prefixes:

#![allow(unused)]
fn main() {
// Environment variables:
// PROCESSOR_BATCH_SIZE=100
// PROCESSOR_TIMEOUT_MS=15000
// PROCESSOR_ENABLE_RETRY=false

let config = ActorConfig::from_env("PROCESSOR")?;

// Access values
let batch_size = config.get_number("batch_size").unwrap(); // 100.0
let timeout = config.get_number("timeout_ms").unwrap();   // 15000.0
let retry = config.get_boolean("enable_retry").unwrap();  // false
}

Configuration Files

#![allow(unused)]
fn main() {
// From YAML file
let config = ActorConfig::from_yaml_file("configs/processor.yaml").await?;

// From JSON file
let config = ActorConfig::from_json_file("configs/processor.json").await?;

// From TOML file
let config = ActorConfig::from_toml_file("configs/processor.toml").await?;
}

Combined Sources with Precedence

#![allow(unused)]
fn main() {
// Build configuration with precedence: CLI args > env vars > config file > defaults
let config = ActorConfig::builder()
    .from_file("configs/defaults.yaml").await?
    .from_env("PROCESSOR")?
    .from_args(std::env::args())?
    .build()?;
}

Configuration Schema and Validation

Defining Configuration Schema

#![allow(unused)]
fn main() {
use reflow_network::actor::{ActorConfigSchema, ConfigField, ConfigType};
use serde::{Deserialize, Serialize};

#[derive(Debug, Serialize, Deserialize)]
struct ProcessorConfigSchema {
    #[serde(default = "default_batch_size")]
    batch_size: u32,
    
    #[serde(default = "default_timeout")]
    timeout_ms: u64,
    
    #[serde(default)]
    enable_retry: bool,
    
    #[serde(default = "default_name")]
    name: String,
    
    processing_mode: ProcessingMode,
}

#[derive(Debug, Serialize, Deserialize)]
enum ProcessingMode {
    Sequential,
    Parallel,
    Batch,
}

fn default_batch_size() -> u32 { 10 }
fn default_timeout() -> u64 { 5000 }
fn default_name() -> String { "processor".to_string() }

impl ActorConfigSchema for ProcessorConfigSchema {
    fn schema() -> Vec<ConfigField> {
        vec![
            ConfigField {
                name: "batch_size".to_string(),
                config_type: ConfigType::Number,
                required: false,
                default_value: Some(serde_json::Value::Number(10.into())),
                description: Some("Number of items to process in each batch".to_string()),
                validation: Some("Must be between 1 and 1000".to_string()),
            },
            ConfigField {
                name: "timeout_ms".to_string(),
                config_type: ConfigType::Number,
                required: false,
                default_value: Some(serde_json::Value::Number(5000.into())),
                description: Some("Processing timeout in milliseconds".to_string()),
                validation: Some("Must be positive".to_string()),
            },
            ConfigField {
                name: "enable_retry".to_string(),
                config_type: ConfigType::Boolean,
                required: false,
                default_value: Some(serde_json::Value::Bool(false)),
                description: Some("Enable automatic retry on failure".to_string()),
                validation: None,
            },
            ConfigField {
                name: "name".to_string(),
                config_type: ConfigType::String,
                required: false,
                default_value: Some(serde_json::Value::String("processor".to_string())),
                description: Some("Actor instance name".to_string()),
                validation: Some("Must be non-empty alphanumeric".to_string()),
            },
            ConfigField {
                name: "processing_mode".to_string(),
                config_type: ConfigType::String,
                required: true,
                default_value: None,
                description: Some("Processing execution mode".to_string()),
                validation: Some("Must be one of: sequential, parallel, batch".to_string()),
            },
        ]
    }
    
    fn validate(&self) -> Result<(), String> {
        if self.batch_size == 0 || self.batch_size > 1000 {
            return Err("batch_size must be between 1 and 1000".to_string());
        }
        
        if self.timeout_ms == 0 {
            return Err("timeout_ms must be positive".to_string());
        }
        
        if self.name.is_empty() {
            return Err("name cannot be empty".to_string());
        }
        
        Ok(())
    }
}
}

Using Typed Configuration

#![allow(unused)]
fn main() {
impl Actor for ProcessorActor {
    fn create_process(&self, config: ActorConfig) -> Pin<Box<dyn Future<Output = ()> + Send + 'static>> {
        // Parse and validate configuration against schema
        let typed_config: ProcessorConfigSchema = config.parse_typed()?;
        
        // Configuration is now type-safe and validated
        let batch_size = typed_config.batch_size;
        let timeout = Duration::from_millis(typed_config.timeout_ms);
        let enable_retry = typed_config.enable_retry;
        let name = typed_config.name;
        let mode = typed_config.processing_mode;
        
        Box::pin(async move {
            match mode {
                ProcessingMode::Sequential => {
                    // Sequential processing logic
                },
                ProcessingMode::Parallel => {
                    // Parallel processing logic
                },
                ProcessingMode::Batch => {
                    // Batch processing logic
                },
            }
        })
    }
}
}

Dynamic Configuration Updates

Runtime Configuration Changes

#![allow(unused)]
fn main() {
use tokio::sync::watch;

struct DynamicProcessorActor {
    config_receiver: watch::Receiver<ActorConfig>,
}

impl DynamicProcessorActor {
    fn new(config_receiver: watch::Receiver<ActorConfig>) -> Self {
        Self { config_receiver }
    }
}

impl Actor for DynamicProcessorActor {
    fn create_process(&self, initial_config: ActorConfig) -> Pin<Box<dyn Future<Output = ()> + Send + 'static>> {
        let mut config_receiver = self.config_receiver.clone();
        
        Box::pin(async move {
            let mut current_config = initial_config;
            
            loop {
                // Check for configuration updates
                if config_receiver.has_changed().unwrap_or(false) {
                    current_config = config_receiver.borrow().clone();
                    println!("Configuration updated: {:?}", current_config);
                    
                    // Apply new configuration
                    let batch_size = current_config.get_number("batch_size").unwrap_or(10.0) as usize;
                    println!("New batch size: {}", batch_size);
                }
                
                // Process with current configuration
                // ... actor logic ...
                
                tokio::time::sleep(Duration::from_millis(100)).await;
            }
        })
    }
}

// Update configuration at runtime
async fn update_actor_config() -> Result<(), Box<dyn std::error::Error>> {
    let (config_sender, config_receiver) = watch::channel(ActorConfig::default());
    
    let actor = DynamicProcessorActor::new(config_receiver);
    tokio::spawn(actor.create_process(ActorConfig::default()));
    
    // Update configuration after 5 seconds
    tokio::time::sleep(Duration::from_secs(5)).await;
    
    let new_config = ActorConfig::from_json(r#"
    {
        "batch_size": 100,
        "timeout_ms": 20000,
        "enable_retry": true
    }
    "#)?;
    
    config_sender.send(new_config)?;
    println!("Configuration updated!");
    
    Ok(())
}
}

Configuration in Networks and Graphs

Network-Level Configuration

#![allow(unused)]
fn main() {
use reflow_network::network::{Network, NetworkConfig};

// Configure network with global defaults
let network_config = NetworkConfig {
    default_actor_config: Some(ActorConfig::from_json(r#"
    {
        "default_timeout_ms": 10000,
        "enable_monitoring": true,
        "log_level": "info"
    }
    "#)?),
    // ... other network config
};

let mut network = Network::new(network_config);

// Add actor with specific configuration
let actor_config = ActorConfig::from_json(r#"
{
    "batch_size": 50,
    "timeout_ms": 15000,
    "name": "data_processor_1"
}
"#)?;

network.add_node_with_config("processor1", "DataProcessorActor", Some(actor_config))?;
}

Graph-Level Configuration

{
  "caseSensitive": false,
  "properties": {
    "name": "data_processing_pipeline"
  },
  "processes": {
    "collector": {
      "component": "DataCollectorActor",
      "metadata": {
        "config": {
          "source_url": "https://api.example.com/data",
          "poll_interval_ms": 5000,
          "batch_size": 100
        }
      }
    },
    "processor": {
      "component": "DataProcessorActor",
      "metadata": {
        "config": {
          "processing_mode": "parallel",
          "worker_count": 4,
          "timeout_ms": 30000
        }
      }
    },
    "validator": {
      "component": "DataValidatorActor",
      "metadata": {
        "config": {
          "strict_validation": true,
          "schema_file": "./schemas/data.json"
        }
      }
    }
  },
  "connections": [
    {
      "from": { "nodeId": "collector", "portId": "Output" },
      "to": { "nodeId": "processor", "portId": "Input" }
    },
    {
      "from": { "nodeId": "processor", "portId": "Output" },
      "to": { "nodeId": "validator", "portId": "Input" }
    }
  ]
}

Loading Graph with Configurations

#![allow(unused)]
fn main() {
use reflow_network::graph::Graph;

// Load graph - configurations are automatically extracted from metadata
let graph = Graph::load_from_file("data_pipeline.graph.json").await?;

// Each actor will receive its specific configuration
// Network automatically extracts config from process metadata
let mut network = Network::new(NetworkConfig::default());
network.load_graph(graph).await?;
}

Environment-Specific Configurations

Development vs Production

#![allow(unused)]
fn main() {
// Development configuration
let dev_config = ActorConfig::from_json(r#"
{
    "log_level": "debug",
    "enable_profiling": true,
    "timeout_ms": 60000,
    "batch_size": 5
}
"#)?;

// Production configuration
let prod_config = ActorConfig::from_json(r#"
{
    "log_level": "warn",
    "enable_profiling": false,
    "timeout_ms": 10000,
    "batch_size": 100
}
"#)?;

// Select configuration based on environment
let config = match std::env::var("ENVIRONMENT").as_deref() {
    Ok("production") => prod_config,
    Ok("staging") => prod_config, // Use prod config for staging
    _ => dev_config, // Default to dev config
};
}

Configuration Profiles

#![allow(unused)]
fn main() {
// Base configuration
let base_config = ActorConfig::from_yaml_file("configs/base.yaml").await?;

// Environment-specific overrides
let env = std::env::var("ENVIRONMENT").unwrap_or_else(|_| "development".to_string());
let env_config_path = format!("configs/{}.yaml", env);

let final_config = if std::path::Path::new(&env_config_path).exists() {
    base_config.merge_with(ActorConfig::from_yaml_file(&env_config_path).await?)?
} else {
    base_config
};
}

Configuration Migration

Migrating from Direct HashMap

Before (Old Pattern):

#![allow(unused)]
fn main() {
// Old approach - direct HashMap usage
impl Actor for OldActor {
    fn set_config(&mut self, config: HashMap<String, serde_json::Value>) {
        self.batch_size = config.get("batch_size")
            .and_then(|v| v.as_f64())
            .unwrap_or(10.0) as usize;
        
        self.timeout = Duration::from_millis(
            config.get("timeout_ms")
                .and_then(|v| v.as_f64())
                .unwrap_or(5000.0) as u64
        );
    }
}
}

After (New Pattern):

#![allow(unused)]
fn main() {
// New approach - ActorConfig
impl Actor for NewActor {
    fn create_process(&self, config: ActorConfig) -> Pin<Box<dyn Future<Output = ()> + Send + 'static>> {
        let batch_size = config.get_number("batch_size").unwrap_or(10.0) as usize;
        let timeout = Duration::from_millis(config.get_number("timeout_ms").unwrap_or(5000.0) as u64);
        
        Box::pin(async move {
            // Actor implementation with configuration
        })
    }
}
}

Migration Helper

#![allow(unused)]
fn main() {
// Helper function to migrate from old HashMap format
impl ActorConfig {
    pub fn from_legacy_hashmap(legacy: HashMap<String, serde_json::Value>) -> Self {
        let mut config = ActorConfig::default();
        
        for (key, value) in legacy {
            config.set(&key, value);
        }
        
        config
    }
}

// Usage in migration
let legacy_config = HashMap::from([
    ("batch_size".to_string(), serde_json::Value::Number(50.into())),
    ("timeout_ms".to_string(), serde_json::Value::Number(10000.into())),
]);

let actor_config = ActorConfig::from_legacy_hashmap(legacy_config);
}

Advanced Features

Conditional Configuration

#![allow(unused)]
fn main() {
#[derive(Debug, Serialize, Deserialize)]
struct ConditionalConfig {
    #[serde(default)]
    enable_cache: bool,
    
    #[serde(skip_serializing_if = "Option::is_none")]
    cache_size_mb: Option<u32>,
    
    #[serde(skip_serializing_if = "Option::is_none")]
    cache_ttl_seconds: Option<u64>,
}

impl ActorConfigSchema for ConditionalConfig {
    fn validate(&self) -> Result<(), String> {
        if self.enable_cache {
            if self.cache_size_mb.is_none() {
                return Err("cache_size_mb is required when cache is enabled".to_string());
            }
            if self.cache_ttl_seconds.is_none() {
                return Err("cache_ttl_seconds is required when cache is enabled".to_string());
            }
        }
        Ok(())
    }
}
}

Configuration Inheritance

#![allow(unused)]
fn main() {
// Base actor configuration
let base_config = ActorConfig::from_json(r#"
{
    "timeout_ms": 10000,
    "enable_logging": true,
    "log_level": "info"
}
"#)?;

// Specialized configuration inheriting from base
let specialized_config = base_config.extend_with(ActorConfig::from_json(r#"
{
    "batch_size": 50,
    "processing_mode": "parallel",
    "timeout_ms": 20000
}
"#)?)?;

// Result combines both configs with specialized values taking precedence
// timeout_ms: 20000 (overridden)
// enable_logging: true (inherited)
// log_level: "info" (inherited)  
// batch_size: 50 (added)
// processing_mode: "parallel" (added)
}

Secret Management

#![allow(unused)]
fn main() {
use reflow_network::actor::SecretResolver;

// Configuration with secret references
let config_with_secrets = ActorConfig::from_json(r#"
{
    "database_url": "${secret:DATABASE_URL}",
    "api_key": "${secret:API_KEY}",
    "batch_size": 100
}
"#)?;

// Resolve secrets from environment or secret store
let secret_resolver = SecretResolver::new()
    .with_env_prefix("SECRET_")
    .with_vault_client(vault_client);

let resolved_config = secret_resolver.resolve(config_with_secrets).await?;

// Secrets are now resolved:
// database_url: "postgresql://user:password@localhost/db"  
// api_key: "sk-1234567890abcdef"
// batch_size: 100
}

Testing with ActorConfig

Test Configuration Helpers

#![allow(unused)]
fn main() {
use reflow_network::actor::testing::TestActorConfig;

#[tokio::test]
async fn test_actor_with_config() {
    let test_config = TestActorConfig::builder()
        .with_number("batch_size", 10.0)
        .with_boolean("enable_retry", false)
        .with_string("name", "test_actor")
        .build();
    
    let actor = MyActor::new();
    let process = actor.create_process(test_config.into());
    
    // Test actor behavior with specific configuration
    // ...
}

#[tokio::test]
async fn test_actor_configuration_validation() {
    let invalid_config = ActorConfig::from_json(r#"
    {
        "batch_size": -1,
        "timeout_ms": 0
    }
    "#).unwrap();
    
    let schema = MyActorConfigSchema::default();
    assert!(schema.validate_config(&invalid_config).is_err());
}
}

Configuration Mocking

#![allow(unused)]
fn main() {
// Mock configuration for testing
struct MockConfigProvider {
    configs: HashMap<String, ActorConfig>,
}

impl MockConfigProvider {
    fn new() -> Self {
        Self {
            configs: HashMap::new(),
        }
    }
    
    fn add_config(&mut self, actor_id: &str, config: ActorConfig) {
        self.configs.insert(actor_id.to_string(), config);
    }
}

impl ConfigProvider for MockConfigProvider {
    async fn get_config(&self, actor_id: &str) -> Result<ActorConfig, ConfigError> {
        self.configs.get(actor_id)
            .cloned()
            .ok_or_else(|| ConfigError::NotFound(actor_id.to_string()))
    }
}
}

Best Practices

Configuration Organization

1. Use Typed Schemas: Define strongly typed configuration schemas for validation
2. Provide Sensible Defaults: Always provide reasonable default values
3. Document Configuration: Include descriptions and validation rules
4. Environment Separation: Use different configurations for different environments
5. Secret Security: Never store secrets in plain text configuration files

Performance Considerations

#![allow(unused)]
fn main() {
// Cache parsed configuration for performance
use std::sync::Arc;
use once_cell::sync::OnceCell;

struct CachedConfigActor {
    cached_config: OnceCell<Arc<ProcessorConfigSchema>>,
}

impl Actor for CachedConfigActor {
    fn create_process(&self, config: ActorConfig) -> Pin<Box<dyn Future<Output = ()> + Send + 'static>> {
        // Parse configuration once and cache it
        let parsed_config = self.cached_config.get_or_init(|| {
            Arc::new(config.parse_typed().expect("Invalid configuration"))
        }).clone();
        
        Box::pin(async move {
            // Use cached configuration
            let batch_size = parsed_config.batch_size;
            // ...
        })
    }
}
}

Error Handling

#![allow(unused)]
fn main() {
use reflow_network::actor::ConfigError;

impl Actor for RobustActor {
    fn create_process(&self, config: ActorConfig) -> Pin<Box<dyn Future<Output = ()> + Send + 'static>> {
        Box::pin(async move {
            // Graceful configuration error handling
            let batch_size = match config.get_number("batch_size") {
                Some(size) if size > 0.0 => size as usize,
                Some(_) => {
                    eprintln!("Invalid batch_size, using default");
                    10
                },
                None => {
                    println!("No batch_size specified, using default");
                    10
                }
            };
            
            // Continue with actor logic
        })
    }
}
}

Next Steps

• Actor Creation Guide - Learn how to create actors that use ActorConfig
• Multi-Graph Composition - Using ActorConfig in multi-graph setups
• ActorConfig Migration Guide - Migrating existing actors

Creating and Managing Graphs

This guide covers the core APIs for creating, modifying, and managing Reflow graphs.

Graph Creation

Basic Graph Creation

#![allow(unused)]
fn main() {
use reflow_network::graph::{Graph, PortType};
use std::collections::HashMap;
use serde_json::json;

// Create a new graph
let mut graph = Graph::new("MyWorkflow", false, None);

// Create with case sensitivity enabled
let mut case_sensitive_graph = Graph::new("CaseSensitive", true, None);

// Create with initial properties
let properties = HashMap::from([
    ("description".to_string(), json!("Data processing workflow")),
    ("version".to_string(), json!("1.0.0")),
    ("author".to_string(), json!("John Doe"))
]);
let mut graph_with_props = Graph::new("WorkflowV1", false, Some(properties));
}

Graph with History Tracking

#![allow(unused)]
fn main() {
// Create graph with unlimited history
let (mut graph, mut history) = Graph::with_history();

// Create graph with limited history (recommended for production)
let (mut graph, mut history) = Graph::with_history_and_limit(100);
}

Node Management

Adding Nodes

#![allow(unused)]
fn main() {
// Basic node addition
graph.add_node("data_source", "FileReader", None);

// Node with position metadata
let metadata = HashMap::from([
    ("x".to_string(), json!(100)),
    ("y".to_string(), json!(200))
]);
graph.add_node("processor", "DataProcessor", Some(metadata));

// Node with comprehensive metadata
let rich_metadata = HashMap::from([
    ("x".to_string(), json!(300)),
    ("y".to_string(), json!(150)),
    ("label".to_string(), json!("CSV Parser")),
    ("color".to_string(), json!("#3498db")),
    ("estimated_time".to_string(), json!(2.5)),
    ("resources".to_string(), json!({
        "memory": 128,
        "cpu": 0.5
    })),
    ("configuration".to_string(), json!({
        "delimiter": ",",
        "has_header": true
    }))
]);
graph.add_node("csv_parser", "CSVParser", Some(rich_metadata));
}

Node Retrieval

#![allow(unused)]
fn main() {
// Get node reference
if let Some(node) = graph.get_node("processor") {
    println!("Node component: {}", node.component);
    if let Some(metadata) = &node.metadata {
        println!("Node metadata: {:?}", metadata);
    }
}

// Get mutable node reference
if let Some(node) = graph.get_node_mut("processor") {
    // Modify node directly (not recommended - use set_node_metadata instead)
}
}

Updating Node Metadata

#![allow(unused)]
fn main() {
// Update metadata (merges with existing)
let updates = HashMap::from([
    ("color".to_string(), json!("#e74c3c")),
    ("priority".to_string(), json!("high"))
]);
graph.set_node_metadata("processor", updates);

// Clear specific metadata field by setting to null
let clear_color = HashMap::from([
    ("color".to_string(), json!(null))
]);
graph.set_node_metadata("processor", clear_color);
}

Node Removal

#![allow(unused)]
fn main() {
// Remove node (automatically removes all connections)
graph.remove_node("old_processor");
}

Node Renaming

#![allow(unused)]
fn main() {
// Rename node (updates all references)
graph.rename_node("old_name", "new_name");
}

Connection Management

Creating Connections

#![allow(unused)]
fn main() {
// Basic connection
graph.add_connection("source", "output", "processor", "input", None);

// Connection with metadata
let conn_metadata = HashMap::from([
    ("weight".to_string(), json!(0.8)),
    ("priority".to_string(), json!("high")),
    ("buffer_size".to_string(), json!(1024))
]);
graph.add_connection("processor", "output", "sink", "input", Some(conn_metadata));
}

Connection Queries

#![allow(unused)]
fn main() {
// Get specific connection
if let Some(connection) = graph.get_connection("source", "output", "processor", "input") {
    println!("Connection metadata: {:?}", connection.metadata);
}

// Get all connections for a node
let incoming = graph.get_incoming_connections("processor");
for (source_node, source_port, connection) in incoming {
    println!("Input from {}:{}", source_node, source_port);
}

let outgoing = graph.get_outgoing_connections("processor");
for (target_node, target_port, connection) in outgoing {
    println!("Output to {}:{}", target_node, target_port);
}

// Get connections for specific port
let port_incoming = graph.get_incoming_connections_for_port("processor", "input");
let port_outgoing = graph.get_outgoing_connections_for_port("processor", "output");
}

Connection Analysis

#![allow(unused)]
fn main() {
// Check if nodes are connected
if graph.are_nodes_connected("source", "processor") {
    println!("Nodes are connected");
}

// Check specific port connections
if graph.are_ports_connected("source", "output", "processor", "input") {
    println!("Ports are connected");
}

// Get connection degrees
let (in_degree, out_degree) = graph.get_connection_degree("processor");
println!("Node has {} inputs and {} outputs", in_degree, out_degree);

// Get port-specific degrees
let (port_in, port_out) = graph.get_port_connection_degree("processor", "data");
}

Connection Updates

#![allow(unused)]
fn main() {
// Update connection metadata
let new_metadata = HashMap::from([
    ("bandwidth".to_string(), json!("high")),
    ("encrypted".to_string(), json!(true))
]);
graph.set_connection_metadata("source", "output", "processor", "input", new_metadata);
}

Connection Removal

#![allow(unused)]
fn main() {
// Remove specific connection
graph.remove_connection("source", "output", "processor", "input");

// Remove all connections for a node (called automatically when removing node)
graph.remove_node_connections("isolated_node");
}

Graph Ports (Inports/Outports)

Graph ports expose internal node ports as external interfaces, making subgraphs reusable.

Adding Input Ports

#![allow(unused)]
fn main() {
// Basic inport
graph.add_inport("data_input", "processor", "input", PortType::Any, None);

// Inport with metadata
let port_metadata = HashMap::from([
    ("description".to_string(), json!("Main data input stream")),
    ("required".to_string(), json!(true)),
    ("default_value".to_string(), json!(null))
]);
graph.add_inport("config", "processor", "config", PortType::Object("Config".to_string()), Some(port_metadata));
}

Adding Output Ports

#![allow(unused)]
fn main() {
// Basic outport
graph.add_outport("processed_data", "processor", "output", PortType::Object("ProcessedData".to_string()), None);

// Outport with metadata
let out_metadata = HashMap::from([
    ("description".to_string(), json!("Processed data stream")),
    ("format".to_string(), json!("json"))
]);
graph.add_outport("results", "processor", "result", PortType::Array(Box::new(PortType::Object("Result".to_string()))), Some(out_metadata));
}

Port Management

#![allow(unused)]
fn main() {
// Update port metadata
let port_updates = HashMap::from([
    ("required".to_string(), json!(false)),
    ("deprecated".to_string(), json!(true))
]);
graph.set_inport_metadata("data_input", port_updates);
graph.set_outport_metadata("results", port_updates);

// Rename ports
graph.rename_inport("old_input", "new_input");
graph.rename_outport("old_output", "new_output");

// Remove ports
graph.remove_inport("unused_input");
graph.remove_outport("unused_output");
}

Initial Information Packets (IIPs)

IIPs provide static data to nodes at startup.

Adding IIPs

#![allow(unused)]
fn main() {
// Basic IIP
graph.add_initial(
    json!("config.yaml"),
    "file_reader",
    "filename",
    None
);

// IIP with metadata
let iip_metadata = HashMap::from([
    ("source".to_string(), json!("configuration")),
    ("priority".to_string(), json!("high"))
]);
graph.add_initial(
    json!({"host": "localhost", "port": 8080}),
    "server",
    "config",
    Some(iip_metadata)
);

// IIP with array index
graph.add_initial_index(
    json!("file1.txt"),
    "multi_reader",
    "files",
    0,
    None
);
}

Graph-level IIPs

When using graph ports, you can add IIPs at the graph level:

#![allow(unused)]
fn main() {
// Add IIP to graph inport
graph.add_graph_initial(
    json!({"mode": "production"}),
    "config_input",  // Graph inport name
    None
);

// Add indexed IIP to graph inport
graph.add_graph_initial_index(
    json!("primary.db"),
    "database_files",  // Graph inport name
    0,
    None
);
}

Removing IIPs

#![allow(unused)]
fn main() {
// Remove node-level IIP
graph.remove_initial("file_reader", "filename");

// Remove graph-level IIP
graph.remove_graph_initial("config_input");
}

Node Groups

Groups provide logical organization of related nodes.

Creating Groups

#![allow(unused)]
fn main() {
// Create basic group
graph.add_group("data_processing", vec!["parser".to_string(), "validator".to_string(), "transformer".to_string()], None);

// Group with metadata
let group_metadata = HashMap::from([
    ("color".to_string(), json!("#2ecc71")),
    ("description".to_string(), json!("Data processing pipeline")),
    ("collapsed".to_string(), json!(false))
]);
graph.add_group("preprocessing", vec!["cleaner".to_string(), "normalizer".to_string()], Some(group_metadata));
}

Managing Group Membership

#![allow(unused)]
fn main() {
// Add node to existing group
graph.add_to_group("data_processing", "formatter");

// Remove node from group
graph.remove_from_group("data_processing", "formatter");
}

Group Metadata

#![allow(unused)]
fn main() {
// Update group metadata
let group_updates = HashMap::from([
    ("collapsed".to_string(), json!(true)),
    ("priority".to_string(), json!("high"))
]);
graph.set_group_metadata("data_processing", group_updates);
}

Removing Groups

#![allow(unused)]
fn main() {
// Remove entire group (nodes remain, just ungrouped)
graph.remove_group("old_group");
}

Graph Properties

Setting Properties

#![allow(unused)]
fn main() {
// Set multiple properties
let properties = HashMap::from([
    ("name".to_string(), json!("Updated Workflow")),
    ("version".to_string(), json!("2.0.0")),
    ("description".to_string(), json!("Enhanced data processing")),
    ("tags".to_string(), json!(["data", "processing", "etl"]))
]);
graph.set_properties(properties);
}

Getting Properties

#![allow(unused)]
fn main() {
// Properties are accessible via graph.properties field
if let Some(name) = graph.properties.get("name") {
    println!("Graph name: {}", name);
}
}

Event Handling

Subscribing to Events

#![allow(unused)]
fn main() {
use reflow_network::graph::GraphEvents;

// Get event receiver
let event_receiver = graph.event_channel.1.clone();

// Handle events in a loop
std::thread::spawn(move || {
    while let Ok(event) = event_receiver.recv() {
        match event {
            GraphEvents::AddNode(data) => {
                println!("Node added: {:?}", data);
            }
            GraphEvents::RemoveNode(data) => {
                println!("Node removed: {:?}", data);
            }
            GraphEvents::AddConnection(data) => {
                println!("Connection added: {:?}", data);
            }
            GraphEvents::RemoveConnection(data) => {
                println!("Connection removed: {:?}", data);
            }
            GraphEvents::ChangeNode(data) => {
                println!("Node changed: {:?}", data);
            }
            // ... handle other events
            _ => {}
        }
    }
});
}

Event Types Reference

Serialization and Loading

Exporting Graphs

#![allow(unused)]
fn main() {
// Export to GraphExport format
let export = graph.export();

// Serialize to JSON
let json_string = serde_json::to_string_pretty(&export)?;
std::fs::write("workflow.json", json_string)?;
}

Loading Graphs

#![allow(unused)]
fn main() {
// Load from JSON
let json_content = std::fs::read_to_string("workflow.json")?;
let export: GraphExport = serde_json::from_str(&json_content)?;

// Create graph from export
let metadata = HashMap::from([
    ("loaded_at".to_string(), json!(chrono::Utc::now().to_rfc3339()))
]);
let loaded_graph = Graph::load(export, Some(metadata));
}

WebAssembly API

When using the graph system in a browser via WebAssembly:

JavaScript/TypeScript Usage

import { Graph, PortType } from 'reflow-network';

// Create graph
const graph = new Graph("WebWorkflow", false, {
    description: "Browser-based workflow"
});

// Add nodes
graph.addNode("input", "InputNode", { x: 0, y: 0 });
graph.addNode("output", "OutputNode", { x: 200, y: 0 });

// Add connections
graph.addConnection("input", "out", "output", "in", {});

// Subscribe to events
graph.subscribe((event) => {
    console.log("Graph event:", event);
    // Update UI based on event
    updateUI(event);
});

// Export for persistence
const exported = graph.toJSON();
localStorage.setItem('workflow', JSON.stringify(exported));

// Load saved workflow
const saved = JSON.parse(localStorage.getItem('workflow'));
const restoredGraph = Graph.load(saved, {});

Error Handling

Common Error Scenarios

#![allow(unused)]
fn main() {
use reflow_network::graph::GraphError;

// Handle node operations
match graph.add_node("duplicate", "TestNode", None) {
    Ok(_) => println!("Node added successfully"),
    Err(GraphError::DuplicateNode(id)) => println!("Node {} already exists", id),
    Err(e) => println!("Error: {}", e),
}

// Handle traversal errors
match graph.traverse_depth_first("nonexistent", |node| {
    println!("Visiting: {}", node.id);
}) {
    Ok(_) => println!("Traversal completed"),
    Err(GraphError::NodeNotFound(id)) => println!("Start node {} not found", id),
    Err(e) => println!("Traversal error: {}", e),
}
}

Error Types

• NodeNotFound(String) - Referenced node doesn't exist
• DuplicateNode(String) - Node with same ID already exists
• InvalidConnection { from: String, to: String } - Connection cannot be created
• CycleDetected - Operation would create a cycle (if validation enabled)
• InvalidOperation(String) - Generic operation error

Best Practices

Performance Tips

1. Batch Operations: Group related changes to minimize events

   #![allow(unused)]
   fn main() {
   // Instead of individual operations, batch them
   graph.add_node("n1", "Node1", None);
   graph.add_node("n2", "Node2", None);
   graph.add_connection("n1", "out", "n2", "in", None);
   }

2. Use Appropriate Data Structures: Store frequently accessed metadata efficiently

   #![allow(unused)]
   fn main() {
   // Good: structured metadata
   let metadata = HashMap::from([
       ("config".to_string(), json!({
           "retries": 3,
           "timeout": 30
       }))
   ]);
   
   // Avoid: flat key-value for complex data
   }

3. Validate Incrementally: Use targeted validation instead of full graph validation

   #![allow(unused)]
   fn main() {
   // Check specific aspects instead of full validation
   if let Some(cycle) = graph.detect_cycles() {
       // Handle cycle
   }
   }

Memory Management

1. Limit History: Use bounded history for production systems

   #![allow(unused)]
   fn main() {
   let (graph, history) = Graph::with_history_and_limit(50);
   }

2. Clean Up Events: Ensure event listeners are properly disposed

   #![allow(unused)]
   fn main() {
   // Store receiver handle to drop when done
   let receiver = graph.event_channel.1.clone();
   // ... use receiver
   drop(receiver); // Clean up
   }

3. Efficient Metadata: Avoid storing large objects in metadata

   #![allow(unused)]
   fn main() {
   // Good: reference to external data
   let metadata = HashMap::from([
       ("data_ref".to_string(), json!("storage://large-dataset-id"))
   ]);
   
   // Avoid: embedding large data
   }

Next Steps

• Graph Analysis - Validation and performance analysis
• Layout System - Positioning and visualization
• Advanced Features - History, subgraphs, and optimization
• Building Visual Editors - Complete tutorial

Graph Analysis and Validation

Reflow's graph system provides extensive analysis capabilities for validation, performance optimization, and structural insights. This guide covers all analysis features available in the graph system.

Flow Validation

Comprehensive Validation

The validate_flow method performs a complete analysis of graph integrity:

#![allow(unused)]
fn main() {
use reflow_network::graph::{FlowValidation, PortMismatch};

// Perform full validation
let validation = graph.validate_flow()?;

// Check for issues
if !validation.cycles.is_empty() {
    for cycle in validation.cycles {
        println!("Cycle detected: {:?}", cycle);
    }
}

if !validation.orphaned_nodes.is_empty() {
    println!("Orphaned nodes: {:?}", validation.orphaned_nodes);
}

if !validation.port_mismatches.is_empty() {
    for mismatch in validation.port_mismatches {
        println!("Port type mismatch: {}", mismatch);
    }
}
}

Validation Results Structure

#![allow(unused)]
fn main() {
pub struct FlowValidation {
    pub cycles: Vec<Vec<String>>,           // Detected cycles
    pub orphaned_nodes: Vec<String>,        // Disconnected nodes
    pub port_mismatches: Vec<PortMismatch>, // Type incompatibilities
}

pub struct PortMismatch {
    pub from_node: String,
    pub from_port: String,
    pub from_type: PortType,
    pub to_node: String,
    pub to_port: String,
    pub to_type: PortType,
    pub reason: String,
}
}

Cycle Detection

Basic Cycle Detection

#![allow(unused)]
fn main() {
// Detect first cycle found
if let Some(cycle) = graph.detect_cycles() {
    println!("Cycle path: {:?}", cycle);
    // cycle is Vec<String> showing the path of the cycle
}

// Check if specific node is in a cycle
if graph.is_node_in_cycle("suspicious_node") {
    println!("Node is part of a cycle");
}
}

Comprehensive Cycle Analysis

#![allow(unused)]
fn main() {
use reflow_network::graph::CycleAnalysis;

let cycle_analysis = graph.analyze_cycles();

println!("Total cycles found: {}", cycle_analysis.total_cycles);
println!("Cycle lengths: {:?}", cycle_analysis.cycle_lengths);
println!("Nodes involved in cycles: {:?}", cycle_analysis.nodes_in_cycles);

if let Some(longest) = cycle_analysis.longest_cycle {
    println!("Longest cycle: {:?} (length: {})", longest, longest.len());
}

if let Some(shortest) = cycle_analysis.shortest_cycle {
    println!("Shortest cycle: {:?} (length: {})", shortest, shortest.len());
}
}

All Cycles Detection

#![allow(unused)]
fn main() {
// Find all cycles in the graph
let all_cycles = graph.detect_all_cycles();

for (i, cycle) in all_cycles.iter().enumerate() {
    println!("Cycle {}: {:?}", i + 1, cycle);
}
}

Orphaned Node Analysis

Basic Orphaned Node Detection

#![allow(unused)]
fn main() {
// Find all orphaned nodes
let orphaned = graph.find_orphaned_nodes();

for node in orphaned {
    println!("Orphaned node: {}", node);
}
}

Detailed Orphaned Analysis

#![allow(unused)]
fn main() {
use reflow_network::graph::OrphanedNodeAnalysis;

let orphan_analysis = graph.analyze_orphaned_nodes();

println!("Total orphaned nodes: {}", orphan_analysis.total_orphaned);

println!("Completely isolated nodes:");
for node in orphan_analysis.completely_isolated {
    println!("  - {}", node);
}

println!("Unreachable nodes (have connections but no path from entry points):");
for node in orphan_analysis.unreachable {
    println!("  - {}", node);
}

println!("Disconnected groups:");
for (i, group) in orphan_analysis.disconnected_groups.iter().enumerate() {
    println!("  Group {}: {:?}", i + 1, group);
}
}

Port Type Validation

Port Compatibility Checking

#![allow(unused)]
fn main() {
// Validate all port types in the graph
let port_mismatches = graph.validate_port_types();

for mismatch in port_mismatches {
    println!("Port mismatch: {} -> {}", 
        format!("{}:{}", mismatch.from_node, mismatch.from_port),
        format!("{}:{}", mismatch.to_node, mismatch.to_port)
    );
    println!("  Types: {:?} -> {:?}", mismatch.from_type, mismatch.to_type);
    println!("  Reason: {}", mismatch.reason);
}
}

Custom Type Compatibility

The graph system includes built-in type compatibility rules:

#![allow(unused)]
fn main() {
// Built-in compatibility rules:
// Any ↔ Any type (always compatible)
// Integer → Float (automatic promotion)
// T → Stream (streaming any type)
// T → Option<T> (wrapping in option)
// Array<T> → Array<U> (if T → U)

// Example of compatible connections:
graph.add_connection("int_source", "out", "float_processor", "in", None);     // Integer → Float ✓
graph.add_connection("data_source", "out", "stream_processor", "in", None);   // Any → Stream ✓
graph.add_connection("value", "out", "optional_sink", "in", None);            // T → Option<T> ✓
}

Performance Analysis

Parallelism Analysis

#![allow(unused)]
fn main() {
use reflow_network::graph::{ParallelismAnalysis, PipelineStage};

let parallelism = graph.analyze_parallelism();

println!("Maximum parallelism: {}", parallelism.max_parallelism);

// Parallel branches that can execute simultaneously
println!("Parallel branches:");
for (i, branch) in parallelism.parallel_branches.iter().enumerate() {
    println!("  Branch {}: {:?}", i + 1, branch.nodes);
    println!("    Entry points: {:?}", branch.entry_points);
    println!("    Exit points: {:?}", branch.exit_points);
}

// Pipeline stages for sequential execution
println!("Pipeline stages:");
for stage in parallelism.pipeline_stages {
    println!("  Stage {}: {:?}", stage.level, stage.nodes);
}
}

Bottleneck Detection

#![allow(unused)]
fn main() {
use reflow_network::graph::Bottleneck;

let bottlenecks = graph.detect_bottlenecks();

for bottleneck in bottlenecks {
    match bottleneck {
        Bottleneck::HighDegree(node) => {
            let (in_deg, out_deg) = graph.get_connection_degree(&node);
            println!("High-degree bottleneck: {} ({} in, {} out)", node, in_deg, out_deg);
        }
        Bottleneck::SequentialChain(chain) => {
            println!("Sequential chain (could be parallelized): {:?}", chain);
        }
    }
}
}

High-Degree Node Analysis

#![allow(unused)]
fn main() {
// Find nodes with unusually high connection counts
let high_degree_nodes = graph.find_high_degree_nodes();

for node in high_degree_nodes {
    let (incoming, outgoing) = graph.get_connection_degree(&node);
    let total_degree = incoming + outgoing;
    
    println!("High-degree node: {} (total degree: {})", node, total_degree);
    println!("  Incoming: {}, Outgoing: {}", incoming, outgoing);
    
    // Analyze connected nodes
    let connected = graph.get_connected_nodes(&node);
    println!("  Connected to {} other nodes", connected.len());
}
}

Sequential Chain Analysis

#![allow(unused)]
fn main() {
// Find chains that could potentially be parallelized
let sequential_chains = graph.find_sequential_chains();

for (i, chain) in sequential_chains.iter().enumerate() {
    println!("Sequential chain {}: {:?}", i + 1, chain);
    println!("  Length: {} nodes", chain.len());
    
    // Analyze chain characteristics
    if chain.len() >= 5 {
        println!("  ⚠️  Long chain - consider breaking into parallel segments");
    }
}
}

Data Flow Analysis

Flow Path Tracing

#![allow(unused)]
fn main() {
use reflow_network::graph::{DataFlowPath, DataTransform};

// Trace data flow from a starting node
let flow_paths = graph.trace_data_flow("input_node")?;

for (i, path) in flow_paths.iter().enumerate() {
    println!("Flow path {}:", i + 1);
    println!("  Nodes: {:?}", path.nodes);
    
    println!("  Transformations:");
    for transform in &path.transforms {
        println!("    {} -> {} ({}: {} -> {})", 
            transform.node, 
            transform.operation,
            transform.node,
            transform.input_type, 
            transform.output_type
        );
    }
}
}

Execution Path Analysis

#![allow(unused)]
fn main() {
use reflow_network::graph::ExecutionPath;

// Find all possible execution paths
let execution_paths = graph.find_execution_paths();

for (i, path) in execution_paths.iter().enumerate() {
    println!("Execution path {}:", i + 1);
    println!("  Nodes: {:?}", path.nodes);
    println!("  Estimated time: {:.2}s", path.estimated_time);
    println!("  Resource requirements: {:?}", path.resource_requirements);
    
    // Check for parallel execution markers
    if path.resource_requirements.contains_key("parallel_branches") {
        let branches = path.resource_requirements["parallel_branches"];
        println!("  ⚡ Contains {} parallel branches", branches);
    }
    
    if path.resource_requirements.contains_key("contains_cycle") {
        println!("  ⚠️  Path contains cycles");
    }
}
}

Resource Requirements Analysis

#![allow(unused)]
fn main() {
// Analyze resource requirements for the entire graph
let resource_analysis = graph.analyze_resource_requirements();

println!("Graph resource requirements:");
for (resource, requirement) in resource_analysis {
    match resource.as_str() {
        "memory" => println!("  Memory: {:.1} MB", requirement),
        "cpu" => println!("  CPU cores: {:.1}", requirement),
        "disk" => println!("  Disk space: {:.1} GB", requirement),
        "network" => println!("  Network bandwidth: {:.1} Mbps", requirement),
        _ => println!("  {}: {:.2}", resource, requirement),
    }
}
}

Runtime Analysis

Comprehensive Runtime Analysis

#![allow(unused)]
fn main() {
use reflow_network::graph::{EnhancedGraphAnalysis, OptimizationSuggestion};

let runtime_analysis = graph.analyze_for_runtime();

println!("=== Runtime Analysis ===");
println!("Estimated execution time: {:.2}s", runtime_analysis.estimated_execution_time);
println!("Resource requirements: {:?}", runtime_analysis.resource_requirements);

// Parallelism opportunities
println!("\nParallelism analysis:");
println!("  Max parallelism: {}", runtime_analysis.parallelism.max_parallelism);
println!("  Parallel branches: {}", runtime_analysis.parallelism.parallel_branches.len());
println!("  Pipeline stages: {}", runtime_analysis.parallelism.pipeline_stages.len());

// Optimization suggestions
println!("\nOptimization suggestions:");
for suggestion in runtime_analysis.optimization_suggestions {
    match suggestion {
        OptimizationSuggestion::ParallelizableChain { nodes } => {
            println!("  ⚡ Parallelize chain: {:?}", nodes);
        }
        OptimizationSuggestion::RedundantNode { node, reason } => {
            println!("  🗑️  Remove redundant node '{}': {}", node, reason);
        }
        OptimizationSuggestion::ResourceBottleneck { resource, severity } => {
            println!("  ⚠️  Resource bottleneck in '{}': {:.1}% usage", resource, severity * 100.0);
        }
        OptimizationSuggestion::DataTypeOptimization { from, to, suggestion } => {
            println!("  🔧 Optimize types {} → {}: {}", from, to, suggestion);
        }
    }
}

// Performance bottlenecks
println!("\nPerformance bottlenecks:");
for bottleneck in runtime_analysis.performance_bottlenecks {
    match bottleneck {
        Bottleneck::HighDegree(node) => {
            println!("  🔥 High-degree node: {}", node);
        }
        Bottleneck::SequentialChain(chain) => {
            println!("  🐌 Sequential bottleneck: {:?}", chain);
        }
    }
}
}

Subgraph Analysis

Extracting Subgraphs

#![allow(unused)]
fn main() {
use reflow_network::graph::{Subgraph, SubgraphAnalysis};

// Get reachable subgraph from a node
if let Some(subgraph) = graph.get_reachable_subgraph("start_node") {
    println!("Subgraph from 'start_node':");
    println!("  Nodes: {:?}", subgraph.nodes);
    println!("  Entry points: {:?}", subgraph.entry_points);
    println!("  Exit points: {:?}", subgraph.exit_points);
    println!("  Internal connections: {}", subgraph.internal_connections.len());
    
    // Analyze subgraph characteristics
    let analysis = graph.analyze_subgraph(&subgraph);
    println!("  Analysis:");
    println!("    Node count: {}", analysis.node_count);
    println!("    Connection count: {}", analysis.connection_count);
    println!("    Max depth: {}", analysis.max_depth);
    println!("    Is cyclic: {}", analysis.is_cyclic);
    println!("    Branching factor: {:.2}", analysis.branching_factor);
}
}

Independent Subgraph Detection

#![allow(unused)]
fn main() {
// Find all independent subgraphs
let subgraphs = graph.find_subgraphs();

println!("Found {} independent subgraphs:", subgraphs.len());
for (i, subgraph) in subgraphs.iter().enumerate() {
    println!("  Subgraph {}: {} nodes", i + 1, subgraph.nodes.len());
    
    let analysis = graph.analyze_subgraph(subgraph);
    if analysis.is_cyclic {
        println!("    ⚠️  Contains cycles");
    }
    
    if subgraph.entry_points.len() > 1 {
        println!("    ⚡ Multiple entry points - potential for parallel input");
    }
    
    if subgraph.exit_points.len() > 1 {
        println!("    📊 Multiple exit points - produces multiple outputs");
    }
}
}

Graph Traversal Analysis

Traversal with Analysis

#![allow(unused)]
fn main() {
use std::collections::HashSet;

// Depth-first traversal with custom analysis
let mut visited_order = Vec::new();
let mut max_depth = 0;
let mut current_depth = 0;

graph.traverse_depth_first("start_node", |node| {
    visited_order.push(node.id.clone());
    current_depth += 1;
    max_depth = max_depth.max(current_depth);
    
    println!("Visiting {} at depth {}", node.id, current_depth);
    
    // Analyze node characteristics
    if let Some(metadata) = &node.metadata {
        if let Some(estimated_time) = metadata.get("estimated_time") {
            println!("  Estimated processing time: {:?}", estimated_time);
        }
    }
})?;

println!("Traversal completed:");
println!("  Visit order: {:?}", visited_order);
println!("  Maximum depth: {}", max_depth);
}

Breadth-First Layer Analysis

#![allow(unused)]
fn main() {
// Breadth-first traversal to analyze layers
let mut layers: HashMap<usize, Vec<String>> = HashMap::new();
let mut current_layer = 0;

graph.traverse_breadth_first("start_node", |node| {
    // In a real implementation, you'd track depth
    layers.entry(current_layer)
        .or_insert_with(Vec::new)
        .push(node.id.clone());
    
    println!("Layer {}: {}", current_layer, node.id);
})?;

// Analyze layer characteristics
for (layer, nodes) in layers {
    println!("Layer {} has {} nodes: {:?}", layer, nodes.len(), nodes);
    
    if nodes.len() > 1 {
        println!("  ⚡ Layer {} can be parallelized", layer);
    }
}
}

Custom Analysis Functions

Building Custom Analyzers

#![allow(unused)]
fn main() {
// Custom analyzer for finding critical paths
fn find_critical_path(graph: &Graph, start: &str, end: &str) -> Option<Vec<String>> {
    let mut longest_path = Vec::new();
    let mut max_weight = 0.0;
    
    // Use path tracing to find all paths
    if let Ok(paths) = graph.trace_data_flow(start) {
        for path in paths {
            if path.nodes.last() == Some(&end.to_string()) {
                // Calculate path weight based on estimated times
                let mut path_weight = 0.0;
                
                for node_id in &path.nodes {
                    if let Some(node) = graph.get_node(node_id) {
                        if let Some(metadata) = &node.metadata {
                            if let Some(time) = metadata.get("estimated_time") {
                                if let Some(t) = time.as_f64() {
                                    path_weight += t;
                                }
                            }
                        }
                    }
                }
                
                if path_weight > max_weight {
                    max_weight = path_weight;
                    longest_path = path.nodes;
                }
            }
        }
    }
    
    if longest_path.is_empty() {
        None
    } else {
        Some(longest_path)
    }
}

// Usage
if let Some(critical_path) = find_critical_path(&graph, "input", "output") {
    println!("Critical path: {:?}", critical_path);
}
}

Performance Metrics Collection

#![allow(unused)]
fn main() {
use std::time::Instant;

// Benchmark graph operations
fn benchmark_graph_operations(graph: &Graph) {
    let start = Instant::now();
    
    // Benchmark cycle detection
    let cycle_start = Instant::now();
    let _cycles = graph.detect_all_cycles();
    let cycle_time = cycle_start.elapsed();
    
    // Benchmark validation
    let validation_start = Instant::now();
    let _validation = graph.validate_flow();
    let validation_time = validation_start.elapsed();
    
    // Benchmark parallelism analysis
    let parallelism_start = Instant::now();
    let _parallelism = graph.analyze_parallelism();
    let parallelism_time = parallelism_start.elapsed();
    
    let total_time = start.elapsed();
    
    println!("=== Performance Metrics ===");
    println!("Graph size: {} nodes, {} connections", 
        graph.nodes.len(), 
        graph.connections.len()
    );
    println!("Cycle detection: {:?}", cycle_time);
    println!("Flow validation: {:?}", validation_time);
    println!("Parallelism analysis: {:?}", parallelism_time);
    println!("Total analysis time: {:?}", total_time);
}
}

Analysis Best Practices

Incremental Analysis

For large graphs, perform incremental analysis:

#![allow(unused)]
fn main() {
// Instead of full validation on every change
let full_validation = graph.validate_flow()?; // Expensive

// Use targeted analysis
if let Some(cycle) = graph.detect_cycles() {
    // Handle cycles specifically
}

// Check only specific node connections
let node_issues = graph.find_orphaned_nodes()
    .into_iter()
    .filter(|n| recently_modified_nodes.contains(n))
    .collect::<Vec<_>>();
}

Caching Analysis Results

#![allow(unused)]
fn main() {
use std::cell::RefCell;

struct CachedAnalyzer {
    graph: Graph,
    cached_validation: RefCell<Option<FlowValidation>>,
    validation_dirty: RefCell<bool>,
}

impl CachedAnalyzer {
    fn get_validation(&self) -> Result<FlowValidation, GraphError> {
        if *self.validation_dirty.borrow() {
            let validation = self.graph.validate_flow()?;
            *self.cached_validation.borrow_mut() = Some(validation.clone());
            *self.validation_dirty.borrow_mut() = false;
            Ok(validation)
        } else {
            Ok(self.cached_validation.borrow().clone().unwrap())
        }
    }
    
    fn invalidate_cache(&self) {
        *self.validation_dirty.borrow_mut() = true;
    }
}
}

Parallel Analysis

For very large graphs, consider parallel analysis:

#![allow(unused)]
fn main() {
use std::thread;

// Analyze different aspects in parallel
let graph_clone = graph.clone();
let cycle_handle = thread::spawn(move || {
    graph_clone.detect_all_cycles()
});

let graph_clone2 = graph.clone();
let orphan_handle = thread::spawn(move || {
    graph_clone2.analyze_orphaned_nodes()
});

let graph_clone3 = graph.clone();
let parallelism_handle = thread::spawn(move || {
    graph_clone3.analyze_parallelism()
});

// Collect results
let cycles = cycle_handle.join().unwrap();
let orphan_analysis = orphan_handle.join().unwrap();
let parallelism_analysis = parallelism_handle.join().unwrap();

println!("Parallel analysis completed:");
println!("  Cycles: {}", cycles.len());
println!("  Orphaned: {}", orphan_analysis.total_orphaned);
println!("  Max parallelism: {}", parallelism_analysis.max_parallelism);
}

Analysis Error Handling

Robust Error Handling

#![allow(unused)]
fn main() {
use reflow_network::graph::GraphError;

fn safe_analysis(graph: &Graph) -> Result<(), Box<dyn std::error::Error>> {
    // Validate graph structure first
    match graph.validate_flow() {
        Ok(validation) => {
            if !validation.cycles.is_empty() {
                println!("⚠️  Cycles detected - some analyses may not be reliable");
            }
        }
        Err(e) => {
            eprintln!("Validation failed: {}", e);
            return Err(Box::new(e));
        }
    }
    
    // Perform safe traversal
    match graph.traverse_depth_first("start", |node| {
        println!("Processing: {}", node.id);
    }) {
        Ok(_) => println!("Traversal completed successfully"),
        Err(GraphError::NodeNotFound(node)) => {
            eprintln!("Start node '{}' not found", node);
        }
        Err(e) => {
            eprintln!("Traversal error: {}", e);
            return Err(Box::new(e));
        }
    }
    
    Ok(())
}
}

Integration with Visual Editors

Real-time Analysis Updates

#![allow(unused)]
fn main() {
// Update UI based on analysis results
fn update_editor_with_analysis(graph: &Graph, ui: &mut GraphEditor) {
    // Highlight cycles
    if let Some(cycle) = graph.detect_cycles() {
        for node in cycle {
            ui.highlight_node(&node, "error");
        }
    }
    
    // Show bottlenecks
    let bottlenecks = graph.detect_bottlenecks();
    for bottleneck in bottlenecks {
        match bottleneck {
            Bottleneck::HighDegree(node) => {
                ui.highlight_node(&node, "bottleneck");
            }
            Bottleneck::SequentialChain(chain) => {
                ui.highlight_chain(&chain, "optimization-opportunity");
            }
        }
    }
    
    // Show parallel opportunities
    let parallelism = graph.analyze_parallelism();
    for branch in parallelism.parallel_branches {
        ui.group_nodes(&branch.nodes, "parallel-group");
    }
}
}

Next Steps

• Layout System - Positioning and visualization
• Advanced Features - History, subgraphs, and optimization
• Creating Graphs - Basic graph operations
• Building Visual Editors - Complete tutorial

Graph Layout System

Reflow's layout system provides intelligent automatic positioning and manual positioning capabilities for graph nodes. The system supports multiple layout algorithms, custom positioning, and integration with visual editors.

Automatic Layout

Basic Auto-Layout

#![allow(unused)]
fn main() {
use reflow_network::graph::Position;

// Calculate optimal positions using default algorithm
let positions = graph.calculate_layout();

for (node_id, position) in positions {
    println!("Node {}: x={:.1}, y={:.1}", node_id, position.x, position.y);
}

// Apply calculated layout to graph metadata
graph.auto_layout()?;
}

Layout Algorithms

The system supports multiple layout algorithms optimized for different graph types:

#![allow(unused)]
fn main() {
use reflow_network::graph::{LayoutAlgorithm, LayoutConfig};

// Hierarchical layout for DAGs (default)
let hierarchical_config = LayoutConfig {
    algorithm: LayoutAlgorithm::Hierarchical,
    node_spacing: 120.0,
    layer_spacing: 80.0,
    edge_spacing: 40.0,
    ..Default::default()
};

let positions = graph.calculate_layout_with_config(&hierarchical_config);

// Force-directed layout for general graphs
let force_config = LayoutConfig {
    algorithm: LayoutAlgorithm::ForceDirected,
    iterations: 100,
    spring_strength: 0.5,
    repulsion_strength: 1000.0,
    ..Default::default()
};

let positions = graph.calculate_layout_with_config(&force_config);

// Grid layout for structured workflows
let grid_config = LayoutConfig {
    algorithm: LayoutAlgorithm::Grid,
    grid_size: 150.0,
    columns: 5,
    align_to_grid: true,
    ..Default::default()
};

let positions = graph.calculate_layout_with_config(&grid_config);
}

Hierarchical Layout

Best for directed acyclic graphs (DAGs) and workflow diagrams:

#![allow(unused)]
fn main() {
use reflow_network::graph::HierarchicalConfig;

let hierarchical = HierarchicalConfig {
    direction: LayoutDirection::TopToBottom,
    layer_spacing: 100.0,
    node_spacing: 80.0,
    edge_routing: EdgeRouting::Orthogonal,
    minimize_crossings: true,
    balance_nodes: true,
};

let positions = graph.hierarchical_layout(&hierarchical);

// Apply with automatic layer detection
graph.auto_layout_hierarchical()?;
}

Force-Directed Layout

Ideal for general graphs with cycles and complex interconnections:

#![allow(unused)]
fn main() {
use reflow_network::graph::ForceDirectedConfig;

let force_config = ForceDirectedConfig {
    iterations: 150,
    cooling_factor: 0.95,
    initial_temperature: 100.0,
    spring_strength: 0.3,
    spring_length: 100.0,
    repulsion_strength: 800.0,
    gravity_strength: 0.1,
    node_charge: -30.0,
};

let positions = graph.force_directed_layout(&force_config);
}

Organic Layout

Creates natural, flowing layouts:

#![allow(unused)]
fn main() {
use reflow_network::graph::OrganicConfig;

let organic_config = OrganicConfig {
    preferred_edge_length: 120.0,
    edge_length_cost_factor: 0.0001,
    node_distribution_cost_factor: 20000.0,
    edge_crossing_cost_factor: 6000.0,
    edge_distance_cost_factor: 15000.0,
    border_line_cost_factor: 100.0,
    max_iterations: 200,
};

let positions = graph.organic_layout(&organic_config);
}

Manual Positioning

Setting Node Positions

#![allow(unused)]
fn main() {
use reflow_network::graph::Position;

// Set specific position
graph.set_node_position("input_node", 0.0, 0.0)?;
graph.set_node_position("processor", 200.0, 100.0)?;
graph.set_node_position("output_node", 400.0, 0.0)?;

// Set position with custom anchor point
let position = Position {
    x: 150.0,
    y: 75.0,
    anchor: Some(Anchor { x: 0.5, y: 0.5 }), // Center anchor
};
graph.set_node_position_with_anchor("centered_node", position)?;
}

Position Metadata Structure

Positions are stored in node metadata following this convention:

#![allow(unused)]
fn main() {
use serde_json::json;
use std::collections::HashMap;

// Standard position metadata
let position_metadata = HashMap::from([
    ("x".to_string(), json!(100)),
    ("y".to_string(), json!(150)),
    ("width".to_string(), json!(120)),
    ("height".to_string(), json!(80)),
    ("anchor".to_string(), json!({
        "x": 0.5,  // Horizontal anchor (0.0 = left, 0.5 = center, 1.0 = right)
        "y": 0.5   // Vertical anchor (0.0 = top, 0.5 = middle, 1.0 = bottom)
    }))
]);

graph.set_node_metadata("positioned_node", position_metadata);
}

Retrieving Positions

#![allow(unused)]
fn main() {
// Get position for a specific node
if let Some(position) = graph.get_node_position("processor") {
    println!("Node position: ({}, {})", position.x, position.y);
}

// Get all node positions
let all_positions = graph.get_all_positions();
for (node_id, position) in all_positions {
    println!("{}: ({:.1}, {:.1})", node_id, position.x, position.y);
}

// Get positions within a bounding box
let bbox = BoundingBox {
    min_x: 0.0,
    min_y: 0.0,
    max_x: 500.0,
    max_y: 300.0,
};
let nodes_in_area = graph.get_nodes_in_area(bbox);
}

Layout Constraints

Alignment Constraints

#![allow(unused)]
fn main() {
use reflow_network::graph::{AlignmentConstraint, ConstraintType};

// Horizontal alignment
let horizontal_alignment = AlignmentConstraint {
    nodes: vec!["node1".to_string(), "node2".to_string(), "node3".to_string()],
    constraint_type: ConstraintType::HorizontalAlignment,
    offset: 0.0,
};

// Vertical alignment
let vertical_alignment = AlignmentConstraint {
    nodes: vec!["input1".to_string(), "input2".to_string()],
    constraint_type: ConstraintType::VerticalAlignment,
    offset: 50.0, // 50 pixels apart
};

// Apply constraints during layout
let config = LayoutConfig {
    algorithm: LayoutAlgorithm::Hierarchical,
    constraints: vec![horizontal_alignment, vertical_alignment],
    ..Default::default()
};

graph.apply_layout_with_constraints(&config)?;
}

Distance Constraints

#![allow(unused)]
fn main() {
use reflow_network::graph::{DistanceConstraint, DistanceType};

// Minimum distance constraint
let min_distance = DistanceConstraint {
    from_node: "source".to_string(),
    to_node: "sink".to_string(),
    distance_type: DistanceType::Minimum,
    distance: 200.0,
};

// Maximum distance constraint
let max_distance = DistanceConstraint {
    from_node: "processor1".to_string(),
    to_node: "processor2".to_string(),
    distance_type: DistanceType::Maximum,
    distance: 300.0,
};

// Fixed distance constraint
let fixed_distance = DistanceConstraint {
    from_node: "controller".to_string(),
    to_node: "display".to_string(),
    distance_type: DistanceType::Fixed,
    distance: 150.0,
};
}

Boundary Constraints

#![allow(unused)]
fn main() {
use reflow_network::graph::BoundaryConstraint;

// Keep nodes within bounds
let boundary = BoundaryConstraint {
    min_x: 0.0,
    min_y: 0.0,
    max_x: 1000.0,
    max_y: 600.0,
    enforce_during_layout: true,
};

// Apply boundary constraint
graph.set_layout_boundary(boundary);
}

Layout Optimization

Minimize Edge Crossings

#![allow(unused)]
fn main() {
// Optimize layout to reduce edge crossings
let optimized_positions = graph.minimize_edge_crossings()?;

// Apply optimization with maximum iterations
let crossings_config = EdgeCrossingConfig {
    max_iterations: 50,
    improvement_threshold: 0.01,
    use_barycenter_heuristic: true,
};

graph.optimize_edge_crossings(&crossings_config)?;
}

Edge Bundling

#![allow(unused)]
fn main() {
use reflow_network::graph::EdgeBundling;

// Enable edge bundling for cleaner layouts
let bundling_config = EdgeBundling {
    enable: true,
    strength: 0.8,
    step_size: 0.1,
    iterations: 60,
    min_distance: 10.0,
};

graph.apply_edge_bundling(&bundling_config)?;
}

Compact Layout

#![allow(unused)]
fn main() {
// Create compact layout by minimizing overall area
let compact_config = CompactLayoutConfig {
    preserve_aspect_ratio: true,
    min_node_spacing: 20.0,
    pack_components: true,
};

graph.create_compact_layout(&compact_config)?;
}

Layer-Based Layout

Automatic Layer Detection

#![allow(unused)]
fn main() {
use reflow_network::graph::{LayerAnalysis, LayerDirection};

// Detect natural layers in the graph
let layer_analysis = graph.analyze_layers();

println!("Detected {} layers:", layer_analysis.layers.len());
for (level, nodes) in layer_analysis.layers.iter().enumerate() {
    println!("  Layer {}: {:?}", level, nodes);
}

// Apply layer-based layout
let layer_config = LayerLayoutConfig {
    direction: LayerDirection::LeftToRight,
    layer_spacing: 150.0,
    node_spacing: 100.0,
    center_nodes_in_layer: true,
};

graph.apply_layer_layout(&layer_config)?;
}

Manual Layer Assignment

#![allow(unused)]
fn main() {
// Manually assign nodes to layers
let layer_assignments = HashMap::from([
    ("input1".to_string(), 0),
    ("input2".to_string(), 0),
    ("processor1".to_string(), 1),
    ("processor2".to_string(), 1),
    ("output".to_string(), 2),
]);

graph.set_layer_assignments(layer_assignments);
graph.apply_layer_layout(&layer_config)?;
}

Group-Based Layout

Layout Node Groups

#![allow(unused)]
fn main() {
use reflow_network::graph::GroupLayoutConfig;

// Layout nodes within groups
let group_config = GroupLayoutConfig {
    group_spacing: 200.0,
    internal_spacing: 50.0,
    group_padding: 20.0,
    layout_algorithm: LayoutAlgorithm::Grid,
};

// Apply group-aware layout
graph.layout_groups(&group_config)?;

// Layout specific group
graph.layout_group("data_processing", &group_config)?;
}

Group Boundaries

#![allow(unused)]
fn main() {
// Calculate group boundaries
let group_bounds = graph.calculate_group_bounds("data_processing");
if let Some(bounds) = group_bounds {
    println!("Group bounds: ({}, {}) to ({}, {})", 
        bounds.min_x, bounds.min_y, bounds.max_x, bounds.max_y);
}

// Set custom group boundary
let custom_bounds = BoundingBox {
    min_x: 100.0,
    min_y: 50.0,
    max_x: 400.0,
    max_y: 250.0,
};
graph.set_group_bounds("data_processing", custom_bounds);
}

Advanced Layout Features

Multi-Level Layout

For very large graphs, use multi-level layout:

#![allow(unused)]
fn main() {
use reflow_network::graph::MultiLevelConfig;

let multilevel_config = MultiLevelConfig {
    coarsening_factor: 0.7,
    max_levels: 5,
    uncoarsening_iterations: 10,
    finest_level_iterations: 20,
};

let positions = graph.multilevel_layout(&multilevel_config)?;
}

Incremental Layout

Update layout incrementally when nodes are added/removed:

#![allow(unused)]
fn main() {
// Add node with incremental layout update
graph.add_node("new_processor", "DataProcessor", None);
graph.add_connection("source", "out", "new_processor", "in", None);

// Update layout incrementally
let incremental_config = IncrementalLayoutConfig {
    stabilization_iterations: 10,
    affected_nodes_only: true,
    preserve_existing_positions: true,
};

graph.incremental_layout_update(&incremental_config)?;
}

Layout Animation Support

#![allow(unused)]
fn main() {
use reflow_network::graph::{LayoutAnimation, AnimationFrame};

// Generate animation frames for smooth transitions
let from_positions = graph.get_all_positions();
let to_positions = graph.calculate_layout();

let animation = LayoutAnimation::new(from_positions, to_positions, 30); // 30 frames

// Get animation frames
for (frame_idx, frame) in animation.frames().enumerate() {
    println!("Frame {}: {} position updates", frame_idx, frame.positions.len());
    
    // Apply frame in UI
    for (node_id, position) in frame.positions {
        // Update UI node position
        ui.set_node_position(&node_id, position.x, position.y);
    }
}
}

Layout Quality Metrics

Measuring Layout Quality

#![allow(unused)]
fn main() {
use reflow_network::graph::{LayoutMetrics, LayoutQuality};

let metrics = graph.calculate_layout_metrics();

println!("Layout Quality Metrics:");
println!("  Edge crossings: {}", metrics.edge_crossings);
println!("  Average edge length: {:.2}", metrics.average_edge_length);
println!("  Node distribution score: {:.2}", metrics.node_distribution_score);
println!("  Aspect ratio: {:.2}", metrics.aspect_ratio);
println!("  Overall score: {:.2}", metrics.overall_quality_score);

// Detailed metrics
println!("\nDetailed Metrics:");
println!("  Minimum edge length: {:.2}", metrics.min_edge_length);
println!("  Maximum edge length: {:.2}", metrics.max_edge_length);
println!("  Edge length variance: {:.2}", metrics.edge_length_variance);
println!("  Node overlap count: {}", metrics.node_overlaps);
println!("  Angular resolution: {:.2}°", metrics.angular_resolution);
}