AnyFS Ecosystem

An open standard for pluggable virtual filesystem backends in Rust.

Overview

AnyFS is an open standard for virtual filesystem backends using a Tower-style middleware pattern for composable functionality.

You get:

• A familiar std::fs-aligned API
• Composable middleware (limits, logging, security)
• Choice of storage: memory, SQLite, host filesystem, or custom
• A developer-first goal: make storage composition easy, safe, and enjoyable

Architecture

┌─────────────────────────────────────────┐
│  FileStorage<B>                         │  ← Ergonomic std::fs-aligned API
├─────────────────────────────────────────┤
│  Middleware (composable):               │
│    Quota<B>                             │  ← Quotas
│    Restrictions<B>                      │  ← Security
│    Tracing<B>                           │  ← Audit
├─────────────────────────────────────────┤
│  Fs                                     │  ← Storage
│  (Memory, SQLite, VRootFs, custom...)   │
└─────────────────────────────────────────┘

Each layer has one job. Compose only what you need.

Two-Crate Structure

Note: Mounting (FsFuse + MountHandle) is part of the anyfs crate behind feature flags (fuse, winfsp), not a separate crate.

Quick Example

#![allow(unused)]
fn main() {
use anyfs::{MemoryBackend, QuotaLayer, RestrictionsLayer, FileStorage};

// Layer-based composition
let backend = MemoryBackend::new()
    .layer(QuotaLayer::builder()
        .max_total_size(100 * 1024 * 1024)
        .build())
    .layer(RestrictionsLayer::builder()
        .deny_permissions()
        .build());

let fs = FileStorage::new(backend);

fs.create_dir_all("/data")?;
fs.write("/data/file.txt", b"hello")?;
}

How to Use This Manual

Future Considerations

These are optional extensions that fit the design but are out of scope for initial release:

• URL-based backend registry and bulk helpers (FsExt/utilities)
• Async adapter for remote backends
• Companion shell for interactive exploration
• Copy-on-write overlay and archive backends (zip/tar)

See Design Overview for the full list and rationale.

Status

Authoritative Documents

1. AGENTS.md (for AI assistants)
2. src/architecture/design-overview.md
3. src/architecture/adrs.md

AnyFS - Executive Summary

One-page overview for stakeholders and decision-makers.

What is it?

AnyFS is an open standard for pluggable virtual filesystem backends in Rust, using a Tower-style middleware pattern for composable functionality.

You get:

• A familiar std::fs-aligned API
• Composable middleware for limits, logging, and security
• Choice of storage: memory, SQLite, host filesystem, or custom

Architecture

┌─────────────────────────────────────────┐
│  FileStorage<B>                         │  ← Ergonomics (std::fs API)
├─────────────────────────────────────────┤
│  Middleware (composable):               │
│    Quota<B>                             │  ← Quotas
│    Restrictions<B>                      │  ← Security
│    Tracing<B>                           │  ← Audit
├─────────────────────────────────────────┤
│  Fs                                     │  ← Storage
└─────────────────────────────────────────┘

Why does it matter?

Key design points

• Two-crate structure

  • anyfs-backend: trait contract
  • anyfs: backends + middleware + ergonomic wrapper

• Middleware pattern (like Axum/Tower)

  • Each middleware has one job
  • Compose only what you need
  • Complete separation of concerns

• std::fs alignment

  • Familiar method names
  • Core traits use &Path; FileStorage/FsExt accept impl AsRef<Path> for ergonomics

• Developer experience first

  • Make storage composition easy, safe, and enjoyable to use

Quick example

#![allow(unused)]
fn main() {
use anyfs::{MemoryBackend, QuotaLayer, RestrictionsLayer, FileStorage};

// Layer-based composition
let backend = MemoryBackend::new()
    .layer(QuotaLayer::builder()
        .max_total_size(100 * 1024 * 1024)
        .build())
    .layer(RestrictionsLayer::builder()
        .deny_permissions()
        .build());

let fs = FileStorage::new(backend);

fs.create_dir_all("/documents")?;
fs.write("/documents/hello.txt", b"Hello!")?;
}

Status

For details, see Design Overview.

AnyFS - Project Structure

Status: Target layout (design spec) Last updated: 2025-12-24

This manual describes the intended code repository layout; this repository contains documentation only.

Repository Layout

anyfs-backend/              # Crate 1: traits + types (minimal dependencies)
  Cargo.toml
  src/
    lib.rs
    traits/
      fs_read.rs            # FsRead trait
      fs_write.rs           # FsWrite trait
      fs_dir.rs             # FsDir trait
      fs_link.rs            # FsLink trait
      fs_permissions.rs     # FsPermissions trait
      fs_sync.rs            # FsSync trait
      fs_stats.rs           # FsStats trait
      fs_path.rs            # FsPath trait (canonicalization, blanket impl)
      fs_inode.rs           # FsInode trait
      fs_handles.rs         # FsHandles trait
      fs_lock.rs            # FsLock trait
      fs_xattr.rs           # FsXattr trait
    layer.rs                # Layer trait (Tower-style)
    ext.rs                  # FsExt (extension methods)
    markers.rs              # SelfResolving marker trait
    path_resolver.rs        # PathResolver trait (pluggable resolution)
    types.rs                # Metadata, DirEntry, Permissions, StatFs
    error.rs                # FsError

anyfs/                      # Crate 2: framework glue (simple backends + middleware + ergonomics)
  Cargo.toml
  src/
    lib.rs
    backends/
      memory.rs             # MemoryBackend (in-memory, simple)
      stdfs.rs              # StdFsBackend (thin std::fs wrapper)
      vrootfs.rs            # VRootFsBackend (std::fs + path containment)
    middleware/
      quota.rs              # Quota<B>
      restrictions.rs       # Restrictions<B>
      path_filter.rs        # PathFilter<B>
      read_only.rs          # ReadOnly<B>
      rate_limit.rs         # RateLimit<B>
      tracing.rs            # Tracing<B>
      dry_run.rs            # DryRun<B>
      cache.rs              # Cache<B>
      overlay.rs            # Overlay<B1, B2>
    resolvers/
      iterative.rs          # IterativeResolver (default)
      noop.rs               # NoOpResolver (for SelfResolving backends)
      caching.rs            # CachingResolver (LRU cache wrapper)
    container.rs            # FileStorage<B>
    stack.rs                # BackendStack builder

Ecosystem Crates (Separate Repositories)

Complex backends with internal runtime requirements:

anyfs-sqlite/               # SQLite backend (connection pooling, WAL, sharding)
anyfs-indexed/              # Hybrid backend (SQLite index + disk blobs)
anyfs-s3/                   # Third-party: S3 backend
anyfs-redis/                # Third-party: Redis backend

Testing Crate

anyfs-test/                 # Conformance test suite for backend implementers
  src/
    lib.rs
    conformance/            # Test generators for each trait level
      fs_tests.rs           # Fs-level tests
      fs_full_tests.rs      # FsFull-level tests
      fs_fuse_tests.rs      # FsFuse-level tests
    prelude.rs              # Re-exports for test files

Dependency Model

anyfs-backend (trait + types)
     ^
     |-- anyfs (backends + middleware + ergonomics)
     |     ^-- vrootfs feature uses strict-path

Key points:

• Custom backends depend only on anyfs-backend
• anyfs provides built-in backends, middleware, mounting (behind feature flags), and the ergonomic FileStorage<B> wrapper

Middleware Pattern

FileStorage<B>
    wraps -> Tracing<B>
        wraps -> Restrictions<B>
            wraps -> Quota<B>
                wraps -> MemoryBackend (or any Fs)

Each layer implements Fs, enabling composition.

Cargo Features

Backends (anyfs crate)

• memory — In-memory storage (default)
• stdfs — Direct std::fs delegation (no containment)
• vrootfs — Host filesystem backend with path containment (uses strict-path)

Ecosystem Backends (Separate Crates)

Complex backends live in their own crates:

• anyfs-sqlite — SQLite-backed persistent storage (pooling, WAL, sharding, encryption)
• anyfs-indexed — SQLite index + disk blobs for large file performance

Middleware (MVP Scope)

Core middleware is always available (no feature flags needed):

• Quota, PathFilter, Restrictions, ReadOnly, RateLimit, Cache, DryRun, Overlay

Optional middleware with external dependencies:

• tracing — Detailed audit logging (requires tracing crate)

Mounting (Platform Features)

• fuse — Mount as filesystem on Linux/macOS (requires fuser crate)
• winfsp — Mount as filesystem on Windows (requires winfsp crate)

Use default-features = false to cherry-pick exactly what you need.

Middleware (Future Scope)

• metrics — Prometheus integration (requires prometheus crate)

Where To Start

• Application usage: Getting Started Guide
• Trait details: Layered Traits
• Middleware: Design Overview
• Decisions: Architecture Decision Records

AnyFS — Getting Started Guide

A practical introduction with examples

Installation

Add to your Cargo.toml:

[dependencies]
anyfs = "0.1"

For additional backends:

[dependencies]
anyfs = { version = "0.1", features = ["stdfs", "vrootfs"] }

# SQLite storage (ecosystem crate)
anyfs-sqlite = "0.1"
anyfs-sqlite = { version = "0.1", features = ["encryption"] }  # With SQLCipher

# Hybrid backend: SQLite metadata + disk blobs (ecosystem crate)
anyfs-indexed = "0.1"

Available anyfs features:

• memory — In-memory storage (default)
• stdfs — Direct std::fs delegation (no containment)
• vrootfs — Host filesystem backend with path containment
• bytes — Zero-copy Bytes support (adds read_bytes() method)
• tracing — Detailed audit logging (requires tracing crate)
• fuse — Mount as filesystem on Linux/macOS
• winfsp — Mount as filesystem on Windows

Core middleware (Quota, PathFilter, Restrictions, ReadOnly, RateLimit, Cache, DryRun, Overlay) is always available.

Quick Start

Examples below use FileStorage, so you can pass paths as &str. If you call core trait methods directly, use &Path.

Hello World

use anyfs::{MemoryBackend, FileStorage};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let fs = FileStorage::new(MemoryBackend::new());

    fs.write("/hello.txt", b"Hello, AnyFS!")?;
    let content = fs.read("/hello.txt")?;
    println!("{}", String::from_utf8_lossy(&content));

    Ok(())
}

With Quotas

use anyfs::{MemoryBackend, QuotaLayer, FileStorage};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let backend = QuotaLayer::builder()
        .max_total_size(100 * 1024 * 1024)  // 100 MB
        .max_file_size(10 * 1024 * 1024)    // 10 MB per file
        .build()
        .layer(MemoryBackend::new());

    let fs = FileStorage::new(backend);

    fs.create_dir_all("/documents")?;
    fs.write("/documents/notes.txt", b"Meeting notes")?;

    Ok(())
}

With Restrictions

use anyfs::{MemoryBackend, RestrictionsLayer, FileStorage};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Block permission changes for untrusted code
    let backend = RestrictionsLayer::builder()
        .deny_permissions()   // Block set_permissions() calls
        .build()
        .layer(MemoryBackend::new());

    let fs = FileStorage::new(backend);

    // All other operations work normally
    fs.write("/file.txt", b"content")?;

    Ok(())
}

Full Stack (Layer-based)

use anyfs::{MemoryBackend, QuotaLayer, RestrictionsLayer, TracingLayer, FileStorage};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let backend = MemoryBackend::new()
        .layer(QuotaLayer::builder()
            .max_total_size(100 * 1024 * 1024)
            .max_file_size(10 * 1024 * 1024)
            .build())
        .layer(RestrictionsLayer::builder()
            .deny_permissions()
            .build())
        .layer(TracingLayer::new());

    let fs = FileStorage::new(backend);

    fs.create_dir_all("/data")?;
    fs.write("/data/file.txt", b"hello")?;

    Ok(())
}

Common Operations

Creating Directories

#![allow(unused)]
fn main() {
fs.create_dir("/documents")?;              // Single level
fs.create_dir_all("/documents/2024/q1")?;  // Recursive
}

Reading and Writing Files

#![allow(unused)]
fn main() {
fs.write("/data.txt", b"line 1\n")?;       // Create or overwrite
fs.append("/data.txt", b"line 2\n")?;      // Append

let content = fs.read("/data.txt")?;                    // Read all
let partial = fs.read_range("/data.txt", 0, 6)?;        // Read range
let text = fs.read_to_string("/data.txt")?;             // Read as String
}

Listing Directories

#![allow(unused)]
fn main() {
for entry in fs.read_dir("/documents")? {
    println!("{}: {:?}", entry.name, entry.file_type);
}
}

Checking Existence and Metadata

#![allow(unused)]
fn main() {
if fs.exists("/file.txt")? {
    let meta = fs.metadata("/file.txt")?;
    println!("Size: {} bytes", meta.size);
}
}

Copying and Moving

#![allow(unused)]
fn main() {
fs.copy("/original.txt", "/copy.txt")?;
fs.rename("/original.txt", "/renamed.txt")?;
}

Deleting

#![allow(unused)]
fn main() {
fs.remove_file("/old-file.txt")?;
fs.remove_dir("/empty-folder")?;
fs.remove_dir_all("/old-folder")?;
}

Middleware

Quota — Resource Limits

#![allow(unused)]
fn main() {
use anyfs::{MemoryBackend, Quota};

let backend = QuotaLayer::builder()
    .max_total_size(500 * 1024 * 1024)   // 500 MB total
    .max_file_size(50 * 1024 * 1024)     // 50 MB per file
    .max_node_count(100_000)             // 100K files/dirs
    .max_dir_entries(5_000)              // 5K per directory
    .max_path_depth(32)                  // Max nesting
    .build()
    .layer(MemoryBackend::new());

// Check usage
let usage = backend.usage();
println!("Using {} bytes", usage.total_size);

// Check remaining
let remaining = backend.remaining();
if !remaining.can_write {
    println!("Storage full!");
}
}

Restrictions — Block Permission Changes

#![allow(unused)]
fn main() {
use anyfs::{MemoryBackend, RestrictionsLayer};

// Restrictions controls permission-related operations.
// Symlink/hard-link capability is determined by trait bounds (FsLink).
let backend = RestrictionsLayer::builder()
    .deny_permissions()   // Block set_permissions() calls
    .build()
    .layer(MemoryBackend::new());

// Blocked operations return FsError::FeatureNotEnabled
}

Tracing — Instrumentation

#![allow(unused)]
fn main() {
use anyfs::{MemoryBackend, TracingLayer};

// TracingLayer uses the global tracing subscriber by default
let backend = MemoryBackend::new().layer(TracingLayer::new());

// Or configure with custom settings
let backend = MemoryBackend::new()
    .layer(TracingLayer::new()
        .with_target("myapp::fs")
        .with_level(tracing::Level::DEBUG));
}

Error Handling

#![allow(unused)]
fn main() {
use anyfs_backend::FsError;

match fs.write("/file.txt", &large_data) {
    Ok(()) => println!("Written"),

    Err(FsError::NotFound { path, .. }) => println!("Not found: {}", path.display()),
    Err(FsError::AlreadyExists { path, .. }) => println!("Exists: {}", path.display()),
    Err(FsError::QuotaExceeded { .. }) => println!("Quota exceeded"),
    Err(FsError::FeatureNotEnabled { feature }) => println!("Feature disabled: {}", feature),

    Err(e) => println!("Error: {}", e),
}
}

Testing

#![allow(unused)]
fn main() {
use anyfs::{MemoryBackend, FileStorage};

#[test]
fn test_write_and_read() {
    let fs = FileStorage::new(MemoryBackend::new());

    fs.write("/test.txt", b"test data").unwrap();
    let content = fs.read("/test.txt").unwrap();

    assert_eq!(content, b"test data");
}
}

With limits:

#![allow(unused)]
fn main() {
use anyfs::{MemoryBackend, Quota, FileStorage};

#[test]
fn test_quota_exceeded() {
    let backend = QuotaLayer::builder()
        .max_total_size(1024)  // 1 KB
        .build()
        .layer(MemoryBackend::new());
    let fs = FileStorage::new(backend);

    let big_data = vec![0u8; 2048];  // 2 KB
    let result = fs.write("/big.bin", &big_data);

    assert!(result.is_err());
}
}

Best Practices

1. Use Appropriate Backend

2. Compose Middleware for Your Needs

#![allow(unused)]
fn main() {
// Minimal: just storage
let fs = FileStorage::new(MemoryBackend::new());

// With limits (layer-based)
let fs = FileStorage::new(
    MemoryBackend::new()
        .layer(QuotaLayer::builder()
            .max_total_size(100 * 1024 * 1024)
            .build())
);

// Sandboxed (layer-based)
let temp_dir = tempfile::tempdir()?;
let fs = FileStorage::new(
    VRootFsBackend::new(temp_dir.path())?
        .layer(QuotaLayer::builder()
            .max_total_size(100 * 1024 * 1024)
            .build())
        .layer(RestrictionsLayer::builder()
            .deny_permissions()
            .build())
);
}

3. Handle Errors Gracefully

Always check for quota exceeded, feature not enabled, and other errors.

Advanced Use Cases

These use cases require the fuse or winfsp feature flags.

Database-Backed Drive with Live Monitoring

Mount a database-backed filesystem and query it directly for real-time analytics:

┌─────────────────────────────────────────────────────────────┐
│  Database (SQLite, PostgreSQL, etc.)                        │
├─────────────────────────────────────────────────────────────┤
│                         │                                   │
│    MountHandle          │         Stats Dashboard           │
│    (write + read)       │         (direct DB queries)       │
│         │               │               │                   │
│         ▼               │               ▼                   │
│  /mnt/workspace         │    SELECT SUM(size) FROM nodes    │
│  $ cp file.txt ./       │    SELECT COUNT(*) FROM nodes     │
│  $ mkdir projects/      │    SELECT * FROM audit_log        │
│                         │               │                   │
│                         │               ▼                   │
│                         │        ┌──────────────┐           │
│                         │        │ Live Graphs  │           │
│                         │        │ - Disk usage │           │
│                         │        │ - File count │           │
│                         │        │ - Recent ops │           │
│                         │        └──────────────┘           │
└─────────────────────────────────────────────────────────────┘

SQLite Example (using ecosystem crate - API sketch, planned):

#![allow(unused)]
fn main() {
use anyfs::{QuotaLayer, TracingLayer, MountHandle};
use anyfs_sqlite::SqliteBackend;  // Ecosystem crate

// Mount the drive
let backend = SqliteBackend::open("tenant.db")?
    .layer(TracingLayer::new())  // Logs operations to tracing subscriber
    .layer(QuotaLayer::builder()
        .max_total_size(1_000_000_000)
        .build());

let mount = MountHandle::mount(backend, "/mnt/workspace")?;
}

#![allow(unused)]
fn main() {
// Meanwhile, in a monitoring dashboard...
// Note: This queries SqliteBackend's internal schema (nodes, audit_log tables).
// See anyfs-sqlite documentation for schema details.
use rusqlite::{Connection, OpenFlags};

let conn = Connection::open_with_flags(
    "tenant.db",
    OpenFlags::SQLITE_OPEN_READ_ONLY,  // Safe concurrent reads
)?;

loop {
    let (file_count, total_bytes): (i64, i64) = conn.query_row(
        "SELECT COUNT(*), COALESCE(SUM(size), 0) FROM nodes WHERE type = 'file'",
        [],
        |row| Ok((row.get(0)?, row.get(1)?)),
    )?;

    let recent_ops: Vec<String> = conn
        .prepare("SELECT operation, path, timestamp FROM audit_log ORDER BY timestamp DESC LIMIT 10")?
        .query_map([], |row| Ok(format!("{}: {}", row.get::<_, String>(0)?, row.get::<_, String>(1)?)))?
        .collect::<Result<_, _>>()?;

    render_dashboard(file_count, total_bytes, &recent_ops);
    std::thread::sleep(Duration::from_secs(1));
}
}

Works with any database backend:

Third-party crates can implement additional database backends (PostgreSQL, MySQL, etc.) following the same pattern.

What you can visualize:

• Real-time storage usage (gauges, bar charts)
• File count over time (line graphs)
• Operations log (live feed)
• Most accessed files (heatmaps)
• Directory tree maps (size visualization)
• Per-tenant usage (multi-tenant dashboards)

This pattern is powerful because the database is the source of truth — you get filesystem semantics via FUSE and SQL analytics via direct queries, from the same data.

RAM Drive

#![allow(unused)]
fn main() {
use anyfs::{MemoryBackend, QuotaLayer, MountHandle};

// 4GB RAM drive
let mount = MountHandle::mount(
    MemoryBackend::new()
        .layer(QuotaLayer::builder()
            .max_total_size(4 * 1024 * 1024 * 1024)
            .build()),
    "/mnt/ramdisk"
)?;

// Use for fast compilation, temp files, etc.
// $ TMPDIR=/mnt/ramdisk cargo build
}

Sandboxed AI Agent Workspace

#![allow(unused)]
fn main() {
use anyfs::{MemoryBackend, QuotaLayer, PathFilterLayer, RestrictionsLayer, TracingLayer, MountHandle};

let mount = MountHandle::mount(
    MemoryBackend::new()
        .layer(PathFilterLayer::builder()
            .allow("/workspace/**")
            .deny("**/..*")           // No hidden files
            .deny("**/.*")            // No dotfiles
            .build())
        .layer(QuotaLayer::builder()
            .max_total_size(100 * 1024 * 1024)
            .max_file_size(10 * 1024 * 1024)
            .build())
        .layer(TracingLayer::new()),  // Full audit trail
    "/mnt/agent"
)?;

// Agent uses standard filesystem APIs
// All operations are sandboxed, quota-limited, and logged
}

Next Steps

• API Quick Reference
• Design Overview
• Backend Implementer’s Guide

AnyFS — API Quick Reference

Condensed reference for developers

Installation

[dependencies]
anyfs = "0.1"

With optional features and ecosystem crates:

anyfs = { version = "0.1", features = ["vrootfs", "bytes"] }
anyfs-sqlite = "0.1"  # SQLite backend (ecosystem crate)

Creating a Backend Stack

#![allow(unused)]
fn main() {
use anyfs::{MemoryBackend, QuotaLayer, RestrictionsLayer, TracingLayer, FileStorage};

// Simple
let fs = FileStorage::new(MemoryBackend::new());

// With limits
let fs = FileStorage::new(
    MemoryBackend::new()
        .layer(QuotaLayer::builder()
            .max_total_size(100 * 1024 * 1024)
            .max_file_size(10 * 1024 * 1024)
            .build())
);

// Full stack (fluent composition)
let backend = MemoryBackend::new()
    .layer(QuotaLayer::builder()
        .max_total_size(100 * 1024 * 1024)
        .build())
    .layer(RestrictionsLayer::builder()
        .deny_permissions()
        .build())
    .layer(TracingLayer::new());

let fs = FileStorage::new(backend);

// BackendStack builder (fluent API)
use anyfs::BackendStack;

let fs = BackendStack::new(MemoryBackend::new())
    .limited(|l| l.max_total_size(100 * 1024 * 1024))
    .restricted(|g| g.deny_permissions())
    .traced()
    .into_container();
}

BackendStack Methods

BackendStack provides a fluent API for building middleware stacks:

#![allow(unused)]
fn main() {
use anyfs::BackendStack;

BackendStack::new(backend)           // Start with any backend
    .limited(|l| l                   // -> Quota<B>
        .max_total_size(bytes)
        .max_file_size(bytes)
        .max_node_count(count))
    .restricted(|g| g                // -> Restrictions<B>
        .deny_permissions())
    .traced()                        // -> Tracing<B>
    .cached(|c| c                    // -> Cache<B>
        .max_size(bytes)
        .ttl(duration))
    .read_only()                     // -> ReadOnly<B>
    .into_container()                // -> FileStorage<...>
}

Quota Methods

#![allow(unused)]
fn main() {
// Builder pattern (required - at least one limit must be set)
QuotaLayer::builder()
    .max_total_size(bytes)      // Total storage limit
    .max_file_size(bytes)       // Per-file limit
    .max_node_count(count)      // Max files/dirs
    .max_dir_entries(count)     // Max entries per dir
    .max_path_depth(depth)      // Max nesting
    .build()
    .layer(backend)

// Query
backend.usage()        // -> Usage { total_size, file_count, ... }
backend.limits()       // -> Limits { max_total_size, ... }
backend.remaining()    // -> Remaining { bytes, can_write, ... }
}

Restrictions Methods

#![allow(unused)]
fn main() {
// Builder pattern
// Restrictions only controls permission-related operations.
// Symlink/hard-link capability is via trait bounds (FsLink), not middleware.
RestrictionsLayer::builder()
    .deny_permissions()    // Block set_permissions() calls
    .build()
    .layer(backend)
}

TracingLayer Methods

#![allow(unused)]
fn main() {
// TracingLayer configuration (applied via .layer())
TracingLayer::new()
    .with_target("anyfs")              // tracing target
    .with_level(tracing::Level::DEBUG)

// Usage
let backend = inner.layer(TracingLayer::new().with_target("anyfs"));
}

PathFilter Methods

#![allow(unused)]
fn main() {
// Builder pattern (required - at least one rule must be set)
PathFilterLayer::builder()
    .allow("/workspace/**")    // Allow glob pattern
    .deny("**/.env")           // Deny glob pattern
    .deny("**/secrets/**")
    .build()
    .layer(backend)

// Rules evaluated in order; first match wins
// No match = denied (deny by default)
}

ReadOnly Methods

#![allow(unused)]
fn main() {
ReadOnly::new(backend)

// All read operations pass through
// All write operations return FsError::ReadOnly
}

RateLimit Methods

#![allow(unused)]
fn main() {
// Builder pattern (required - must set ops and window)
RateLimitLayer::builder()
    .max_ops(1000)           // Operation limit
    .per_second()            // Window: 1 second
    // or
    .per_minute()            // Window: 60 seconds
    // or
    .per(Duration::from_millis(500))  // Custom window
    .build()
    .layer(backend)
}

DryRun Methods

#![allow(unused)]
fn main() {
let dry_run = DryRun::new(backend);
let fs = FileStorage::new(dry_run);

// Read operations execute normally
// Write operations are logged but not executed
fs.write("/file.txt", b"data")?;  // Logged, returns Ok

// Inspect logged operations (returns Vec<String>)
let ops: Vec<String> = dry_run.operations();
// e.g., ["write /file.txt (4 bytes)", "remove_file /old.txt"]

dry_run.clear();  // Clear the log
}

Cache Methods

#![allow(unused)]
fn main() {
// Builder pattern (required - at least max_entries must be set)
CacheLayer::builder()
    .max_entries(1000)                          // LRU cache size
    .max_entry_size(1024 * 1024)               // 1MB max per entry
    .ttl(Duration::from_secs(300))             // Optional: entry lifetime (default: no expiry)
    .build()
    .layer(backend)
}

IndexLayer Methods (Future)

#![allow(unused)]
fn main() {
// Builder pattern (required - set index path)
IndexLayer::builder()
    .index_file("index.db")             // Sidecar index file (SQLite default)
    .consistency(IndexConsistency::Strict)
    .track_reads(false)                 // Optional
    .build()
    .layer(backend)
}

Overlay Methods

#![allow(unused)]
fn main() {
use anyfs::{VRootFsBackend, MemoryBackend, Overlay};

let base = VRootFsBackend::new("/var/templates")?;  // Read-only base
let upper = MemoryBackend::new();                    // Writable upper

let overlay = Overlay::new(base, upper);

// Read: check upper first, fall back to base
// Write: always to upper layer
// Delete: whiteout marker in upper
}

FsExt Methods

Extension methods available on all backends:

#![allow(unused)]
fn main() {
use anyfs_backend::FsExt;

// JSON support (requires `serde` feature on anyfs-backend)
let config: Config = fs.read_json("/config.json")?;
fs.write_json("/config.json", &config)?;

// Type checks
if fs.is_file("/path")? { ... }
if fs.is_dir("/path")? { ... }
}

Note: JSON methods require anyfs-backend = { version = "0.1", features = ["serde"] }

File Operations

Examples below assume FileStorage (std::fs-style paths). If you call core trait methods directly, pass &Path.

#![allow(unused)]
fn main() {
// Check existence
fs.exists("/path")?                     // -> bool

// Metadata
let meta = fs.metadata("/path")?;
meta.inode                               // unique identifier
meta.nlink                               // hard link count
meta.file_type                           // File | Directory | Symlink
meta.size                                // file size in bytes
meta.permissions                         // Permissions (default if unsupported)
meta.created                             // SystemTime (UNIX_EPOCH if unsupported)
meta.modified                            // SystemTime (UNIX_EPOCH if unsupported)
meta.accessed                            // SystemTime (UNIX_EPOCH if unsupported)

// Read
let bytes = fs.read("/path")?;           // -> Vec<u8>
let text = fs.read_to_string("/path")?;  // -> String
let chunk = fs.read_range("/path", 0, 1024)?;

// List directory
for entry in fs.read_dir("/path")? {
    let entry = entry?;
    entry.name                           // String (file/dir name only)
    entry.path                           // PathBuf (full path)
    entry.file_type                      // File | Directory | Symlink
    entry.size                           // u64 (0 for directories)
    entry.inode                          // u64 (0 if unsupported)
}

// Write
fs.write("/path", b"content")?;          // Create or overwrite
fs.append("/path", b"more")?;            // Append

// Directories
fs.create_dir("/path")?;
fs.create_dir_all("/path")?;

// Delete
fs.remove_file("/path")?;
fs.remove_dir("/path")?;                 // Empty only
fs.remove_dir_all("/path")?;             // Recursive

// Move/Copy
fs.rename("/from", "/to")?;
fs.copy("/from", "/to")?;

// Links
fs.symlink("/target", "/link")?;
fs.hard_link("/original", "/link")?;
fs.read_link("/link")?;                  // -> PathBuf
fs.symlink_metadata("/link")?;           // Metadata of link itself, not target
// Symlink capability is determined by FsLink trait bounds, not middleware.

// Permissions (requires FsPermissions)
fs.set_permissions("/path", perms)?;

// File size
fs.truncate("/path", 1024)?;             // Resize to 1024 bytes

// Durability
fs.sync()?;                              // Flush all writes
fs.fsync("/path")?;                      // Flush writes for one file
}

Path Canonicalization

FileStorage provides path canonicalization that works on the virtual filesystem.

Note: Canonicalization requires FsLink because symlink resolution needs read_link() and symlink_metadata(). Backends that only implement Fs (without FsLink) cannot use these methods.

#![allow(unused)]
fn main() {
// Strict canonicalization - path must exist
let canonical = fs.canonicalize("/some/../path/./file.txt")?;
// Returns fully resolved absolute path, follows symlinks

// Soft canonicalization - handles non-existent paths
let resolved = fs.soft_canonicalize("/existing/dir/new_file.txt")?;
// Resolves existing components, appends non-existent remainder lexically

// Anchored canonicalization - sandboxed resolution
let safe = fs.anchored_canonicalize("/workspace/../etc/passwd", "/workspace")?;
// Clamps result within anchor directory (returns error if escape attempted)
}

Standalone utility (no backend needed):

#![allow(unused)]
fn main() {
use anyfs::normalize;

// Lexical path cleanup only
let clean = normalize("//foo///bar//");  // -> "/foo/bar"
// Does NOT resolve . or .. (those require filesystem context)
// Does NOT follow symlinks
}

Comparison:

Inode Operations (FsInode trait)

Backends implementing FsInode track inodes internally for hardlink support and FUSE mounting:

#![allow(unused)]
fn main() {
use anyfs::FileStorage;

// Convert between paths and inodes
let fs = FileStorage::new(backend);
let inode: u64 = fs.path_to_inode("/some/path")?;
let path: PathBuf = fs.inode_to_path(inode)?;

// Lookup child by name within a directory (FUSE-style)
let root_inode = fs.path_to_inode("/")?;
let child_inode = fs.lookup(root_inode, "filename.txt")?;

// Get metadata by inode (avoids path resolution)
let meta = fs.metadata_by_inode(inode)?;

// Hardlinks share the same inode
fs.hard_link("/original", "/link")?;
let ino1 = fs.path_to_inode("/original")?;
let ino2 = fs.path_to_inode("/link")?;
assert_eq!(ino1, ino2);  // Same inode!
}

Inode sources by backend:

Error Handling

#![allow(unused)]
fn main() {
use anyfs_backend::FsError;

match result {
    // Path errors
    Err(FsError::NotFound { path }) => {
        // e.g., path="/file.txt"
    }
    Err(FsError::AlreadyExists { path, operation }) => ...
    Err(FsError::NotADirectory { path }) => ...
    Err(FsError::NotAFile { path }) => ...
    Err(FsError::DirectoryNotEmpty { path }) => ...
    Err(FsError::SymlinkLoop { path }) => ...  // Circular symlink detected

    // Quota middleware errors\n    Err(FsError::QuotaExceeded { path, limit, requested, usage }) => ...\n    Err(FsError::FileSizeExceeded { path, size, limit }) => ...\n\n    // Restrictions middleware errors\n    Err(FsError::FeatureNotEnabled { path, feature, operation }) => ...\n\n    // PathFilter middleware errors\n    Err(FsError::AccessDenied { path, reason }) => ...\n\n    // ReadOnly middleware errors\n    Err(FsError::ReadOnly { path, operation }) => ...\n\n    // RateLimit middleware errors\n    Err(FsError::RateLimitExceeded { path, limit, window_secs }) => ...

    // FsExt errors
    Err(FsError::Serialization(msg)) => ...
    Err(FsError::Deserialization(msg)) => ...

    // Optional feature not supported
    Err(FsError::NotSupported { operation }) => ...

    Err(e) => ...
}
}

Built-in Backends (anyfs crate)

Ecosystem Backends (Separate Crates)

SqliteBackend Encryption (Ecosystem Crate)

Crate: anyfs-sqlite with encryption feature

#![allow(unused)]
fn main() {
use anyfs_sqlite::SqliteBackend;

// Standard (no encryption)
let backend = SqliteBackend::open("data.db")?;

// With encryption (requires `encryption` feature)
let backend = SqliteBackend::open_encrypted("encrypted.db", "password")?;

// With raw 256-bit key
let backend = SqliteBackend::open_with_key("encrypted.db", &key)?;

// Change password on open encrypted database
backend.change_password("new-password")?;
}

Without the correct password, the .db file appears as random bytes.

Feature: anyfs-sqlite = { version = "0.1", features = ["encryption"] }

Middleware

Layers

Note: Overlay<B1, B2> is constructed directly via Overlay::new(base, upper) rather than using a Layer, because it takes two backends.

Type Reference

From anyfs-backend

Core Traits (Layer 1):

Extended Traits (Layer 2):

Inode Trait (Layer 3):

POSIX Traits (Layer 4):

Convenience Supertraits:

Other Types:

From anyfs

From Ecosystem Crates

Ergonomic Wrappers (in anyfs):

AnyFS - Design Overview

Status: Current Last updated: 2025-12-24

What This Project Is

AnyFS is an open standard for pluggable virtual filesystem backends in Rust. It uses a middleware/decorator pattern (like Axum/Tower) for composable functionality with complete separation of concerns.

Philosophy: Focused App, Smart Storage

It decouples application logic from storage policy, enabling a Data Mesh at the filesystem level.

• The App focuses on business value (“save the document”).
• The Storage Layer enforces non-functional requirements (“encrypt, audit, limit, index”).

Anyone can:

• Control how a drive acts, looks, and protects itself.
• Implement a custom backend for their specific storage needs (Cloud, DB, RAM).
• Compose middleware to add limits, logging, and security.
• Use the ergonomic FileStorage<B> wrapper for a standard std::fs-like API.

Architecture (Tower-style Middleware)

┌─────────────────────────────────────────┐
│  FileStorage<B>                         │  ← Ergonomic std::fs-aligned API
├─────────────────────────────────────────┤
│  Middleware (optional, composable):     │
│                                         │
│  Policy:                                │
│    Quota<B>         - Resource limits   │
│    Restrictions<B>  - Least privilege   │
│    PathFilter<B>    - Sandbox paths     │
│    ReadOnly<B>      - Prevent writes    │
│    RateLimit<B>     - Ops/sec limit     │
│                                         │
│  Observability:                         │
│    Tracing<B>       - Instrumentation   │
│    DryRun<B>        - Test mode         │
│                                         │
│  Performance:                           │
│    Cache<B>         - LRU caching       │
│                                         │
│  Composition:                           │
│    Overlay<B1,B2>   - Layered FS        │
│                                         │
├─────────────────────────────────────────┤
│  Backend (implements Fs, FsFull,        │  ← Pure storage + fs semantics
│           FsFuse, or FsPosix)           │
│  (Memory, SQLite, VRootFs, custom...)   │
└─────────────────────────────────────────┘

Each layer has exactly one responsibility:

Design Principle: Predictable Defaults, Opt-in Security

The Fs traits mimic std::fs with predictable, permissive defaults.

See ADR-027 for the decision rationale.

The traits are low-level interfaces that any backend can implement - memory, SQLite, real filesystem, network storage, etc. To maintain consistent behavior across all backends:

• All operations work by default (symlink(), hard_link(), set_permissions())
• No security restrictions at the trait level
• Behavior matches what you’d expect from a real filesystem

Why not secure-by-default at this layer?

1. Predictability: A backend should behave like a filesystem. Surprising restrictions break expectations.
2. Backend-agnostic: The traits don’t know if they’re wrapping a sandboxed memory store or a real filesystem. Restrictions that make sense for one may not for another.
3. Composition: Security is achieved by layering middleware, not by baking it into the storage layer.

Security is the responsibility of higher-level APIs:

Example: Secure AI Agent Sandbox

#![allow(unused)]
fn main() {
use anyfs::{MemoryBackend, QuotaLayer, PathFilterLayer, FileStorage};

// Create wrapper type for type-safe sandbox
struct AiSandbox(FileStorage<MemoryBackend>);

impl AiSandbox {
    fn new() -> Self {
        AiSandbox(FileStorage::new(
            MemoryBackend::new()
                .layer(QuotaLayer::builder()
                    .max_total_size(50 * 1024 * 1024)
                    .build())
                .layer(PathFilterLayer::builder()
                    .allow("/workspace/**")
                    .deny("**/.env")
                    .build())
        ))
    }
}
}

The backend is permissive. The application adds restrictions appropriate for its use case.

Crates

Dependency Graph

anyfs-backend (trait + types)
     ^
     |-- anyfs (backends + middleware + ergonomics)
           ^-- vrootfs feature may use strict-path

Future Considerations

These are optional extensions to explore after the core is stable.

Keep (add-ons that fit the current design):

• URL-based backend registry (sqlite://, mem://, stdfs://) as a helper crate, not in core APIs.
• Bulk operation helpers (read_many, write_many, copy_many, glob, walk) as FsExt or a utilities crate.
• Early async adapter crate (anyfs-async) to support remote backends without changing sync traits.
• Bash-style shell (example app or anyfs-shell crate) that routes ls/cd/cat/cp/mv/rm/mkdir/stat through FileStorage to demonstrate middleware and backend neutrality (navigation and file management only, not full bash scripting).
• Copy-on-write overlay middleware (Afero-style CopyOnWriteFs) as a specialized Overlay variant.
• Archive backends (zip/tar) as separate crates implementing Fs (inspired by PyFilesystem/fsspec).
• Indexing middleware (Indexing<B> + IndexLayer) with pluggable index engines (SQLite default). See Indexing Middleware.

Defer (valuable, but needs data or wider review):

• Range/block caching middleware for read_range heavy workloads (fsspec-style block cache).
• Runtime capability discovery (Capabilities struct) for feature detection (symlink control, case sensitivity, max path length).
• Lint/analyzer to discourage direct std::fs usage in app code (System.IO.Abstractions-style).
• Retry/timeout middleware for remote backends (when network backends are real).

Drop for now (adds noise or cross-platform complexity):

• Change notification support (optional FsWatch trait or polling middleware).

Detailed rationale lives in src/comparisons/prior-art-analysis.md.

Language Bindings (Python, C, etc.)

The AnyFS design is FFI-friendly and can be exposed to other languages with minimal friction.

Why the design works well for FFI:

Recommended approach for Python (PyO3):

#![allow(unused)]
fn main() {
// anyfs-python/src/lib.rs
use pyo3::prelude::*;
use anyfs::{FileStorage, MemoryBackend, Fs};
use anyfs_sqlite::SqliteBackend;  // Ecosystem crate

#[pyclass]
struct PyFileStorage {
    // Type-erased for FFI
    inner: FileStorage<Box<dyn Fs>>,
}

#[pymethods]
impl PyFileStorage {
    #[staticmethod]
    fn memory() -> Self {
        Self { inner: FileStorage::new(MemoryBackend::new()).boxed() }
    }

    #[staticmethod]
    fn sqlite(path: &str) -> PyResult<Self> {
        let backend = SqliteBackend::open(path)
            .map_err(|e| PyErr::new::<pyo3::exceptions::PyIOError, _>(e.to_string()))?;
        Ok(Self { inner: FileStorage::new(backend).boxed() })
    }

    fn read(&self, path: &str) -> PyResult<Vec<u8>> {
        self.inner.read(path)
            .map_err(|e| PyErr::new::<pyo3::exceptions::PyIOError, _>(e.to_string()))
    }

    fn write(&self, path: &str, data: &[u8]) -> PyResult<()> {
        self.inner.write(path, data)
            .map_err(|e| PyErr::new::<pyo3::exceptions::PyIOError, _>(e.to_string()))
    }
}

#[pymodule]
fn anyfs_python(_py: Python, m: &PyModule) -> PyResult<()> {
    m.add_class::<PyFileStorage>()?;
    Ok(())
}
}

Python usage:

from anyfs_python import PyFileStorage

fs = PyFileStorage.memory()
fs.write("/hello.txt", b"Hello from Python!")
data = fs.read("/hello.txt")
print(data)  # b"Hello from Python!"

Key considerations for FFI:

Future crate: anyfs-python

Dynamic Middleware

The current design uses compile-time generics for zero-cost middleware composition:

#![allow(unused)]
fn main() {
// Static: type known at compile time
let fs: Tracing<Quota<MemoryBackend>> = MemoryBackend::new()
    .layer(QuotaLayer::builder().max_total_size(100).build())
    .layer(TracingLayer::new());
}

For runtime-configured middleware (e.g., based on config files), use Box<dyn Fs>:

#![allow(unused)]
fn main() {
fn build_from_config(config: &Config) -> FileStorage<Box<dyn Fs>> {
    let mut backend: Box<dyn Fs> = Box::new(MemoryBackend::new());

    if config.enable_quota {
        let quota_config = QuotaConfig {
            max_total_size: Some(config.quota_limit),
            ..Default::default()
        };
        backend = Box::new(Quota::with_config(backend, quota_config)
            .expect("quota initialization failed"));
    }

    if config.enable_antivirus {
        backend = Box::new(AntivirusMiddleware::new(backend, config.av_scanner_path));
    }

    if config.enable_tracing {
        backend = Box::new(Tracing::new(backend));
    }

    FileStorage::new(backend)
}
}

Trade-off: One Box allocation per layer + vtable dispatch. For I/O-bound workloads, this overhead is negligible (<1% of operation time).

Example: Antivirus Middleware

#![allow(unused)]
fn main() {
pub struct Antivirus<B> {
    inner: B,
    scanner: Arc<dyn VirusScanner + Send + Sync>,
}

pub trait VirusScanner: Send + Sync {
    fn scan(&self, data: &[u8]) -> Option<String>;  // Returns threat name if detected
}

impl<B: FsWrite> FsWrite for Antivirus<B> {
    fn write(&self, path: &Path, data: &[u8]) -> Result<(), FsError> {
        if let Some(threat) = self.scanner.scan(data) {
            return Err(FsError::ThreatDetected { 
                path: path.to_path_buf(), 
                reason: threat,
            });
        }
        self.inner.write(path, data)
    }

    fn open_write(&self, path: &Path) -> Result<Box<dyn Write + Send>, FsError> {
        let inner = self.inner.open_write(path)?;
        Ok(Box::new(ScanningWriter::new(inner, self.scanner.clone())))
    }
}
}

Future: Plugin System

For true runtime-loaded plugins (.so/.dll), a future MiddlewarePlugin trait could enable:

#![allow(unused)]
fn main() {
pub trait MiddlewarePlugin: Send + Sync {
    fn name(&self) -> &str;
    fn wrap(&self, backend: Box<dyn Fs>) -> Box<dyn Fs>;
}

// Load at runtime
let plugin = libloading::Library::new("antivirus_plugin.so")?;
let create_plugin: fn() -> Box<dyn MiddlewarePlugin> = plugin.get(b"create_plugin")?;
let av_plugin = create_plugin();

let backend = av_plugin.wrap(backend);
}

When to use each approach:

Verdict: The current design supports dynamic middleware via Box<dyn Fs>. A formal MiddlewarePlugin trait for hot-loading is a future enhancement.

Middleware with Configurable Backends

Some middleware benefit from pluggable backends for their own storage or output. The pattern is to inject a trait object or configuration at construction time.

Metrics Middleware with Prometheus Exporter: (Requires features = ["metrics"])

#![allow(unused)]
fn main() {
use prometheus::{Counter, Histogram, Registry};

pub struct Metrics<B> {
    inner: B,
    reads: Counter,
    writes: Counter,
    read_bytes: Counter,
    write_bytes: Counter,
    latency: Histogram,
}

impl<B> Metrics<B> {
    /// Creates a new Metrics middleware.
    ///
    /// # Panics
    /// Panics if metric registration fails (indicates duplicate metric names - programmer error).
    /// This is acceptable at initialization time per the No Panic Policy, which applies to
    /// runtime operations. Initialization failures are configuration errors that should fail fast.
    pub fn new(inner: B, registry: &Registry) -> Self {
        let reads = Counter::new("anyfs_reads_total", "Total read operations")
            .expect("metric creation failed");
        let writes = Counter::new("anyfs_writes_total", "Total write operations")
            .expect("metric creation failed");
        registry.register(Box::new(reads.clone()))
            .expect("metric registration failed");
        registry.register(Box::new(writes.clone()))
            .expect("metric registration failed");
        // ... register all metrics
        Self { inner, reads, writes, /* ... */ }
    }
}

impl<B: FsRead> FsRead for Metrics<B> {
    fn read(&self, path: &Path) -> Result<Vec<u8>, FsError> {
        self.reads.inc();
        let start = Instant::now();
        let result = self.inner.read(path);
        self.latency.observe(start.elapsed().as_secs_f64());
        if let Ok(ref data) = result {
            self.read_bytes.inc_by(data.len() as u64);
        }
        result
    }
}

// Expose via HTTP endpoint
async fn metrics_handler(registry: web::Data<Registry>) -> impl Responder {
    let encoder = TextEncoder::new();
    let metrics = registry.gather();
    encoder.encode_to_string(&metrics)
        .unwrap_or_else(|e| format!("# Encoding error: {}", e))
}
}

Indexing Middleware with Remote Database:

#![allow(unused)]
fn main() {
pub trait IndexBackend: Send + Sync {
    fn record_write(&self, path: &Path, size: u64, hash: &str) -> Result<(), IndexError>;
    fn record_delete(&self, path: &Path) -> Result<(), IndexError>;
    fn query(&self, pattern: &str) -> Result<Vec<IndexEntry>, IndexError>;
}

// SQLite implementation
pub struct SqliteIndex { conn: Connection }

// PostgreSQL implementation  
pub struct PostgresIndex { pool: PgPool }

// MariaDB implementation
pub struct MariaDbIndex { pool: MySqlPool }

pub struct Indexing<B, I: IndexBackend> {
    inner: B,
    index: I,
}

impl<B: FsWrite, I: IndexBackend> FsWrite for Indexing<B, I> {
    fn write(&self, path: &Path, data: &[u8]) -> Result<(), FsError> {
        self.inner.write(path, data)?;
        let hash = sha256(data);
        self.index.record_write(path, data.len() as u64, &hash)
            .map_err(|e| FsError::Backend(e.to_string()))?;
        Ok(())
    }
}

// Usage with PostgreSQL
let index = PostgresIndex::connect("postgres://user:pass@db.example.com/files").await?;
let backend = MemoryBackend::new()
    .layer(IndexLayer::builder()
        .index(index)
        .build());
}

Configurable Tracing with Multiple Sinks:

#![allow(unused)]
fn main() {
pub trait TraceSink: Send + Sync {
    fn log_operation(&self, op: &Operation);
}

// Structured JSON logs
pub struct JsonSink { writer: Box<dyn Write + Send> }

// CEF (Common Event Format) for SIEM integration
pub struct CefSink { 
    host: String,
    port: u16,
    device_vendor: String 
}

impl TraceSink for CefSink {
    fn log_operation(&self, op: &Operation) {
        let cef = format!(
            "CEF:0|AnyFS|FileStorage|1.0|{}|{}|{}|src={} dst={}",
            op.event_id, op.name, op.severity, op.source_path, op.dest_path
        );
        self.send_syslog(&cef);
    }
}

// Remote sink (e.g., Loki, Elasticsearch)
pub struct RemoteSink { endpoint: String, client: reqwest::Client }

pub struct Tracing<B, S: TraceSink> {
    inner: B,
    sink: S,
}
}

Performance: Strategic Boxing (ADR-025)

AnyFS follows Tower/Axum’s approach to dynamic dispatch: zero-cost on the hot path, box at boundaries where flexibility is needed. We avoid heap allocations and dynamic dispatch unless they add flexibility without meaningful performance impact.

Hot-loop guidance: If you open many small files and care about micro-overhead (especially on virtual backends), prefer read()/write() or the typed streaming extension (FsReadTyped/FsWriteTyped) when the backend type is known. These are the zero-allocation fast paths.

Why box streams and iterators?

1. Middleware needs to wrap them (QuotaWriter counts bytes, PathFilter filters entries)
2. Box allocation (~50ns) is <1% of actual I/O time
3. Avoids type explosion: QuotaReader<PathFilterReader<TracingReader<Cursor<...>>>>

Why NOT box bulk operations?

1. read() and write() are the most common operations
2. They return concrete types (Vec<u8>, ())
3. Zero overhead for the typical use case

See ADR-025 and Zero-Cost Alternatives for full analysis.

Trait Architecture (in anyfs-backend)

AnyFS uses layered traits for maximum flexibility with minimal complexity.

See ADR-030 for the rationale behind the layered hierarchy.

                        FsPosix
                           │
            ┌──────────────┼──────────────┐
            │              │              │
       FsHandles      FsLock       FsXattr
            │              │              │
            └──────────────┴──────────────┘
                           │
                        FsFuse ← FsFull + FsInode
                           │
            ┌──────────────┴──────────────┐
            │                             │
         FsFull                       FsInode
            │
            │
            ├──────┬───────┬───────┬──────┐
            │      │       │       │      │
       FsLink  FsPerm  FsSync FsStats │
            │      │       │       │      │
            └──────┴───────┴───────┴──────┘
                           │
                           Fs  ← Most users only need this
                           │
               ┌───────────┼───────────┐
               │           │           │
            FsRead    FsWrite     FsDir

Simple rule: Import Fs for basic use. Add traits as needed for advanced features.

Core Traits (Layer 1)

FsRead - Read Operations

#![allow(unused)]
fn main() {
pub trait FsRead: Send + Sync {
    fn read(&self, path: &Path) -> Result<Vec<u8>, FsError>;
    fn read_to_string(&self, path: &Path) -> Result<String, FsError>;
    fn read_range(&self, path: &Path, offset: u64, len: usize) -> Result<Vec<u8>, FsError>;
    fn exists(&self, path: &Path) -> Result<bool, FsError>;
    fn metadata(&self, path: &Path) -> Result<Metadata, FsError>;
    fn open_read(&self, path: &Path) -> Result<Box<dyn Read + Send>, FsError>;
}
}

FsWrite - Write Operations

#![allow(unused)]
fn main() {
pub trait FsWrite: Send + Sync {
    fn write(&self, path: &Path, data: &[u8]) -> Result<(), FsError>;
    fn append(&self, path: &Path, data: &[u8]) -> Result<(), FsError>;
    fn remove_file(&self, path: &Path) -> Result<(), FsError>;
    fn rename(&self, from: &Path, to: &Path) -> Result<(), FsError>;
    fn copy(&self, from: &Path, to: &Path) -> Result<(), FsError>;
    fn truncate(&self, path: &Path, size: u64) -> Result<(), FsError>;
    fn open_write(&self, path: &Path) -> Result<Box<dyn Write + Send>, FsError>;
}
}

Note: All methods use &self (interior mutability). Backends manage their own synchronization. See ADR-023.

FsDir - Directory Operations

#![allow(unused)]
fn main() {
pub trait FsDir: Send + Sync {
    fn read_dir(&self, path: &Path) -> Result<ReadDirIter, FsError>;
    fn create_dir(&self, path: &Path) -> Result<(), FsError>;
    fn create_dir_all(&self, path: &Path) -> Result<(), FsError>;
    fn remove_dir(&self, path: &Path) -> Result<(), FsError>;
    fn remove_dir_all(&self, path: &Path) -> Result<(), FsError>;
}
}

Extended Traits (Layer 2 - Optional)

#![allow(unused)]
fn main() {
pub trait FsLink: Send + Sync {
    fn symlink(&self, original: &Path, link: &Path) -> Result<(), FsError>;
    fn hard_link(&self, original: &Path, link: &Path) -> Result<(), FsError>;
    fn read_link(&self, path: &Path) -> Result<PathBuf, FsError>;
    fn symlink_metadata(&self, path: &Path) -> Result<Metadata, FsError>;
}

pub trait FsPermissions: Send + Sync {
    fn set_permissions(&self, path: &Path, perm: Permissions) -> Result<(), FsError>;
}

pub trait FsSync: Send + Sync {
    fn sync(&self) -> Result<(), FsError>;
    fn fsync(&self, path: &Path) -> Result<(), FsError>;
}

pub trait FsStats: Send + Sync {
    fn statfs(&self) -> Result<StatFs, FsError>;
}
}

Inode Traits (Layer 3 - For FUSE)

#![allow(unused)]
fn main() {
pub trait FsInode: Send + Sync {
    fn path_to_inode(&self, path: &Path) -> Result<u64, FsError>;
    fn inode_to_path(&self, inode: u64) -> Result<PathBuf, FsError>;
    fn lookup(&self, parent_inode: u64, name: &OsStr) -> Result<u64, FsError>;
    fn metadata_by_inode(&self, inode: u64) -> Result<Metadata, FsError>;
}
}

POSIX Traits (Layer 4 - Full POSIX)

#![allow(unused)]
fn main() {
pub trait FsHandles: Send + Sync {
    fn open(&self, path: &Path, flags: OpenFlags) -> Result<Handle, FsError>;
    fn read_at(&self, handle: Handle, buf: &mut [u8], offset: u64) -> Result<usize, FsError>;
    fn write_at(&self, handle: Handle, data: &[u8], offset: u64) -> Result<usize, FsError>;
    fn close(&self, handle: Handle) -> Result<(), FsError>;
}

pub trait FsLock: Send + Sync {
    fn lock(&self, handle: Handle, lock: LockType) -> Result<(), FsError>;
    fn try_lock(&self, handle: Handle, lock: LockType) -> Result<bool, FsError>;
    fn unlock(&self, handle: Handle) -> Result<(), FsError>;
}

pub trait FsXattr: Send + Sync {
    fn get_xattr(&self, path: &Path, name: &str) -> Result<Vec<u8>, FsError>;
    fn set_xattr(&self, path: &Path, name: &str, value: &[u8]) -> Result<(), FsError>;
    fn remove_xattr(&self, path: &Path, name: &str) -> Result<(), FsError>;
    fn list_xattr(&self, path: &Path) -> Result<Vec<String>, FsError>;
}
}

Convenience Supertraits (Simple API)

#![allow(unused)]
fn main() {
/// Basic filesystem - covers 90% of use cases
pub trait Fs: FsRead + FsWrite + FsDir {}
impl<T: FsRead + FsWrite + FsDir> Fs for T {}

/// Full filesystem with all std::fs features
pub trait FsFull: Fs + FsLink + FsPermissions + FsSync + FsStats {}
impl<T: Fs + FsLink + FsPermissions + FsSync + FsStats> FsFull for T {}

/// FUSE-mountable filesystem
pub trait FsFuse: FsFull + FsInode {}
impl<T: FsFull + FsInode> FsFuse for T {}

/// Full POSIX filesystem
pub trait FsPosix: FsFuse + FsHandles + FsLock + FsXattr {}
impl<T: FsFuse + FsHandles + FsLock + FsXattr> FsPosix for T {}
}

Usage Examples

Application code should use FileStorage for the std::fs-style DX (string paths). Core trait examples are shown separately for implementers and generic code.

Most Users: FileStorage

#![allow(unused)]
fn main() {
use anyfs::{FileStorage, MemoryBackend};

fn process_files() -> Result<(), Box<dyn std::error::Error>> {
    let fs = FileStorage::new(MemoryBackend::new());
    let data = fs.read("/input.txt")?;
    fs.write("/output.txt", &processed(data))?;
    Ok(())
}
}

Generic Code over Core Traits

#![allow(unused)]
fn main() {
use anyfs::{FileStorage, Fs, FsError};

fn process_files<B: Fs>(fs: &FileStorage<B>) -> Result<(), FsError> {
    let data = fs.read("/input.txt")?;
    fs.write("/output.txt", &processed(data))?;
    Ok(())
}
}

Need Links? Add the Trait

#![allow(unused)]
fn main() {
use anyfs::{FileStorage, Fs, FsLink, FsError};

fn with_symlinks<B: Fs + FsLink>(fs: &FileStorage<B>) -> Result<(), FsError> {
    fs.write("/target.txt", b"content")?;
    fs.symlink("/target.txt", "/link.txt")?;
    Ok(())
}
}

FUSE Mount

Mounting is part of anyfs crate with fuse and winfsp feature flags; see src/guides/mounting.md.

#![allow(unused)]
fn main() {
use anyfs::{FsFuse, MountHandle, MountError};

fn mount_filesystem(fs: impl FsFuse) -> Result<(), MountError> {
    MountHandle::mount(fs, "/mnt/myfs")?;
    Ok(())
}
}

Full POSIX Application

#![allow(unused)]
fn main() {
use anyfs::{FileStorage, FsPosix, FsError, OpenFlags, LockType, Handle};

fn database_app<B: FsPosix>(fs: &FileStorage<B>, data: &[u8], offset: u64) -> Result<(), FsError> {
    let handle: Handle = fs.open("/data.db", OpenFlags::READ_WRITE)?;
    fs.lock(handle, LockType::Exclusive)?;
    fs.write_at(handle, data, offset)?;
    fs.unlock(handle)?;
    fs.close(handle)?;
    Ok(())
}
}

Core Types (in anyfs-backend)

Constants

#![allow(unused)]
fn main() {
/// Root directory inode. FUSE convention.
pub const ROOT_INODE: u64 = 1;
}

Metadata

#![allow(unused)]
fn main() {
/// File or directory metadata.
#[derive(Debug, Clone)]
pub struct Metadata {
    /// Type: File, Directory, or Symlink.
    pub file_type: FileType,

    /// Size in bytes (0 for directories).
    pub size: u64,

    /// Permission mode bits. Default to 0o755/0o644 if unsupported.
    pub permissions: Permissions,

    /// Creation time (UNIX_EPOCH if unsupported).
    pub created: SystemTime,

    /// Last modification time.
    pub modified: SystemTime,

    /// Last access time.
    pub accessed: SystemTime,

    /// Inode number (0 if unsupported).
    pub inode: u64,

    /// Number of hard links (1 if unsupported).
    pub nlink: u64,
}

impl Metadata {
    /// Check if this is a file.
    pub fn is_file(&self) -> bool { self.file_type == FileType::File }

    /// Check if this is a directory.
    pub fn is_dir(&self) -> bool { self.file_type == FileType::Directory }

    /// Check if this is a symlink.
    pub fn is_symlink(&self) -> bool { self.file_type == FileType::Symlink }
}
}

FileType

#![allow(unused)]
fn main() {
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum FileType {
    File,
    Directory,
    Symlink,
}
}

DirEntry

#![allow(unused)]
fn main() {
/// Entry in a directory listing.
#[derive(Debug, Clone)]
pub struct DirEntry {
    /// File or directory name (not full path).
    pub name: String,

    /// Full path to the entry.
    pub path: PathBuf,

    /// Type: File, Directory, or Symlink.
    pub file_type: FileType,

    /// Size in bytes (0 for directories, can be lazy).
    pub size: u64,

    /// Inode number (0 if unsupported).
    pub inode: u64,
}
}

Permissions

#![allow(unused)]
fn main() {
/// Unix-style permission bits.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct Permissions(u32);

impl Permissions {
    /// Create permissions from a mode (e.g., 0o755).
    pub fn from_mode(mode: u32) -> Self { Permissions(mode) }

    /// Get the mode bits.
    pub fn mode(&self) -> u32 { self.0 }

    /// Read-only permissions (0o444).
    pub fn readonly() -> Self { Permissions(0o444) }

    /// Default file permissions (0o644).
    pub fn default_file() -> Self { Permissions(0o644) }

    /// Default directory permissions (0o755).
    pub fn default_dir() -> Self { Permissions(0o755) }
}
}

StatFs

#![allow(unused)]
fn main() {
/// Filesystem statistics.
#[derive(Debug, Clone)]
pub struct StatFs {
    /// Total size in bytes (0 = unlimited).
    pub total_bytes: u64,

    /// Used bytes.
    pub used_bytes: u64,

    /// Available bytes.
    pub available_bytes: u64,

    /// Total number of inodes (0 = unlimited).
    pub total_inodes: u64,

    /// Used inodes.
    pub used_inodes: u64,

    /// Available inodes.
    pub available_inodes: u64,

    /// Filesystem block size.
    pub block_size: u64,

    /// Maximum filename length.
    pub max_name_len: u64,
}
}

Middleware (in anyfs)

Each middleware implements the same traits as its inner backend. This enables composition while preserving capabilities.

Quota

Enforces quota limits. Tracks usage and rejects operations that would exceed limits.

#![allow(unused)]
fn main() {
use anyfs::{MemoryBackend, QuotaLayer};

let backend = QuotaLayer::builder()
    .max_total_size(100 * 1024 * 1024)   // 100 MB
    .max_file_size(10 * 1024 * 1024)     // 10 MB per file
    .max_node_count(10_000)              // 10K files/dirs
    .max_dir_entries(1_000)              // 1K entries per dir
    .max_path_depth(64)
    .build()
    .layer(MemoryBackend::new());

// Check usage
let usage = backend.usage();
let remaining = backend.remaining();
}

Restrictions

Blocks permission-related operations when needed.

#![allow(unused)]
fn main() {
use anyfs::{MemoryBackend, Restrictions};

// Symlink/hard-link capability is determined by trait bounds (FsLink).
// Restrictions only controls permission changes.
let backend = RestrictionsLayer::builder()
    .deny_permissions()    // Block set_permissions() calls
    .build()
    .layer(MemoryBackend::new());
}

When blocked, operations return FsError::FeatureNotEnabled.

Tracing

Integrates with the tracing ecosystem for structured logging and instrumentation.

#![allow(unused)]
fn main() {
use anyfs::{MemoryBackend, TracingLayer};

let backend = MemoryBackend::new()
    .layer(TracingLayer::new()
        .with_target("anyfs")
        .with_level(tracing::Level::DEBUG));

// Users configure tracing subscribers as they prefer
tracing_subscriber::fmt::init();
}

Why tracing instead of custom logging?

• Works with existing tracing infrastructure
• Structured logging with spans
• Compatible with OpenTelemetry, Jaeger, etc.
• Users choose their subscriber (console, file, distributed tracing)

PathFilter

Restricts access to specific paths. Essential for sandboxing.

#![allow(unused)]
fn main() {
use anyfs::{MemoryBackend, PathFilterLayer};

let backend = PathFilterLayer::builder()
    .allow("/workspace/**")           // Allow all under /workspace
    .allow("/tmp/**")                  // Allow temp files
    .deny("/workspace/.env")           // But deny .env files
    .deny("**/.git/**")               // Deny all .git directories
    .build()
    .layer(MemoryBackend::new());
}

When a path is denied, operations return FsError::AccessDenied.

ReadOnly

Prevents all write operations. Useful for publishing immutable data.

#![allow(unused)]
fn main() {
use anyfs::{VRootFsBackend, ReadOnly, FileStorage};

// Wrap any backend to make it read-only
let backend = ReadOnly::new(VRootFsBackend::new("/var/published")?);
let fs = FileStorage::new(backend);

fs.read("/doc.txt")?;     // OK
fs.write("/doc.txt", b"x"); // Error: FsError::ReadOnly
}

RateLimit

Limits operations per second. Prevents runaway agents.

#![allow(unused)]
fn main() {
use anyfs::{MemoryBackend, RateLimitLayer};

let backend = RateLimitLayer::builder()
    .max_ops(100)       // 100 ops per window
    .per_second()       // 1 second window
    .build()
    .layer(MemoryBackend::new());

// When rate exceeded: FsError::RateLimitExceeded
}

DryRun

Logs operations without executing writes. Great for testing and debugging.

#![allow(unused)]
fn main() {
use anyfs::{MemoryBackend, DryRun, FileStorage};

let backend = DryRun::new(MemoryBackend::new());
let fs = FileStorage::new(backend);

fs.write("/test.txt", b"hello")?;  // Logged but not written
let _ = fs.read("/test.txt");       // Error: file doesn't exist

// To inspect recorded operations, keep the DryRun handle before wrapping it.
}

Cache

LRU cache for read operations. Essential for slow backends (S3, network).

#![allow(unused)]
fn main() {
use anyfs::{MemoryBackend, CacheLayer, FileStorage};

let backend = MemoryBackend::new()
    .layer(CacheLayer::builder()
        .max_entries(10_000)              // Max 10K entries in cache
        .max_entry_size(10 * 1024 * 1024) // 10 MB max per entry
        .build());
let fs = FileStorage::new(backend);

// First read: hits backend, caches result
let data = fs.read("/file.txt")?;

// Second read: served from cache (fast!)
let data = fs.read("/file.txt")?;
}

Overlay<Base, Upper>

Union filesystem with a read-only base and writable upper layer. Like Docker.

#![allow(unused)]
fn main() {
use anyfs::{VRootFsBackend, MemoryBackend, Overlay};

// Base: read-only template
let base = VRootFsBackend::new("/var/templates")?;

// Upper: writable layer for changes
let upper = MemoryBackend::new();

let backend = Overlay::new(base, upper);

// Reads check upper first, then base
// Writes always go to upper
// Deletes in upper "shadow" base files
}

Use cases:

• Container images (base image + writable layer)
• Template filesystems with per-user modifications
• Testing with rollback capability

FileStorage (in anyfs)

FileStorage<B> is an ergonomic wrapper with a single generic parameter:

• B - Backend type (the only generic)
• Resolver is boxed internally (cold path, per ADR-025)

Axum-style design: Simple by default, type erasure opt-in via .boxed().

Basic Usage

#![allow(unused)]
fn main() {
use anyfs::{MemoryBackend, FileStorage};

// Type is inferred - no need to write it out
let fs = FileStorage::new(MemoryBackend::new());

fs.create_dir_all("/documents")?;
fs.write("/documents/hello.txt", b"Hello!")?;
let content = fs.read("/documents/hello.txt")?;
}

Type-Safe Wrappers (User-Defined)

If you need compile-time safety to prevent mixing filesystems, create wrapper types:

#![allow(unused)]
fn main() {
use anyfs::{MemoryBackend, FileStorage};
use anyfs_sqlite::SqliteBackend;  // Ecosystem crate

// Define wrapper types for your domains
struct SandboxFs(FileStorage<MemoryBackend>);
struct UserDataFs(FileStorage<SqliteBackend>);

// Type-safe function signatures prevent mixing
fn process_sandbox(fs: &SandboxFs) {
    // Can only accept SandboxFs
}

fn save_user_file(fs: &UserDataFs, name: &str, data: &[u8]) {
    // Can only accept UserDataFs
}

// Compile-time safety:
let sandbox = SandboxFs(FileStorage::new(MemoryBackend::new()));
process_sandbox(&sandbox);   // OK
// process_sandbox(&userdata);  // Compile error! Wrong type
}

Type Aliases for Clean Code

#![allow(unused)]
fn main() {
use anyfs_sqlite::SqliteBackend;  // Ecosystem crate

// Define your standard secure stack
type SecureBackend = Tracing<Restrictions<Quota<SqliteBackend>>>;

// Type aliases for common combinations
type SandboxFs = FileStorage<MemoryBackend>;
type UserDataFs = FileStorage<SecureBackend>;

// Clean function signatures
fn run_agent(fs: &SandboxFs) { ... }
}

FileStorage Implementation

#![allow(unused)]
fn main() {
use anyfs_backend::PathResolver;

/// Ergonomic wrapper with single generic.
pub struct FileStorage<B> {
    backend: B,
    resolver: Box<dyn PathResolver>,  // Boxed: cold path
}

impl<B: Fs> FileStorage<B> {
    /// Create with default resolver (IterativeResolver).
    pub fn new(backend: B) -> Self { ... }

    /// Create with custom path resolver.
    pub fn with_resolver(backend: B, resolver: impl PathResolver + 'static) -> Self { ... }

    /// Type-erase the backend (opt-in boxing).
    pub fn boxed(self) -> FileStorage<Box<dyn Fs>> { ... }
}
}

Type Erasure (Opt-in)

When you need uniform types (e.g., collections), use .boxed():

#![allow(unused)]
fn main() {
use anyfs_sqlite::SqliteBackend;  // Ecosystem crate

// Type-erased for uniform storage
let filesystems: Vec<FileStorage<Box<dyn Fs>>> = vec![
    FileStorage::new(MemoryBackend::new()).boxed(),
    FileStorage::new(SqliteBackend::open("a.db")?).boxed(),
];
}

Layer Trait (in anyfs-backend)

The Layer trait (inspired by Tower) standardizes middleware composition:

#![allow(unused)]
fn main() {
/// A layer that wraps a backend to add functionality.
pub trait Layer<B: Fs> {
    type Backend: Fs;
    fn layer(self, backend: B) -> Self::Backend;
}

/// Extension trait enabling fluent `.layer()` method on any Fs.
/// This is how `backend.layer(QuotaLayer::builder()...build())` works.
pub trait LayerExt: Fs + Sized {
    fn layer<L: Layer<Self>>(self, layer: L) -> L::Backend {
        layer.layer(self)
    }
}

// Blanket impl: any Fs gets .layer() for free
impl<B: Fs> LayerExt for B {}
}

Each middleware provides a corresponding Layer implementation:

#![allow(unused)]
fn main() {
// QuotaLayer wraps QuotaConfig (not a separate QuotaLimits type)
pub struct QuotaLayer {
    config: QuotaConfig,
}

impl<B: Fs> Layer<B> for QuotaLayer {
    type Backend = Quota<B>;
    fn layer(self, backend: B) -> Self::Backend {
        Quota::with_config(backend, self.config)
            .expect("quota initialization failed")
    }
}
}

Note: Middleware that implements additional traits (like FsInode) can use more specific bounds to preserve capabilities through the layer.

Composing Middleware

Middleware composes by wrapping. Order matters - innermost applies first.

Fluent Composition

Use the .layer() extension method for Axum-style composition:

#![allow(unused)]
fn main() {
use anyfs::{MemoryBackend, QuotaLayer, RestrictionsLayer, TracingLayer};

let backend = MemoryBackend::new()
    .layer(QuotaLayer::builder()
        .max_total_size(100 * 1024 * 1024)
        .build())
    .layer(RestrictionsLayer::builder()
        .deny_permissions()  // Block set_permissions()
        .build())
    .layer(TracingLayer::new());
}

BackendStack Builder

For complex stacks, use BackendStack for a fluent API:

#![allow(unused)]
fn main() {
use anyfs::BackendStack;

let fs = BackendStack::new(MemoryBackend::new())
    .limited(|l| l
        .max_total_size(100 * 1024 * 1024)
        .max_file_size(10 * 1024 * 1024))
    .restricted(|g| g
        .deny_permissions())    // Block set_permissions() calls
    .traced()
    .into_container();
}

Built-in Backends (anyfs crate)

Ecosystem Backends (Separate Crates)

Complex backends with internal runtime requirements live in their own crates:

Why separate crates? Complex backends need internal runtimes (connection pools, sharding, chunking). Keeps anyfs lightweight and focused on framework glue.

Path Handling

Core traits take &Path so they are object-safe (dyn Fs works). The ergonomic layer (FileStorage and FsExt) accepts impl AsRef<Path>:

#![allow(unused)]
fn main() {
// These work via FileStorage/FsExt
fs.write("/file.txt", data)?;
fs.write(String::from("/file.txt"), data)?;
fs.write(PathBuf::from("/file.txt"), data)?;
}

Path Resolution

Path resolution (walking directory structure, following symlinks) operates on the Fs abstraction, not reimplemented per-backend.

See ADR-029 for the path-resolution decision.

Why Abstract Path Resolution?

We simulate inodes - that’s the whole point of virtualizing a filesystem. Path resolution must work on that abstraction:

• /foo/../bar cannot be resolved lexically - foo might be a symlink to /other/place, making .. resolve to /other
• Resolution requires following the actual directory structure (inodes)
• The Fs traits have the needed methods: metadata(), read_link(), read_dir()

Path Resolution via PathResolver Trait

FileStorage delegates path resolution to a pluggable PathResolver (see ADR-033). The default IterativeResolver walks paths component by component:

#![allow(unused)]
fn main() {
/// Default resolver algorithm (simplified):
/// - Walk path component by component
/// - Use backend.metadata() to check node types
/// - If backend implements FsLink, use read_link() to follow symlinks
/// - Detect circular symlinks (max depth: 40)
/// - Return fully resolved canonical path
pub struct IterativeResolver {
    max_symlink_depth: usize,  // Default: 40
}
}

Resolution behavior depends on the resolver used. The default IterativeResolver follows symlinks when the backend implements FsLink. For backends without FsLink, it traverses directories but treats symlinks as regular files. Users can provide custom resolvers for case-insensitive matching, caching, or other behaviors.

Note: Built-in virtual backends (MemoryBackend) and ecosystem backends (SqliteBackend) implement FsLink, so symlink-aware resolution works out of the box.

When Resolution Is Needed

Opt-out Mechanism

Virtual backends need resolution by default. Real filesystem backends opt out via a marker trait:

#![allow(unused)]
fn main() {
/// Marker trait for backends that handle their own path resolution.
/// VRootFsBackend implements this because the OS handles resolution.
pub trait SelfResolving {}

impl SelfResolving for VRootFsBackend {}
}

Important: FileStorage does NOT auto-detect SelfResolving. You must explicitly use NoOpResolver:

#![allow(unused)]
fn main() {
// For SelfResolving backends, use NoOpResolver explicitly
let fs = FileStorage::with_resolver(VRootFsBackend::new("/data")?, NoOpResolver);
}

The default IterativeResolver follows symlinks when FsLink is available. Custom resolvers can implement different behaviors (e.g., no symlink following, caching, case-insensitivity).

#![allow(unused)]
fn main() {
impl<B: Fs> FileStorage<B> {
    pub fn new(backend: B) -> Self { /* uses IterativeResolver */ }
    pub fn with_resolver(backend: B, resolver: impl PathResolver + 'static) -> Self { /* custom resolver */ }
}
}

Path Canonicalization Utilities

FileStorage provides path canonicalization methods modeled after the soft-canonicalize crate, adapted to work on the virtual filesystem abstraction.

Why We Need Our Own Canonicalization

std::fs::canonicalize operates on the real filesystem. For virtual backends (MemoryBackend, SqliteBackend), there is no real filesystem - we need canonicalization that queries the virtual structure via metadata() and read_link().

Core Methods

#![allow(unused)]
fn main() {
impl<B: Fs> FileStorage<B> {
    /// Strict canonicalization - entire path must exist.
    ///
    /// Delegates to the PathResolver to resolve symlinks and normalize the path.
    /// Returns error if any component doesn't exist.
    pub fn canonicalize(&self, path: impl AsRef<Path>) -> Result<PathBuf, FsError> {
        self.resolver.canonicalize(path.as_ref(), &self.backend as &dyn Fs)
    }

    /// Soft canonicalization - resolves existing components,
    /// appends non-existent remainder lexically.
    ///
    /// Delegates to the PathResolver.
    pub fn soft_canonicalize(&self, path: impl AsRef<Path>) -> Result<PathBuf, FsError> {
        self.resolver.soft_canonicalize(path.as_ref(), &self.backend as &dyn Fs)
    }

    /// Anchored soft canonicalization - like soft_canonicalize but
    /// clamps result within a boundary directory.
    ///
    /// Useful for sandboxing: ensures the resolved path never escapes
    /// the anchor directory, even via symlinks or `..` traversal.
    pub fn anchored_canonicalize(
        &self,
        path: impl AsRef<Path>,
        anchor: impl AsRef<Path>
    ) -> Result<PathBuf, FsError>;
}

/// Standalone lexical normalization (no backend needed).
///
/// Pure string manipulation:
/// - Collapses `//` to `/`
/// - Removes trailing slashes
/// - Does NOT resolve `.` or `..` (those require filesystem context)
/// - Does NOT follow symlinks
pub fn normalize(path: impl AsRef<Path>) -> PathBuf;
}

Algorithm: Component-by-Component Resolution

The canonicalization algorithm walks the path one component at a time:

Input: /a/b/c/d/e

1. Start at root (/)
2. Check /a exists?
   - Yes, and it's a symlink → follow to target
   - Yes, and it's a directory → continue
3. Check /a/b exists?
   - Yes → continue
4. Check /a/b/c exists?
   - No → stop resolution, append "c/d/e" lexically
5. Result: /resolved/path/to/b/c/d/e

Key behaviors:

• Symlink following: Existing symlinks are resolved to their targets
• Non-existent handling: When a component doesn’t exist, the remainder is appended as-is
• Cycle detection: Bounded depth tracking prevents infinite loops from circular symlinks
• Root boundary: Never ascends past the filesystem root

Comparison with std::fs

Security Considerations

For virtual backends: Canonicalization happens entirely within the virtual structure. There is no host filesystem to escape to.

For VRootFsBackend: Delegates to OS canonicalization + strict-path containment. The anchored_canonicalize provides additional safety by clamping paths within a boundary.

Platform Notes (VRootFsBackend only)

When delegating to OS canonicalization:

• Windows: Returns extended-length UNC paths (\\?\C:\path) by default
• Linux/macOS: Standard canonical paths

Windows UNC Path Simplification

The dunce crate provides simplified() - a lexical function that converts UNC paths to regular paths without filesystem access:

#![allow(unused)]
fn main() {
use dunce::simplified;

// \\?\C:\Users\foo\bar.txt → C:\Users\foo\bar.txt
let path = simplified(r"\\?\C:\Users\foo\bar.txt");
}

Why this matters for soft_canonicalize:

• soft_canonicalize works with non-existent paths
• We can’t use dunce::canonicalize (requires path to exist)
• dunce::simplified is pure string manipulation - works on any path

When UNC can be simplified:

• Path is on a local drive (C:, D:, etc.)
• Path doesn’t exceed MAX_PATH (260 chars)
• No reserved names (CON, PRN, etc.)

When UNC must be kept:

• Network paths (\\?\UNC\server\share)
• Paths exceeding MAX_PATH
• Paths with reserved device names

Virtual backends have no platform differences - paths are just strings.

Filesystem Semantics: Linux-like by Default

Design principle: Simple, secure defaults. Don’t close doors for alternative semantics.

See ADR-028 for the decision rationale.

Default Behavior (Virtual Backends)

Virtual backends (MemoryBackend, SqliteBackend) use Linux-like semantics:

Real filesystem backends (StdFsBackend, VRootFsBackend) follow OS semantics—case-insensitive on Windows/macOS, case-sensitive on Linux.

Trait is Agnostic

The Fs trait doesn’t enforce filesystem semantics - backends decide their behavior:

#![allow(unused)]
fn main() {
use anyfs::{FileStorage, MemoryBackend};
use std::path::Path;

// Virtual backends: Linux-like (case-sensitive)
let linux_fs = FileStorage::new(MemoryBackend::new());
assert!(linux_fs.exists("/Foo.txt")? != linux_fs.exists("/foo.txt")?);

// For case-insensitive behavior, implement a custom PathResolver:
// (Not built-in because real-world demand is minimal - VRootFsBackend on 
// Windows/macOS already gets case-insensitivity from the OS)
struct CaseFoldingResolver;
impl PathResolver for CaseFoldingResolver {
    fn canonicalize(&self, path: &Path, fs: &dyn Fs) -> Result<PathBuf, FsError> {
        // Normalize path components to lowercase during lookup
        todo!()
    }
    
    fn soft_canonicalize(&self, path: &Path, fs: &dyn Fs) -> Result<PathBuf, FsError> {
        // Same but allows non-existent final component
        todo!()
    }
}

let ntfs_like = FileStorage::with_resolver(
    MemoryBackend::new(),
    CaseFoldingResolver  // User-implemented
);
}

FUSE Mount: Report What You Support

When mounting, the FUSE layer reports backend capabilities to the OS:

#![allow(unused)]
fn main() {
impl FuseOps for AnyFsFuse<B> {
    fn get_volume_params(&self) -> VolumeParams {
        VolumeParams {
            case_sensitive: self.backend.is_case_sensitive(),
            supports_hard_links: /* check if B: FsLink */,
            supports_symlinks: /* check if B: FsLink */,
            // ...
        }
    }
}
}

Windows respects these flags - a case-sensitive mounted filesystem works correctly (modern Windows/WSL handle this).

Illustrative: Custom Middleware for Windows Compatibility

For users who need Windows-safe paths in virtual backends, here are example middleware patterns (not built-in - implement as needed):

#![allow(unused)]
fn main() {
/// Example: Middleware that validates paths are Windows-compatible.
/// Rejects: CON, PRN, NUL, COM1-9, LPT1-9, trailing dots/spaces, ADS.
pub struct NtfsValidation<B> { /* user-implemented */ }

/// Example: Middleware that makes a backend case-insensitive.
/// Stores canonical (lowercase) keys, preserves original case in metadata.
pub struct CaseInsensitive<B> { /* user-implemented */ }
}

Not built-in - these are illustrative patterns for users who need NTFS-like behavior.

Security Model

Security is achieved through composition:

Defense in depth: Compose multiple middleware layers for comprehensive security.

AI Agent Sandbox Example

#![allow(unused)]
fn main() {
use anyfs::{MemoryBackend, Quota, PathFilter, RateLimit, Tracing};

// Build a secure sandbox for an AI agent
let sandbox = MemoryBackend::new()
    .layer(QuotaLayer::builder()
        .max_total_size(50 * 1024 * 1024)  // 50 MB
        .max_file_size(5 * 1024 * 1024)    // 5 MB per file
        .build())
    .layer(PathFilterLayer::builder()
        .allow("/workspace/**")
        .deny("**/.env")
        .deny("**/secrets/**")
        .build())
    .layer(RateLimitLayer::builder()
        .max_ops(1000)
        .per_second()
        .build())
    .layer(TracingLayer::new());
}

Extension Traits (in anyfs-backend)

The FsExt trait provides convenience methods for any Fs backend:

#![allow(unused)]
fn main() {
/// Extension methods for Fs (auto-implemented for all backends).
pub trait FsExt: Fs {
    /// Check if path is a file.
    fn is_file(&self, path: impl AsRef<Path>) -> Result<bool, FsError> {
        self.metadata(path.as_ref()).map(|m| m.file_type == FileType::File)
    }

    /// Check if path is a directory.
    fn is_dir(&self, path: impl AsRef<Path>) -> Result<bool, FsError> {
        self.metadata(path.as_ref()).map(|m| m.file_type == FileType::Directory)
    }

    // JSON methods require `serde` feature (see below)
    #[cfg(feature = "serde")]
    fn read_json<T: DeserializeOwned>(&self, path: impl AsRef<Path>) -> Result<T, FsError>;
    #[cfg(feature = "serde")]
    fn write_json<T: Serialize>(&self, path: impl AsRef<Path>, value: &T) -> Result<(), FsError>;
}

// Blanket implementation for all Fs backends
impl<B: Fs> FsExt for B {}
}

JSON Methods (feature: serde)

The read_json and write_json methods require the serde feature:

anyfs-backend = { version = "0.1", features = ["serde"] }

#![allow(unused)]
fn main() {
use serde::{Serialize, de::DeserializeOwned};

#[cfg(feature = "serde")]
impl<B: Fs> FsExt for B {
    fn read_json<T: DeserializeOwned>(&self, path: impl AsRef<Path>) -> Result<T, FsError> {
        let bytes = self.read(path.as_ref())?;
        serde_json::from_slice(&bytes).map_err(|e| FsError::Deserialization(e.to_string()))
    }

    fn write_json<T: Serialize>(&self, path: impl AsRef<Path>, value: &T) -> Result<(), FsError> {
        let bytes = serde_json::to_vec(value).map_err(|e| FsError::Serialization(e.to_string()))?;
        self.write(path.as_ref(), &bytes)
    }
}
}

Users can define their own extension traits for domain-specific operations.

Optional Features

Bytes Support (feature: bytes)

For zero-copy efficiency, enable the bytes feature to get Bytes-returning convenience methods on FileStorage:

anyfs = { version = "0.1", features = ["bytes"] }

#![allow(unused)]
fn main() {
use anyfs::{FileStorage, MemoryBackend};
use bytes::Bytes;

let fs = FileStorage::new(MemoryBackend::new());

// With bytes feature, FileStorage provides read_bytes() convenience method
let data: Bytes = fs.read_bytes("/large-file.bin")?;
let slice = data.slice(1000..2000);  // Zero-copy!

// Core trait still uses Vec<u8> for object safety
// read_bytes() wraps the Vec<u8> in Bytes::from()
}

Note: Core traits (FsRead, etc.) always use Vec<u8> for object safety (dyn Fs). The bytes feature adds convenience methods to FileStorage that wrap results in Bytes.

When to use:

• Large file handling with frequent slicing
• Network-backed storage
• Streaming scenarios

Default: Vec<u8> (no extra dependency)

Error Types

FsError includes context for better debugging. It implements std::error::Error via thiserror and uses #[non_exhaustive] for forward compatibility.

#![allow(unused)]
fn main() {
/// Filesystem error with context.
///
/// All variants include enough information for meaningful error messages.
/// Use `#[non_exhaustive]` to allow adding variants in minor versions.
#[non_exhaustive]
#[derive(Debug, thiserror::Error)]
pub enum FsError {
    // ========================================================================
    // Path/File Errors
    // ========================================================================

    /// Path not found.
    #[error("not found: {path}")]
    NotFound {
        path: PathBuf,
    },

    /// Circular symlink detected during path resolution.
    #[error("symlink loop detected: {path}")]
    SymlinkLoop {
        path: PathBuf,
    },

    /// Security threat detected (e.g., virus).
    /// Note: This variant supports the Antivirus middleware example.
    /// Custom middleware can use this or define domain-specific error types.
    #[error("threat detected: {reason} in {path}")]
    ThreatDetected {
        path: PathBuf,
        reason: String,
    },

    /// Path already exists.
    #[error("{operation}: already exists: {path}")]
    AlreadyExists {
        path: PathBuf,
        operation: &'static str,
    },

    /// Expected a file, found directory.
    NotAFile { path: PathBuf },

    /// Expected a directory, found file.
    NotADirectory { path: PathBuf },

    /// Directory not empty (for remove_dir).
    DirectoryNotEmpty { path: PathBuf },

    // ========================================================================
    // Permission/Access Errors
    // ========================================================================

    /// Permission denied (general filesystem permission error).
    PermissionDenied {
        path: PathBuf,
        operation: &'static str,
    },

    /// Access denied (from PathFilter or RBAC).
    AccessDenied {
        path: PathBuf,
        reason: String,  // Dynamic reason string
    },

    /// Read-only filesystem (from ReadOnly middleware).
    ReadOnly {
        path: PathBuf,
        operation: &'static str,
    },

    /// Feature not enabled (from Restrictions middleware).
    /// Note: Symlink/hard-link capability is determined by trait bounds (FsLink),
    /// not middleware. Restrictions only controls "permissions".
    FeatureNotEnabled {
        path: PathBuf,
        feature: &'static str,  // "permissions"
        operation: &'static str,
    },

    // ========================================================================
    // Resource Limit Errors
    // ========================================================================

    /// Quota exceeded (total storage).
    QuotaExceeded {
        path: PathBuf,
        limit: u64,
        requested: u64,
        usage: u64,
    },

    /// File size limit exceeded.
    FileSizeExceeded {
        path: PathBuf,
        size: u64,
        limit: u64,
    },

    /// Rate limit exceeded (from RateLimit middleware).
    RateLimitExceeded {
        path: PathBuf,
        limit: u32,
        window_secs: u64,
    },

    // ========================================================================
    // Data Errors
    // ========================================================================

    /// Invalid data (e.g., not valid UTF-8 when string expected).
    InvalidData {
        path: PathBuf,
        details: String,
    },

    /// Corrupted data (e.g., failed checksum, parse error).
    CorruptedData {
        path: PathBuf,
        details: String,
    },

    /// Data integrity verification failed (AEAD tag mismatch, HMAC failure).
    IntegrityError {
        path: PathBuf,
    },

    /// Serialization error (from FsExt JSON methods).
    Serialization(String),

    /// Deserialization error (from FsExt JSON methods).
    Deserialization(String),

    // ========================================================================
    // Backend/Operation Errors
    // ========================================================================

    /// Operation not supported by this backend.
    NotSupported {
        operation: &'static str,
    },

    /// Invalid password or encryption key (from SqliteBackend with encryption).
    InvalidPassword,

    /// Conflict during sync (from offline mode).
    Conflict {
        path: PathBuf,
    },

    /// Backend-specific error (catch-all for custom backends).
    Backend {
        message: String,
    },

    /// I/O error wrapper.
    Io {
        operation: &'static str,
        path: PathBuf,
        source: std::io::Error,
    },
}

// Required implementations
impl From<std::io::Error> for FsError {
    fn from(err: std::io::Error) -> Self {
        FsError::Io {
            operation: "io",
            path: PathBuf::new(),
            source: err,
        }
    }
}
}

Implementation notes:

• All variants have #[error("...")] attributes (shown for first two, omitted for brevity)
• #[non_exhaustive] allows adding variants in minor versions without breaking changes
• From<std::io::Error> enables ? operator with std::io functions
• Consider #[must_use] on functions returning Result<_, FsError>

Cross-Platform Compatibility

AnyFS is designed for cross-platform use. Virtual backends work everywhere; real filesystem backends have platform considerations.

Backend Compatibility

*SQLiteBackend on WASM requires wasm32 build of rusqlite with bundled SQLite. Encryption feature not available on WASM.

Feature Compatibility

Platform-Specific Notes

Virtual Backends (MemoryBackend, SqliteBackend)

Fully cross-platform. All features work identically everywhere because:

• Paths are just strings/keys - no OS path resolution
• Symlinks are stored data, not OS constructs
• Permissions are metadata, not enforced by OS
• No filesystem syscalls involved

#![allow(unused)]
fn main() {
// This works identically on Windows, Linux, macOS, and WASM
let fs = FileStorage::new(MemoryBackend::new());
fs.symlink("/target", "/link")?;                          // Just stores the link
fs.set_permissions("/file", Permissions::from_mode(0o755))?; // Just stores metadata
}

VRootFsBackend (Real Filesystem)

Wraps the host filesystem. Platform differences apply:

*Windows requires SeCreateSymbolicLinkPrivilege or Developer Mode for symlinks.

FUSE Mounting

Path Handling

Virtual backends use / as separator internally, regardless of platform:

#![allow(unused)]
fn main() {
// Always use forward slashes with virtual backends
fs.write("/project/src/main.rs", code)?;  // Works everywhere
}

VRootFsBackend translates to native paths internally:

• Linux/macOS: / stays /
• Windows: /project/file.txt → C:\root\project\file.txt

Recommendations

Feature Detection

Check platform capabilities at runtime if needed:

#![allow(unused)]
fn main() {
/// Check if symlinks are supported on the current platform.
pub fn symlinks_available() -> bool {
    #[cfg(unix)]
    return true;

    #[cfg(windows)]
    {
        // Check for Developer Mode or symlink privilege
        // ...
    }
}
}

On platforms without symlink support, use a backend that doesn’t implement FsLink, or check symlinks_available() before calling symlink operations.

Layered Design: Backends + Middleware + Ergonomics

AnyFS uses a layered architecture that separates concerns:

1. Backends: Pure storage + filesystem semantics
2. Middleware: Composable policy layers
3. FileStorage: Ergonomic wrapper

Architecture

┌─────────────────────────────────────────┐
│  FileStorage                         │  ← Ergonomics only
├─────────────────────────────────────────┤
│  Middleware Stack (composable):         │  ← Policy enforcement
│    Tracing → PathFilter → Restrictions  │
│    → Quota → Backend                    │
├─────────────────────────────────────────┤
│  Fs                             │  ← Pure storage
│  (Memory, SQLite, VRootFs, custom)      │
└─────────────────────────────────────────┘

Layer Responsibilities

Core traits use &Path for object safety; FileStorage/FsExt provide impl AsRef<Path> ergonomics. Path resolution is pluggable via PathResolver trait (see ADR-033). Backends that wrap a real filesystem implement SelfResolving so FileStorage can skip resolution.

Policy via Middleware

Old design (rejected): FileStorage contained quota/feature logic.

Current design: Policy is handled by composable middleware:

#![allow(unused)]
fn main() {
// Middleware enforces policy
let backend = MemoryBackend::new()
    .layer(QuotaLayer::builder()
        .max_total_size(100 * 1024 * 1024)
        .build())
    .layer(PathFilterLayer::builder()
        .allow("/workspace/**")
        .build())
    .layer(TracingLayer::new());

// FileStorage is ergonomics + path resolution (no policy)
let fs = FileStorage::new(backend);
}

Path Containment

For VRootFsBackend (real filesystem), path containment uses strict-path::VirtualRoot internally:

#![allow(unused)]
fn main() {
// VRootFsBackend implements FsRead, FsWrite, FsDir (and thus Fs via blanket impl)
impl FsRead for VRootFsBackend {
    fn read(&self, path: &Path) -> Result<Vec<u8>, FsError> {
        // VirtualRoot ensures paths can't escape
        let safe_path = self.root.join(path)?;
        std::fs::read(safe_path).map_err(Into::into)
    }
}
}

For virtual backends (Memory, SQLite), paths are just keys - no OS path traversal possible. FileStorage performs symlink-aware resolution for these backends so normalization is consistent across virtual implementations.

For sandboxing across all backends, use PathFilter middleware:

#![allow(unused)]
fn main() {
PathFilterLayer::builder()
    .allow("/workspace/**")
    .deny("**/.env")
    .build()
    .layer(backend)
}

Why This Matters

• Separation of concerns: Backends focus on storage, middleware handles policy
• Composability: Add/remove policies without touching storage code
• Flexibility: Same middleware works with any backend
• Simplicity: Each layer has one job

AnyFS - Architecture Decision Records

This file captures the decisions for the current AnyFS design.

Decision Map

Primary docs are where each decision is explained in narrative form. ADRs remain the source of truth for the decision itself.

ADR Index

ADR-001: Path-based Fs trait

Decision: Backends implement a path-based trait aligned with std::fs method naming.

Why: Filesystem operations are naturally path-oriented; a single, familiar trait surface is easier to implement and adopt than graph-store or inode models.

ADR-002: Two-crate structure

Decision:

Why:

• Backend authors only need anyfs-backend (no heavy dependencies).
• Middleware is composable and lives with backends in anyfs.
• FileStorage provides ergonomics plus centralized path resolution for virtual backends - no policy logic - included in anyfs for convenience.

ADR-003: Object-safe path parameters

Decision: Core Fs traits take &Path so they remain object-safe (dyn Fs works). For ergonomics, FileStorage and FsExt accept impl AsRef<Path> and forward to the core traits.

Why:

• Object safety enables opt-in type erasure (FileStorage::boxed()).
• Keeps hot-path calls zero-cost; dynamic dispatch is explicit and optional.
• Ergonomics preserved via FileStorage/FsExt (&str, String, PathBuf).

ADR-004: Tower-style middleware pattern

Decision: Use composable middleware (decorator pattern) for cross-cutting concerns like limits, logging, and feature gates. Each middleware implements Fs by wrapping another Fs.

Why:

• Complete separation of concerns - each layer has one job.
• Composable - use only what you need.
• Familiar pattern (Axum/Tower use the same approach).
• No code duplication - middleware written once, works with any backend.
• Testable - each layer can be tested in isolation.

Example:

#![allow(unused)]
fn main() {
let backend = SqliteBackend::open("data.db")?
    .layer(QuotaLayer::builder()
        .max_total_size(100 * 1024 * 1024)
        .build())
    .layer(PathFilterLayer::builder()
        .allow("/workspace/**")
        .build())
    .layer(TracingLayer::new());
}

ADR-005: std::fs-aligned method names

Decision: Prefer read_dir, create_dir_all, remove_file, etc.

Why: Familiarity and reduced cognitive overhead.

ADR-006: Quota for quota enforcement

Decision: Quota/limit enforcement is handled by Quota<B> middleware, not by backends or FileStorage.

Configuration:

• with_max_total_size(bytes) - total storage limit
• with_max_file_size(bytes) - per-file limit
• with_max_node_count(count) - max files/directories
• with_max_dir_entries(count) - max entries per directory
• with_max_path_depth(depth) - max directory nesting

Why:

• Limits are policy, not storage semantics.
• Written once, works with any backend.
• Optional - users who don’t need limits skip this middleware.

Implementation notes:

• On construction, scan existing backend to initialize usage counters.
• Wrap open_write streams with CountingWriter to track streamed bytes.
• Check limits before operations, update usage after successful operations.

ADR-007: Capability via Trait Bounds

Decision: Symlink and hard-link capability is determined by whether the backend implements FsLink. The default PathResolver (IterativeResolver) follows symlinks when FsLink is available.

The Rule

Examples

#![allow(unused)]
fn main() {
// MemoryBackend implements FsLink
let fs = FileStorage::new(MemoryBackend::new());
fs.symlink("/target", "/link")?;  // ✅ Works

// Custom backend that doesn't implement FsLink
let fs = FileStorage::new(MySimpleBackend::new());
fs.symlink("/target", "/link")?;  // ❌ Won't compile - no FsLink impl
}

Why Not Runtime Blocking?

A hypothetical deny_symlinks() middleware would create type/behavior mismatch:

• Type says “I implement FsLink”
• Runtime says “but symlink() errors”

This is confusing and defeats the purpose of Rust’s type system. Instead, symlink capability is determined at compile time by trait bounds.

Restrictions Middleware

Restrictions<B> is limited to operations where runtime policy makes sense:

#![allow(unused)]
fn main() {
let backend = RestrictionsLayer::builder()
    .deny_permissions()  // Prevent metadata changes
    .build()
    .layer(backend);
}

Symlink following: Controlled by the PathResolver. The default IterativeResolver follows symlinks when FsLink is available. Custom resolvers can implement different behaviors. OS-backed backends delegate to the OS (strict-path prevents escapes).

ADR-008: FileStorage as thin ergonomic wrapper

Decision: FileStorage<B> is a thin wrapper that provides std::fs-aligned ergonomics and path resolution for virtual backends. It contains NO policy logic.

Context: Earlier designs used FileStorage<B, R, M> with three type parameters:

• B - Backend type
• R - PathResolver type (default: IterativeResolver)
• M - Marker type for compile-time container differentiation

This was over-engineered. We simplified to a single generic.

Why only one generic parameter?

What it does:

• Provides familiar method names
• Accepts impl AsRef<Path> for convenience and forwards to the core &Path traits
• Delegates path resolution to a boxed PathResolver (cold path, boxing OK per ADR-025)
• Delegates all operations to the wrapped backend

What it does NOT do:

• Quota enforcement (use Quota)
• Feature gating (use Restrictions)
• Instrumentation (use Tracing)
• Marker types (users create wrapper newtypes if needed)
• Any other policy

Why this design:

• Single responsibility - ergonomics + path resolution (no policy).
• One generic parameter keeps the API simple for 90% of users.
• Resolver is boxed because path resolution is a cold path.
• Users who need type-safe markers can create their own wrapper types.
• Policy is composable via middleware, not hardcoded.

User-defined type safety pattern:

#![allow(unused)]
fn main() {
// Instead of FileStorage<_, _, Sandbox>, users create:
struct SandboxFs(FileStorage<MemoryBackend>);
struct UserDataFs(FileStorage<SqliteBackend>);

fn process_sandbox(fs: &SandboxFs) { /* only accepts SandboxFs */ }
}

ADR-009: Simple backends in anyfs, complex backends as ecosystem crates

Decision: Simple backends (MemoryBackend, StdFsBackend, VRootFsBackend) are built into anyfs with feature flags. Complex backends (SqliteBackend, IndexedBackend) live in separate ecosystem crates.

Built-in backends (anyfs features):

• memory (default)
• stdfs (optional)
• vrootfs (optional)

Ecosystem crates:

• anyfs-sqlite — SqliteBackend with optional encryption feature
• anyfs-indexed — IndexedBackend (SQLite metadata + disk blobs)

Why:

• Simple backends have minimal dependencies (just std)
• Complex backends need internal runtimes (connection pools, sharding, chunking)
• Follows Tower/Axum pattern: framework is minimal, complex implementations in their own crates
• Reduces compile time and binary size for users who don’t need complex backends

ADR-010: Sync-first, async-ready design

Decision: Fs traits are synchronous. The API is designed to allow adding AsyncFs later without breaking changes.

Rationale:

• Built-in backends are naturally synchronous:
  • MemoryBackend - in-memory, instant
  • StdFsBackend / VRootFsBackend - std::fs is sync
• Ecosystem backends are also sync (e.g., SqliteBackend uses rusqlite which is sync)
• Sync is simpler - no runtime dependency (tokio/async-std)
• Users can wrap sync backends in spawn_blocking if needed

Async-ready design principles:

• Traits require Send - compatible with async executors
• Return types are Result<T, FsError> - works with async
• No internal blocking assumptions
• Methods are stateless per-call - no hidden blocking state

Future async path (Option 2): When async is needed (e.g., network-backed storage), add a parallel trait:

#![allow(unused)]
fn main() {
// In anyfs-backend
pub trait AsyncFs: Send + Sync {
    async fn read(&self, path: &Path) -> Result<Vec<u8>, FsError>;
    async fn write(&self, path: &Path, data: &[u8]) -> Result<(), FsError>;
    // ... mirrors Fs with async

    // Streaming uses AsyncRead/AsyncWrite
    async fn open_read(&self, path: &Path)
        -> Result<Box<dyn AsyncRead + Send + Unpin>, FsError>;
}
}

Migration notes:

• AsyncFs would be a separate trait, not replacing Fs
• Blanket impl possible: impl<T: Fs> AsyncFs for T using spawn_blocking
• Middleware would need async variants: AsyncQuota<B>, etc.
• No breaking changes to existing sync API

Why not async now:

• Complexity without benefit - all current backends are sync
• Rust 1.75 makes async traits easy, so adding later is low-cost
• Better to wait for real async backend requirements

ADR-011: Layer trait for standardized composition

Decision: Provide a Layer trait (inspired by Tower) that standardizes middleware composition.

#![allow(unused)]
fn main() {
pub trait Layer<B: Fs> {
    type Backend: Fs;
    fn layer(self, backend: B) -> Self::Backend;
}
}

Note: For async compatibility, the trait is unbounded: Layer<B> without B: Fs. This allows the same layer types to implement both sync (impl<B: Fs> Layer<B>) and async (impl<B: AsyncFs> Layer<B>). See ADR-027.

Why:

• Standardized composition pattern familiar to Tower/Axum users.
• IDE autocomplete for available layers.
• Enables BackendStack fluent builder in anyfs.
• Each middleware provides a corresponding *Layer type.

Example:

#![allow(unused)]
fn main() {
// SqliteBackend from anyfs-sqlite crate
let backend = SqliteBackend::open("data.db")?
    .layer(QuotaLayer::builder()
        .max_total_size(100_000)
        .build())
    .layer(TracingLayer::new());
}

ADR-012: Tracing for instrumentation

Decision: Use Tracing<B> integrated with the tracing ecosystem instead of a custom logging solution.

Why:

• Works with existing tracing infrastructure (tracing-subscriber, OpenTelemetry, Jaeger).
• Structured logging with spans for each operation.
• Users choose their subscriber - no logging framework lock-in.
• Consistent with modern Rust ecosystem practices.

Configuration:

#![allow(unused)]
fn main() {
backend.layer(TracingLayer::new()
    .with_target("anyfs")
    .with_level(tracing::Level::DEBUG))
}

ADR-013: FsExt for extension methods

Decision: Provide FsExt trait with convenience methods, auto-implemented for all backends.

#![allow(unused)]
fn main() {
pub trait FsExt: Fs {
    fn is_file(&self, path: impl AsRef<Path>) -> Result<bool, FsError>;
    fn is_dir(&self, path: impl AsRef<Path>) -> Result<bool, FsError>;

    // JSON methods require `serde` feature
    #[cfg(feature = "serde")]
    fn read_json<T: DeserializeOwned>(&self, path: impl AsRef<Path>) -> Result<T, FsError>;
    #[cfg(feature = "serde")]
    fn write_json<T: Serialize>(&self, path: impl AsRef<Path>, value: &T) -> Result<(), FsError>;
}

impl<B: Fs> FsExt for B {}
}

Feature gating:

• is_file() and is_dir() are always available.
• read_json() and write_json() require anyfs-backend = { features = ["serde"] }.

Why:

• Adds convenience without bloating Fs trait.
• Blanket impl means all backends get these methods for free.
• Users can define their own extension traits for domain-specific operations.
• Follows Rust convention (e.g., IteratorExt, StreamExt).
• Serde is optional - users who don’t need JSON avoid the dependency.

ADR-014: Optional Bytes support

Decision: Support the bytes crate via an optional feature for zero-copy efficiency.

anyfs = { version = "0.1", features = ["bytes"] }

Why:

• Bytes provides O(1) slicing via reference counting.
• Beneficial for large file handling, network backends, streaming.
• Optional - users who don’t need it avoid the dependency.
• Core traits remain Vec<u8> for simplicity and Send + Sync compliance.

Implementation: The bytes feature adds a convenience method to FileStorage, not a core trait change:

#![allow(unused)]
fn main() {
// In anyfs/src/container.rs (behind `bytes` feature)
impl<B: Fs> FileStorage<B> {
    #[cfg(feature = "bytes")]
    pub fn read_bytes(&self, path: impl AsRef<Path>) -> Result<bytes::Bytes, FsError> {
        Ok(bytes::Bytes::from(self.read(path)?))
    }
}
}

Core traits unchanged: FsRead::read() returns Vec<u8>. The bytes feature only adds ergonomic wrappers.

ADR-015: Contextual FsError

Decision: FsError variants include context for better debugging.

#![allow(unused)]
fn main() {
FsError::NotFound {
    path: PathBuf,
}

FsError::QuotaExceeded {
    limit: u64,
    requested: u64,
    usage: u64,
}
}

Why:

• Error messages include enough context to understand what failed.
• No need for separate error context crate (like anyhow) for basic usage.
• Path is sufficient for NotFound - the call site knows the operation.
• Quota errors include all relevant numbers for debugging.

ADR-016: PathFilter for path-based access control

Decision: Provide PathFilter<B> middleware for glob-based path access control.

Configuration:

#![allow(unused)]
fn main() {
PathFilterLayer::builder()
    .allow("/workspace/**")    // Allow workspace access
    .deny("**/.env")           // Deny .env files anywhere
    .deny("**/secrets/**")     // Deny secrets directories
    .build()
    .layer(backend)
}

Semantics:

• Deny rules are evaluated first and take precedence over allow rules.
• If path matches any deny rule, access is denied.
• If path matches an allow rule (and no deny), access is granted.
• If no rules match, access is denied (deny by default).
• Uses glob patterns (e.g., ** for recursive, * for single segment).
• Returns FsError::AccessDenied for denied paths.

Why:

• Essential for AI agent sandboxing - restrict to specific directories.
• Prevents access to sensitive files (.env, secrets, credentials).
• Separate from backend - works with any backend.
• Inspired by AgentFS and similar AI sandbox patterns.

Implementation notes:

• Use globset crate for efficient glob pattern matching.
• read_dir filters out denied entries from results (don’t expose existence of denied files).
• Check path at operation start, then delegate to inner backend.

ADR-017: ReadOnly for preventing writes

Decision: Provide ReadOnly<B> middleware that blocks all write operations.

Usage:

#![allow(unused)]
fn main() {
let readonly_fs = ReadOnly::new(backend);
}

Semantics:

• All read operations pass through to inner backend.
• All write operations return FsError::ReadOnly.
• Simple, no configuration needed.

Why:

• Safe browsing of container contents without modification risk.
• Useful for debugging, inspection, auditing.
• Simpler than configuring Restrictions for read-only use case.

ADR-018: RateLimit for operation throttling

Decision: Provide RateLimit<B> middleware to limit operations per time window.

Configuration:

#![allow(unused)]
fn main() {
RateLimitLayer::builder()
    .max_ops(1000)
    .per_second()
    .build()
    .layer(backend)
}

Semantics:

• Tracks operation count in fixed time window (simpler than sliding window, sufficient for most use cases).
• Returns FsError::RateLimitExceeded when limit exceeded.
• Counter resets when window expires.

Why:

• Protects against runaway processes consuming resources.
• Essential for multi-tenant environments.
• Prevents denial-of-service from misbehaving code.

Implementation notes:

• Use std::time::Instant for timing.
• Store window start time and counter; reset when window expires.
• Count operation calls (including open_read/open_write), not bytes transferred.
• Return error immediately when limit exceeded (no blocking/waiting).

ADR-019: DryRun for testing and debugging

Decision: Provide DryRun<B> middleware that logs write operations without executing them.

Usage:

#![allow(unused)]
fn main() {
let dry_run = DryRun::new(backend);
let fs = FileStorage::new(dry_run);

fs.write("/test.txt", b"hello")?;  // Logged but not written
// To inspect recorded operations, keep the DryRun handle before wrapping it.
}

Semantics:

• Read operations execute normally against inner backend.
• Write operations are logged but return Ok(()) without executing.
• Operations log can be inspected for verification.

Why:

• Test code paths without side effects.
• Debug complex operation sequences.
• Audit what would happen before committing.

Implementation notes:

• Read operations delegate to inner backend (test against real state).
• Write operations log and return Ok(()) without executing.
• open_write returns std::io::sink() - writes are discarded.
• Useful for: “What would this code do?” not “Run this in isolation.”

ADR-020: Cache for read performance

Decision: Provide Cache<B> middleware with LRU caching for read operations.

Configuration:

#![allow(unused)]
fn main() {
CacheLayer::builder()
    .max_entries(1000)
    .max_entry_size(1024 * 1024)  // 1MB max per entry
    .build()
    .layer(backend)
}

Semantics:

• Read operations check cache first, populate on miss.
• Write operations invalidate relevant cache entries.
• LRU eviction when max entries exceeded.

Why:

• Improves performance for repeated reads.
• Reduces load on underlying backend (especially for SQLite/network).
• Configurable to balance memory vs performance.

Implementation notes:

• Cache bulk reads only: read(), read_to_string(), read_range(), metadata(), exists().
• Do NOT cache open_read() - streams are for large files that shouldn’t be cached.
• Invalidate cache entry on any write to that path.
• Use lru crate or similar for LRU eviction.

ADR-021: Overlay for union filesystem

Decision: Provide Overlay<B1, B2> middleware for copy-on-write layered filesystems.

Usage:

#![allow(unused)]
fn main() {
// SqliteBackend from anyfs-sqlite crate
let base = SqliteBackend::open("base.db")?;  // Read-only base
let upper = MemoryBackend::new();             // Writable upper layer

let overlay = Overlay::new(base, upper);
}

Semantics:

• Read: check upper layer first, fall back to base if not found.
• Write: always to upper layer (copy-on-write).
• Delete: create whiteout marker in upper layer (file appears deleted but base unchanged).
• Directory listing: merge results from both layers.

Why:

• Docker-like layered filesystem for containers.
• Base image with per-instance modifications.
• Testing with isolated changes over shared baseline.
• Inspired by OverlayFS and VFS crate patterns.

Implementation notes:

• Whiteout convention: .wh.<filename> marks deleted files from base layer.
• read_dir must merge results from both layers, excluding whiteouts and whited-out files.
• exists checks upper first, then base (respecting whiteouts).
• All writes go to upper layer; base is never modified.
• Consider opaque directories (.wh..wh..opq) to hide entire base directories.

ADR-022: Builder pattern for configurable middleware

Decision: Middleware that requires configuration MUST use a builder pattern that prevents construction without meaningful values. ::new() constructors are NOT allowed for middleware where a default configuration is nonsensical.

Problem: A constructor like QuotaLayer::new() raises the question: “What quota?” An unlimited quota is pointless - you wouldn’t use QuotaLayer at all. Similarly, RestrictionsLayer::new() with no restrictions, PathFilterLayer::new() with no rules, and RateLimitLayer::new() with no rate limit are all nonsensical.

Solution: Use builders that enforce at least one meaningful configuration:

#![allow(unused)]
fn main() {
// QuotaLayer - requires at least one limit
let quota = QuotaLayer::builder()
    .max_total_size(100 * 1024 * 1024)
    .build();

// Can also set multiple limits
let quota = QuotaLayer::builder()
    .max_total_size(1_000_000)
    .max_file_size(100_000)
    .max_node_count(1000)
    .build();

// RestrictionsLayer - requires at least one restriction
let restrictions = RestrictionsLayer::builder()
    .deny_permissions()
    .build();

// PathFilterLayer - requires at least one rule
let filter = PathFilterLayer::builder()
    .allow("/workspace/**")
    .deny("**/.env")
    .build();

// RateLimitLayer - requires rate limit parameters
let rate_limit = RateLimitLayer::builder()
    .max_ops(1000)
    .per_second()
    .build();

// CacheLayer - requires cache configuration
let cache = CacheLayer::builder()
    .max_entries(1000)
    .build();
}

Middleware that MAY keep ::new():

Implementation:

#![allow(unused)]
fn main() {
// Builder with typestate pattern for compile-time enforcement
pub struct QuotaLayerBuilder<State = Unconfigured> {
    max_total_size: Option<u64>,
    max_file_size: Option<u64>,
    max_node_count: Option<u64>,
    _state: PhantomData<State>,
}

pub struct Unconfigured;
pub struct Configured;

impl QuotaLayerBuilder<Unconfigured> {
    pub fn max_total_size(mut self, bytes: u64) -> QuotaLayerBuilder<Configured> {
        self.max_total_size = Some(bytes);
        QuotaLayerBuilder {
            max_total_size: self.max_total_size,
            max_file_size: self.max_file_size,
            max_node_count: self.max_node_count,
            _state: PhantomData,
        }
    }

    pub fn max_file_size(mut self, bytes: u64) -> QuotaLayerBuilder<Configured> {
        // Similar transition to Configured state
    }

    pub fn max_node_count(mut self, count: u64) -> QuotaLayerBuilder<Configured> {
        // Similar transition to Configured state
    }

    // Note: NO build() method on Unconfigured state!
}

impl QuotaLayerBuilder<Configured> {
    // Additional configuration methods stay in Configured state
    pub fn max_total_size(mut self, bytes: u64) -> Self {
        self.max_total_size = Some(bytes);
        self
    }

    // Only Configured state has build()
    pub fn build(self) -> QuotaLayer {
        QuotaLayer { /* ... */ }
    }
}

impl QuotaLayer {
    pub fn builder() -> QuotaLayerBuilder<Unconfigured> {
        QuotaLayerBuilder {
            max_total_size: None,
            max_file_size: None,
            max_node_count: None,
            _state: PhantomData,
        }
    }
}
}

Why:

• Compile-time safety: Invalid configurations don’t compile.
• Self-documenting API: Users must explicitly choose configuration.
• No meaningless defaults: Eliminates “what does this default to?” confusion.
• IDE guidance: Autocomplete shows required methods before build().
• Familiar pattern: Rust builders are idiomatic and widely understood.

Error prevention:

#![allow(unused)]
fn main() {
// This won't compile - no build() on Unconfigured
let quota = QuotaLayer::builder().build();  // ❌ Error!

// This compiles - at least one limit set
let quota = QuotaLayer::builder()
    .max_total_size(1_000_000)
    .build();  // ✅ OK
}

ADR-023: Interior mutability for all trait methods

Decision: All Fs trait methods use &self, not &mut self. Backends manage their own synchronization internally (interior mutability).

Previous design:

#![allow(unused)]
fn main() {
pub trait FsRead: Send {
    fn read(&self, path: &Path) -> Result<Vec<u8>, FsError>;
}

pub trait FsWrite: Send {
    fn write(&mut self, path: &Path, data: &[u8]) -> Result<(), FsError>;
}
}

New design:

#![allow(unused)]
fn main() {
pub trait FsRead: Send + Sync {
    fn read(&self, path: &Path) -> Result<Vec<u8>, FsError>;
}

pub trait FsWrite: Send + Sync {
    fn write(&self, path: &Path, data: &[u8]) -> Result<(), FsError>;
}
}

Why:

1. Filesystems are conceptually always mutable. A filesystem doesn’t become “borrowed” when you write to it - the underlying storage manages concurrency itself.

2. Enables concurrent access patterns. With &mut self, you cannot have concurrent readers and writers even when the backend supports it (e.g., SQLite with WAL mode, real filesystems).

3. Matches real-world filesystem semantics. std::fs::write() takes a path, not a mutable reference to some filesystem object. Files are shared resources.

4. Simplifies middleware implementation. Middleware no longer needs to worry about propagating mutability - all operations use &self.

5. Common pattern in Rust. Many I/O abstractions use interior mutability: std::io::Write for File (via OS handles), tokio::fs, database connection pools, etc.

Implementation:

Backends use appropriate synchronization primitives:

#![allow(unused)]
fn main() {
pub struct MemoryBackend {
    // Interior mutability via Mutex/RwLock
    data: RwLock<HashMap<PathBuf, Vec<u8>>>,
}

impl FsWrite for MemoryBackend {
    fn write(&self, path: &Path, data: &[u8]) -> Result<(), FsError> {
        let mut guard = self.data.write().unwrap();
        guard.insert(path.as_ref().to_path_buf(), data.to_vec());
        Ok(())
    }
}

pub struct SqliteBackend {
    // SQLite handles its own locking
    conn: Connection,  // rusqlite::Connection is internally synchronized
}
}

Trade-offs:

Backend implementer responsibility:

Backends MUST use interior mutability (RwLock, Mutex, etc.) to ensure thread-safe concurrent access. This guarantees:

• Memory safety (no data corruption)
• Atomic operations (a single write() won’t produce partial results)

This does NOT guarantee:

• Order of concurrent writes to the same path (last write wins - standard FS behavior)

Conclusion: The benefits of matching filesystem semantics and enabling concurrent access outweigh the loss of compile-time single-writer enforcement. Backends are responsible for their own thread safety via interior mutability.

ADR-024: Async Strategy

Status: Accepted Context: Async/await is prevalent in Rust networking and I/O. While AnyFS is primarily sync-focused (matching std::fs), we may need async support in the future for:

• Network-backed storage (S3, WebDAV, etc.)
• High-concurrency scenarios
• Integration with async runtimes (tokio, async-std)

Decision: Plan for a parallel async trait hierarchy that mirrors the sync traits.

Strategy:

Sync Traits          Async Traits
-----------          ------------
FsRead        →      AsyncFsRead
FsWrite       →      AsyncFsWrite
FsDir         →      AsyncFsDir
Fs            →      AsyncFs
FsFull        →      AsyncFsFull
FsFuse        →      AsyncFsFuse
FsPosix       →      AsyncFsPosix

Design principles:

1. Separate crate: Async traits live in anyfs-async to avoid pulling async dependencies into the core.

2. Method parity: Each async trait method corresponds 1:1 with its sync counterpart:

   #![allow(unused)]
   fn main() {
   // Sync (anyfs-backend)
   pub trait FsRead: Send + Sync {
       fn read(&self, path: &Path) -> Result<Vec<u8>, FsError>;
   }
   
   // Async (anyfs-async)
   #[async_trait]
   pub trait AsyncFsRead: Send + Sync {
       async fn read(&self, path: &Path) -> Result<Vec<u8>, FsError>;
   }
   }

3. Layer trait compatibility: The Layer trait works for both sync and async:

   #![allow(unused)]
   fn main() {
   pub trait Layer<B> {
       type Backend;
       fn layer(self, backend: B) -> Self::Backend;
   }
   
   // Middleware can implement for both:
   impl<B: Fs> Layer<B> for QuotaLayer {
       type Backend = Quota<B>;
       fn layer(self, backend: B) -> Self::Backend { ... }
   }
   
   impl<B: AsyncFs> Layer<B> for QuotaLayer {
       type Backend = AsyncQuota<B>;
       fn layer(self, backend: B) -> Self::Backend { ... }
   }
   }

4. Sync-to-async bridge: Provide adapters for using sync backends in async contexts:

   #![allow(unused)]
   fn main() {
   // Wraps sync backend for use in async code (uses spawn_blocking)
   pub struct SyncToAsync<B>(B);
   
   impl<B: Fs> AsyncFsRead for SyncToAsync<B> {
       async fn read(&self, path: &Path) -> Result<Vec<u8>, FsError> {
           let path = path.as_ref().to_path_buf();
           let backend = self.0.clone(); // requires Clone
           tokio::task::spawn_blocking(move || backend.read(&path)).await?
       }
   }
   }

5. No async-to-sync bridge: We intentionally don’t provide async-to-sync adapters (would require blocking on async runtime, which is problematic).

Implementation phases:

Why parallel traits (not feature flags):

• No conditional compilation complexity - sync and async are separate, clean codebases
• No trait object issues - async traits have different object safety requirements
• Clear dependency boundaries - sync code doesn’t pull in tokio/async-std
• Ecosystem alignment - mirrors how std::io vs tokio::io work

Trade-offs:

Conclusion: Parallel async traits provide the best balance of simplicity now (sync-only core) with a clear migration path for async support later. The Layer trait design already accommodates this pattern.

ADR-025: Strategic Boxing (Tower-style)

Status: Accepted

Context: Dynamic dispatch (Box<dyn Trait>) adds heap allocation and vtable indirection. We need to decide where boxing is acceptable vs. where zero-cost abstractions are required.

Decision: Follow Tower/Axum’s battle-tested strategy: zero-cost on the hot path, box at boundaries where flexibility is needed and I/O cost dominates.

Principle: Avoid heap allocations and dynamic dispatch unless they buy real flexibility with negligible performance impact. Box only at cold boundaries (streams/iterators), and make type erasure explicit and opt-in.

DX stance: Application code uses FileStorage/FsExt (std::fs-style paths). Core traits stay object-safe for dyn Fs. For hot loops on known concrete backends, we provide a typed streaming extension as the first-class zero-alloc fast path.

Boxing Strategy

HOT PATH (many calls per operation - must be zero-cost):
┌─────────────────────────────────────────────────────┐
│  read(), write(), metadata(), exists()              │  ← Returns concrete types
│  Read::read() / Write::write() on streams           │  ← Vtable dispatch only
│  Iterator::next() on ReadDirIter                    │  ← Vtable dispatch only
│  Middleware composition                             │  ← Generics, monomorphized
└─────────────────────────────────────────────────────┘

COLD PATH (once per operation - boxing acceptable):
┌─────────────────────────────────────────────────────┐
│  open_read(), open_write()                          │  ← Box<dyn Read/Write>
│  read_dir()                                         │  ← ReadDirIter (boxed inner)
└─────────────────────────────────────────────────────┘

SETUP (once at startup - zero-cost):
┌─────────────────────────────────────────────────────┐
│  Middleware stacking: Quota<Tracing<B>>             │  ← Generics, no boxing
│  FileStorage::new(backend)                          │  ← Zero-cost wrapper
└─────────────────────────────────────────────────────┘

OPT-IN TYPE ERASURE (when explicitly needed):
┌─────────────────────────────────────────────────────────────┐
│  FileStorage::boxed() -> FileStorage<Box<dyn Fs>>          │  ← Like Tower's BoxService
│  (Resolver already boxed internally - this boxes backend)  │
└─────────────────────────────────────────────────────────────┘

What Gets Boxed and Why

Why This Works

1. Bulk operations are the common case: Most code uses read() and write(), not streaming. These are zero-cost.

2. Streaming is for large files: open_read() / open_write() are for files too large to load into memory. For large files, I/O time (1-100ms) dwarfs box allocation (~50ns).

3. Box once, vtable many: After open_read() allocates once, subsequent Read::read() calls are just vtable dispatch - no further allocations.

4. Middleware needs flexibility:

• Quota wraps streams with QuotaWriter to count bytes
• PathFilter filters ReadDirIter to hide denied entries
• Overlay merges directory listings from two backends Boxing enables this without type explosion.

Comparison to Tower/Axum

Tower’s Timeout middleware uses Pin<Box<dyn Future>> in practice. Axum’s Router uses BoxedIntoRoute to store handlers. We follow the same pattern.

Cost Analysis

The boxing cost is negligible relative to actual I/O.

Alternatives Considered

1. Associated types everywhere:

#![allow(unused)]
fn main() {
pub trait FsRead {
    type Reader: Read + Send;
    fn open_read(&self, path: &Path) -> Result<Self::Reader, FsError>;
}
}

Rejected: Causes type explosion. QuotaReader<PathFilterReader<TracingReader<Cursor<Vec<u8>>>>> is unwieldy and every middleware needs a custom wrapper type.

2. RPITIT (Rust 1.75+):

#![allow(unused)]
fn main() {
fn open_read(&self, path: &Path) -> Result<impl Read + Send, FsError>;
}

Rejected as default: Loses object safety. Can’t use dyn Fs for runtime backend selection.

3. Always box everything: Rejected: Unnecessary overhead on hot path operations like read().

Future Considerations

If profiling shows stream boxing is a bottleneck (unlikely), we can add:

#![allow(unused)]
fn main() {
/// Extension trait for zero-cost streaming when backend type is known
pub trait FsReadTyped: FsRead {
    type Reader: Read + Send;
    fn open_read_typed(&self, path: &Path) -> Result<Self::Reader, FsError>;
}
}

This follows Tower’s pattern of providing both Service (with associated types) and BoxService (with type erasure).

Conclusion

Our boxing strategy mirrors Tower/Axum’s production-proven approach:

• Zero-cost where it matters (hot path bulk operations, middleware composition)
• Box where flexibility is needed (streaming I/O, iterator filtering)
• Opt-in type erasure (explicit boxed() method)

The performance cost is negligible (<1% of I/O time), while the ergonomic and flexibility benefits are substantial.

ADR-026: Companion shell (anyfs-shell)

Status: Accepted (Future)

Context: Users want a low-friction way to explore how different backends and middleware behave without writing a full application.

Decision: Provide a separate companion crate (e.g., anyfs-shell) that exposes a bash-style navigation and file management interface built on FileStorage.

Scope:

• Commands: ls, cd, cat, cp, mv, rm, mkdir, stat.
• Navigation and file management only; no full bash scripting, pipes, or job control.
• All operations route through FileStorage to exercise middleware and backend composition.

Why:

• Demonstrates backend neutrality and middleware effects in a tangible way.
• Useful for docs, demos, and quick validation.
• Keeps the core crates free of CLI/UI dependencies.

ADR-027: Permissive core; security via middleware

Status: Accepted

Context: We need predictable filesystem semantics across backends. Some use cases require strict sandboxing, while others expect full filesystem behavior. Baking security restrictions into core traits would make behavior surprising and backend-dependent.

Decision: Core traits are permissive: all operations supported by a backend are allowed by default. Security controls (limits, access control, read-only, rate limiting, audit) are applied via middleware such as Restrictions, PathFilter, ReadOnly, Quota, RateLimit, and Tracing.

Why:

• Predictability: core behavior matches std::fs expectations.
• Backend-agnostic: virtual and host backends share the same contract.
• Separation of concerns: policy lives in middleware, not storage semantics.
• Explicit security posture: applications opt in to the protections they need.

ADR-028: Linux-like semantics for virtual backends

Status: Accepted

Context: Cross-platform filesystems differ in case sensitivity, separators, reserved names, and path length limits. Virtual backends need a consistent model that does not inherit OS quirks.

Decision: Virtual backends use Linux-like semantics by default:

• Case-sensitive paths
• / as the internal separator
• No reserved names
• No max path length
• No ADS (:stream) support

Why:

• Cross-platform consistency for the same data.
• Fewer surprises and reduced security footguns.
• Simplifies backend implementation and testing.
• Custom semantics remain possible via middleware or custom backends.

ADR-029: Path resolution in FileStorage

Status: Accepted

Context: Path normalization (//, ., ..) and symlink resolution must be consistent across backends. Implementing this logic in every backend is error-prone and leads to divergent behavior.

Decision: FileStorage performs canonicalization and normalization for virtual backends. Backends receive resolved paths. Real filesystem backends (e.g., VRootFsBackend) delegate to OS resolution plus strict-path containment. FileStorage exposes canonicalize, soft_canonicalize, and anchored_canonicalize for explicit use.

Why:

• Consistent semantics across all backends.
• Centralizes security-critical path handling.
• Simplifies backend implementations.
• Makes conformance testing straightforward.

ADR-030: Layered trait hierarchy

Status: Accepted

Context: Not all backends can or should implement full POSIX behavior. Forcing a single large trait would make simple backends harder to implement and would obscure capabilities.

Decision: Split the API into layered traits:

• Core: FsRead, FsWrite, FsDir (combined as Fs)
• Extensions: FsLink, FsPermissions, FsSync, FsStats
• FUSE: FsInode
• POSIX: FsHandles, FsLock, FsXattr
• Convenience supertraits: Fs, FsFull, FsFuse, FsPosix

Why:

• Implement the lowest level you need.
• Clear capability boundaries and trait bounds.
• Avoids forcing unsupported features on backends.
• Enables middleware to target specific capabilities.

ADR-031: Indexing as middleware

Status: Accepted (Future)

Context: We want a durable, queryable index of file activity and metadata (for audit trails, drive management tools, and statistics). This indexing should be optional, configurable, and work across all backends.

Decision: Indexing is implemented as middleware (Indexing<B> with IndexLayer), not as a specialized backend. The middleware writes to a sidecar index (SQLite by default) and can evolve to support alternate index engines.

Naming: Use IndexLayer (builder) and Indexing<B> (middleware), consistent with existing layer naming.

Why:

• Separation of concerns: Indexing is policy/analytics, not storage semantics.
• Backend-agnostic: Works with Memory, SQLite, VRootFs, and custom backends.
• Composability: Users opt in and configure it like other middleware (Quota, Tracing).
• Flexibility: Allows future index engines without changing core traits.
• DX consistency: Keeps std::fs-style usage via FileStorage with no API changes.

Trade-offs:

• External OS changes: Not captured unless a future watcher/scan helper is added.
• Index failures: Choose between strict mode (fail the op) and best-effort mode.

Implementation sketch:

• IndexLayer::builder().index_file("index.db").consistency(IndexConsistency::Strict)...
• Wraps open_write() with a counting writer to record final size on close.
• Updates a nodes table and logs ops entries per operation.

ADR-032: Path Canonicalization via FsPath Trait

Status: Accepted

Context: Path canonicalization (resolving .., ., and symlinks) is needed for consistent path handling. The naive approach of baking this into FileStorage has issues:

• It’s not testable in isolation
• It can’t be optimized per-backend
• N+1 queries for paths like /a/b/c/d/e (each component = separate call)

Decision: Introduce an FsPath trait with canonicalize() and soft_canonicalize() methods that have default implementations but allow backend-specific optimizations.

The Pattern:

#![allow(unused)]
fn main() {
pub trait FsPath: FsRead + FsLink {
    /// Resolve all symlinks and normalize path components.
    /// Returns error if final path doesn't exist.
    fn canonicalize(&self, path: &Path) -> Result<PathBuf, FsError> {
        default_canonicalize(self, path)
    }

    /// Like canonicalize, but allows non-existent final component.
    fn soft_canonicalize(&self, path: &Path) -> Result<PathBuf, FsError> {
        default_soft_canonicalize(self, path)
    }
}

// Auto-implement for all FsLink implementors
impl<T: FsRead + FsLink> FsPath for T {}
}

Default Implementation:

#![allow(unused)]
fn main() {
fn default_canonicalize<F: FsRead + FsLink>(fs: &F, path: &Path) -> Result<PathBuf, FsError> {
    let mut resolved = PathBuf::from("/");
    for component in path.components() {
        match component {
            Component::RootDir => resolved = PathBuf::from("/"),
            Component::ParentDir => { resolved.pop(); },
            Component::CurDir => {},
            Component::Normal(name) => {
                resolved.push(name);
                if let Ok(meta) = fs.symlink_metadata(&resolved) {
                    if meta.file_type.is_symlink() {
                        let target = fs.read_link(&resolved)?;
                        resolved.pop();
                        resolved = resolve_relative(&resolved, &target);
                    }
                }
            },
            _ => {},
        }
    }
    // Verify final path exists
    if !fs.exists(&resolved)? {
        return Err(FsError::NotFound { path: resolved, operation: "canonicalize" });
    }
    Ok(resolved)
}
}

Backend Optimization Examples:

SQLite Optimized Implementation:

#![allow(unused)]
fn main() {
impl FsPath for SqliteBackend {
    fn canonicalize(&self, path: &Path) -> Result<PathBuf, FsError> {
        // Single query with recursive CTE
        self.conn.query_row(
            r#"
            WITH RECURSIVE path_resolve(segment, remaining, resolved, depth) AS (
                -- Initial: split path into segments
                SELECT ..., 0
                UNION ALL
                -- Recursive: resolve each segment, following symlinks
                SELECT ...
                FROM path_resolve
                JOIN nodes ON ...
                WHERE depth < 40  -- Loop protection
            )
            SELECT resolved FROM path_resolve 
            WHERE remaining = '' 
            ORDER BY depth DESC LIMIT 1
            "#,
            params![path.to_string_lossy()],
            |row| Ok(PathBuf::from(row.get::<_, String>(0)?))
        ).map_err(|e| FsError::NotFound { path: path.into(), operation: "canonicalize" })
    }
}
}

Why This Design:

FileStorage Integration:

Note: ADR-033 introduces PathResolver as the primary resolution strategy. FsPath remains as an optional backend optimization hook. When a backend implements both FsPath and the default traits, the backend can choose to delegate to its resolver or provide fully custom logic (e.g., SQLite CTE queries).

FileStorage uses a boxed PathResolver internally for resolution (see ADR-033):

#![allow(unused)]
fn main() {
impl<B: Fs> FileStorage<B> {
    pub fn canonicalize(&self, path: impl AsRef<Path>) -> Result<PathBuf, FsError> {
        self.resolver.canonicalize(path.as_ref(), &self.backend as &dyn Fs)
    }

    pub fn soft_canonicalize(&self, path: impl AsRef<Path>) -> Result<PathBuf, FsError> {
        self.resolver.soft_canonicalize(path.as_ref(), &self.backend as &dyn Fs)
    }
}
}

Backends implementing FsPath can provide optimized implementations that the resolver MAY use:

#![allow(unused)]
fn main() {
impl FsPath for SqliteBackend {
    fn canonicalize(&self, path: &Path) -> Result<PathBuf, FsError> {
        // Optimized: single CTE query instead of iterative resolution
        self.conn.query_row(/* ... */)
    }
}
}

Trade-offs:

Conclusion: The FsPath trait provides a clean abstraction that works everywhere but can be optimized where it matters. This follows Rust’s “zero-cost abstractions” philosophy: you don’t pay for what you don’t use, and you can optimize hot paths when needed.

ADR-033: PathResolver Trait for Pluggable Resolution

Status: Accepted

Context: Path resolution (normalizing .., ., and following symlinks) is currently handled in two places:

1. FsPath trait methods (canonicalize, soft_canonicalize) with backend-specific optimizations
2. FileStorage performs pre-resolution for non-SelfResolving backends
3. SelfResolving marker trait opts out of FileStorage resolution

This works, but the resolution algorithm is not a first-class, testable unit. The logic is spread across components, making it harder to:

• Test path resolution in isolation
• Benchmark/profile resolution performance
• Provide third-party custom resolvers
• Explore alternative resolution strategies (case-insensitive, caching, etc.)

Decision: Introduce a PathResolver trait that encapsulates the path resolution algorithm as a standalone, pluggable component.

The Pattern:

#![allow(unused)]
fn main() {
// In anyfs-backend (trait definition)
/// Strategy trait for path resolution algorithms.
///
/// Encapsulates path normalization, `..`/`.` resolution, and optionally symlink following.
///
/// **Symlink handling:** The base trait works with `&dyn Fs` (no symlink awareness).
/// For symlink-aware resolution, use `PathResolverWithLinks` which accepts `&dyn FsLink`.
/// `IterativeResolver` implements both traits - call the appropriate method based on
/// what your backend supports.
///
/// **Implementation note:** Only `canonicalize` is required. `soft_canonicalize` has a
/// default implementation that canonicalizes the parent and appends the final component.
pub trait PathResolver: Send + Sync {
    /// Resolve path to canonical form (no symlink following).
    /// Normalizes `.` and `..` components only.
    fn canonicalize(&self, path: &Path, fs: &dyn Fs) -> Result<PathBuf, FsError>;
    
    /// Like canonicalize, but allows non-existent final component.
    /// Default: canonicalize parent, append final component.
    fn soft_canonicalize(&self, path: &Path, fs: &dyn Fs) -> Result<PathBuf, FsError> {
        match path.parent() {
            Some(parent) if !parent.as_os_str().is_empty() => {
                let canonical_parent = self.canonicalize(parent, fs)?;
                match path.file_name() {
                    Some(name) => Ok(canonical_parent.join(name)),
                    None => Ok(canonical_parent),
                }
            }
            _ => self.canonicalize(path, fs),  // Root or single component
        }
    }
}

/// Extension trait for symlink-aware resolution.
/// Backends implementing FsLink can use this for full resolution.
pub trait PathResolverWithLinks: PathResolver {
    /// Resolve path following symlinks (requires FsLink backend).
    fn canonicalize_following_links(&self, path: &Path, fs: &dyn FsLink) -> Result<PathBuf, FsError>;
    
    /// Like canonicalize_following_links, but allows non-existent final component.
    /// Default: canonicalize parent following links, append final component.
    fn soft_canonicalize_following_links(&self, path: &Path, fs: &dyn FsLink) -> Result<PathBuf, FsError> {
        match path.parent() {
            Some(parent) if !parent.as_os_str().is_empty() => {
                let canonical_parent = self.canonicalize_following_links(parent, fs)?;
                match path.file_name() {
                    Some(name) => Ok(canonical_parent.join(name)),
                    None => Ok(canonical_parent),
                }
            }
            _ => self.canonicalize_following_links(path, fs),
        }
    }
}
}

Built-in Implementations (in anyfs crate):

#![allow(unused)]
fn main() {
/// Default iterative resolver - walks path component by component.
/// Implements both PathResolver and PathResolverWithLinks.
pub struct IterativeResolver {
    max_symlink_depth: usize,  // Default: 40
}

impl PathResolver for IterativeResolver {
    fn canonicalize(&self, path: &Path, fs: &dyn Fs) -> Result<PathBuf, FsError> {
        // Normalize `.` and `..` only - no symlink following
        self.normalize_path(path, fs)
    }
    // ...
}

impl PathResolverWithLinks for IterativeResolver {
    fn canonicalize_following_links(&self, path: &Path, fs: &dyn FsLink) -> Result<PathBuf, FsError> {
        // Full resolution with symlink following
        self.resolve_with_symlinks(path, fs, self.max_symlink_depth)
    }
    // ...
}

/// No-op resolver for SelfResolving backends (OS handles resolution).
pub struct NoOpResolver;

/// LRU cache wrapper around another resolver.
pub struct CachingResolver<R: PathResolver> {
    inner: R,
    cache: Cache<PathBuf, PathBuf>,  // LRU cache, bounded size
}

// Case-folding resolver is NOT built-in. Users can implement one via PathResolver
// trait if needed, but real-world demand is minimal since VRootFsBackend on
// Windows/macOS already gets case-insensitivity from the OS.
}

Integration with FileStorage:

#![allow(unused)]
fn main() {
pub struct FileStorage<B> {
    backend: B,
    resolver: Box<dyn PathResolver>,  // Boxed: resolution is cold path
}

impl<B: Fs> FileStorage<B> {
    pub fn new(backend: B) -> Self {
        Self { backend, resolver: Box::new(IterativeResolver::default()) }
    }

    pub fn with_resolver(backend: B, resolver: impl PathResolver + 'static) -> Self {
        Self { backend, resolver: Box::new(resolver) }
    }
}

// Usage
let fs = FileStorage::new(backend);  // Uses IterativeResolver
let fs = FileStorage::with_resolver(backend, CachingResolver::new(IterativeResolver::default()));
}

Relationship with FsPath Trait:

FsPath can delegate to the resolver:

#![allow(unused)]
fn main() {
pub trait FsPath: FsRead + FsLink {
    fn resolver(&self) -> &dyn PathResolver {
        static DEFAULT: IterativeResolver = IterativeResolver::new();
        &DEFAULT
    }
    
    fn canonicalize(&self, path: &Path) -> Result<PathBuf, FsError> {
        self.resolver().canonicalize(path, self)
    }
}
}

Or backends can override entirely for optimized implementations (e.g., SQLite CTE).

Why This Design:

Crate Placement:

Note: Case-folding resolvers are NOT built-in. The PathResolver trait allows users to implement custom resolvers if needed, but we don’t ship speculative features.

Example Use Cases:

#![allow(unused)]
fn main() {
// Default: case-sensitive, symlink-aware (IterativeResolver is ZST, zero-cost)
let fs = FileStorage::new(MemoryBackend::new());

// Caching for read-heavy workloads
let fs = FileStorage::with_resolver(
    backend,
    CachingResolver::new(IterativeResolver::default())
);

// Custom resolver (user-implemented)
let fs = FileStorage::with_resolver(backend, MyCustomResolver::new());

// Testing: verify resolution behavior in isolation
#[test]
fn test_symlink_loop_detection() {
    let resolver = IterativeResolver::new();
    let mock_fs = MockFs::with_symlink_loop();
    let result = resolver.canonicalize(Path::new("/loop"), &mock_fs);
    assert!(matches!(result, Err(FsError::InvalidData { .. })));
}
}

Conclusion: The PathResolver trait provides clean separation of concerns, making path resolution testable, benchmarkable, and extensible. It complements FsPath (backend optimization hook) and can replace or work alongside SelfResolving (via NoOpResolver).

ADR-034: LLM-Oriented Architecture (LOA)

Status: Accepted

Context: AnyFS is being developed with significant LLM assistance (GitHub Copilot, Claude, etc.). Traditional software architecture prioritizes maintainability, testability, and extensibility for human developers. However, when LLMs are part of the development workflow, additional constraints become essential:

1. LLMs work best with limited context windows - they can’t “understand” an entire codebase
2. LLMs excel at pattern matching - consistent structure enables better assistance
3. LLMs need clear contracts - well-documented interfaces reduce hallucination
4. LLMs benefit from isolated components - fixing one thing shouldn’t require understanding everything

These same properties also benefit:

• Open source contributors (quick onboarding)
• Code review (focused changes)
• Parallel development (independent components)
• AI-generated tests and documentation

Decision: Structure AnyFS using LLM-Oriented Architecture (LOA) - a methodology where every component is independently understandable, testable, and fixable with only local context.

The Five Pillars:

File Structure Convention:

#![allow(unused)]
fn main() {
//! # Component Name
//!
//! ## Responsibility
//! - Single bullet point
//!
//! ## Dependencies  
//! - Traits/types only
//!
//! ## Usage
//! ```rust
//! // Minimal example
//! ```

// ============================================================================
// Types
// ============================================================================

// ============================================================================
// Trait Implementations
// ============================================================================

// ============================================================================
// Public API
// ============================================================================

// ============================================================================
// Private Helpers
// ============================================================================

// ============================================================================
// Tests
// ============================================================================
}

Component Isolation Checklist:

• Single file per component
• Implements a trait with documented invariants
• Dependencies are traits/types, not implementations
• Tests use mocks, not real backends
• Error messages explain what went wrong and how to fix
• Doc comment shows standalone usage example
• No global state
• Send + Sync where required

LLM Prompting Patterns:

The architecture enables these clean prompts:

# Implement (user-provided resolver example)
Implement a case-folding resolver in your project.
Contract: Implement `PathResolver` trait from anyfs-backend.
Test: "/Foo/BAR" → "/foo/bar"

# Fix
Bug: Quota<B> doesn't account for existing file size.
File: src/middleware/quota.rs
Error: QuotaExceeded writing 50 bytes to 30-byte file with 100-byte limit.

# Review
Does this change maintain the PathResolver contract?
Are edge cases handled?
Are error messages informative?

Deliverables:

1. AGENTS.md - Instructions for LLMs contributing to the codebase
2. LLM Development Methodology Guide - Full methodology documentation
3. llm-context.md - Context7-style API guide for LLMs using the library

Why This Design:

Trade-offs:

Relationship to Other ADRs:

• ADR-030 (Layered traits): LOA extends this with per-file isolation
• ADR-033 (PathResolver): Example of LOA - resolver is isolated, testable, replaceable
• ADR-025 (Strategic Boxing): LOA prefers simplicity over micro-optimization

Conclusion: LLM-Oriented Architecture is not just about AI. It’s about creating a codebase where any component can be understood, tested, fixed, or replaced with only local context. This benefits LLMs, open source contributors, code reviewers, and future maintainers equally. As AI-assisted development becomes standard, LOA positions AnyFS as a reference implementation for sustainable human-AI collaboration.

See Also: LLM Development Methodology Guide

IndexedBackend Pattern

SQLite Metadata + Content-Addressed Blob Storage

This document describes the IndexedBackend architecture pattern: separating filesystem metadata (stored in SQLite) from file content (stored as blobs). This enables efficient queries, large file support, and flexible storage backends.

Ecosystem Implementation: The anyfs-indexed crate provides IndexedBackend as a production-ready implementation using local disk blobs. See the Backends Guide for usage. This document covers the underlying design pattern for those building custom implementations (e.g., with S3, cloud storage, or custom blob stores).

Overview

The IndexedBackend pattern separates:

• Metadata (directory structure, inodes, permissions) → SQLite
• Content (file bytes) → Content-Addressed Storage (CAS)

┌─────────────────────────────────────────────────────────┐
│            IndexedBackend (pattern)                     │
│  ┌─────────────────────┐    ┌────────────────────────┐  │
│  │   SQLite Metadata   │    │   Blob Store (CAS)     │  │
│  │                     │    │                        │  │
│  │  - inodes           │    │  - content-addressed   │  │
│  │  - dir_entries      │    │  - deduplicated        │  │
│  │  - blob references  │    │  - S3, local, etc.     │  │
│  │  - audit log        │    │                        │  │
│  └─────────────────────┘    └────────────────────────┘  │
└─────────────────────────────────────────────────────────┘

Custom backends can use S3, cloud storage, or other blob stores.
IndexedBackend implements a simpler variant with UUID-named local blobs
(optimized for streaming; see note below on storage models).

Why this pattern?

• SQLite is great for metadata queries (directory listings, stats, audit)
• Blob stores scale better for large file content
• Content-addressing enables deduplication
• Separating concerns enables independent scaling

Storage Model Variants

IndexedBackend uses UUID+Timestamp naming because:

• Large files can be streamed without buffering the entire file to compute a hash
• Write latency is consistent (no hash computation)
• Simpler garbage collection (delete blob when reference removed)

Custom implementations may prefer content-addressed storage when:

• Deduplication is valuable (many users uploading same files)
• Using cloud blob stores with native CAS support (S3, GCS)
• Building archival systems where write latency is acceptable

Framework Validation

Do Current Traits Support This?

Yes. The Fs traits define operations, not storage implementation.

The traits don’t care where bytes come from - that’s the backend’s business.

Thread Safety

Current design requires &self methods with interior mutability. For hybrid:

#![allow(unused)]
fn main() {
pub struct CustomIndexedBackend {
    // SQLite needs single-writer (see "Write Queue" below)
    metadata: Arc<Mutex<Connection>>,

    // Blob store is typically already thread-safe
    blobs: Arc<dyn BlobStore>,

    // Write queue for serializing SQLite writes
    write_tx: mpsc::Sender<WriteCmd>,
}
}

This aligns with ADR-023 (interior mutability).

Data Model

SQLite Schema

-- Inode table (one row per file/directory/symlink)
CREATE TABLE nodes (
    inode       INTEGER PRIMARY KEY,
    parent      INTEGER NOT NULL,
    name        TEXT NOT NULL,
    node_type   TEXT NOT NULL,  -- 'file', 'dir', 'symlink'
    size        INTEGER NOT NULL DEFAULT 0,
    mode        INTEGER NOT NULL DEFAULT 420,  -- 0o644
    nlink       INTEGER NOT NULL DEFAULT 1,
    blob_id     TEXT,           -- NULL for directories
    symlink_target TEXT,        -- NULL unless symlink
    created_at  INTEGER NOT NULL,
    modified_at INTEGER NOT NULL,
    accessed_at INTEGER NOT NULL,

    UNIQUE(parent, name)
);

-- Root directory (inode 1)
INSERT INTO nodes (inode, parent, name, node_type, size, mode, created_at, modified_at, accessed_at)
VALUES (1, 1, '', 'dir', 0, 493, strftime('%s', 'now'), strftime('%s', 'now'), strftime('%s', 'now'));

-- Blob reference tracking (for dedup + GC)
CREATE TABLE blobs (
    blob_id     TEXT PRIMARY KEY,  -- sha256 hex
    size        INTEGER NOT NULL,
    refcount    INTEGER NOT NULL DEFAULT 0,
    created_at  INTEGER NOT NULL
);

-- Audit log (optional but recommended)
CREATE TABLE audit (
    seq         INTEGER PRIMARY KEY AUTOINCREMENT,
    timestamp   INTEGER NOT NULL,
    operation   TEXT NOT NULL,
    path        TEXT,
    actor       TEXT,
    details     TEXT  -- JSON
);

-- Indexes
CREATE INDEX idx_nodes_parent ON nodes(parent);
CREATE INDEX idx_nodes_blob ON nodes(blob_id) WHERE blob_id IS NOT NULL;
CREATE INDEX idx_blobs_refcount ON blobs(refcount) WHERE refcount = 0;

Blob Store Interface

#![allow(unused)]
fn main() {
/// Content-addressed blob storage.
pub trait BlobStore: Send + Sync {
    /// Store bytes, returns content hash (blob_id).
    fn put(&self, data: &[u8]) -> Result<String, BlobError>;

    /// Retrieve bytes by content hash.
    fn get(&self, blob_id: &str) -> Result<Vec<u8>, BlobError>;

    /// Check if blob exists.
    fn exists(&self, blob_id: &str) -> Result<bool, BlobError>;

    /// Delete blob (only call after refcount reaches 0).
    fn delete(&self, blob_id: &str) -> Result<(), BlobError>;

    /// Streaming read for large files.
    fn open_read(&self, blob_id: &str) -> Result<Box<dyn Read + Send>, BlobError>;

    /// Streaming write, returns blob_id on completion.
    fn open_write(&self) -> Result<Box<dyn BlobWriter>, BlobError>;
}

pub trait BlobWriter: Write + Send {
    /// Finalize the blob and return its content hash.
    fn finalize(self: Box<Self>) -> Result<String, BlobError>;
}
}

Implementations could be:

• LocalCasBackend - local directory with content-addressed files
• S3BlobStore - S3-compatible object storage
• MemoryBlobStore - in-memory for testing

Implementation Sketch

Core Structure

#![allow(unused)]
fn main() {
use anyfs_backend::{FsRead, FsWrite, FsDir, FsError, Metadata, ReadDirIter, DirEntry, FileType};
use rusqlite::Connection;
use std::sync::{Arc, Mutex};
use std::path::{Path, PathBuf};
use tokio::sync::mpsc;

pub struct CustomIndexedBackend {
    /// SQLite connection (metadata)
    db: Arc<Mutex<Connection>>,

    /// Content-addressed blob storage
    blobs: Arc<dyn BlobStore>,

    /// Write command queue (single-writer pattern)
    write_tx: mpsc::UnboundedSender<WriteCmd>,

    /// Background writer handle
    _writer_handle: Arc<WriterHandle>,
}

enum WriteCmd {
    Write {
        path: PathBuf,
        blob_id: String,
        size: u64,
        reply: oneshot::Sender<Result<(), FsError>>,
    },
    Remove {
        path: PathBuf,
        reply: oneshot::Sender<Result<(), FsError>>,
    },
    CreateDir {
        path: PathBuf,
        reply: oneshot::Sender<Result<(), FsError>>,
    },
    // ... other write operations
}
}

Read Operations (Direct)

Read operations can query SQLite and blob store directly (no queue needed):

#![allow(unused)]
fn main() {
impl FsRead for CustomIndexedBackend {
    fn read(&self, path: &Path) -> Result<Vec<u8>, FsError> {
        let path = path.as_ref();

        // 1. Query SQLite for blob_id
        let db = self.db.lock().map_err(|_| FsError::Backend("lock poisoned".into()))?;

        let (blob_id, node_type): (Option<String>, String) = db.query_row(
            "SELECT blob_id, node_type FROM nodes WHERE inode = (
                SELECT inode FROM nodes WHERE parent = ? AND name = ?
            )",
            // ... path resolution params
            |row| Ok((row.get(0)?, row.get(1)?)),
        ).map_err(|_| FsError::NotFound { path: path.to_path_buf() })?;

        if node_type != "file" {
            return Err(FsError::NotAFile { path: path.to_path_buf() });
        }

        let blob_id = blob_id.ok_or_else(|| FsError::NotFound { path: path.to_path_buf() })?;

        drop(db);  // Release lock before blob fetch

        // 2. Fetch from blob store
        self.blobs.get(&blob_id)
            .map_err(|e| FsError::Backend(e.to_string()))
    }

    fn exists(&self, path: &Path) -> Result<bool, FsError> {
        let path = path.as_ref();
        let db = self.db.lock().map_err(|_| FsError::Backend("lock poisoned".into()))?;

        // Pure SQLite query
        let exists: bool = db.query_row(
            "SELECT EXISTS(SELECT 1 FROM nodes WHERE parent = ? AND name = ?)",
            // ... params
            |row| row.get(0),
        ).unwrap_or(false);

        Ok(exists)
    }

    fn metadata(&self, path: &Path) -> Result<Metadata, FsError> {
        let path = path.as_ref();
        let db = self.db.lock().map_err(|_| FsError::Backend("lock poisoned".into()))?;

        // Pure SQLite query - no blob store needed
        db.query_row(
            "SELECT node_type, size, mode, nlink, created_at, modified_at, accessed_at, inode
             FROM nodes WHERE parent = ? AND name = ?",
            // ... params
            |row| {
                let node_type: String = row.get(0)?;
                Ok(Metadata {
                    file_type: match node_type.as_str() {
                        "file" => FileType::File,
                        "dir" => FileType::Directory,
                        "symlink" => FileType::Symlink,
                        _ => FileType::File,
                    },
                    size: row.get(1)?,
                    permissions: Some(row.get(2)?),
                    // ... other fields
                })
            },
        ).map_err(|_| FsError::NotFound { path: path.to_path_buf() })
    }

    // ... other FsRead methods
}
}

Write Operations (Two-Phase Commit)

Writes use a two-phase pattern: upload blob first, then commit SQLite:

#![allow(unused)]
fn main() {
impl FsWrite for CustomIndexedBackend {
    fn write(&self, path: &Path, data: &[u8]) -> Result<(), FsError> {
        let path = path.as_ref().to_path_buf();

        // Phase 1: Upload blob (can fail independently)
        let blob_id = self.blobs.put(data)
            .map_err(|e| FsError::Backend(format!("blob upload failed: {}", e)))?;

        // Phase 2: Commit metadata (via write queue)
        let (tx, rx) = oneshot::channel();

        self.write_tx.send(WriteCmd::Write {
            path,
            blob_id,
            size: data.len() as u64,
            reply: tx,
        }).map_err(|_| FsError::Backend("write queue closed".into()))?;

        // Wait for SQLite commit
        rx.blocking_recv()
            .map_err(|_| FsError::Backend("write cancelled".into()))?
    }

    fn remove_file(&self, path: &Path) -> Result<(), FsError> {
        let path = path.as_ref().to_path_buf();

        // Queue the removal (blob cleanup happens in background via GC)
        let (tx, rx) = oneshot::channel();

        self.write_tx.send(WriteCmd::Remove { path, reply: tx })
            .map_err(|_| FsError::Backend("write queue closed".into()))?;

        rx.blocking_recv()
            .map_err(|_| FsError::Backend("remove cancelled".into()))?
    }

    fn copy(&self, from: &Path, to: &Path) -> Result<(), FsError> {
        // Copy is just a metadata operation - increment refcount, no blob copy!
        let (tx, rx) = oneshot::channel();

        self.write_tx.send(WriteCmd::Copy {
            from: from.as_ref().to_path_buf(),
            to: to.as_ref().to_path_buf(),
            reply: tx,
        }).map_err(|_| FsError::Backend("write queue closed".into()))?;

        rx.blocking_recv()
            .map_err(|_| FsError::Backend("copy cancelled".into()))?
    }

    // ... other FsWrite methods
}
}

Write Queue Worker

The single-writer pattern for SQLite:

#![allow(unused)]
fn main() {
async fn write_worker(
    db: Arc<Mutex<Connection>>,
    blobs: Arc<dyn BlobStore>,
    mut rx: mpsc::UnboundedReceiver<WriteCmd>,
) {
    while let Some(cmd) = rx.recv().await {
        let result = {
            let mut db = db.lock().unwrap();

            match cmd {
                WriteCmd::Write { path, blob_id, size, reply } => {
                    let result = db.execute_batch(&format!(r#"
                        BEGIN;

                        -- Upsert blob record
                        INSERT INTO blobs (blob_id, size, refcount, created_at)
                        VALUES ('{blob_id}', {size}, 1, strftime('%s', 'now'))
                        ON CONFLICT(blob_id) DO UPDATE SET refcount = refcount + 1;

                        -- Update or insert node
                        -- (simplified - real impl needs path resolution)

                        -- Audit log
                        INSERT INTO audit (timestamp, operation, path)
                        VALUES (strftime('%s', 'now'), 'write', '{path}');

                        COMMIT;
                    "#));

                    let _ = reply.send(result.map_err(|e| FsError::Backend(e.to_string())));
                }

                WriteCmd::Remove { path, reply } => {
                    // Decrement refcount (GC cleans up when refcount = 0)
                    let result = db.execute_batch(&format!(r#"
                        BEGIN;

                        -- Get blob_id before delete
                        -- Decrement refcount
                        -- Remove node
                        -- Audit log

                        COMMIT;
                    "#));

                    let _ = reply.send(result.map_err(|e| FsError::Backend(e.to_string())));
                }

                // ... other commands
            }
        };
    }
}
}

Deduplication

Content-addressing gives you dedup for free:

#![allow(unused)]
fn main() {
impl BlobStore for LocalCasBackend {
    fn put(&self, data: &[u8]) -> Result<String, BlobError> {
        // Hash the content
        let hash = sha256(data);
        let blob_id = hex::encode(hash);

        // Check if already exists
        let blob_path = self.root.join(&blob_id[0..2]).join(&blob_id);

        if blob_path.exists() {
            // Already have this content - dedup!
            return Ok(blob_id);
        }

        // Store new blob
        std::fs::create_dir_all(blob_path.parent().unwrap())?;
        std::fs::write(&blob_path, data)?;

        Ok(blob_id)
    }
}
}

Dedup in action:

• User A writes report.pdf (10 MB) → blob abc123, refcount = 1
• User B writes identical report.pdf → same blob abc123, refcount = 2
• Physical storage: 10 MB (not 20 MB)

Refcount Management

-- On file write (new reference to blob)
UPDATE blobs SET refcount = refcount + 1 WHERE blob_id = ?;

-- On file delete
UPDATE blobs SET refcount = refcount - 1 WHERE blob_id = ?;

-- On copy (no blob copy needed!)
UPDATE blobs SET refcount = refcount + 1 WHERE blob_id = ?;

SQLite Performance

The SQLite metadata database benefits from the same tuning as SqliteBackend:

Why FULL synchronous: Index corruption means paths no longer resolve to blobs—blobs become orphaned and unreachable. Use FULL as the safe default; opt-in to NORMAL only with battery-backed storage or when index can be rebuilt.

SQL Indexes (critical):

CREATE INDEX idx_nodes_parent ON nodes(parent);
CREATE INDEX idx_nodes_blob ON nodes(blob_id) WHERE blob_id IS NOT NULL;
CREATE INDEX idx_blobs_refcount ON blobs(refcount) WHERE refcount = 0;

Without proper indexes, path lookups become full table scans—catastrophic for large filesystems.

Connection pooling: 4-8 reader connections for concurrent metadata queries; single writer for updates. See SQLite Operations Guide for detailed patterns.

Garbage Collection

Blobs with refcount = 0 are orphans and can be deleted:

#![allow(unused)]
fn main() {
impl CustomIndexedBackend {
    /// Run garbage collection (call periodically or on-demand).
    pub fn gc(&self) -> Result<GcStats, FsError> {
        let db = self.db.lock().map_err(|_| FsError::Backend("lock".into()))?;

        // Find orphaned blobs
        let orphans: Vec<String> = db.prepare(
            "SELECT blob_id FROM blobs WHERE refcount = 0"
        )?.query_map([], |row| row.get(0))?
          .filter_map(|r| r.ok())
          .collect();

        drop(db);

        // Delete from blob store
        let mut deleted = 0;
        for blob_id in &orphans {
            if self.blobs.delete(blob_id).is_ok() {
                deleted += 1;
            }
        }

        // Remove from SQLite
        let db = self.db.lock().unwrap();
        db.execute(
            "DELETE FROM blobs WHERE refcount = 0",
            [],
        )?;

        Ok(GcStats { orphans_found: orphans.len(), blobs_deleted: deleted })
    }
}
}

GC Safety:

• Never delete blobs referenced by snapshots
• Add snapshot_refs table or use refcount that includes snapshot references
• Run GC in background, not during writes

Snapshots and Backup

Creating a Snapshot

#![allow(unused)]
fn main() {
impl CustomIndexedBackend {
    /// Create a point-in-time snapshot.
    pub fn snapshot(&self, name: &str) -> Result<SnapshotId, FsError> {
        let db = self.db.lock().unwrap();

        db.execute_batch(&format!(r#"
            BEGIN;

            -- Record snapshot
            INSERT INTO snapshots (name, created_at, root_manifest)
            VALUES ('{name}', strftime('%s', 'now'),
                    (SELECT json_group_array(blob_id) FROM blobs WHERE refcount > 0));

            -- Pin all current blobs (prevent GC)
            UPDATE blobs SET refcount = refcount + 1
            WHERE blob_id IN (SELECT blob_id FROM nodes WHERE blob_id IS NOT NULL);

            COMMIT;
        "#))?;

        Ok(SnapshotId(name.to_string()))
    }

    /// Export as single portable artifact.
    pub fn export(&self, dest: impl AsRef<Path>) -> Result<(), FsError> {
        // 1. SQLite backup API for metadata
        let db = self.db.lock().unwrap();
        let backup_db = Connection::open(dest.as_ref().join("metadata.db"))?;
        db.backup(rusqlite::DatabaseName::Main, &backup_db, None)?;

        // 2. Copy referenced blobs
        let blob_ids: Vec<String> = db.prepare(
            "SELECT DISTINCT blob_id FROM nodes WHERE blob_id IS NOT NULL"
        )?.query_map([], |row| row.get(0))?
          .filter_map(|r| r.ok())
          .collect();

        drop(db);

        let blobs_dir = dest.as_ref().join("blobs");
        std::fs::create_dir_all(&blobs_dir)?;

        for blob_id in blob_ids {
            let data = self.blobs.get(&blob_id)?;
            std::fs::write(blobs_dir.join(&blob_id), data)?;
        }

        Ok(())
    }
}
}

Middleware Integration

Middleware works unchanged - it wraps the hybrid backend like any other:

#![allow(unused)]
fn main() {
use anyfs::{FileStorage, QuotaLayer, TracingLayer, PathFilterLayer};

let backend = CustomIndexedBackend::open("drive.db", LocalCasBackend::new("./blobs"))?;

// Standard middleware stack
let backend = backend
    .layer(QuotaLayer::builder()
        .max_total_size(50 * 1024 * 1024 * 1024)  // 50 GB
        .build())
    .layer(PathFilterLayer::builder()
        .deny("**/.env")
        .build())
    .layer(TracingLayer::new());

let fs = FileStorage::new(backend);

// Use like any other filesystem
fs.write("/documents/report.pdf", &pdf_bytes)?;
}

Quota tracking note: QuotaLayer tracks logical size (what users see), not physical size (with dedup). For physical tracking, the backend could expose physical_usage() separately.

Async Considerations

The hybrid pattern benefits significantly from async (ADR-024):

When AsyncFs traits exist (ADR-024), the hybrid backend can use them naturally:

#![allow(unused)]
fn main() {
#[async_trait]
impl AsyncFsRead for CustomIndexedBackend {
    async fn read(&self, path: &Path) -> Result<Vec<u8>, FsError> {
        let blob_id = self.lookup_blob_id(path).await?;
        self.blobs.get_async(&blob_id).await  // Non-blocking!
    }
}
}

Identified Gaps

Areas where the current framework could be enhanced:

Summary

Framework validation: PASSED

The current AnyFS trait design supports hybrid backends:

• Traits define operations, not storage
• Interior mutability allows single-writer patterns
• Middleware composes unchanged
• Async strategy (ADR-024) enhances this pattern

Key patterns for hybrid backends:

1. Single-writer queue for SQLite
2. Two-phase commit (blob upload → SQLite commit)
3. Content-addressing for dedup
4. Refcounting for GC safety
5. Snapshot pinning for backup safety

This validates that AnyFS is flexible enough for advanced storage architectures while maintaining its simple middleware composition model.

Zero-Cost Alternatives for I/O Operations

This document analyzes alternatives to dynamic dispatch (Box<dyn Trait>) for streaming I/O and directory iteration.

Decision: See ADR-025: Strategic Boxing for the formal decision.

TL;DR: We follow Tower/Axum’s approach - zero-cost on hot path (read(), write()), box at cold path boundaries (open_read(), read_dir()). We avoid heap allocations and dynamic dispatch unless they buy flexibility with negligible performance impact.

Current Design (Dynamic Dispatch)

#![allow(unused)]
fn main() {
pub trait FsRead: Send + Sync {
    fn open_read(&self, path: &Path) -> Result<Box<dyn Read + Send>, FsError>;
}

pub trait FsDir: Send + Sync {
    fn read_dir(&self, path: &Path) -> Result<ReadDirIter, FsError>;
}

// Where ReadDirIter is:
pub struct ReadDirIter(Box<dyn Iterator<Item = Result<DirEntry, FsError>> + Send>);
}

Cost: One heap allocation per open_read(), open_write(), or read_dir() call.

Option 1: Associated Types (Classic Approach)

#![allow(unused)]
fn main() {
pub trait FsRead: Send + Sync {
    type Reader: Read + Send;

    fn open_read(&self, path: &Path) -> Result<Self::Reader, FsError>;
}

pub trait FsDir: Send + Sync {
    type DirIter: Iterator<Item = Result<DirEntry, FsError>> + Send;

    fn read_dir(&self, path: &Path) -> Result<Self::DirIter, FsError>;
}
}

Implementation

#![allow(unused)]
fn main() {
impl FsRead for MemoryBackend {
    type Reader = std::io::Cursor<Vec<u8>>;

    fn open_read(&self, path: &Path) -> Result<Self::Reader, FsError> {
        let data = self.read(path)?;
        Ok(std::io::Cursor::new(data))
    }
}

impl FsDir for MemoryBackend {
    type DirIter = std::vec::IntoIter<Result<DirEntry, FsError>>;

    fn read_dir(&self, path: &Path) -> Result<Self::DirIter, FsError> {
        let entries = self.collect_entries(path)?;
        Ok(entries.into_iter())
    }
}
}

Middleware Propagation Problem

#![allow(unused)]
fn main() {
impl<B: FsRead> FsRead for Quota<B> {
    // Must define our own Reader type that wraps B::Reader
    type Reader = QuotaReader<B::Reader>;

    fn open_read(&self, path: &Path) -> Result<Self::Reader, FsError> {
        let inner = self.inner.open_read(path)?;
        Ok(QuotaReader::new(inner, self.usage.clone()))
    }
}

// Every middleware needs a custom wrapper type
struct QuotaReader<R> {
    inner: R,
    usage: Arc<RwLock<QuotaUsage>>,
}

impl<R: Read> Read for QuotaReader<R> {
    fn read(&mut self, buf: &mut [u8]) -> std::io::Result<usize> {
        // Track bytes read if needed
        self.inner.read(buf)
    }
}
}

The Type Explosion

With a middleware stack like Quota<PathFilter<Tracing<MemoryBackend>>>:

#![allow(unused)]
fn main() {
type FinalReader = QuotaReader<PathFilterReader<TracingReader<Cursor<Vec<u8>>>>>;
type FinalDirIter = QuotaIter<PathFilterIter<TracingIter<IntoIter<Result<DirEntry, FsError>>>>>;
}

Verdict

Not recommended as the primary API due to complexity explosion.

Option 2: RPITIT (Rust 1.75+)

Return Position Impl Trait in Traits allows:

#![allow(unused)]
fn main() {
pub trait FsRead: Send + Sync {
    fn open_read(&self, path: &Path) -> Result<impl Read + Send, FsError>;
}

pub trait FsDir: Send + Sync {
    fn read_dir(&self, path: &Path)
        -> Result<impl Iterator<Item = Result<DirEntry, FsError>> + Send, FsError>;
}
}

How It Works

The compiler infers a unique anonymous type for each implementor:

#![allow(unused)]
fn main() {
impl FsRead for MemoryBackend {
    fn open_read(&self, path: &Path) -> Result<impl Read + Send, FsError> {
        let data = self.read(path)?;
        Ok(std::io::Cursor::new(data))  // Returns Cursor<Vec<u8>>, but caller sees impl Read
    }
}

impl FsRead for SqliteBackend {
    fn open_read(&self, path: &Path) -> Result<impl Read + Send, FsError> {
        Ok(SqliteReader::new(self.conn.clone(), path))  // Different type, same interface
    }
}
}

Middleware Still Works

#![allow(unused)]
fn main() {
impl<B: FsRead> FsRead for Tracing<B> {
    fn open_read(&self, path: &Path) -> Result<impl Read + Send, FsError> {
        let span = tracing::span!(Level::DEBUG, "open_read");
        let _guard = span.enter();
        self.inner.open_read(path)  // Just forward - return type is inferred
    }
}
}

The Catch: Object Safety

RPITIT makes traits non-object-safe. You cannot do:

#![allow(unused)]
fn main() {
// This won't compile with RPITIT
let backends: Vec<Box<dyn FsRead>> = vec![...];
}

Verdict

Good for performance-critical paths but sacrifices dyn usage.

Option 3: Generic Associated Types (GATs)

For readers that borrow from the backend:

#![allow(unused)]
fn main() {
pub trait FsRead: Send + Sync {
    type Reader<'a>: Read + Send where Self: 'a;

    fn open_read(&self, path: &Path) -> Result<Self::Reader<'_>, FsError>;
}
}

Use Case: Zero-Copy Reads

#![allow(unused)]
fn main() {
impl FsRead for MemoryBackend {
    type Reader<'a> = &'a [u8];  // Borrow directly from internal storage!

    fn open_read(&self, path: &Path) -> Result<Self::Reader<'_>, FsError> {
        let data = self.storage.read().unwrap();
        let bytes = data.get(path.as_ref())
            .ok_or(FsError::NotFound { path: path.as_ref().to_path_buf() })?;
        Ok(bytes.as_slice())
    }
}
}

Complexity

GATs are powerful but add significant complexity:

• Lifetime parameters propagate through middleware
• Not all backends can provide borrowed data (SQLite must copy)
• Makes trait definitions harder to understand

Verdict

Overkill for most use cases. Consider only for specialized zero-copy scenarios.

Option 4: Hybrid Approach (Recommended)

Provide both dynamic and static APIs:

#![allow(unused)]
fn main() {
pub trait FsRead: Send + Sync {
    /// Dynamic dispatch version (simple, flexible)
    fn open_read(&self, path: &Path) -> Result<Box<dyn Read + Send>, FsError>;
}

/// Extension trait for zero-cost static dispatch
pub trait FsReadTyped: FsRead {
    type Reader: Read + Send;

    /// Static dispatch version (zero-cost, less flexible)
    fn open_read_typed(&self, path: &Path) -> Result<Self::Reader, FsError>;
}

// Blanket impl for convenience when types align
impl<T: FsReadTyped> FsRead for T {
    fn open_read(&self, path: &Path) -> Result<Box<dyn Read + Send>, FsError> {
        Ok(Box::new(self.open_read_typed(path)?))
    }
}
}

Usage

#![allow(unused)]
fn main() {
// Default: dynamic dispatch (works everywhere)
let reader = fs.open_read("/file.txt")?;

// Performance-critical: static dispatch
let reader: MemoryReader = fs.open_read_typed("/file.txt")?;
}

Verdict

Best of both worlds - simple default, zero-cost opt-in.

Option 5: Callback-Based Iteration

Avoid returning iterators entirely:

#![allow(unused)]
fn main() {
pub trait FsDir: Send + Sync {
    fn for_each_entry<F>(&self, path: &Path, f: F) -> Result<(), FsError>
    where
        F: FnMut(DirEntry) -> ControlFlow<(), ()>;
}
}

Usage

#![allow(unused)]
fn main() {
fs.for_each_entry("/dir", |entry| {
    println!("{}", entry.name);
    ControlFlow::Continue(())
})?;
}

Verdict

Not recommended as primary API. Could be added as optimization option.

Option 6: Stack-Allocated Small Buffer

For directory iteration, most directories are small:

#![allow(unused)]
fn main() {
use smallvec::SmallVec;

pub struct ReadDirIter {
    // Stack-allocate up to 32 entries, heap only if larger
    entries: SmallVec<[Result<DirEntry, FsError>; 32]>,
    index: usize,
}
}

Verdict

Reasonable optimization for directory iteration specifically.

Recommendation

Primary API: Keep Dynamic Dispatch

#![allow(unused)]
fn main() {
pub trait FsRead: Send + Sync {
    fn open_read(&self, path: &Path) -> Result<Box<dyn Read + Send>, FsError>;
}

pub trait FsDir: Send + Sync {
    fn read_dir(&self, path: &Path) -> Result<ReadDirIter, FsError>;
}
}

Why:

1. Simplicity - One type to learn, one API
2. Object safety - Can use Box<dyn Fs> for runtime polymorphism
3. Middleware simplicity - No wrapper types needed
4. Actual cost is low - One allocation per stream open, not per read

Optional: Static Dispatch Extension (Fast Path)

For performance-critical code, offer typed variants. This is the first-class fast path for hot loops when the backend type is known:

#![allow(unused)]
fn main() {
pub trait FsReadTyped: FsRead {
    type Reader: Read + Send;
    fn open_read_typed(&self, path: &Path) -> Result<Self::Reader, FsError>;
}
}

Future: RPITIT When Object Safety Not Needed

If a user doesn’t need dyn Fs, they can define their own trait:

#![allow(unused)]
fn main() {
pub trait FsReadStatic: Send + Sync {
    fn open_read(&self, path: &Path) -> Result<impl Read + Send, FsError>;
}
}

Cost Analysis: Is It Actually a Problem?

Heap Allocation Cost

The allocation is dwarfed by actual I/O time. For a 4KB file read from SQLite or disk, the Box allocation is <0.1% of total time.

When It Matters

Conclusion

Dynamic dispatch is the right default. The cost is negligible for real workloads, and the ergonomic benefits are substantial. Offer static dispatch as an opt-in escape hatch for the rare cases where it matters.

Summary Decision Matrix

Indexing Middleware (Design Plan)

Status: Accepted (Future) — See ADR-031 Scope: Design plan only (no API break)

Summary

Provide a consistent, queryable index of file activity and metadata for real filesystems (SQLite default). The index tracks operations (create, write, rename, delete) and maintains a catalog of files for fast queries and statistics. This enables workflows like “manage a flash drive and query every change” and “mount a drive and get an implicit audit trail.”

Direction: Middleware-only. Indexing is a composable layer users opt into when they want a queryable catalog of file activity.

Goals

• Preserve std::fs-style DX via FileStorage (no change to core traits).
• Track file operations with timestamps in a durable index (SQLite default).
• Provide fast queries (by path, prefix, mtime, size, hash).
• Keep index consistent for operations executed through AnyFS.
• Keep the design open to future index engines via a small trait (SQLite default).

Non-Goals

• Full OS-level auditing outside AnyFS (requires kernel hooks).
• Mandatory hashing of all files (optional and expensive).
• Replacing Tracing (indexing is storage + query, tracing is instrumentation).

Architecture (Middleware-Only)

Indexing Middleware (Primary)

Shape: Indexing<B> where B: Fs

Layer: IndexLayer (builder-based, like QuotaLayer, TracingLayer)

Behavior: Intercepts operations and writes entries into an index (SQLite by default). Works on all backends (Memory, SQLite, VRootFs, custom). Guarantees apply only to operations that flow through AnyFS.

Pros

• Backend-agnostic.
• Useful for virtual backends too.
• No special-case OS behavior.

Cons

• External changes on real FS are not captured (unless a watcher/scan helper is added later).

Where It Fits

Consistency Model

• Through AnyFS: Strong consistency for index updates in strict mode.
• External OS changes: Not captured by default. A future watcher/scan helper can reconcile.

Modes:

#![allow(unused)]
fn main() {
enum IndexConsistency {
    Strict,      // If index update fails, return error from FS op
    BestEffort,  // FS op succeeds even if index update fails
}
}

Index Schema (SQLite)

Minimal schema focused on query speed and durability:

CREATE TABLE IF NOT EXISTS nodes (
  path TEXT PRIMARY KEY,
  parent_path TEXT NOT NULL,
  file_type INTEGER NOT NULL,      -- 0=file, 1=dir, 2=symlink
  size INTEGER NOT NULL DEFAULT 0,
  inode INTEGER,
  nlink INTEGER,
  permissions INTEGER,
  created_at INTEGER,
  modified_at INTEGER,
  accessed_at INTEGER,
  symlink_target TEXT,
  hash BLOB,                        -- optional
  exists INTEGER NOT NULL DEFAULT 1,
  last_seen_at INTEGER NOT NULL
);

CREATE INDEX IF NOT EXISTS idx_nodes_parent ON nodes(parent_path);
CREATE INDEX IF NOT EXISTS idx_nodes_mtime ON nodes(modified_at);
CREATE INDEX IF NOT EXISTS idx_nodes_hash ON nodes(hash);

CREATE TABLE IF NOT EXISTS ops (
  id INTEGER PRIMARY KEY,
  ts INTEGER NOT NULL,
  op TEXT NOT NULL,                 -- "write", "rename", "remove", ...
  path TEXT,
  path_to TEXT,
  bytes INTEGER,
  status TEXT NOT NULL,             -- "ok" | "err"
  error TEXT
);

CREATE TABLE IF NOT EXISTS config (
  key TEXT PRIMARY KEY,
  value TEXT NOT NULL
);

Path normalization: Store virtual paths (what the user sees), not host paths. For host FS, optionally store host paths in a separate table if needed.

Operation Mapping

Streaming writes: Wrap open_write() with a counting writer that records final size and timestamps on close.

Configuration

#![allow(unused)]
fn main() {
pub struct IndexConfig {
    pub index_file: PathBuf,            // sidecar index file (SQLite default)
    pub consistency: IndexConsistency,
    pub track_reads: bool,
    pub track_errors: bool,
    pub track_metadata: bool,
    pub content_hashing: ContentHashing, // None | OnWrite | OnDemand
    pub initial_scan: InitialScan,       // None | OnDemand | FullScan
}
}

Naming

• Middleware type: Indexing<B>
• Layer: IndexLayer
• Builder methods emphasize intent: index_file, consistency, track_*, content_hashing, initial_scan

Example Configuration

#![allow(unused)]
fn main() {
let backend = MemoryBackend::new()
    .layer(IndexLayer::builder()
        .index_file("index.db")
        .consistency(IndexConsistency::Strict)
        .track_reads(false)
        .build());
}

Index Engine Abstraction (Future)

To keep the middleware ergonomic while enabling alternate engines, define a small storage trait and keep SQLite as the default implementation:

#![allow(unused)]
fn main() {
pub trait IndexStore: Send + Sync {
    fn upsert_node(&self, node: IndexNode) -> Result<(), IndexError>;
    fn mark_removed(&self, path: &Path) -> Result<(), IndexError>;
    fn rename_prefix(&self, from: &Path, to: &Path) -> Result<(), IndexError>;
    fn record_op(&self, entry: OpEntry) -> Result<(), IndexError>;
}

}

The default implementation uses SQLite at index_file. If/when alternate engines are needed, the IndexLayer builder can accept a boxed IndexStore for advanced use without introducing an enum.

Performance Notes

• Use WAL mode for concurrency.
• Batch updates for recursive operations (rename/remove_dir_all).
• Hashing is optional and should be off by default.
• Keep op logs bounded (optional retention policy).

For detailed SQLite tuning (pragmas, connection pooling, checkpointing), see the SQLite Operations Guide.

Security and Containment

• Index file should live outside the root path by default.
• For mounted drives, use a dedicated index path per mount.
• Respect PathFilter and Restrictions when operating through middleware.

Mounting Scenario

When mounted via anyfs (with fuse or winfsp feature flags), all access goes through AnyFS. The index becomes an implicit audit trail:

• Every file operation is logged.
• Queries reflect all operations routed through AnyFS.

Open Questions

• Should op logs be bounded by size/time by default?
• Do we need a query API in anyfs or a separate anyfs-index crate?
• Should middleware expose a read-only IndexStore handle for queries?
• Should we add a companion watcher/scan tool for external changes on real FS?

Layered Traits (anyfs-backend)

AnyFS uses a layered trait architecture for maximum flexibility with minimal complexity.

See ADR-030 for the design rationale.

Trait Hierarchy

                    FsPosix
                       │
        ┌──────────────┼──────────────┐
        │              │              │
   FsHandles       FsLock        FsXattr
        │              │              │
        └──────────────┴──────────────┘
                       │
                    FsFuse ← FsFull + FsInode
                       │
        ┌──────────────┴──────────────┐
        │                             │
     FsFull                       FsInode
        │
        │
        ├──────┬───────┬───────┬──────┐
        │      │       │       │      │
   FsLink   FsPerm  FsSync  FsStats   │
        │      │       │       │      │
        └──────┴───────┴───────┴──────┘
                       │
                      Fs   ← Most users only need this
                       │
           ┌───────────┼───────────┐
           │           │           │
        FsRead      FsWrite     FsDir

                                              Derived Traits (auto-impl)
                                              ───────────────────────────
                                              FsPath: FsRead + FsLink
                                                (path canonicalization)

Simple rule: Import Fs for basic use. Add traits as needed for advanced features.

Note: FsPath is a derived trait with a blanket impl. Any type implementing FsRead + FsLink automatically gets FsPath. SelfResolving is a marker trait - backends that implement it should be used with NoOpResolver in FileStorage (there is no automatic detection; use FileStorage::with_resolver(backend, NoOpResolver) explicitly). PathResolver is a strategy trait for pluggable path resolution (see ADR-033).

Layer 1: Core Traits (Required)

Thread Safety: All traits require Send + Sync and use &self for all methods. Backend implementers MUST use interior mutability (RwLock, Mutex, etc.) to ensure thread-safe concurrent access. See ADR-023 for rationale.

Path Parameters: Core traits use &Path so they are object-safe (dyn Fs works). For ergonomics, FileStorage and FsExt accept impl AsRef<Path> and forward to the core traits.

FsRead

#![allow(unused)]
fn main() {
pub trait FsRead: Send + Sync {
    fn read(&self, path: &Path) -> Result<Vec<u8>, FsError>;
    fn read_to_string(&self, path: &Path) -> Result<String, FsError>;
    fn read_range(&self, path: &Path, offset: u64, len: usize) -> Result<Vec<u8>, FsError>;
    fn exists(&self, path: &Path) -> Result<bool, FsError>;
    fn metadata(&self, path: &Path) -> Result<Metadata, FsError>;
    fn open_read(&self, path: &Path) -> Result<Box<dyn Read + Send>, FsError>;
}
}

FsWrite

#![allow(unused)]
fn main() {
pub trait FsWrite: Send + Sync {
    fn write(&self, path: &Path, data: &[u8]) -> Result<(), FsError>;
    fn append(&self, path: &Path, data: &[u8]) -> Result<(), FsError>;
    fn remove_file(&self, path: &Path) -> Result<(), FsError>;
    fn rename(&self, from: &Path, to: &Path) -> Result<(), FsError>;
    fn copy(&self, from: &Path, to: &Path) -> Result<(), FsError>;
    fn truncate(&self, path: &Path, size: u64) -> Result<(), FsError>;
    fn open_write(&self, path: &Path) -> Result<Box<dyn Write + Send>, FsError>;
}
}

FsDir

#![allow(unused)]
fn main() {
pub trait FsDir: Send + Sync {
    fn read_dir(&self, path: &Path) -> Result<ReadDirIter, FsError>;
    fn create_dir(&self, path: &Path) -> Result<(), FsError>;
    fn create_dir_all(&self, path: &Path) -> Result<(), FsError>;
    fn remove_dir(&self, path: &Path) -> Result<(), FsError>;
    fn remove_dir_all(&self, path: &Path) -> Result<(), FsError>;
}

/// Iterator over directory entries. Wraps a boxed iterator for flexibility.
///
/// - Outer `Result` (from `read_dir()`) = "can I open this directory?"
/// - Inner `Result` (per item) = "can I read this entry?"
pub struct ReadDirIter(Box<dyn Iterator<Item = Result<DirEntry, FsError>> + Send + 'static>);

impl Iterator for ReadDirIter {
    type Item = Result<DirEntry, FsError>;
    fn next(&mut self) -> Option<Self::Item> { self.0.next() }
}

impl ReadDirIter {
    pub fn new(iter: impl Iterator<Item = Result<DirEntry, FsError>> + Send + 'static) -> Self {
        Self(Box::new(iter))
    }

    /// Create from a pre-collected vector (useful for middleware like Overlay).
    pub fn from_vec(entries: Vec<Result<DirEntry, FsError>>) -> Self {
        Self(Box::new(entries.into_iter()))
    }

    /// Collect all entries, short-circuiting on first error.
    pub fn collect_all(self) -> Result<Vec<DirEntry>, FsError> {
        self.collect()
    }
}
}

Layer 2: Extended Traits (Optional)

FsLink

#![allow(unused)]
fn main() {
pub trait FsLink: Send + Sync {
    fn symlink(&self, target: &Path, link: &Path) -> Result<(), FsError>;
    fn hard_link(&self, original: &Path, link: &Path) -> Result<(), FsError>;
    fn read_link(&self, path: &Path) -> Result<PathBuf, FsError>;
    fn symlink_metadata(&self, path: &Path) -> Result<Metadata, FsError>;
}
}

FsPermissions

#![allow(unused)]
fn main() {
pub trait FsPermissions: Send + Sync {
    fn set_permissions(&self, path: &Path, perm: Permissions) -> Result<(), FsError>;
}
}

FsSync

#![allow(unused)]
fn main() {
pub trait FsSync: Send + Sync {
    fn sync(&self) -> Result<(), FsError>;
    fn fsync(&self, path: &Path) -> Result<(), FsError>;
}
}

FsStats

#![allow(unused)]
fn main() {
pub trait FsStats: Send + Sync {
    fn statfs(&self) -> Result<StatFs, FsError>;
}
}

FsPath (Optimizable)

Path canonicalization with a default implementation. Backends can override for optimized resolution.

#![allow(unused)]
fn main() {
pub trait FsPath: FsRead + FsLink {
    /// Resolve all symlinks and normalize path (.., .).
    /// Default: iterative resolution via read_link() and symlink_metadata().
    fn canonicalize(&self, path: &Path) -> Result<PathBuf, FsError> {
        // ... default impl ...
    }

    /// Like canonicalize, but allows non-existent final component.
    fn soft_canonicalize(&self, path: &Path) -> Result<PathBuf, FsError> {
        // ... default impl ...
    }
}
impl<T: FsRead + FsLink> FsPath for T {}
}

SelfResolving (Marker)

Marker trait for backends that handle their own path resolution (e.g., VRootFsBackend, StdFsBackend). When using these backends, use NoOpResolver explicitly:

#![allow(unused)]
fn main() {
pub trait SelfResolving {}

// Usage:
let fs = FileStorage::with_resolver(VRootFsBackend::new("/data")?, NoOpResolver);
}

Layer 3: Inode Trait (For FUSE)

FsInode

Required for FUSE mounting (FUSE operates on inodes, not paths). Also enables correct hardlink reporting (same inode = same file, proper nlink count).

Note: FsLink defines hardlink creation (hard_link()). FsInode enables FUSE to track hardlinks via inode identity.

#![allow(unused)]
fn main() {
pub trait FsInode: Send + Sync {
    fn path_to_inode(&self, path: &Path) -> Result<u64, FsError>;
    fn inode_to_path(&self, inode: u64) -> Result<PathBuf, FsError>;
    fn lookup(&self, parent_inode: u64, name: &OsStr) -> Result<u64, FsError>;
    fn metadata_by_inode(&self, inode: u64) -> Result<Metadata, FsError>;
}
}

Layer 4: POSIX Traits (Full POSIX)

POSIX Types

#![allow(unused)]
fn main() {
/// Opaque file handle (inode-based for efficiency)
pub struct Handle(pub u64);

/// File open flags (mirrors POSIX)
#[derive(Clone, Copy, Debug)]
pub struct OpenFlags {
    pub read: bool,
    pub write: bool,
    pub create: bool,
    pub truncate: bool,
    pub append: bool,
}

impl OpenFlags {
    pub const READ: Self = Self { read: true, write: false, create: false, truncate: false, append: false };
    pub const WRITE: Self = Self { read: false, write: true, create: true, truncate: true, append: false };
    pub const READ_WRITE: Self = Self { read: true, write: true, create: false, truncate: false, append: false };
    pub const APPEND: Self = Self { read: false, write: true, create: true, truncate: false, append: true };
}

/// File lock type (mirrors POSIX flock)
#[derive(Clone, Copy, Debug)]
pub enum LockType {
    Shared,     // Multiple readers
    Exclusive,  // Single writer
}
}

FsHandles

#![allow(unused)]
fn main() {
pub trait FsHandles: Send + Sync {
    fn open(&self, path: &Path, flags: OpenFlags) -> Result<Handle, FsError>;
    fn read_at(&self, handle: Handle, buf: &mut [u8], offset: u64) -> Result<usize, FsError>;
    fn write_at(&self, handle: Handle, data: &[u8], offset: u64) -> Result<usize, FsError>;
    fn close(&self, handle: Handle) -> Result<(), FsError>;
}
}

FsLock

#![allow(unused)]
fn main() {
pub trait FsLock: Send + Sync {
    fn lock(&self, handle: Handle, lock: LockType) -> Result<(), FsError>;
    fn try_lock(&self, handle: Handle, lock: LockType) -> Result<bool, FsError>;
    fn unlock(&self, handle: Handle) -> Result<(), FsError>;
}
}

FsXattr

#![allow(unused)]
fn main() {
pub trait FsXattr: Send + Sync {
    fn get_xattr(&self, path: &Path, name: &str) -> Result<Vec<u8>, FsError>;
    fn set_xattr(&self, path: &Path, name: &str, value: &[u8]) -> Result<(), FsError>;
    fn remove_xattr(&self, path: &Path, name: &str) -> Result<(), FsError>;
    fn list_xattr(&self, path: &Path) -> Result<Vec<String>, FsError>;
}
}

Convenience Supertraits

These are automatically implemented via blanket impls:

#![allow(unused)]
fn main() {
/// Basic filesystem - covers 90% of use cases
pub trait Fs: FsRead + FsWrite + FsDir {}
impl<T: FsRead + FsWrite + FsDir> Fs for T {}

/// Full filesystem with all std::fs features
pub trait FsFull: Fs + FsLink + FsPermissions + FsSync + FsStats {}
impl<T: Fs + FsLink + FsPermissions + FsSync + FsStats> FsFull for T {}

/// FUSE-mountable filesystem
pub trait FsFuse: FsFull + FsInode {}
impl<T: FsFull + FsInode> FsFuse for T {}

/// Full POSIX filesystem
pub trait FsPosix: FsFuse + FsHandles + FsLock + FsXattr {}
impl<T: FsFuse + FsHandles + FsLock + FsXattr> FsPosix for T {}
}

When to Use Each Level

Implementing Functions

Use trait bounds to specify requirements:

#![allow(unused)]
fn main() {
use anyfs::FileStorage;

// Works with any backend, keeps std::fs-style paths
fn process_files<B: Fs>(fs: &FileStorage<B>) -> Result<(), FsError> {
    let data = fs.read("/input.txt")?;
    fs.write("/output.txt", &data)?;
    Ok(())
}

// Requires link support
fn create_backup<B: Fs + FsLink>(fs: &FileStorage<B>) -> Result<(), FsError> {
    fs.hard_link("/data.txt", "/data.txt.bak")?;
    Ok(())
}

// Requires FsFuse trait + fuse/winfsp feature
fn mount_filesystem(fs: impl FsFuse) -> Result<(), MountError> {
    anyfs::MountHandle::mount(fs, "/mnt/myfs")?;
    Ok(())
}
}

Extension Trait

FsExt provides convenience methods for any Fs backend:

#![allow(unused)]
fn main() {
pub trait FsExt: Fs {
    /// Check if path is a file.
    fn is_file(&self, path: impl AsRef<Path>) -> Result<bool, FsError>;

    /// Check if path is a directory.
    fn is_dir(&self, path: impl AsRef<Path>) -> Result<bool, FsError>;

    /// JSON methods (require optional `serde` feature in anyfs-backend)
    #[cfg(feature = "serde")]
    fn read_json<T: DeserializeOwned>(&self, path: impl AsRef<Path>) -> Result<T, FsError>;
    #[cfg(feature = "serde")]
    fn write_json<T: Serialize>(&self, path: impl AsRef<Path>, value: &T) -> Result<(), FsError>;
}

// Blanket implementation
impl<B: Fs> FsExt for B {}
}

FileStorage (anyfs)

Ergonomic wrapper for std::fs-aligned API

Overview

FileStorage<B> is a thin wrapper that provides a familiar std::fs-aligned API with:

• B - Backend type (the only generic)
• Built-in path resolution via boxed PathResolver (swappable at runtime)

It is the intended application-facing API: std::fs-style paths with object-safe core traits under the hood.

It does TWO things:

1. Ergonomics (std::fs-aligned API with impl AsRef<Path> convenience)
2. Path resolution for virtual backends (via boxed PathResolver - cold path, boxing is acceptable)

All policy (limits, feature gates, logging) is handled by middleware, not FileStorage.

Why Only One Generic?

Previous designs used FileStorage<B, R, M> with three type parameters. We simplified to FileStorage<B>:

Result: Simpler API for 90% of users. Those who need type-safe markers wrap FileStorage themselves.

Creating a Container

#![allow(unused)]
fn main() {
use anyfs::{MemoryBackend, FileStorage};

// Simple: ergonomics + default path resolution
let fs = FileStorage::new(MemoryBackend::new());
}

With middleware (layer-based):

#![allow(unused)]
fn main() {
use anyfs::{MemoryBackend, QuotaLayer, RestrictionsLayer, FileStorage};

let fs = FileStorage::new(
    MemoryBackend::new()
        .layer(QuotaLayer::builder()
            .max_total_size(100 * 1024 * 1024)
            .build())
        .layer(RestrictionsLayer::builder()
            .deny_permissions()
            .build())
);
}

With custom resolver:

#![allow(unused)]
fn main() {
use anyfs::{MemoryBackend, FileStorage};
use anyfs::resolvers::{CachingResolver, IterativeResolver};

// Custom resolver for read-heavy workloads
let fs = FileStorage::with_resolver(
    MemoryBackend::new(),
    CachingResolver::new(IterativeResolver::default())
);
}

Type-Safe Markers (User-Defined Wrappers)

If you need compile-time safety to prevent mixing filesystems, create wrapper newtypes:

#![allow(unused)]
fn main() {
use anyfs::{MemoryBackend, FileStorage};
use anyfs_sqlite::SqliteBackend;  // Ecosystem crate

// Define your own wrapper types
struct SandboxFs(FileStorage<MemoryBackend>);
struct UserDataFs(FileStorage<SqliteBackend>);

impl SandboxFs {
    fn new() -> Self {
        SandboxFs(FileStorage::new(MemoryBackend::new()))
    }
}

// Type-safe function signatures prevent mixing
fn process_sandbox(fs: &SandboxFs) {
    // Can only accept SandboxFs
}

fn save_user_file(fs: &UserDataFs, name: &str, data: &[u8]) {
    // Can only accept UserDataFs
}

// Compile-time safety:
let sandbox = SandboxFs::new();
process_sandbox(&sandbox);     // OK
// process_sandbox(&userdata); // Compile error! Wrong type
}

When to Use Wrapper Types

std::fs-aligned Methods

FileStorage mirrors std::fs naming:

When the backend implements extended traits (e.g., FsLink, FsInode, FsHandles), FileStorage forwards those methods too and keeps the same impl AsRef<Path> ergonomics for path parameters.

What FileStorage Does NOT Do

FileStorage is not a policy layer. If you need policy, compose middleware.

FileStorage Implementation

#![allow(unused)]
fn main() {
use anyfs_backend::{Fs, PathResolver};
use anyfs::resolvers::IterativeResolver;

/// Ergonomic wrapper with single generic parameter.
pub struct FileStorage<B> {
    backend: B,
    resolver: Box<dyn PathResolver>,  // Boxed: resolution is cold path
}

impl<B: Fs> FileStorage<B> {
    /// Create with default resolver (IterativeResolver).
    pub fn new(backend: B) -> Self {
        FileStorage {
            backend,
            resolver: Box::new(IterativeResolver::new()),
        }
    }

    /// Create with custom resolver.
    pub fn with_resolver(backend: B, resolver: impl PathResolver + 'static) -> Self {
        FileStorage {
            backend,
            resolver: Box::new(resolver),
        }
    }

    /// Type-erase the backend for simpler types (opt-in boxing).
    pub fn boxed(self) -> FileStorage<Box<dyn Fs>> {
        FileStorage {
            backend: Box::new(self.backend),
            resolver: self.resolver,
        }
    }
}
}

Path Resolution

FileStorage handles path resolution for virtual backends via the boxed PathResolver. The default IterativeResolver provides symlink-aware canonicalization.

Backends implementing SelfResolving (like VRootFsBackend) skip resolution since the OS handles it.

Type Erasure (Opt-in)

When you need simpler types (e.g., storing in collections), use .boxed():

#![allow(unused)]
fn main() {
use anyfs::{MemoryBackend, FileStorage, Fs};
use anyfs_sqlite::SqliteBackend;  // Ecosystem crate

// Type-erased for uniform storage
let filesystems: Vec<FileStorage<Box<dyn Fs>>> = vec![
    FileStorage::new(MemoryBackend::new()).boxed(),
    FileStorage::new(SqliteBackend::open("a.db")?).boxed(),
    FileStorage::new(SqliteBackend::open("b.db")?).boxed(),
];
}

When to use .boxed():

Direct Backend Access

If you don’t need the wrapper, use backends directly:

#![allow(unused)]
fn main() {
use anyfs::{MemoryBackend, QuotaLayer, FileStorage};

let backend = MemoryBackend::new()
    .layer(QuotaLayer::builder()
        .max_total_size(100 * 1024 * 1024)
        .build());

// Use FileStorage for std::fs-style paths
let fs = FileStorage::new(backend);
fs.write("/file.txt", b"data")?;
}

FileStorage<B> is part of the anyfs crate, not a separate crate.

Backends Guide

This guide explains each backend available for AnyFS—both built-in (in anyfs) and ecosystem crates (anyfs-sqlite, anyfs-indexed)—how they work internally, when to use them, and the trade-offs involved.

Quick Reference: Which Backend Should You Use?

TL;DR — Pick the first match from top to bottom:

⚠️ Security Warning: StdFsBackend provides NO isolation. Never use with untrusted input.

Ecosystem Crates: Complex backends like SqliteBackend and IndexedBackend live in separate crates (anyfs-sqlite, anyfs-indexed) because they require internal runtime complexity (connection pooling, sharding, chunking).

Backend Categories

AnyFS backends fall into two fundamental categories based on who resolves paths:

Type 1: Virtual Filesystem Backends

These backends store filesystem data in an abstract format (memory, database, etc.). AnyFS handles path resolution via pluggable PathResolver (see ADR-033), including:

• Path traversal (.., .)
• Symlink following (simulated)
• Hard link tracking (simulated)
• Path normalization

Key benefit: Complete isolation from the host OS. Identical behavior across all platforms.

Type 2: Real Filesystem Backends

These backends delegate operations to the actual operating system. The OS handles path resolution, which means:

• Native symlink behavior
• Native permission enforcement
• Platform-specific edge cases
• Potential security considerations (path escapes)

Key benefit: Native performance and compatibility with existing files.

Type 1: Virtual Filesystem Backends

MemoryBackend

An in-memory filesystem. All data lives in RAM and is lost when the process exits.

#![allow(unused)]
fn main() {
use anyfs::{MemoryBackend, FileStorage};

let fs = FileStorage::new(MemoryBackend::new());
fs.write("/data.txt", b"Hello, World!")?;
}

How It Works

• Files and directories stored in a tree structure (HashMap or similar)
• Symlinks stored as data pointing to target paths
• Hard links share the same underlying data node
• All operations are memory-only (no disk I/O)
• Supports snapshots via Clone and persistence via save_to()/load_from()

Performance

Advantages

• Fastest backend - no disk I/O overhead
• Deterministic - perfect for testing
• Portable - works on all platforms including WASM
• Snapshots - Clone creates instant backups
• No cleanup - no temp files to delete

Disadvantages

• Volatile - data lost on process exit (unless serialized)
• Memory-limited - large filesystems consume RAM
• No persistence - must explicitly save/load state

When to Use

✅ USE MemoryBackend when:

• Writing unit tests (fast, isolated, deterministic)
• Writing integration tests (no filesystem pollution)
• Building temporary workspaces or scratch space
• Caching data that fits in memory
• Running in WASM/browser environments (simplest option)
• Need instant snapshots via Clone

❌ DON’T USE MemoryBackend when:

• Storing files larger than available RAM
• Data must survive process restart (use anyfs-sqlite)
• Working with existing files on disk (use VRootFsBackend)

SqliteBackend (Ecosystem Crate)

Crate: anyfs-sqlite

Complex backends live in separate crates. See AGENTS.md “Crate Ecosystem” section.

Stores the entire filesystem in a single SQLite database file.

#![allow(unused)]
fn main() {
use anyfs_sqlite::SqliteBackend;
use anyfs::FileStorage;

let fs = FileStorage::new(SqliteBackend::open("myfs.db")?);
fs.write("/documents/report.txt", b"Annual Report")?;
}

How It Works

• Single .db file contains all files, directories, and metadata
• Schema: nodes table (path, type, content, permissions, timestamps)
• Symlinks stored as rows with target path in content
• Hard links share the same inode (row ID)
• Uses WAL mode for concurrent read access
• Connection pooling: multiple readers, single writer with batching
• Write batching: groups operations into transactions for efficiency
• Transactions ensure atomic operations

Key insight: “Writes are expensive.” SqliteBackend batches writes internally because one transaction per batch is far more efficient than one transaction per operation.

Performance

Large file behavior: SQLite streams BLOB content incrementally via sqlite3_blob_read/write, so files don’t need to fit entirely in RAM. However, very large BLOBs (>100MB) can cause higher memory pressure during I/O operations due to SQLite’s internal buffering and page management. For frequent large file operations, consider IndexedBackend which uses native file I/O.

Performance note: SQLite performance varies significantly based on hardware, configuration, and workload. With proper tuning (WAL mode, connection pooling, write batching), a single SQLite database on modern hardware can achieve high throughput. See sqlite-operations.md for tuning guidance.

Advantages

• Single-file portability - entire filesystem in one .db file
• ACID transactions - atomic operations, crash recovery
• Cross-platform - works on all platforms including WASM
• Complete isolation - no interaction with host filesystem
• Queryable - can inspect with SQLite tools
• Optional encryption - AES-256 via SQLCipher with encryption feature

Disadvantages

• Slower than memory - database overhead on every operation
• Single-writer - SQLite’s write lock limits concurrency
• Large file overhead - very large BLOBs (>100MB) have higher memory pressure due to SQLite buffering

When to Use

✅ USE SqliteBackend when:

• Need portable, single-file storage (easy to copy, backup, share)
• Building embedded/self-contained applications
• Complete isolation from host filesystem is required
• Want encryption (use open_encrypted() with encryption feature)
• Need ACID transactions and crash recovery
• Cross-platform consistency is critical

❌ DON’T USE SqliteBackend when:

• Files must be accessible to external tools (use VRootFsBackend)
• Minimizing memory pressure for very large files is critical (use anyfs-indexed: IndexedBackend)

💡 SqliteBackend vs IndexedBackend: Both provide complete path isolation. Choose SqliteBackend for single-file portability and portable storage. Choose IndexedBackend (anyfs-indexed) for very large files (>100MB) that need native disk streaming performance.

IndexedBackend (Ecosystem Crate)

Crate: anyfs-indexed

Complex backends live in separate crates. See AGENTS.md “Crate Ecosystem” section.

A hybrid backend: virtual paths with disk-based content storage. Paths, directories, symlinks, and metadata are stored in an index database. File content is stored on the real filesystem as opaque blobs.

Key insight: Same isolation model as SqliteBackend, but file content stored externally for native I/O performance with large files.

#![allow(unused)]
fn main() {
use anyfs_indexed::IndexedBackend;
use anyfs::FileStorage;

// Files stored in ./storage/, index in ./storage/index.db
let fs = FileStorage::new(IndexedBackend::open("./storage")?);
fs.write("/documents/report.pdf", &pdf_bytes)?;
// Actually stored as: ./storage/a1b2c3d4-5678-...-1704067200.bin
}

How It Works

Virtual Path                    Real Storage
─────────────────────────────────────────────────────
/documents/report.pdf    →    ./storage/blobs/a1b2c3d4-...-1704067200.bin
/images/photo.jpg        →    ./storage/blobs/b2c3d4e5-...-1704067201.bin
/config.json             →    ./storage/blobs/c3d4e5f6-...-1704067202.bin

index.db contains:
┌─────────────────────────┬──────────────────────────────┬──────────┐
│ virtual_path            │ blob_name                    │ metadata │
├─────────────────────────┼──────────────────────────────┼──────────┤
│ /documents/report.pdf   │ a1b2c3d4-...-1704067200.bin  │ {...}    │
│ /images/photo.jpg       │ b2c3d4e5-...-1704067201.bin  │ {...}    │
└─────────────────────────┴──────────────────────────────┴──────────┘

• Virtual filesystem, real content: Directory structure, paths, symlinks, and metadata are virtual (stored in index.db). Only raw file content lives on disk as opaque blobs.
• Files stored with UUID + timestamp names (flat, meaningless filenames)
• index.db SQLite database maps virtual paths to blob names
• Symlinks and hard links are simulated in the index (not OS symlinks)
• Path resolution handled by AnyFS framework (Type 1 backend)
• File content streamed directly from disk (native I/O performance)

Performance

Index Optimization

The SQLite index benefits from the same performance tuning as SqliteBackend:

Connection pooling: 4-8 reader connections for concurrent index queries; single writer for metadata updates. Blob I/O bypasses SQLite entirely, so the bottleneck is typically blob disk throughput, not index performance.

See anyfs-indexed#9 for detailed performance guidance.

Advantages

• Native file I/O - content stored as raw files, fast streaming
• Large file support - uses OS file I/O, avoids SQLite BLOB buffering overhead
• Complete path isolation - virtual paths, same as SqliteBackend
• Inspectable - can see blob files on disk (though with opaque names)
• Cross-platform - works identically on all platforms

Disadvantages

• Index dependency - losing index.db = losing virtual structure (blobs become orphaned)
• Two-component backup - must copy directory + index.db together
• Content exposure - blob files are readable on disk (paths are hidden, content is not)
• Not single-file portable - unlike SqliteBackend

When to Use

✅ USE IndexedBackend when:

• Storing large files (videos, images, documents >100MB)
• Need native I/O performance for streaming content
• Building media libraries or document management systems
• Want virtual path isolation but with real disk performance
• Files are large but path structure should be sandboxed

❌ DON’T USE IndexedBackend when:

• Need single-file portability (use anyfs-sqlite: SqliteBackend)
• Content must be hidden from host filesystem (use anyfs-sqlite: SqliteBackend with encryption)
• Need WASM/browser support (use MemoryBackend)

🔒 Encryption Tip: If you need large file performance but content confidentiality matters, you can implement an Encryption<B> middleware wrapper to encrypt blob contents at rest. This is a user-defined middleware pattern (not built-in) - see the middleware implementation guide for how to create custom middleware. Alternatively, use SqliteBackend with SQLCipher encryption for simpler encrypted storage.

Type 2: Real Filesystem Backends

StdFsBackend

Direct delegation to std::fs. Every call maps 1:1 to the standard library.

#![allow(unused)]
fn main() {
use anyfs::{StdFsBackend, FileStorage, NoOpResolver};

// SelfResolving backends require explicit NoOpResolver
let fs = FileStorage::with_resolver(StdFsBackend::new(), NoOpResolver);
fs.write("/tmp/data.txt", b"Hello")?; // Actually writes to /tmp/data.txt
}

How It Works

• Every method directly calls the equivalent std::fs function
• Paths passed through unchanged
• OS handles all resolution, symlinks, permissions
• Implements SelfResolving marker (use NoOpResolver to skip virtual resolution)

Performance

Advantages

• Zero overhead - direct OS calls
• Full compatibility - works with all existing files
• Native features - OS permissions, ACLs, xattrs
• Middleware-ready - add Quota, Tracing, etc. to real filesystem

Disadvantages

• No isolation - full filesystem access
• No containment - paths can escape anywhere
• Platform differences - Windows vs Unix behavior
• Security risk - must trust path inputs

When to Use

✅ USE StdFsBackend when:

• Adding middleware (Quota, Tracing, etc.) to real filesystem operations
• Operating in a fully trusted environment with controlled inputs
• Migrating existing code to AnyFS incrementally
• Need full access to host filesystem features (ACLs, xattrs)
• Building tools that work with user’s actual files

❌ DON’T USE StdFsBackend when:

• Handling untrusted path inputs (use VRootFsBackend)
• Any form of sandboxing is required (no containment!)
• Building multi-tenant systems (use virtual backends)
• Security isolation matters at all

⚠️ Security Warning: StdFsBackend provides ZERO isolation. Paths like ../../etc/passwd will work. Only use with fully trusted, controlled inputs.

VRootFsBackend

Sets a directory as a virtual root. All operations are contained within it.

Feature: vrootfs

#![allow(unused)]
fn main() {
use anyfs::{VRootFsBackend, FileStorage, NoOpResolver};

// /home/user/sandbox becomes the virtual "/"
// SelfResolving backends require explicit NoOpResolver
let fs = FileStorage::with_resolver(
    VRootFsBackend::new("/home/user/sandbox")?,
    NoOpResolver
);

fs.write("/data.txt", b"Hello")?; 
// Actually writes to: /home/user/sandbox/data.txt

fs.read("/../../../etc/passwd")?;
// Resolves to: /home/user/sandbox/etc/passwd (clamped!)
}

How It Works

• Configured with a real directory as the “virtual root”
• All paths are validated and clamped to stay within root
• Uses strict-path crate for escape prevention
• Symlinks are followed but targets validated
• Implements SelfResolving marker (OS handles resolution after validation)

Virtual Path          Validation              Real Path
───────────────────────────────────────────────────────────────
/data.txt        →   validate & join    →   /home/user/sandbox/data.txt
/../../../etc    →   clamp to root      →   /home/user/sandbox/etc
/link → /tmp     →   validate target    →   ERROR or clamped

Performance

Advantages

• Path containment - cannot escape virtual root
• Real file access - native OS performance for content
• Symlink safety - targets validated against root
• Drop-in sandboxing - wrap existing directories

Disadvantages

• Performance overhead - validation on every operation
• Extra I/O - symlink following requires lstat calls
• Platform quirks - symlink behavior varies (especially Windows)
• Theoretical edge cases - TOCTOU races exist but are difficult to exploit

When to Use

✅ USE VRootFsBackend when:

• Containing user-uploaded content to a specific directory
• Sandboxing plugins, extensions, or untrusted code
• Need chroot-like isolation without actual chroot privileges
• Building AI agent workspaces with filesystem boundaries
• Want real filesystem performance with path containment

❌ DON’T USE VRootFsBackend when:

• Maximum security required (theoretical TOCTOU edge cases exist - use MemoryBackend)
• Need highest I/O performance (validation adds overhead)
• Cross-platform symlink consistency is critical (Windows differs)
• Want complete isolation from host (use SqliteBackend)

🔒 Encryption Tip: For sensitive data in sandboxed directories (user uploads, plugin workspaces, AI agent data), consider implementing an Encryption<B> middleware wrapper. This is a user-defined middleware pattern - you would create a custom Layer that encrypts data before delegating to the inner backend. See the middleware implementation guide for the pattern.

Composition Middleware

Overlay<Base, Upper>

Union filesystem middleware combining a read-only base with a writable upper layer.

Note: Overlay is middleware (in anyfs/middleware/overlay.rs), not a standalone backend. It composes two backends into a layered view.

#![allow(unused)]
fn main() {
use anyfs::{VRootFsBackend, MemoryBackend, Overlay, FileStorage};

// Base: read-only template
let base = VRootFsBackend::new("/var/templates")?;

// Upper: writable scratch layer  
let upper = MemoryBackend::new();

let fs = FileStorage::new(Overlay::new(base, upper));

// Read: checks upper first, falls back to base
let data = fs.read("/config.txt")?;

// Write: always goes to upper
fs.write("/config.txt", b"modified")?;

// Delete: creates "whiteout" in upper, shadows base
fs.remove_file("/unwanted.txt")?;
}

How It Works

┌─────────────────────────────────────────────────┐
│                  Overlay<B, U>                  │
├─────────────────────────────────────────────────┤
│  Read:   upper.exists(path)?                    │
│            → upper.read(path)                   │
│            : base.read(path)                    │
│                                                 │
│  Write:  upper.write(path, data)                │
│          (base unchanged)                       │
│                                                 │
│  Delete: upper.mark_whiteout(path)              │
│          (shadows base, doesn't delete it)      │
│                                                 │
│  List:   merge(base.read_dir(), upper.read_dir())│
│          - exclude whiteouts                    │
└─────────────────────────────────────────────────┘

         ┌──────────────┐
         │    Upper     │  ← Writes go here
         │ (MemoryFs)   │  ← Modifications stored here
         │              │  ← Whiteouts (deletions) here
         └──────┬───────┘
                │ if not found
                ▼
         ┌──────────────┐
         │     Base     │  ← Read-only layer
         │ (SqliteFs)   │  ← Original/template data
         │              │  ← Never modified
         └──────────────┘

• Reads: Check upper layer first, fall back to base
• Writes: Always go to upper layer (base is read-only)
• Deletes: Create “whiteout” marker in upper (shadows base file)
• Directory listing: Merge both layers, exclude whiteouts

Performance

Advantages

• Copy-on-write semantics - modifications don’t affect base
• Instant rollback - discard upper layer to reset
• Space efficient - only changes stored in upper
• Template pattern - share base across multiple instances
• Testing isolation - test against real data without modifying it

Disadvantages

• Complexity - whiteout handling, merge logic
• Directory listing overhead - must combine and filter
• Two backends to manage - lifecycle of both layers
• Not true CoW - doesn’t deduplicate at block level

When to Use

✅ USE Overlay when:

• Building container-like systems (base image + writable layer)
• Sharing a template filesystem across multiple instances
• Testing against production data without modifying it
• Need instant rollback capability (discard upper layer)
• Implementing git-like branching at filesystem level

❌ DON’T USE Overlay when:

• Simple, single-purpose filesystem (unnecessary complexity)
• Need block-level copy-on-write (Overlay is file-level)
• Directory listing performance is critical (merge overhead)
• Don’t need layered semantics (use single backend)

Backend Selection Guide

Quick Decision Tree

Do you need persistence?
├─ No → MemoryBackend
└─ Yes
   ├─ Single portable file? → SqliteBackend
   ├─ Large files + path isolation? → IndexedBackend
   └─ Access existing files on disk?
      ├─ Need containment? → VRootFsBackend  
      └─ Trusted environment? → StdFsBackend

Comparison Matrix

†Overlay is middleware that composes two backends; characteristics depend on the backends used.

By Use Case

Platform Compatibility