Atomic Zettel: System-Test Distributed Ruby (DRb) Harness and Test Schema Corpus

Controlling Claim

The system-test repository implements an out-of-process distributed orchestration harness using Ruby Distributed Object Protocol (DRb) to control multi-node Vespa cluster deployments, execute fault injection, and validate end-to-end data plane convergence.

Code and Architectural Evidence

1. DRb Endpoint Architecture (system-test/lib/drb_endpoint.rb):
   • Manages inter-process RPC communication between test runner processes and cluster node server instances (system-test/lib/node_server.rb).
   • Handles connection lifecycle, remote process supervision, and command serialization over TLS/TCP.
2. Schema Test Corpus:
   • Contains 807 specialized .sd schema files in system-test/tests/ exercising edge cases in indexing language syntax, grouping operations, tensor attribute configurations, and schema upgrades.
3. Performance Result Model (system-test/lib/performance/resultmodel.rb):
   • Records latency percentiles, throughput counters, and resource utilization metrics, providing empirical verification for cluster performance baselines.

Operational Relevance

System test executions validate invariants that cannot be fully asserted in unit tests, particularly multi-node bucket re-distribution during node failure and live schema migrations.