Domain Guide — Document Feeding, Storage, and Operations

Document writes (put, update, remove) flow from high-throughput HTTP/2 clients through
container-jdisc document processing chains into the elastic C++ storage layer in content-nodes.

End-to-End Write Path

1. Ingress and Document/v1 (vespa/vespaclient-container-plugin, vespa/vespa-feed-client):
   • Clients stream JSON document operations over HTTP/2 to /document/v1/ handled by jDisc.
2. Document Processing (vespa/docproc, vespa/docprocs):
   • Operations traverse chainable Java DocumentProcessor components (including custom enrichers,
     embedders, and the IndexingProcessor executing vespa/indexinglanguage scripts; see
     schemas-app-packages).
3. MessageBus and Distributor Routing (vespa/messagebus, vespa/documentapi, vespa/storage):
   • Documents are hashed into buckets and routed via vespa/messagebus to the responsible C++
     distributor node (vespa/storage/src/vespa/storage/distributor/).
4. Persistence and Proton Indexing (vespa/searchcore):
   • The distributor replicates writes to content node persistence units (proton) according to
     the cluster state maintained by vespa/clustercontroller-core.

Operational Controls

• Reindexing and Visiting: Background visiting iterates bucket ranges for reindexing or dump
  operations (vespa/storage/src/vespa/storage/visiting/).
• Cluster State Convergence: vespa/clustercontroller-core arbitrates node up/down/maintenance
  transitions via ZooKeeper and triggers automatic bucket redistribution.