MOC — Content Nodes (C++ data plane)

Content nodes store all Vespa data, maintain reverse and forward indexes, and perform the
distributed parts of query execution: matching, ranking, grouping/aggregation. Mostly C++
(cluster controller is Java). (Description from vespa/Code-map.md; sizes from first-hand
find/wc inspection.)

Modules

  • vespa/searchcore — index maintenance, matching, data storage, grouping, and the content
    node server “proton” (1210 files, ~131k LOC).
  • vespa/searchlib — libraries invoked by searchcore: ranking (fef, rank features,
    expressions), index and btree implementations, attributes (forward indexes), plus the Java
    ranking libraries (2492 files, ~305k LOC — largest module in the codebase by LOC).
  • vespa/storage/src/vespa/storage — elastic, auto-recovering data storage over node clusters
    (716 files, ~108k LOC). Subdirs observed: distributor, persistence, storageserver,
    bucketdb, visiting, common, config, frameworkimpl, storageutil, subscriptions.
  • vespa/eval — efficient evaluation of ranking expressions; tensor API and implementation
    (352 files, ~50k LOC).
  • vespa/storageapi — message bus messages/implementation for the document API.
  • vespa/clustercontroller-core — Java cluster controller for storage; singular node-level
    decisions via ZooKeeper (215 files, ~31k LOC). Related unmapped siblings observed:
    clustercontroller-apps, clustercontroller-reindexer, clustercontroller-utils.

Anchor files

  • vespa/searchcore/src/vespa/searchcore/proton/ — the proton server (observed subdir).
  • vespa/searchcore/src/vespa/searchcore/grouping/ — distributed grouping.
  • vespa/searchlib/src/ — index, attribute, ranking libraries.
  • vespa/storage/src/vespa/storage/distributor/ — bucket distribution logic.
  • vespa/clustercontroller-core/src/ — cluster state decisions.

Open questions

  • Exact proton entry class/file (vespa/searchcore/src/vespa/searchcore/proton/server/?) —
    human to confirm.
  • Why is clustercontroller-core grouped under content nodes rather than config-admin?
  • Do vespa/vespa-ann-benchmark/vespa-search (workspace siblings outside vespa/) relate
    here?