Summary
Slobrok is Vespa’s Service Location Broker, a small naming service: processes register name plus connection-spec pairs, brokers mirror the service map among themselves, and clients keep a local mirror for fast pattern lookups. Filedistribution moves the payloads behind FileReference tokens (created by the config system when configs reference bundled files or URLs) from config servers to nodes over jrt RPC; fileacquirer is the component-facing API that blocks until a referenced file is available on local disk. Consumers include content nodes; see content-nodes.
Key mechanisms
- Registration: slobrok/README is “Service Location Broker (a simple naming service)”. The client-side RegisterAPI queues register/unregister operations in todo lists and performs them asynchronously with a BackOff against the current slobrok target (slobrok/src/vespa/slobrok/sbregister.h:25-56); wire methods are slobrok.registerRpcServer and slobrok.unregisterRpcServer (sbregister.cpp:221, 213). The broker re-verifies ownership by calling back slobrok.callback.listNamesServed and can revoke via slobrok.callback.notifyUnregistered (sbregister.cpp:263-268).
- Server map: ServiceMapMirror holds the name->spec map with a vespalib::GenCnt generation and supports MapListener incremental updates (slobrok/src/vespa/slobrok/server/service_map_mirror.h:17-40). Point lookup is jrt method slobrok.lookupRpcServer (server/rpchooks.cpp:207); mirror clients pull incremental diffs with slobrok.incremental.fetch (rpchooks.cpp:194; sbmirror.cpp:268). IMirrorAPI::lookup(pattern) returns (name, “tcp/host:port”) pairs, where ’*’ matches within one ’/‘-delimited segment, and updates() is a wrap-around counter used to detect any change cheaply (imirrorapi.h:29-56). MirrorAPI maintains that local mirror in the background (sbmirror.h:26).
- Broker mesh and liveness: ExchangeManager tracks RemoteSlobrok partner objects and operates on all remote brokers in parallel (server/exchange_manager.h:22-28); peers exchange state via slobrok.internal.fetchLocalView (remote_slobrok.cpp:61) and manage membership via slobrok.admin.addPeer / removePeer (rpchooks.cpp:159-165; remote_slobrok.cpp:189). Registered services are health-monitored: RpcMappingMonitor creates a ManagedRpcServer per mapping that polls the service’s slobrok.callback.listNamesServed (managed_rpc_server.cpp:36) and reports up() / down() through MappingMonitorOwner (mapping_monitor.h:12-27).
- File distribution protocol: filedistribution/README says it is used by config proxy and config server to distribute files from config servers to nodes “using RPC”. A FileReference is an immutable record rejecting control characters and leading ’..’ segments (config-lib/src/main/java/com/yahoo/config/FileReference.java:12-25); payloads live on the config server’s disk in its file references directory (see GDN-NOTE-0037). Transfer is server-pushed back over the requesting connection: filedistribution.receiveFileMeta / receiveFilePart / receiveFileEof (filedistribution/src/main/java/com/yahoo/vespa/filedistribution/FileReceiver.java:42-44, method table 170-188). Content carries Type file|compressed, CompressionType gzip|lz4|none|zstd, and an xxhash64 checksum (FileReferenceData.java:15-16, 57). FileDownloader writes into $VESPA_HOME/var/db/vespa/filedistribution by default, with env-tunable backoff (FileDownloader.java:32-33, 47-53).
- Acquisition: components call FileAcquirer.waitFor(FileReference, timeout) (fileacquirer/src/main/java/com/yahoo/filedistribution/fileacquirer/FileAcquirer.java:13-23); the implementation asks the local config proxy on localhost:19090 (FileAcquirerImpl.java:55-58) and maps custom error codes for fileReferenceNotFound and fileReferencePermissionDenied (lines 42-45).
Open questions
- Peer-consistency window: a slobrok.missing.consensus metric exists (server/metrics_producer.cpp:90) but convergence timing is not pinned down here.
- Whether the config proxy relays file downloads or clients always hit config servers (config-proxy’s filedistribution package not read in this pass).
- URL-based downloads (UrlDownloader) unexamined.