Circuit vs. Consul

Circuit and Consul look largely the same except for one semantic difference.

Similarities

• Both are truly distributed (no host is distinguishably more important to the circuit than any other)
• Both support host membership and health information in real-time
• Both afford a simple configuration-less start command (for hardware provisioning engineers)
• Both afford communication and synchronization primitives, which are abstractly attached to a globally-visible programmable namespace.

Differences

Terminology equation: Circuit elements and their paths are Consul values and their keys. I'll use the key/value language below.

• Circuit uniformly treats processes, channels, data, etc. as values of different types -- i.e. with different sets of methods -- but placed in the same global namespace. This means that Circuit supports programmability (for the operations engineers) of any host via the local server daemon on any other host. In Consul there's no programmability of any other hosts in the sense that no processes can be triggered in another host. Even though there's a distributed storage where values can be put, this is only useful for fixed information, namely strings, and no other types like channels or processes exist. Globally speaking, in a Consul cluster you can register new services (typically a process running on a host accessible through the network) and modify keys in the distributed data storage, that's as far as the global API reaches. Consul doesn't include something like a RPC API or event triggers resembling Circuit's processes.

• Every key/value pair is stored and maintained by exactly one circuit server. This is a far-reaching difference:

  • Circuit exposes the physical placement (which circuit server maintains the object) of a value as part of its key name (it is the first directory on the path from / to the key).

  • While both Circuit and Consul have a consistent views of the entire namespace, as seen from any one server, the circuit does not have to worry about replication of values, as it is the user's responsibility to make placement choices. This means that even though both systems are strongly consistent, Circuit values are only as durable as the host where they are placed. On Consul, committing a new value guarantees durability as well through multi-node replication.

  • As a result, the circuit networking, health, discovery and consistency protocols are much simpler than Consul's, while just as robust in preserving consistency during high host churn.

More technical aspects

• It seems that because the key/value pairs' placement is not exposed to the Consul user (and therefore Consul is responsible for its replication), Consul requires more complex/heterogenous protocols for the one- vs multi-datacenter case.

• The circuit handles one- and multi- datacenter scenarios out-of-the-box and with no different treatment (i.e. deployment configuration) required.

• From the perspective of a developer, Circuit exposes more low level primitives that enable distributed execution of unix processes and the global namespace (even though similar to a key value storage) is used as a representation of the "state" of those primitives and not as an arbitrary data storage. Consul on the other side, exposes a global key-value storage that is distributed and is intended, in practice, to save configuration data. Besides this data storage, it exposes a DNS interface for service discovery that works more like a cluster directory to simplify coordination of running services in specific hosts rather than Circuit's capability of distributing processes in existing hosts. To put it simply, Consul reflects a full fledged product and Circuit can be used as a library if desired.