Doc: faq: ship faq from release-0.9; remove 0.9 feature flag loosen-f…

…ollower-log-revert

openraft/src/docs/faq/faq.md

@@ -1,8 +1,101 @@
 # FAQ
 
--   Q: 🤔 Why is log id `(term, node_id, log_index)`, while standard Raft uses just
-    `(term, log_index)`?
+### What are the differences between Openraft and standard Raft?
 
-    A: The log id `(term, node_id, log_index)` is used to minimize the chance of election conflicts.
-    This way in every term there could be more than one leaders elected, and the last one is valid.
-    See: [`leader-id`](`crate::docs::data::leader_id`) for details.
+- Optionally, In one term there could be more than one leaders to be established, in order to reduce election conflict. See: std mode and adv mode leader id: [`leader_id`][];
+- Openraft stores committed log id: See: [`RaftLogStorage::save_committed()`][];
+- Openraft optimized `ReadIndex`: no `blank log` check: [`Linearizable Read`][].
+- A restarted Leader will stay in Leader state if possible;
+- Does not support single step memebership change. Only joint is supported.
+
+
+### Why is log id a tuple of `(term, node_id, log_index)`?
+
+In standard Raft log id is `(term, log_index)`, in Openraft he log id `(term,
+node_id, log_index)` is used to minimize the chance of election conflicts.
+This way in every term there could be more than one leaders elected, and the last one is valid.
+See: [`leader-id`](`crate::docs::data::leader_id`) for details.
+
+
+### How to initialize a cluster?
+
+There are two ways to initialize a raft cluster, assuming there are three nodes, `n1, n2, n3`:
+
+1. Single-step method: Call `Raft::initialize()` on any one of the nodes with the configuration of all 3 nodes, e.g. `n2.initialize(btreeset! {1,2,3})`.
+2. Incremental method: First, call `Raft::initialize()` on `n1` with configuraion containing `n1` itself, e.g., `n0.initialize(btreeset! {1})`. Subsequently use `Raft::change_membership()` on `n1` to add `n2` and `n3` into the cluster.
+
+Employing the second method provides the flexibility to start with a single-node cluster for testing purposes and subsequently expand it to a three-node cluster for deployment in a production environment.
+
+
+### Are there any issues with running a single node service?
+
+Not at all.
+
+Running a cluster with just one node is a standard approach for testing or as an initial step in setting up a cluster.
+
+A single node functions exactly the same as cluster mode.
+It will consistently maintain the `Leader` status and never transition to `Candidate` or `Follower` states.
+
+
+### How to remove node-2 safely from a cluster `{1, 2, 3}`?
+
+Call `Raft::change_membership(btreeset!{1, 3})` to exclude node-2 from
+the cluster. Then wipe out node-2 data.
+**NEVER** modify/erase the data of any node that is still in a raft cluster, unless you know what you are doing.
+
+
+### What actions are required when a node restarts?
+
+None. No calls, e.g., to either [`add_learner()`][] or [`change_membership()`][]
+are necessary.
+
+Openraft maintains the membership configuration in [`Membership`][] for for all
+nodes in the cluster, including voters and non-voters (learners).  When a
+`follower` or `learner` restarts, the leader will automatically re-establish
+replication.
+
+
+### What will happen when data gets lost?
+
+Raft operates on the presumption that the storage medium (i.e., the disk) is
+secure and reliable.
+
+If this presumption is violated, e.g., the raft logs are lost or the snapshot is
+damaged, no predictable outcome can be assured. In other words, the resulting
+behavior is **undefined**.
+
+
+### Is Openraft resilient to incorrectly configured clusters?
+
+No, Openraft, like standard raft, cannot identify errors in cluster configuration.
+
+A common error is the assigning a wrong network addresses to a node. In such
+a scenario, if this node becomes the leader, it will attempt to replicate
+logs to itself. This will cause Openraft to panic because replication
+messages can only be received by a follower.
+
+```text
+thread 'main' panicked at openraft/src/engine/engine_impl.rs:793:9:
+assertion failed: self.internal_server_state.is_following()
+```
+
+```ignore
+// openraft/src/engine/engine_impl.rs:793
+pub(crate) fn following_handler(&mut self) -> FollowingHandler<C> {
+    debug_assert!(self.internal_server_state.is_following());
+    // ...
+}
+```
+
+
+[`single-term-leader`]:         `crate::docs::feature_flags#single_term_leader`
+
+[`Linearizable Read`]: `crate::docs::protocol::read`
+[`leader_id`]:         `crate::docs::data::leader_id`
+
+[`RaftLogStorage::save_committed()`]: `crate::storage::RaftLogStorage::save_committed`
+
+[`add_learner()`]: `crate::Raft::add_learner`
+[`change_membership()`]: `crate::Raft::change_membership`
+
+[`Membership`]: `crate::Membership`

openraft/src/docs/feature_flags/feature-flags.md

@@ -12,21 +12,15 @@ By default openraft enables no features.
   attaches backtrace to generated errors. This feature works ONLY with nightly rust, because it requires unstable feature `error_generic_member_access`.
   <br/><br/>
 
-- `loosen-follower-log-revert`:
-  Permit the follower's log to roll back to an earlier state without causing the leader to panic.
-  Although log state reversion is typically seen as a bug, enabling it can be useful for testing or other special scenarios.
-  For instance, in an even number nodes cluster, erasing a node's data and then rebooting it(log reverts to empty) will not result in data loss.
-
-  **Do not use it unless you know what you are doing**.
-  <br/><br/>
-
 - `serde`: derives `serde::Serialize, serde::Deserialize` for type that are used
   in storage and network, such as `Vote` or `AppendEntriesRequest`.
+  <br/><br/>
 
 - `single-term-leader`: allows only one leader to be elected in each `term`.
   This is the standard raft policy, which increases election confliction rate
   but reduce `LogId`(`(term, node_id, index)` to `(term, index)`) size.
   Read more about how it is implemented in [`vote`](./vote.md)
+  <br/><br/>
 
 - `compat-07`: provides additional data types to build v0.7 compatible RaftStorage.
 
@@ -38,5 +32,4 @@ By default openraft enables no features.
 - `storage-v2`: enables `RaftLogStorage` and `RaftStateMachine` as the v2 storage
   This is a temporary feature flag, and will be removed in the future, when v2 storage is stable.
   This feature disables `Adapter`, which is for v1 storage to be used as v2.
-  V2 storage separates log store and state machine store so that log IO and state machine IO can be parallelized naturally. 
-
+  V2 storage separates log store and state machine store so that log IO and state machine IO can be parallelized naturally.