Merge pull request #162 from biaslab/rx-compare-docs

Add comparison to other packages

docs/make.jl

@@ -111,6 +111,7 @@ makedocs(;
         "User guide" => [
             # "Background: variational inference" => "manuals/background.md",
             "Getting started"           => "manuals/getting-started.md",
+            "RxInfer.jl vs. Others"     => "manuals/comparison.md",
             "Model specification"       => "manuals/model-specification.md",
             "Constraints specification" => "manuals/constraints-specification.md",
             "Meta specification"        => "manuals/meta-specification.md",

docs/src/index.md

@@ -57,6 +57,8 @@ taking advantage of local conjugate relationships in probabilistic models, and f
 various Bayesian inference algorithms in different parts of the factor graph of a single probabilistic model. This makes it easier
 to explore different "what-if" scenarios and enables very efficient inference in specific cases.
 
+**Curious about how RxInfer compares to other tools you might be considering?** We invite you to view a detailed [comparison](@ref user-guide-debugging), where we put RxInfer head-to-head with other popular packages in the field.
+
 ## How to get started?
 
 Head to the [Getting started](@ref user-guide-getting-started) section to get up and running with RxInfer. Alternatively, explore various [examples](@ref examples-overview) in the documentation.
@@ -66,6 +68,7 @@ Head to the [Getting started](@ref user-guide-getting-started) section to get up
 ```@contents
 Pages = [
   "manuals/background.md",
+  "manuals/comparison.md",
   "manuals/getting-started.md",
   "manuals/model-specification.md",
   "manuals/constraints-specification.md",
@@ -103,3 +106,4 @@ The `RxInfer` unites 3 core packages into one powerful reactive message passing-
 
 ```@index
 ```
+

docs/src/manuals/comparison.md

@@ -0,0 +1,87 @@
+# [Comparison to other packages](@id comparison)
+
+Nowadays there's plenty of probabilistic programming languages and packages available. While all of them are based on Bayesian Inference, their methodologies vary. This section compares `RxInfer.jl` against other renowned probabilistic programming languages and packages. The goal is to enlighten potential users about the nuances and guide them in choosing the package that best suits their requirements.
+
+**DISCLAIMER**: 
+1. This comparison isn't exhaustive and mirrors the author's hands-on experience with the packages. Some might have undergone more rigorous testing than others. If you're an author of one of these packages and believe the comparison doesn't do justice, please reach out, and we'll be more than willing to rectify.
+2. The comparison is more qualitative than quantitative, considering the intricacies of upkeeping benchmarking code for perpetually evolving packages.
+
+
+
+| Toolbox                                                              | Universality | Efficiency | Expressiveness | Debugging & Visualization | Modularity | Inference Engine | Language | Community & Ecosystem |
+| -------------------------------------------------------------------- | ------------ | ---------- | -------------- | ------------------------- | ---------- | ---------------- | -------- | --------------------- |
+| [**RxInfer.jl**](https://rxinfer.ml/)                                | ~            | ✓          | ✓              | ~                         | ✗          | Message-passing  | Julia    | ✗                     |
+| [**ForneyLab.jl**](https://github.com/biaslab/ForneyLab.jl)          | ✗            | ~          | ✗              | ~                         | ✗          | Message-passing  | Julia    | ✗                     |
+| [**Infer.net**](https://dotnet.github.io/infer/)                     | ~            | ✓          | ✗              | ✓                         | ✗          | Message-passing  | C#       | ✗                     |
+| [**PGMax**](https://github.com/google-deepmind/PGMax)                | ✗            | ✓          | ✗              | ✓                         | ✗          | Message-passing  | Python   | ✗                     |
+| [**Turing.jl**](https://turing.ml/)                                  | ✓            | ✗          | ✓              | ~                         | ✗          | Sampling         | Julia    | ✓                     |
+| [**PyMC**](https://www.pymc.io/welcome.html)                         | ✓            | ✗          | ✓              | ✓                         | ✗          | Sampling         | Python   | ✓                     |
+| [**NumPyro**](https://num.pyro.ai/en/stable/)                        | ✓            | ✓          | ~              | ✓                         | ✗          | Sampling         | Python   | ✓                     |
+| [**TensorFlow Probability**](https://www.tensorflow.org/probability) | ✓            | ✗          | ~              | ✓                         | ✗          | Sampling         | Python   | ✓                     |
+| [**Stan**](https://mc-stan.org/)                                     | ✓            | ✗          | ✓              | ✓                         | ✗          | Sampling         | Stan     | ✓                     |
+(Date of creation: 20/10/2023)
+
+**Legend**
+- `✓` : Full capability or feature is present.
+- `~` : Partial capability or feature is present.
+- `✗` : No capability or feature.
+
+**Notes**:
+- **Universality**: Denotes the capability to depict a vast array of probabilistic models.
+- **Efficiency**: Highlights computational competence. A "–" in this context suggests perceived slowness.
+- **Expressiveness**: Assesses the ability to concisely formulate intricate probabilistic models.
+- **Debugging & Visualization**: Evaluates the suite of tools for model debugging and visualization.
+- **Modularity**: Reflects the potential to create models by integrating smaller models.
+- **Inference Engines**: Pinpoints the primary inference strategy employed by the toolbox.
+- **Language**: Identifies the programming language integral to the toolbox.
+- **Community & Ecosystem**: Signifies the vibrancy of the ecosystem, inclusive of tools, libraries, and community backing.
+---
+
+# RxInfer.jl breakdown
+
+- **Universality**: `RxInfer.jl` shines in formulating intricate models derived from the exponential family distributions. The package encompasses not only commonly used distributions such as Gaussian or Bernoulli, but also specialized stochastic nodes that represents prevalent probabilistic models like [Autoregressive models](@ref ARmodel), [Gamma Mixture models](@ref GammaMixturemodel), among others. Furthermore, `RxInfer.jl` proficiently manages deterministic transformations of variables from the exponential family, see [Delta node](@ref delta-node-manual). Nevertheless, for models outside the exponential family, `RxInfer.jl` might not be the good choice. Such models would require the creation of novel nodes and corresponding rules, as illustrated [in](@id create-node).
+
+- **Efficiency**: `RxInfer.jl` distinguishes itself with its inference engine rooted in reactive message passing. This approach is supremely efficient, facilitating real-time propagation of updates across the system, supporting parallelization, interruptibility, and more. However, the current version of `RxInfer.jl` hasn't harnessed all these potentials.
+
+- **Modularity**: Broadly, the toolboxes in the table aren't modular in the truest sense. They don't offer the fusion of models by integrating smaller models. While `RxInfer.jl` currently doesn't support this, a solution is on the horizon:
+
+```julia
+@model function inner_inner(τ, y)
+    y ~ Normal(τ[1], τ[2])
+end
+
+@model function inner(θ, α)
+    β ~ Normal(0, 1)
+    α ~ Gamma(β, 1)
+    α ~ inner_inner(τ = θ)
+end
+
+@model function outer()
+    local w
+    for i = 1:5
+        w[i] ~ inner(θ = Gamma(1, 1))
+    end
+    y ~ inner(θ = w[2:3])
+end
+```
+
+- **Expressiveness**: `RxInfer.jl` empowers users to elegantly and concisely craft models, closely mirroring probabilistic notation, thanks to Julia's macro capabilities. To illustrate this, let's consider the following model:
+
+$$\begin{aligned}
+ x & \sim \mathrm{Normal}(0.0, 1.0)\\
+ w & \sim \mathrm{InverseGamma}(1.0, 1.0)\\
+ y & \sim \mathrm{Normal}(x, w)
+\end{aligned}$$
+
+The model then is expressed in `RxInfer.jl` as follows:
+```@example comparisondoc
+using RxInfer
+
+@model function example_model()
+    x ~ Normal(mean=0.0, var=1.0)
+    w ~ InverseGamma(α = 1, θ = 1)
+    y ~ Normal(mean = x, var = w)
+end
+```
+
+- **Debugging & Visualization**: While `RxInfer.jl` struggles with Julia's early-stage debugging system, it does provide a mechanism to debug the inference [procedure](@ref user-guide-debugging), even though not as seamlessly as some other packages.

docs/src/manuals/custom-node.md

@@ -1,6 +1,4 @@
-
-# Creating your own custom nodes
-
+# [Creating your own custom nodes](@id create-node)
 
 Welcome to the `RxInfer` documentation on creating custom factor graph nodes. In `RxInfer`, factor nodes represent functional relationships between variables, also known as factors. Together, these factors define your probabilistic model. Quite often these factors represent distributions, denoting how a certain parameter affects another. However, other factors are also possible, such as ones specifying linear or non-linear relationships. `RxInfer` already supports a lot of factor nodes, however, depending on the problem that you are trying to solve, you may need to create a custom node that better fits the specific requirements of your model. This tutorial will guide you through the process of defining a custom node in `RxInfer`, step by step. By the end of this tutorial, you will be able to create your own custom node and integrate it into your model.
 

docs/src/manuals/delta-node.md

@@ -1,4 +1,4 @@
-# Delta node manual
+# [Delta node manual](@id delta-node-manual)
 
 RxInfer.jl offers a comprehensive set of stochastic nodes, with a primary emphasis on distributions from the exponential family and its associated compositions, such as Gaussian with controlled variance (GCV) or autoregressive (AR) nodes. The `DeltaNode` stands out in this package, representing a deterministic transformation of either a single random variable or a group of them. This guide provides insights into the `DeltaNode` and its functionalities.