JuliaDynamics · Datseris · Sep 19, 2022 · Sep 6, 2022 · Sep 6, 2022 · Sep 6, 2022
diff --git a/docs/make.jl b/docs/make.jl
@@ -8,7 +8,7 @@ using DelayEmbeddings
 using Documenter
 using DocumenterTools: Themes
 using Entropies
-using PyPlot
+using CairoMakie
 using DynamicalSystems
 using Wavelets
 
@@ -29,19 +29,20 @@ Themes.compile(joinpath(@__DIR__, "juliadynamics-light.scss"), joinpath(@__DIR__
 Themes.compile(joinpath(@__DIR__, "juliadynamics-dark.scss"), joinpath(@__DIR__, "src/assets/themes/documenter-dark.css"))
 
 # %% Build docs
-PyPlot.ioff()
 cd(@__DIR__)
 ENV["JULIA_DEBUG"] = "Documenter"
 
 PAGES = [
     "Entropies.jl" => "index.md",
-    "Estimators" => "estimators.md",
-    "Remaining" => [
-        "DispersionEntropy.md",
-    ],
+    "Probabilities" => "probabilities.md",
+    "Generalized entropy" => "generalized_entropies.md",
+    "Shannon entropy" => "shannon_entropies.md",
+    "Complexity measures" => "complexity_measures.md",
     "Utility methods" => "utils.md",
 ]
 
+include("style.jl")
+
 makedocs(
     modules = [Entropies],
     format = Documenter.HTML(
@@ -62,5 +63,3 @@ if CI
         push_preview = true
     )
 end
-PyPlot.close("all")
-PyPlot.ion()
diff --git a/docs/src/DispersionEntropy.md b/docs/src/DispersionEntropy.md
diff --git a/docs/src/complexity_measures.md b/docs/src/complexity_measures.md
@@ -0,0 +1,7 @@
+# Complexity measures
+
+## Sample entropy
+
+## Approximate entropy
+
+## Disequilibrium
diff --git a/docs/src/generalized_entropies.md b/docs/src/generalized_entropies.md
@@ -0,0 +1,15 @@
+# [Generalized entropies](@id generalized_entropies)
+
+Computing entropy boils down to one thing: first estimating a probability distribution, then applying one of the generalized entropy formulas ([`entropy_renyi`](@ref) or [`entropy_tsallis`](@ref)) to these distributions. Thus, any of the implemented [probabilities estimators](@ref estimators) can be used to compute generalized entropies.
+
+## Rényi (generalized) entropy
+
+```@docs
+Entropies.entropy_renyi
+```
+
+## Tsallis (generalized) entropy
+
+```@docs
+Entropies.entropy_tsallis
+```
diff --git a/docs/src/index.md b/docs/src/index.md
@@ -2,41 +2,37 @@
 
 This package provides estimators for probabilities, entropies, and complexity measures for timeseries, nonlinear dynamics and complex systems. It is used in the [CausalityTools.jl](https://github.com/JuliaDynamics/CausalityTools.jl) and [DynamicalSystems.jl](https://github.com/JuliaDynamics/DynamicalSystems.jl) packages.
 
-## Input data
+## API & terminology
 
-This package assumes that your data is represented by the `Dataset`-type from [`DelayEmbeddings.jl`](https://github.com/JuliaDynamics/DelayEmbeddings.jl), where each observation is a D-dimensional data point. See the [`DynamicalSystems.jl` documentation](https://juliadynamics.github.io/DynamicalSystems.jl/dev/) for more info. Univariate timeseries given as
-`AbstractVector{<:Real}` also work with some estimators, but are treated differently
-based on which method for probability/entropy estimation is applied.
+In the literature, the term "entropy" is used (and abused) in multiple contexts. We use the following distinctions.
 
-## API
+- [Generalized Rényi and Tsallis entropies](@ref generalized_entropies) are theoretically well-founded concepts that are functions of *probability distributions*.
 
-The main **API** of this package is contained in three functions:
+- [Shannon entropy](@ref shannon_entropies) is a special case of Rényi and Tsallis entropies. We provide convenience functions for most common Shannon entropy estimators.
 
-* [`probabilities`](@ref) which computes probability distributions of given datasets
-* [`genentropy`](@ref) which uses the output of [`probabilities`](@ref), or a set of
-    pre-computed [`Probabilities`](@ref), to calculate entropies.
-* [`tsallisentropy`](@ref) which uses the output of [`probabilities`](@ref), or a set of
-    pre-computed [`Probabilities`](@ref), to calculate Tsallis entropies.
+- [*Probability estimation*](@ref estimators) is a separate but required step to compute entropies. We provide a range of probability estimators. These estimators can be used in isolation for estimating probability distributions, or for computing generalized Rényi and Tsallis entropies.
 
-These functions dispatch estimators listed [here](@ref estimators).
+- "Entropy-like" complexity measures, which strictly speaking don't compute entropies, and may or may not explicitly compute probability distributions, appear in the [Complexity measures](@ref) section.
 
-## Probabilities
+The main **API** of this package is thus contained in three functions:
 
-```@docs
-Probabilities
-probabilities
-probabilities!
-ProbabilitiesEstimator
-```
+- [`probabilities`](@ref), which computes probability distributions of given datasets.
+- [`entropy_renyi`](@ref), which uses the output of [`probabilities`](@ref), or a set of pre-computed [`Probabilities`](@ref), to calculate entropies.
+- [`entropy_tsallis`](@ref), which uses the output of [`probabilities`](@ref), or a set of pre-computed [`Probabilities`](@ref), to calculate Tsallis entropies.
+- Convenience functions for commonly used methods appear throughout the documentation.
 
-## Rényi (generalized) entropy
+These functions dispatch on the probability estimators listed [here](@ref estimators), and are used behind the scenes by many of the [Shannon entropy](@ref shannon_entropy) convenience methods.
 
-```@docs
-Entropies.genentropy
-```
+*Note: there are fewer probability estimators than there are Shannon entropy estimators, because some Shannon entropy estimators are indirect, in the sense that they don't explicitly compute probability distributions*.
+
+## Input data
+
+### Temporal (1D) data
+
+In this package, [probability](@ref estimators), [generalized entropy](@ref generalized_entropy) and [Shannon entropy](@ref shannon_entropy) estimators assume that temporal data is represented by the `Dataset`-type from [`DelayEmbeddings.jl`](https://github.com/JuliaDynamics/DelayEmbeddings.jl), where each observation is a D-dimensional data point. See the [`DynamicalSystems.jl` documentation](https://juliadynamics.github.io/DynamicalSystems.jl/dev/) for more info. Univariate timeseries given as
+`AbstractVector{<:Real}` also work with some estimators, but are treated differently
+based on which method for probability/entropy estimation is applied.
 
-## Tsallis entropy
+### Spatiotemporal (2D and higher) data
 
-```@docs
-Entropies.tsallisentropy
-```
+The [`SpatialSymbolicPermutation`](@ref) probability estimator handles probability and entropy computations for arbitrary -dimensional data (e.g. 2D images, 3D images). These data should be provided as `AbstractArray{T, N}` where `N` is the dimension of the data.
diff --git a/docs/src/probabilities.md b/docs/src/probabilities.md
@@ -0,0 +1,86 @@
+# [Probabilities](@id estimators)
+
+For categorical or integer-valued data, probabilities can be estimated by directly counting relative frequencies of data elements. For such data, use `probabilities(x::Array_or_Dataset) → p::Probabilities`.
+
+More advanced estimators computing probabilities by first either discretizing, symbolizing or transforming the data in a way that quantifies some useful properties about the underlying data (e.g. visitation frequencies, wavelet energies, or permutation patterns), from which probability distributions can be estimated. Use `probabilities(x::Array_or_Dataset, est::ProbabilitiesEstimator)` in combination with any of the estimators listed below.
+
+```@docs
+Probabilities
+probabilities
+probabilities!
+ProbabilitiesEstimator
+```
+
+## Estimators
+
+### Count occurrences (counting)
+
+```@docs
+CountOccurrences
+```
+
+### Permutation (symbolic)
+
+```@docs
+SymbolicPermutation
+SpatialSymbolicPermutation
+```
+
+### Visitation frequency (binning)
+
+```@docs
+VisitationFrequency
+```
+
+#### Specifying binning/boxes
+
+```@docs
+RectangularBinning
+```
+
+### Transfer operator (binning)
+
+```@docs
+TransferOperator
+```
+
+#### Utility methods/types
+
+```@docs
+InvariantMeasure
+invariantmeasure
+transfermatrix
+```
+
+### Kernel density
+
+```@docs
+NaiveKernel
+```
+
+#### Example
+
+Here, we draw some random points from a 2D normal distribution. Then, we use kernel 
+density estimation to associate a probability to each point `p`, measured by how many 
+points are within radius `1.5` of `p`. Plotting the actual points, along with their 
+associated probabilities estimated by the KDE procedure, we get the following surface 
+plot.
+
+```@example MAIN
+using DynamicalSystems, CairoMakie, Distributions
+𝒩 = MvNormal([1, -4], 2)
+N = 500
+D = Dataset(sort([rand(𝒩) for i = 1:N]))
+x, y = columns(D)
+p = probabilities(D, NaiveKernel(1.5))
+fig, ax = surface(x, y, p.p; axis=(type=Axis3,))
+ax.zlabel = "P"
+ax.zticklabelsvisible = false
+fig
+```
+
+### Wavelet
+
+```@docs
+WaveletOverlap
+```