Merge pull request #87 from yuehhua/doc

Add doc for GAE and VGAE

Author: yuehhua

docs/make.jl

@@ -6,6 +6,13 @@ makedocs(
     format = Documenter.HTML(),
     modules = [GeometricFlux],
     pages = ["Home" => "index.md",
+             "Get started" => "start.md",
+             "Basics" =>
+               ["Building layers" => "basics/layers.md",
+                "Graph passing" => "basics/passgraph.md"],
+             "Abstractions" =>
+               ["Message passing scheme" => "abstractions/msgpass.md",
+                "Graph network block" => "abstractions/gn.md"],
              "Manual" =>
                ["Convolutional Layers" => "manual/conv.md",
                 "Pooling Layers" => "manual/pool.md",

docs/src/abstractions/gn.md

@@ -0,0 +1 @@
+# Graph network block

docs/src/abstractions/msgpass.md

@@ -0,0 +1 @@
+# Message passing scheme

docs/src/basics/layers.md

@@ -0,0 +1 @@
+# Building layers

docs/src/basics/passgraph.md

@@ -0,0 +1,9 @@
+# Graph passing
+
+Graph is an input data structure for graph neural network. Passing graph and input feature into 
+
+## Static graph
+
+## Variable graph
+
+## Cached graph

docs/src/index.md

@@ -1,6 +1,12 @@
 # GeometricFlux: The Geometric Deep Learning Library in Julia
 
-Documentation for GeometricFlux.jl
+GeometricFlux is a framework for geometric deep learning/machine learning.
+
+* It extends Flux machine learning library for geometric deep learning.
+* It supports of CUDA GPU with CUDA.jl
+* It integrates with JuliaGraphs ecosystems.
+* It supports generic graph neural network architectures (i.g. message passing scheme and graph network block)
+* It contains built-in GNN benchmark datasets (WIP)
 
 ## Installation
 
@@ -9,3 +15,99 @@ Documentation for GeometricFlux.jl
 ```
 
 ## Quick start
+
+The basic graph convolutional network (GCN) is constructed as follow.
+
+```
+model = Chain(GCNConv(adj_mat, num_features=>hidden, relu),
+              GCNConv(adj_mat, hidden=>target_catg),
+              softmax)
+```
+
+### Load dataset
+
+Load cora dataset from GeometricFlux.
+
+```
+using JLD2
+using SparseArrays
+
+@load joinpath(pkgdir(GeometricFlux), "data/cora_features.jld2") features
+@load joinpath(pkgdir(GeometricFlux), "data/cora_labels.jld2") labels
+@load joinpath(pkgdir(GeometricFlux), "data/cora_graph.jld2") g
+```
+
+### Training/testing data
+
+Data is stored in sparse array, thus, we have to convert it into normal array.
+
+```
+train_X = Matrix{Float32}(features)
+train_y = Matrix{Float32}(labels)
+adj_mat = Matrix{Float32}(adjacency_matrix(g))
+```
+
+### Loss function
+
+```
+loss(x, y) = logitcrossentropy(model(x), y)
+accuracy(x, y) = mean(onecold(model(x)) .== onecold(y))
+```
+
+### Training
+
+```
+ps = Flux.params(model)
+train_data = [(train_X, train_y)]
+opt = ADAM()
+evalcb() = @show(accuracy(train_X, train_y))
+
+@epochs epochs Flux.train!(loss, ps, train_data, opt, cb=throttle(evalcb, 10))
+```
+
+### Logs
+
+```
+[ Info: Epoch 1
+accuracy(train_X, train_y) = 0.11669128508124077
+[ Info: Epoch 2
+accuracy(train_X, train_y) = 0.19608567208271788
+[ Info: Epoch 3
+accuracy(train_X, train_y) = 0.3098227474150665
+[ Info: Epoch 4
+accuracy(train_X, train_y) = 0.387370753323486
+[ Info: Epoch 5
+accuracy(train_X, train_y) = 0.44645494830132937
+[ Info: Epoch 6
+accuracy(train_X, train_y) = 0.46824224519940916
+[ Info: Epoch 7
+accuracy(train_X, train_y) = 0.48892171344165436
+[ Info: Epoch 8
+accuracy(train_X, train_y) = 0.5025849335302807
+[ Info: Epoch 9
+accuracy(train_X, train_y) = 0.5151403249630724
+[ Info: Epoch 10
+accuracy(train_X, train_y) = 0.5291728212703102
+[ Info: Epoch 11
+accuracy(train_X, train_y) = 0.543205317577548
+[ Info: Epoch 12
+accuracy(train_X, train_y) = 0.5550221565731167
+[ Info: Epoch 13
+accuracy(train_X, train_y) = 0.5638847858197932
+[ Info: Epoch 14
+accuracy(train_X, train_y) = 0.5657311669128509
+[ Info: Epoch 15
+accuracy(train_X, train_y) = 0.5749630723781388
+[ Info: Epoch 16
+accuracy(train_X, train_y) = 0.5834564254062038
+[ Info: Epoch 17
+accuracy(train_X, train_y) = 0.5919497784342689
+[ Info: Epoch 18
+accuracy(train_X, train_y) = 0.5978581979320532
+[ Info: Epoch 19
+accuracy(train_X, train_y) = 0.6019202363367799
+[ Info: Epoch 20
+accuracy(train_X, train_y) = 0.6067208271787297
+```
+
+Check `examples/gcn.jl` for details.

docs/src/manual/linalg.md

@@ -2,21 +2,21 @@
 
 
 ```@docs
-GeometricFlux.degrees
+GraphSignals.degrees
 ```
 
 ```@docs
-GeometricFlux.degree_matrix
+GraphSignals.degree_matrix
 ```
 
 ```@docs
-GeometricFlux.inv_sqrt_degree_matrix
+GraphSignals.inv_sqrt_degree_matrix
 ```
 
 ```@docs
-GeometricFlux.laplacian_matrix
+GraphSignals.laplacian_matrix
 ```
 
 ```@docs
-GeometricFlux.normalized_laplacian
+GraphSignals.normalized_laplacian
 ```

docs/src/manual/models.md

@@ -1 +1,68 @@
 # Models
+
+## Autoencoders
+
+### Graph Autoencoder
+
+```math
+Z = enc(X, A) \\
+\hat{A} = \sigma (ZZ^T)
+```
+
+where ``A`` denotes the adjacency matrix.
+
+```@docs
+GAE
+```
+
+Reference: [Variational Graph Auto-Encoders](https://arxiv.org/abs/1611.07308)
+
+---
+
+### Variational Graph Autoencoder
+
+```math
+X' = \sigma(\hat{D}^{-1/2} \hat{A} \hat{D}^{-1/2} X \Theta)
+```
+
+where ``\hat{A} = A + I``, ``A`` denotes the adjacency matrix, and
+``\hat{D} = [\hat{d}_{ij}] = \sum_{j=0} [\hat{a}_{ij}]`` is degree matrix.
+
+```@docs
+VGAE
+```
+
+Reference: [Variational Graph Auto-Encoders](https://arxiv.org/abs/1611.07308)
+
+---
+
+## Special Layers
+
+### Inner-product Decoder
+
+```math
+\hat{A} = \sigma (ZZ^T)
+```
+
+where ``Z`` denotes the input matrix from encoder.
+
+```@docs
+InnerProductDecoder
+```
+
+Reference: [Variational Graph Auto-Encoders](https://arxiv.org/abs/1611.07308)
+
+---
+
+### Variational Encoder
+
+```math
+H = f(X)
+μ, logσ = μ(H), Σ(H)
+```
+
+```@docs
+VariationalEncoder
+```
+
+Reference: [Variational Graph Auto-Encoders](https://arxiv.org/abs/1611.07308)

docs/src/manual/utils.md

@@ -1,5 +1 @@
 # Utilities
-
-```@docs
-GeometricFlux.save_div
-```

docs/src/start.md

@@ -0,0 +1 @@
+# Get started