Merge pull request #105 from yuehhua/doc

Doc

Author: yuehhua

README.md

@@ -19,7 +19,6 @@ Suggestions, issues and pull requsts are welcome.
 
 ```
 ]add GeometricFlux
-]add GraphSignals@0.1.1
 ```
 
 ## Features

docs/make.jl

@@ -3,7 +3,9 @@ using GeometricFlux
 
 makedocs(
     sitename = "GeometricFlux",
-    format = Documenter.HTML(),
+    format = Documenter.HTML(
+      canonical = "https://yuehhua.github.io/GeometricFlux.jl/stable"
+    ),
     modules = [GeometricFlux],
     pages = ["Home" => "index.md",
              "Get started" => "start.md",

docs/src/abstractions/gn.md

@@ -1 +1,27 @@
 # Graph network block
+
+Graph network (GN) is a more generic model for graph neural network. It describes an update order: edge, node and then global. There are three corresponding update functions for edge, node and then global, respectively. Three update functions return their default values as follow:
+
+```
+update_edge(gn, e, vi, vj, u) = e
+update_vertex(gn, ē, vi, u) = vi
+update_global(gn, ē, v̄, u) = u
+```
+
+Information propagation between different levels are achieved by aggregate functions. Three aggregate functions `aggregate_neighbors`, `aggregate_edges` and `aggregate_vertices` are defined to aggregate states.
+
+GN block is realized into a abstract type `GraphNet`. User can make a subtype of `GraphNet` to customize GN block. Thus, a GN block is defined as a layer in GNN. `MessagePassing` is a subtype of `GraphNet`.
+
+## Update functions
+
+`update_edge` acts as the first update function to apply to edge states. It takes edge state `e`, node `i` state `vi`, node `j` state `vj` and global state `u`. It is expected to return a feature vector for new edge state. `update_vertex` updates nodes state by taking aggregated edge state `ē`, node `i` state `vi` and global state `u`. It is expected to return a feature vector for new node state. `update_global` updates global state with aggregated information from edge and node. It takes aggregated edge state `ē`, aggregated node state `v̄` and global state `u`. It is expected to return a feature vector for new global state. User can define their own behavior by overriding update functions.
+
+## Aggregate functions
+
+An aggregate function `aggregate_neighbors` aggregates edge states for edges incident to some node `i` into node-level information. Aggregate function `aggregate_edges` aggregates all edge states into global-level information. The last aggregate function `aggregate_vertices` aggregates all vertex states into global-level information. It is avaible for assigning aggregate function by assigning aggregate operations to `propagate` function.
+
+```
+propagate(gn, fg::FeaturedGraph, naggr=nothing, eaggr=nothing, vaggr=nothing)
+```
+
+`naggr`, `eaggr` and `vaggr` are arguments for `aggregate_neighbors`, `aggregate_edges` and `aggregate_vertices`, respectively. Avaible aggregate functions are assigned by following symbols to them: `:add`, `:sub`, `:mul`, `:div`, `:max`, `:min` and `:mean`.

docs/src/abstractions/msgpass.md

@@ -1 +1,40 @@
 # Message passing scheme
+
+Message passing scheme is a popular GNN scheme in many frameworks. It adapts the property of connectivity of neighbors and form a general approach for spatial grpah convolutional neural network. It comprises two user-defined functions and one aggregate function. A message function is defined to process information from edge states and node states from neighbors and itself. Messages from each node are obtained and aggregated by aggregate function to provide node-level information for update function. Update function takes current node state and aggregated message and gives a new node state.
+
+Message passing scheme is realized into a abstract type `MessagePassing`. Any subtype of `MessagePassing` is a message passing layer which utilize default message and update functions:
+
+```
+message(mp, x_i, x_j, e_ij) = x_j
+update(mp, m, x) = m
+```
+
+`mp` denoates a message passing layer. `message` accepts node state `x_i` for node `i` and its neighbor state `x_j` for node `j`, as well as corresponding edge state `e_ij` for edge `(i,j)`. The default message function gives all the neighbor state `x_j` for neighbor of node `i`. `update` takes aggregated message `m` and current node state `x`, and then outputs `m`.
+
+## Message function
+
+A message function accepts feature vector representing node state `x_i`, feature vectors for neighbor state `x_j` and corresponding edge state `e_ij`. A vector is expected to output from `message` for message. User can override `message` for customized message passing layer to provide desired behavior.
+
+## Aggregate messages
+
+Messages from message function are aggregated by an aggregate function. An aggregated message is passed to update function for node-level computation. An aggregate function is given by the following:
+
+```
+propagate(mp, fg::FeaturedGraph, aggr::Symbol=:add)
+```
+
+`propagate` function calls the whole message passing layer. `fg` acts as an input for message passing layer and `aggr` represents assignment of aggregate function to `propagate` function. `:add` represents an aggregate function of addition of all messages.
+
+The following `aggr` are avaible aggregate functions:
+
+`:add`: sum over all messages
+`:sub`: negative of sum over all messages
+`:mul`: multiplication over all messages
+`:div`: inverse of multiplication over all messages
+`:max`: the maximum of all messages
+`:min`: the minimum of all messages
+`:mean`: the average of all messages
+
+## Update function
+
+An update function takes aggregated message `m` and current node state `x` as arguments. An output vector is expected to be the new node state for next layer.User can override `update` for customized message passing layer to provide desired behavior.

docs/src/basics/layers.md

@@ -1 +1,14 @@
-# Building layers
+# Building graph neural networks
+
+Building GNN is as simple as building neural network in Flux. The synatex here is the same as Flux. `Chain` is used to stack layers into a GNN. A simple example is shown here:
+
+```
+model = Chain(GCNConv(adj_mat, feat=>h1),
+              GCNConv(adj_mat, h1=>h2, relu))
+```
+
+`GCNConv` is used for layer construction for neural network. The first argument `adj_mat` is the representation of a graph in form of adjacency matrix. The feature dimension in first layer is mapped from `feat` to `h1`. In seceond layer, `h1` is then mapped to `h2`. Default activation function is given as identity if it is not specified by users.
+
+## Customize layers
+
+Customizing your own GNN layers are the same as customizing layers in Flux. You may want to reference [Flux documentation](https://fluxml.ai/Flux.jl/stable/models/basics/#Building-Layers-1).

docs/src/basics/passgraph.md

@@ -1,9 +1,33 @@
 # Graph passing
 
-Graph is an input data structure for graph neural network. Passing graph and input feature into 
+Graph is an input data structure for graph neural network. Passing graph into GNN layer can have different behaviors. If graph remains fixed across samples, that is, all samples utilize the same graph structure, a static graph is used. Graphs can be carried within `FeaturedGraph` to provide varible graph to GNN layer. Users have the flexibility to pick a adequate approach for their own needs.
 
 ## Static graph
 
+A static graph is used to reduce redundent computation during passing through layers. A static graph can be set in graph convolutional layers in prior such that graph in layers is used first for computation. A adjacency matrix `adj_mat` is given to represent a graph and is put into a graph convolutional layer as follow:
+
+```
+GCNConv(adj_mat, feat=>h1, relu)
+```
+
+`Simple(Di)Graph`, `SimpleWeighted(Di)Graph` or `Meta(Di)Graph` provided by LightGraphs, SimpleWeightedGraphs and MetaGraphs, respectively, are acceptable for passing to layer as a static graph. A adjacency list is also accepted in the type of `Vector{Vector}`.
+
 ## Variable graph
 
-## Cached graph
+A variable graph is supported by `FeaturedGraph`. Each `FeaturedGraph` contains different graph structure and its features. Data of `FeaturedGraph` are feed directly to graph convolutional layer or graph neural network to let each feature be learn on different graph strucutre. A adjacency matrix `adj_mat` is given to construct a `FeaturedGraph` as follow:
+
+```
+FeaturedGraph(adj_mat, features)
+```
+
+`Simple(Di)Graph`, `SimpleWeighted(Di)Graph` or `Meta(Di)Graph` provided by LightGraphs, SimpleWeightedGraphs and MetaGraphs, respectively, are acceptable for constructing a `FeaturedGraph`. A adjacency list is also accepted, too.
+
+## Cached graph in layers
+
+While a variable graph is given by `FeaturedGraph`, a GNN layer don't need a static graph anymore. Besides taking off the static graph from arguments of a layer, remember to turn off the cache mechanism. A cache mechanism is designed to cache static graph to reduce computation. A cached graph is gotten from layer and computation is then performed. For each time, it will assign current computating graph back to layer. Assignment operation is not differentiable, so we must turn off the cache mechanism as follow:
+
+```
+GCNConv(feat=>h1, relu, cached=false)
+```
+
+This ensures layer function as expected.

docs/src/index.md

@@ -6,7 +6,7 @@ GeometricFlux is a framework for geometric deep learning/machine learning.
 * It supports of CUDA GPU with CUDA.jl
 * It integrates with JuliaGraphs ecosystems.
 * It supports generic graph neural network architectures (i.g. message passing scheme and graph network block)
-* It contains built-in GNN benchmark datasets (WIP)
+* It contains built-in GNN benchmark datasets (provided by GraphMLDatasets)
 
 ## Installation