Skip to content

Commit bd01d70

Browse files
authored
Merge pull request #71 from asprionj/literate
Servedocs improvement + literate jl docs
2 parents a46926d + b726278 commit bd01d70

File tree

8 files changed

+268
-52
lines changed

8 files changed

+268
-52
lines changed

docs/make.jl

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,8 @@ makedocs(
1414
"Home" => "index.md",
1515
"Manual" => [
1616
"Functionalities" => "man/functionalities.md",
17-
"Extending LiveServer" => "man/extending_ls.md"
17+
"Extending LiveServer" => "man/extending_ls.md",
18+
"LiveServer + Literate" => "man/ls+lit.md"
1819
],
1920
"Library" => [
2021
"Public" => "lib/public.md",

docs/src/assets/testlit.png

34.3 KB
Loading

docs/src/assets/testlit2.png

33.5 KB
Loading

docs/src/index.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,8 @@ julia> servedocs()
4949
5050
Open a browser and go to `http://localhost:8000/` to see your docs being rendered; try modifying files (e.g. `docs/index.md`) and watch the changes being rendered in the browser.
5151
52+
You can also use LiveServer with both Documenter and [Literate.jl](https://github.com/fredrikekre/Literate.jl).
53+
This is explained [here](man/ls+lit.md).
5254
5355
## How it works
5456

docs/src/lib/internals.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -70,7 +70,7 @@ LiveServer.update_and_close_viewers!
7070
#### Helper functions associated with `servedocs`
7171

7272
```@docs
73-
LiveServer.servedocs_callback
73+
LiveServer.servedocs_callback!
7474
LiveServer.scan_docs!
7575
```
7676

docs/src/man/ls+lit.md

Lines changed: 170 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,170 @@
1+
# LiveServer + Literate
2+
3+
(_Thanks to [Fredrik Ekre](https://github.com/fredrikekre) and [Benoit Pasquier](https://github.com/briochemc) for their input; a lot of this section is drawn from an early prototype suggested by Fredrik._)
4+
5+
You've likely already seen how LiveServer could be used along with Documenter to have live updating documentation (see [`servedocs`](/man/functionalities/#servedocs-1) if not).
6+
7+
It is also easy to use LiveServer with both Documenter and [Literate.jl](https://github.com/fredrikekre/Literate.jl), a package for literate programming written by Fredrik Ekre that can convert julia script files into markdown.
8+
This can be particularly convenient for documentation pages with a lot of code examples.
9+
10+
Only two steps are required to have this working (assuming you have already added Literate to your environment):
11+
12+
1. pick a folder structure
13+
1. modify the `docs/make.jl` file to contain a line calling Literate
14+
15+
### Folder structure
16+
17+
There are effectively two recommended ways, pick whichever one you prefer.
18+
In the first case, the script files `.jl` to be compiled by Literate are at the _same location_ as the output file so that you would have:
19+
20+
```
21+
docs
22+
└── src
23+
├── index.jl
24+
└── index.md
25+
```
26+
27+
if you're happy with this, then you can jump to the [next step](#Modifying-the-make-file-1) to change the make file.
28+
29+
However you may not be happy with this, in particular if you have lots of such files and a mix of files which are generated by `Literate` and some which aren't, then typically you might prefer to keep all scripts in a separate folder.
30+
You would just have to make sure that the output is properly redirected to `docs/src`.
31+
Your folder structure would then look something like:
32+
33+
```
34+
docs
35+
├── lit
36+
│   └── index.jl
37+
└── src
38+
└── index.md
39+
```
40+
41+
The only thing you have to do in this case is to specify to `servedocs` where the "literate folder" is; this is a keyword argument and for the example above we would have:
42+
43+
```julia
44+
servedocs(literate=joinpath("docs", "lit"))
45+
```
46+
47+
### Modifying the make file
48+
49+
The only thing you have to do here is add a few lines to specify which files should be compiled by `Literate`.
50+
Assuming you have taken the second path in the situation above, your `make.jl` file should look like:
51+
52+
```julia
53+
using Documenter, Literate
54+
55+
src = joinpath(@__DIR__, "src")
56+
lit = joinpath(@__DIR__, "lit")
57+
58+
for (root, _, files) walkdir(lit), file files
59+
splitext(file)[2] == ".jl" || continue
60+
ipath = joinpath(root, file)
61+
opath = splitdir(replace(ipath, lit=>src))[1]
62+
Literate.markdown(ipath, opath)
63+
end
64+
65+
makedocs(
66+
sitename = "Test",
67+
modules = [Test],
68+
pages = ["Home" => "index.md"]
69+
)
70+
```
71+
72+
If you were happy with the `.jl` and `.md` files being in the same location, simply replace the `lit = ` line by
73+
74+
```julia
75+
lit = src
76+
```
77+
78+
What the for loop does is simple: it loops over the files in the folder where it's likely to encounter `.jl` files and for those it encounters:
79+
80+
1. it retrieves the path to the file (`ipath`)
81+
1. it constructs the output path in `docs/src` (`opath`)
82+
1. it compiles the file `ipath` and saves the output at `opath`
83+
84+
## Complete example
85+
86+
Here's a step-by-step example to get started which should help put all the pieces together.
87+
88+
Let's start by creating a dummy repo
89+
90+
```julia-repl
91+
pkg> generate testlit
92+
julia> cd("testlit")
93+
pkg> activate testlit
94+
pkg> add Documenter Literate LiveServer
95+
pkg> dev .
96+
```
97+
98+
add a `docs/` folder with the appropriate structure so that the `testlit` folder ends up like
99+
100+
```
101+
.
102+
├── Manifest.toml
103+
├── Project.toml
104+
├── docs
105+
│   ├── literate
106+
│   │   └── man
107+
│   │   └── pg1.jl
108+
│   ├── make.jl
109+
│   └── src
110+
│   ├── index.md
111+
│   └── man
112+
└── src
113+
└── testlit.jl
114+
```
115+
116+
where the file `pg1.jl` contains
117+
118+
```julia
119+
# # Test literate
120+
121+
# We can include some code like so:
122+
123+
f(x) = x^5
124+
f(5)
125+
```
126+
127+
the file `index.md` contains
128+
129+
```
130+
# Test
131+
132+
A link to the [other page](/man/pg1.md)
133+
```
134+
135+
and the file `make.jl` contains
136+
137+
```julia
138+
using Documenter, Literate
139+
140+
src = joinpath(@__DIR__, "src")
141+
lit = joinpath(@__DIR__, "literate")
142+
143+
for (root, _, files) walkdir(lit), file files
144+
splitext(file)[2] == ".jl" || continue
145+
ipath = joinpath(root, file)
146+
opath = splitdir(replace(ipath, lit=>src))[1]
147+
Literate.markdown(ipath, opath)
148+
end
149+
150+
makedocs(
151+
sitename = "testlit",
152+
modules = [testlit],
153+
pages = ["Home" => "index.md",
154+
"Other page" => "man/pg1.md"]
155+
)
156+
```
157+
158+
Now `cd("testlit/")` and do
159+
160+
```julia-repl
161+
julia> servedocs(literate=joinpath("docs", "literate"))
162+
```
163+
164+
if you navigate to `localhost:8000` you should end up with
165+
166+
![](/assets/testlit.png)
167+
168+
if you modify `testlit/docs/literate/man/pg1.jl` for instance writing `f(4)` it will be applied directly:
169+
170+
![](/assets/testlit2.png)

src/utils.jl

Lines changed: 74 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
"""
2-
servedocs_callback(filepath, watchedfiles, path2makejl)
2+
servedocs_callback!(docwatcher, filepath, path2makejl, literate)
33
44
Custom callback used in [`servedocs`](@ref) triggered when the file corresponding to `filepath`
55
is changed. If that file is `docs/make.jl`, the callback will check whether any new files have
@@ -9,82 +9,119 @@ Otherwise, if the modified file is in `docs/src` or is `docs/make.jl`, a pass of
99
triggered to regenerate the documents, subsequently the LiveServer will render the produced pages
1010
in `docs/build`.
1111
"""
12-
function servedocs_callback(fp::AbstractString, vwf::Vector{WatchedFile}, makejl::AbstractString)
13-
ismakejl = (fp == makejl)
14-
# if the file that was changed is the `make.jl` file,
15-
# assume that maybe new files are referenced and so refresh the
16-
# vector of watched files as a result.
17-
if ismakejl
18-
watchedpaths = (wf.path for wf vwf)
19-
for (root, _, files) walkdir(joinpath("docs", "src")), file files
20-
fpath = joinpath(root, file)
21-
fpath watchedpaths || push!(vwf, WatchedFile(fpath))
22-
end
23-
# check if any file that was watched has died
24-
deadfiles = Int[]
25-
for (i, wf) enumerate(vwf)
26-
isfile(wf.path) || push!(deadfiles, i)
27-
end
28-
deleteat!(vwf, deadfiles)
12+
function servedocs_callback!(dw::SimpleWatcher, fp::AbstractString, makejl::AbstractString,
13+
literate::String="")
14+
# if the file that was changed is the `make.jl` file, assume that maybe new files are # referenced and so refresh the vector of watched files as a result.
15+
if fp == makejl
16+
# it's easier to start from scratch (takes negligible time)
17+
empty!(dw.watchedfiles)
18+
scan_docs!(dw, literate)
2919
end
30-
# only trigger for changes appearing in `docs/src` otherwise a loop gets triggered
31-
# changes from docs/src create change in docs/build which trigger a pass which
32-
# regenerates files in docs/build etc...
33-
if ismakejl || occursin(joinpath("docs", "src"), fp)
20+
if splitext(fp)[2] (".md", ".jl")
3421
Main.include(makejl)
3522
file_changed_callback(fp)
3623
end
3724
return nothing
3825
end
3926

27+
4028
"""
41-
scan_docs!(dw::SimpleWatcher)
29+
scan_docs!(dw::SimpleWatcher, literate="")
4230
4331
Scans the `docs/` folder in order to recover the path to all files that have to be watched and add
44-
those files to `dw.watchedfiles`. The function returns the path to `docs/make.jl`.
32+
those files to `dw.watchedfiles`. The function returns the path to `docs/make.jl`. A list of
33+
folders and file paths can also be given for files that should be watched in addition to the
34+
content of `docs/src`.
4535
"""
46-
function scan_docs!(dw::SimpleWatcher)
36+
function scan_docs!(dw::SimpleWatcher, literate::String="")
4737
src = joinpath("docs", "src")
4838
if !(isdir("docs") && isdir(src))
4939
@error "I didn't find a docs/ or docs/src/ folder."
5040
end
5141
makejl = joinpath("docs", "make.jl")
5242
push!(dw.watchedfiles, WatchedFile(makejl))
5343
if isdir("docs")
54-
# add all files in `docs/src` to watched files
55-
for (root, _, files) walkdir(joinpath("docs", "src")), file files
56-
push!(dw.watchedfiles, WatchedFile(joinpath(root, file)))
57-
end
44+
# add all files in `docs/src` to watched files
45+
for (root, _, files) walkdir(joinpath("docs", "src")), file files
46+
push!(dw.watchedfiles, WatchedFile(joinpath(root, file)))
47+
end
48+
end
49+
if !isempty(literate)
50+
isdir(literate) || @error "I didn't find the provided literate folder $literate."
51+
for (root, _, files) walkdir(literate), file files
52+
push!(dw.watchedfiles, WatchedFile(joinpath(root, file)))
53+
end
5854
end
55+
56+
# When using literate.jl, we should only watch the source file otherwise we would double
57+
# trigger: first when the script.jl is modified then again when the script.md is created
58+
# which would cause an infinite loop if both `script.jl` and `script.md` are watched.
59+
# So here we remove from the watchlist all files.md that have a files.jl with the same path.
60+
remove = Int[]
61+
if isempty(literate)
62+
# assumption is that the scripts are in `docs/src/...` and that the generated markdown
63+
# goes in exactly the same spot so for instance:
64+
# docs
65+
# └── src
66+
# ├── index.jl
67+
# └── index.md
68+
for wf dw.watchedfiles
69+
spath = splitext(wf.path)
70+
spath[2] == ".jl" || continue
71+
k = findfirst(e -> splitext(e.path) == (spath[1], ".md"), dw.watchedfiles)
72+
k === nothing || push!(remove, k)
73+
end
74+
else
75+
# assumption is that the scripts are in `literate/` and that the generated markdown goes
76+
# in `docs/src` with the same relative paths so for instance:
77+
# docs
78+
# ├── lit
79+
# │   └── index.jl
80+
# └── src
81+
# └── index.md
82+
for (root, _, files) walkdir(literate), file files
83+
spath = splitext(joinpath(root, file))
84+
spath[2] == ".jl" || continue
85+
path = replace(spath[1], Regex("^$literate") => joinpath("docs", "src"))
86+
k = findfirst(e -> splitext(e.path) == (path, ".md"), dw.watchedfiles)
87+
k === nothing || push!(remove, k)
88+
end
89+
end
90+
deleteat!(dw.watchedfiles, remove)
5991
return makejl
6092
end
6193

94+
6295
"""
63-
servedocs(; verbose=false)
96+
servedocs(; verbose=false, literate="")
6497
6598
Can be used when developing a package to run the `docs/make.jl` file from Documenter.jl and
6699
then serve the `docs/build` folder with LiveServer.jl. This function assumes you are in the
67100
directory `[MyPackage].jl` with a subfolder `docs`.
68101
69-
* `verbose` is a boolean switch to make the server print information about file changes and connections.
102+
* `verbose` is a boolean switch to make the server print information about file changes and
103+
connections.
104+
* `literate` is the path to the folder containing the literate scripts, if left empty, it will be
105+
assumed that they are in `docs/src`.
70106
"""
71-
function servedocs(; verbose::Bool=false)
107+
function servedocs(; verbose::Bool=false, literate::String="")
72108
# Custom file watcher: it's the standard `SimpleWatcher` but with a custom callback.
73109
docwatcher = SimpleWatcher()
74-
set_callback!(docwatcher, fp->servedocs_callback(fp, docwatcher.watchedfiles, makejl))
110+
set_callback!(docwatcher, fp->servedocs_callback!(docwatcher, fp, makejl, literate))
75111

76-
makejl = scan_docs!(docwatcher)
112+
# Retrieve files to watch
113+
makejl = scan_docs!(docwatcher, literate)
77114

78-
# trigger a first pass of Documenter
115+
# trigger a first pass of Documenter (& possibly Literate)
79116
Main.include(makejl)
80117

81118
# note the `docs/build` exists here given that if we're here it means the documenter
82119
# pass did not error and therefore that a docs/build has been generated.
83120
serve(docwatcher, dir=joinpath("docs", "build"), verbose=verbose)
84-
85121
return nothing
86122
end
87123

124+
88125
#
89126
# Miscellaneous utils
90127
#
@@ -96,6 +133,7 @@ Set the verbosity of LiveServer to either `true` (showing messages upon events)
96133
"""
97134
setverbose(b::Bool) = (VERBOSE.x = b)
98135

136+
99137
"""
100138
example()
101139

0 commit comments

Comments
 (0)