fixes for skip dir and adjustment to the docs (#169)

Author: tlienart

docs/src/man/ls+lit.md

@@ -7,96 +7,160 @@ You've likely already seen how LiveServer could be used along with Documenter to
 It is also easy to use LiveServer with both Documenter and [Literate.jl](https://github.com/fredrikekre/Literate.jl), a package for literate programming written by Fredrik Ekre that can convert julia script files into markdown.
 This can be particularly convenient for documentation pages with a lot of code examples.
 
-Only two steps are required to have this working (assuming you have already added Literate to your environment):
+There are mainly two steps 
 
-1. pick a folder structure
-1. modify the `docs/make.jl` file to contain a line calling Literate
+1. have the `make.jl` file process the literate files to go from `.jl` to `.md` files,
+2. call `servedocs` with appropriate keywords.
 
-### Folder structure
-
-There are effectively two recommended ways, pick whichever one you prefer.
-In the first case, the script files `.jl` to be compiled by Literate are at the _same location_ as the output file so that you would have:
+The function `LiveServer.servedocs_literate_example` generates a directory which has the right structure that you can copy for your package.
+To experiment, do:
 
-```
-docs
-└── src
-    ├── index.jl
-    └── index.md
+```julia-repl
+julia> using LiveServer
+julia> LiveServer.servedocs_literate_example("test_dir")
+julia> cd("test_dir")
+julia> servedocs(literate_dir=joinpath("docs", "literate"))
 ```
 
-if you're happy with this, then you can jump to the [next step](#Modifying-the-make-file-1) to change the make file.
+if you then navigate to `localhost:8000` you should end up with
 
-However you may not be happy with this, in particular if you have lots of such files and a mix of files which are generated by `Literate` and some which aren't, then typically you might prefer to keep all scripts in a separate folder.
-You would just have to make sure that the output is properly redirected to `docs/src`.
-Your folder structure would then look something like:
+![](/assets/testlit.png)
 
-```
-docs
-├── lit
-│   └── index.jl
-└── src
-    └── index.md
-```
+if you modify `test_dir/docs/literate/man/pg1.jl` for instance writing `f(4)` it will be applied directly:
 
-The only thing you have to do in this case is to specify to `servedocs` where the "literate folder" is; this is a keyword argument and for the example above we would have:
+![](/assets/testlit2.png)
+
+
+In the explanations below we assume you have defined
+
+* `LITERATE_INPUT` the directory where the literate files are,
+* `LITERATE_OUTPUT` the directory where the generated markdown files will be.
 
-```julia
-servedocs(literate_dir=joinpath("docs", "lit"))
-```
 
-### Modifying the make file
+## Having the make file call Literate
 
-The only thing you have to do here is add a few lines to specify which files should be compiled by `Literate`.
-Assuming you have taken the second path in the situation above, your `make.jl` file should look like:
+Here's a basic `make.jl` file which loops over the files in `LITERATE_INPUT` to generate files
+in `LITERATE_OUTPUT` 
 
 ```julia
 using Documenter, Literate
 
-src = joinpath(@__DIR__, "src")
-lit = joinpath(@__DIR__, "lit")
+LITERATE_INPUT = ...
+LITERATE_OUTPUT = ...
 
-for (root, _, files) ∈ walkdir(lit), file ∈ files
+for (root, _, files) ∈ walkdir(LITERATE_INPUT), file ∈ files
+    # ignore non julia files
     splitext(file)[2] == ".jl" || continue
+    # full path to a literate script
     ipath = joinpath(root, file)
-    opath = splitdir(replace(ipath, lit=>src))[1]
+    # generated output path
+    opath = splitdir(replace(ipath, LITERATE_INPUT=>LITERATE_OUTPUT))[1]
+    # generate the markdown file calling Literate
     Literate.markdown(ipath, opath)
 end
 
 makedocs(
-    sitename = "Test",
-    modules = [Test],
-    pages = ["Home" => "index.md"]
+    ...
     )
 ```
 
-If you were happy with the `.jl` and `.md` files being in the same location, simply replace the `lit = ` line by
+## Calling servedocs with the right arguments
+
+`LiveServer.servedocs` needs to know two things to work with literate scripts properly:
+
+* where the scripts are
+* where the generated files will be
+
+it can make assumptions for some basic cases but, in general, you'll have to provide both.
+
+Doing so improperly may lead to an infinite loop where:
+* the first `make.jl` call generates markdown files with Literate
+* these generated markdown files themselves trigger `make.jl`
+* (infinite loop)
+
+To avoid this, you must generally call `servedocs` as follows when working with literate files:
+
+```
+servedocs(
+    literate_dir = LITERATE_INPUT
+    skip_dir = LITERATE_OUTPUT
+)
+```
+
+where
+
+* `literate_dir` is the parent directory of the literate scripts, and
+* `skip_dir` is the parent directory where the generated markdown files are placed.
+
+**Special cases**:
+
+* if the literate scripts are located in `docs/src` you can just specify `literate_dir=""`,
+* if the literate scripts are generated with in `docs/src` with the exact same relative path, you
+do not need to specify `skip_dir`.
+
+
+### Example 1 <!-- checked 10/7/2023 | 1.2.1 -->
+
+```
+docs
+└── src
+    ├── literate_script.jl
+    └── literate_script.md
+```
+
+in this case we can call 
 
 ```julia
-lit = src
+servedocs(literate="")
 ```
 
-What the for loop does is simple: it loops over the files in the folder where it's likely to encounter `.jl` files and for those it encounters:
+since
 
-1. it retrieves the path to the file (`ipath`)
-1. it constructs the output path in `docs/src` (`opath`)
-1. it compiles the file `ipath` and saves the output at `opath`
+1. the literate scripts are under `docs/src`
+2. the generated markdown files have exactly the same relative path.
 
-## Complete example
 
-The function `LiveServer.servedocs_literate_example` generates a directory which has the right structure that you can copy for your package.
-To experiment, do:
+### Example 2 <!-- checked 10/7/2023 | 1.2.1 -->
 
-```julia-repl
-julia> using LiveServer
-julia> LiveServer.servedocs_literate_example("test_dir")
-julia> cd("test_dir")
-julia> servedocs(literate_dir=joinpath("docs", "literate"))
+```
+docs
+├── literate
+│   └── literate_script.jl
+└── src
+    └── literate_script.md
 ```
 
-if you then navigate to `localhost:8000` you should end up with
+in this case we can call 
 
-![](/assets/testlit.png)
+```julia
+servedocs(literate=joinpath("docs", "literate"))
+```
 
-if you modify `test_dir/docs/literate/man/pg1.jl` for instance writing `f(4)` it will be applied directly:
+since
+
+1. the literate scripts are under a dedicated folder,
+2. the generated markdown files have exactly the same relative path.
+
+### Example 3 <!-- checked 10/7/2023 | 1.2.1 -->
+
+```
+foo
+├── literate
+│   └── literate_script.jl
+docs
+└── src
+    └── generated
+        └── literate_script.md
+```
+
+in this case we can call 
+
+```julia
+servedocs(literate=joinpath("foo", "literate"), skip_dir=joinpath())
+```
+
+since
+
+1. the literate scripts are under a dedicated folder,
+2. the generated markdown files do not have exactly the same relative path.
 
-![](/assets/testlit2.png)

src/utils.jl

@@ -180,24 +180,13 @@ subfolder `docs`.
 * `doc_env=false`: a boolean switch to make the server start by activating the
 doc environment or not (i.e. the `Project.toml` in `docs/`).
 * `literate=nothing`: see `literate_dir`.
-* `literate_dir=nothing`: Path to a directory containing Literate scripts. If
-                          `nothing`, it's assumed there are no such scripts.
-                          Any `*.jl` file in the folder (or subfolders) is
-                          checked for changes and is assumed to generate a
-                          `*.md` file with the same name, in the same location.
-                          It is necessary to indicate this path to avoid a
-                          recursive trigger loop where the generated `*.md` file
-                          triggers, causing the literate script to be
-                          re-evaluated which, in turn, re-generates the `*.md`
-                          file.
-                          If the generated `*.md` file are in fact not located
-                          in the same location as their source `*.jl` file, 
-                          then the user must indicate that these `*.md` files
-                          should be ignored (should not trigger) by using
-                          `skip_dir` or `skip_files`.
-* `skip_dir=""`: a subpath of `docs/` where modifications should not trigger
-                 the generation of the docs, this is useful for instance if
-                 you're using Weave and Weave generates some files in
+* `literate_dir=nothing`: Path to a directory containing Literate scripts if
+                          these are not simply under `docs/src`. 
+                          See also the docs for information about how to
+                          work with Literate.
+* `skip_dir=""`: a path starting with `docs/` where modifications should not
+                 trigger the generation of the docs, this is useful for instance
+                 if you're using Weave and Weave generates some files in
                  `docs/src/examples` in which case you should set
                  `skip_dir=joinpath("docs","src","examples")`.
 * `skip_dirs=[]`: same as `skip_dir` but for a list of such dirs. Takes
@@ -239,6 +228,7 @@ function servedocs(;
     if isempty(skip_dirs) && !isempty(skip_dir)
         skip_dirs = [skip_dir]
     end
+    push!(skip_dirs, joinpath("docs", "build"))
     skip_dirs     = abspath.(skip_dirs)
     skip_files    = abspath.(skip_files)
     include_dirs  = abspath.(include_dirs)
@@ -248,16 +238,19 @@ function servedocs(;
     if isnothing(literate_dir) && !isnothing(literate)
         literate_dir = literate
     end
+    if !isnothing(literate_dir)
+        literate_dir = abspath(literate_dir)
+    end
 
-    path2makejl = joinpath(foldername, makejl)
+    path2makejl  = joinpath(foldername, makejl)
 
     # The file watcher is a default SimpleWatcher with a custom
     # callback
     docwatcher = SimpleWatcher()
     set_callback!(
         docwatcher,
         fp -> servedocs_callback!(
-                docwatcher, fp, path2makejl,
+                docwatcher, abspath(fp), path2makejl,
                 literate_dir,
                 skip_dirs, skip_files, include_dirs, include_files,
                 foldername, buildfoldername