chore: Rename tket-c-api and add static linking (#953)

After a lot more trials, I have concluded that we should support both
static and dynamic linking of `tket1-passes` (which I am renaming to
`tket-c-api`, see below).

Shared VS static `tket-c-api` library.

### Static linking

For users, static linking is much simpler: dynamic linking leads to
problems at runtime, when the dynamic linker cannot find the
`tket-c-api` library unless it is installed system-wide. [There is no
way to tell cargo](rust-lang/cargo#9016) to
set the rpath accordingly on every binary it builds, and so the users
would have to point the linker to the right library with an environment
variable whenever using `cargo` (or some `.cargo/config.toml` setting).

However, static linking is only feasible through a dependency management
tool such as conan. This is because when linking statically, all
recursive dependencies have to be linked in (tket, symengine, boost,
etc.). Doing this manually is a nightmare.

So conan is a must, but is terrible for CI speeds: just installing
`conan` and fetching the cache takes ~30 seconds :/

### Dynamic linking

With dynamic linking, on the other hand, it suffices to build against
`tket-c-api`; no recursive dependencies are required. This can be cached
in CI, and the `LD_LIBRARY_PATH` can be set so that the library can be
found at runtime.

## Solution

`tket-c-api` will by default be built using `conan` and linked in
statically. Conan can be called directly from `cargo`, and so no
separate build step other than `cargo build` should be necessary
(provided `conan` has been installed). I have renamed the special
purpose `tket1-passes` library into `tket-c-api`. I suggest the latter
should be integrated into `tket1` and compiled libraries could be
published to our JFrog artifactory, for faster local builds. Until this
is done, a separate `cd tket1-passes/tket-c-api && conan create .` step
will be required.

If the user (mostly: CI) sets the `TKET_C_API_PATH`, this folder will be
used instead to locate the `tket-c-api` library, bypassing conan
altogether. This should be a shared library, otherwise dependencies will
be missing at link time. It is then the user's responsibility to ensure
that the library can be loaded at runtime (by setting `LD_LIBRARY_PATH`
or equivalent)

Author: lmondada

.github/actions/tket-c-api/action.yml

@@ -0,0 +1,46 @@
+name: "Install tket-c-api C++ library"
+description: "Retrieve cached tket-c-api C++ library or build if needed"
+
+inputs:
+  install-path:
+    description: "Path where the tket-c-api library will be installed"
+    required: true
+
+runs:
+  using: composite
+  steps:
+    - name: Attempt to retrieve cached tket-c-api
+      uses: actions/cache/restore@v4
+      id: cache
+      with:
+        path: ${{ inputs.install-path }}
+        key: ${{ runner.os }}-${{ hashFiles('tket1-passes/tket-c-api/**') }}
+
+    - name: Install conan
+      if: steps.cache.outputs.cache-hit != 'true'
+      uses: conan-io/setup-conan@v1
+      with:
+        cache_packages: true
+
+    - name: Set up conan remote
+      if: steps.cache.outputs.cache-hit != 'true'
+      shell: bash
+      run: |
+        conan remote add tket-libs https://quantinuumsw.jfrog.io/artifactory/api/conan/tket1-libs --index 0
+
+    - name: Build and install tket-c-api
+      if: steps.cache.outputs.cache-hit != 'true'
+      shell: bash
+      run: |
+        cd tket1-passes/tket-c-api
+        PACKAGE_NAME="tket-c-api/$(conan inspect . --format=json | jq -r '.version')"
+        CONAN_OUTPUT=$(conan create . --build=missing --options="$PACKAGE_NAME:shared=True" --format=json)
+        CONAN_LIB_FOLDER=$(echo "$CONAN_OUTPUT" | jq -r ".graph.nodes.\"1\".package_folder")
+        cp -r $CONAN_LIB_FOLDER ${{ inputs.install-path }}
+
+    - name: Upload compiled library to cache
+      if: steps.cache.outputs.cache-hit != 'true'
+      uses: actions/cache/save@v4
+      with:
+        path: ${{ inputs.install-path }}
+        key: ${{ steps.cache.outputs.cache-primary-key }}

.github/actions/tket1-passes/action.yml

@@ -29,6 +29,9 @@ env:
   LLVM_VERSION: "14.0"
   LLVM_FEATURE_NAME: "14-0"
 
+  TKET_C_API_PATH: "${{ github.workspace }}/tket-c-api"
+  LD_LIBRARY_PATH: ${{ github.workspace }}/tket-c-api/lib
+
 jobs:
   # Check if changes were made to the relevant files.
   # Always returns true if running on the default branch, to ensure all changes are thoroughly checked.
@@ -75,8 +78,10 @@ jobs:
         uses: dtolnay/rust-toolchain@stable
         with:
           components: rustfmt, clippy
-      - name: Retrieve tket1-passes library
-        uses: ./.github/actions/tket1-passes
+      - name: Install tket-c-api library
+        uses: ./.github/actions/tket-c-api
+        with:
+          install-path: ${{ env.TKET_C_API_PATH }}
       - name: Check formatting
         run: cargo fmt -- --check
       - name: Install LLVM and Clang
@@ -178,8 +183,10 @@ jobs:
         uses: KyleMayes/install-llvm-action@v2
         with:
           version: ${{ env.LLVM_VERSION }}
-      - name: Retrieve tket1-passes library
-        uses: ./.github/actions/tket1-passes
+      - name: Install tket-c-api library
+        uses: ./.github/actions/tket-c-api
+        with:
+          install-path: ${{ env.TKET_C_API_PATH }}
       - name: Build with all features
         run: cargo test --verbose --workspace --all-features --no-run
       - name: Tests with all features
@@ -208,8 +215,10 @@ jobs:
         uses: KyleMayes/install-llvm-action@v2
         with:
           version: ${{ env.LLVM_VERSION }}
-      - name: Retrieve tket1-passes library
-        uses: ./.github/actions/tket1-passes
+      - name: Install tket-c-api library
+        uses: ./.github/actions/tket-c-api
+        with:
+          install-path: ${{ env.TKET_C_API_PATH }}
       - name: Configure default rust toolchain
         run: rustup override set ${{steps.toolchain.outputs.name}}
       - name: Build with no features
@@ -240,8 +249,10 @@ jobs:
         uses: KyleMayes/install-llvm-action@v2
         with:
           version: ${{ env.LLVM_VERSION }}
-      - name: Retrieve tket1-passes library
-        uses: ./.github/actions/tket1-passes
+      - name: Install tket-c-api library
+        uses: ./.github/actions/tket-c-api
+        with:
+          install-path: ${{ env.TKET_C_API_PATH }}
       - name: Install cargo-llvm-cov
         uses: taiki-e/install-action@cargo-llvm-cov
       - name: Run tests with coverage instrumentation

.github/workflows/ci.yml

@@ -43,19 +43,22 @@ jobs:
     needs: [changes]
     if: ${{ needs.changes.outputs.rust == 'true' }}
     runs-on: ubuntu-latest
+    env:
+      TKET_C_API_PATH: ${{ github.workspace }}/tket-c-api
+      LD_LIBRARY_PATH: ${{ github.workspace }}/tket-c-api/lib
     steps:
       - name: Checkout tket2
         uses: actions/checkout@v4
-      - name: Retrieve tket1-passes library
-        uses: ./.github/actions/tket1-passes
+      - name: Install tket-c-api library
+        uses: ./.github/actions/tket-c-api
         with:
-          lib-path: ${{ github.workspace }}/tket1-lib
+          install-path: ${{ env.TKET_C_API_PATH }}
       - name: Install LLVM from apt
         run: |
           echo "Installing apt dependencies: llvm-14"
           sudo apt-get install -y llvm-14
       - uses: CQCL/hugrverse-actions/rs-semver-checks@main
         env:
-          TKET_LIB_PATH: ${{ github.workspace }}/tket1-lib
+          TKET_C_API_PATH: ${{ env.TKET_C_API_PATH }}
         with:
           token: ${{ secrets.HUGRBOT_PAT }}

.github/workflows/semver-checks.yml

@@ -9,7 +9,7 @@ members = [
     "compile-rewriter",
     "badger-optimiser",
     "tket2-hseries",
-    "tket1-passes/rust",
+    "tket1-passes",
 ]
 default-members = ["tket2", "tket2-hseries"]
 

Cargo.lock

@@ -12,6 +12,7 @@ dev-dependencies = [
     "hypothesis >=6.111.1,<7",
     "graphviz >=0.20,<0.21",
     "pre-commit >=3.8.0,<4",
+    "conan >= 2.0.0,<3",
     # Required to run `maturin develop`
     "pip >=24",
 ]
@@ -35,6 +36,7 @@ exclude = '''(?x)(
     ^target/        # Build artifacts
     | ^notebooks/   # Examples
     | ^test_files/  # Test files
+    | ^tket1-passes/tket-c-api/ # Do not check conan files
 )'''
 
 # TODO: Fix the lint errors and enable this

Cargo.toml

@@ -1 +1 @@
-lib
+lib

pyproject.toml

@@ -21,6 +21,7 @@ tket-json-rs.workspace = true
 [build-dependencies]
 cc = "1.2"
 bindgen = "0.72"
+conan2 = "0.1.7"
 
 [dev-dependencies]
 rstest.workspace = true

tket1-passes/.gitignore

@@ -1,53 +1,31 @@
 # TKET1-Passes
 
-A bridge between C++ TKET1 and Rust land. The folder `cpp` provides a minimal
-C-interface to a subset of TKET1's functionality. The folder `rust` then wraps
-this interface into a Rust-friendly API.
+A bridge between C++ TKET1 and Rust land. The folder `tket-c-api`
+provides a minimal C-interface to a subset of TKET1's functionality. The
+crate in this directory then wraps this interface into a Rust-friendly
+API.
 
-## Building the C++ library
+## Building instructions
 
-This requires standard modern C++ tooling:
-
-- Conan 2.0+
-- CMake 3.23+
-- C++20 compiler
-
-as well as the GMP (GNU Multiple Precision Arithmetic Library). This is usually
-already installed on Linux; on macos you can use `brew install gmp`. You must
-further set up `conan` to use the artifactory containing the tket package:
+Building `tket1-passes` is done using `conan >= 2.0`. After installing `conan`,
+(`pip install conan` or use `pipx`), you must set up `conan` to use the
+artifactory containing the tket package:
 
 ```
 conan remote add tket-libs https://quantinuumsw.jfrog.io/artifactory/api/conan/tket1-libs --index 0
 ```
 
-Finally, if you have not used `conan` before, you will need to set up a profile:
+For the time being, one extra step is required to export `tket-c-api` to the conan cache:
 
 ```
-conan profile detect
-```
-
-Building the `tket1-passes` library is then as simple as running the following script:
-
-```bash
-cd cpp
-./build.sh
+cd tket-c-api && conan create . --build=missing"
 ```
 
-Upon successfully building the C++ library, the compiled library should be found
-in the `lib` directory. The location the library is installed to can be changed
-by setting the `TKET_LIB_PATH` environment variable.
-
-## Building the Rust library
-
-The Rust library can then be built using:
-
-```bash
-cd rust
-cargo build
-```
+That's it! You can now build the library using `cargo build`.
 
-If you are using a non-default library path, make sure to set the `TKET_LIB_PATH`
-environment variable accordingly.
+If conan is unable to fetch all dependencies as pre-compiled binaries, you will
+also need standard C++ tooling to compile the dependencies (i.e. a reasonably
+recent version of cmake and a C++ compiler).
 
 ## Currently supported TKET1 features