Repositories list

• ukwa-site

  Public
  Using static site generation for parts of the our site.

  SCSS

  •

  Apache License 2.0

  •4•0•12•1•Updated Aug 18, 2025Aug 18, 2025

• webarchive-discovery

  Public
  Please note that the warc-indexer tool & code is now supported by NetArchiveSuite. The 'warc-indexer' directory and code that exists in this repo is now only for reference. For support and issues of 'warc-indexer', please communicate with NetArchiveSuite.

  Java

  •26•128•90•3•Updated Jul 24, 2025Jul 24, 2025

• httpfs

  Public
  Apache Hadoop HttpFS for cdh3

  Java

  •17•1•1•1•Updated Jun 19, 2025Jun 19, 2025

• ukwa-monitor

  Public
  Dashboard and monitoring system for the UK Web Archive

  Python

  •4•0•8•0•Updated Jan 9, 2025Jan 9, 2025

• domain-crawler

  Public
  Shell

  •1•0•0•0•Updated Jan 8, 2025Jan 8, 2025

• ukwa-services

  Public
  Deployment configuration for all UKWA services stacks.

  Python

  •

  Apache License 2.0

  •5•5•41•0•Updated Dec 6, 2024Dec 6, 2024

• ukwa-heritrix

  Public
  The UKWA Heritrix3 custom modules and Docker builder.

  Java

  •7•11•45•1•Updated Dec 2, 2024Dec 2, 2024

• npld-player

  Public
  Secured browser for accessing NPLD content in Legal Deposit Library reading rooms.

  TypeScript

  •

  MIT License

  •1•0•3•0•Updated Sep 27, 2024Sep 27, 2024

• pywb-install

  Public
  Shell

  •0•0•0•0•Updated Aug 29, 2024Aug 29, 2024

• cdx-install

  Public
  Shell

  •0•0•0•0•Updated Aug 1, 2024Aug 1, 2024

• node_exporter-install

  Public
  Shell

  •0•0•0•0•Updated Aug 1, 2024Aug 1, 2024

• crawl-streams

  Public
  Tools for working with UKWA crawler event streams

  Python

  •

  Apache License 2.0

  •3•0•5•5•Updated Jul 6, 2024Jul 6, 2024

• crawl-db

  Public
  A standalone database for crawl events.

  Python

  •

  Apache License 2.0

  •1•1•0•4•Updated Jul 5, 2024Jul 5, 2024

• ukwa-website

  Public
  Static basic UKWA website

  HTML

  •0•0•0•0•Updated Jul 4, 2024Jul 4, 2024

• docker-clamd

  Public
  ClamD in a container

  Dockerfile

  •

  Apache License 2.0

  •2•0•0•0•Updated May 20, 2024May 20, 2024

• docker-airflow

  Public
  Apache Airflow with a few additional dependencies

  Dockerfile

  •

  Apache License 2.0

  •2•0•0•0•Updated Apr 25, 2024Apr 25, 2024

• docker-robot-framework

  Public
  A Dockerised Robot Framework execution environment.

  RobotFramework

  •

  Apache License 2.0

  •2•0•4•0•Updated Apr 24, 2024Apr 24, 2024

• docker-hadoop

  Public
  Hadoop running in a container.

  Dockerfile

  •

  Apache License 2.0

  •2•1•0•0•Updated Apr 24, 2024Apr 24, 2024

• acid-crawl

  Public
  An acid test suite for crawlers.

  PHP

  •3•4•7•0•Updated Apr 24, 2024Apr 24, 2024

• python-w3act

  Public
  Python clients for W3ACT and Heritrix3

  Python

  •4•0•13•2•Updated Apr 19, 2024Apr 19, 2024

• webrender-puppeteer

  Public
  Web page rendering service based on Google's Puppeteer

  JavaScript

  •3•2•1•2•Updated Apr 11, 2024Apr 11, 2024

• ukwa-ui-collections-solr

  Public
  Containerised version of the Solr service used to generate the UKWA UI collections browser

  Python

  •3•0•0•1•Updated Apr 11, 2024Apr 11, 2024

• ukwa-ui

  Public
  A new user interface for the UK Web Archive

  web-archivingweb-archives

  Java

  •

  BSD 3-Clause "New" or "Revised" License

  •6•0•118•3•Updated Apr 10, 2024Apr 10, 2024

• ukwa-notebook-apps

  Public
  UKWA web apps for working with internal APIs, build on Jupyter notebooks and Voila.

  Jupyter Notebook

  •2•0•0•6•Updated Jan 31, 2024Jan 31, 2024

• crawl-log-viewer

  Public
  A simple web service for viewing crawl logs.

  Python

  •

  Apache License 2.0

  •2•0•1•3•Updated Jan 30, 2024Jan 30, 2024

• backstage

  Public
  UI for searching across internal services

  Ruby

  •2•0•1•7•Updated Jan 30, 2024Jan 30, 2024

• docker-pdf2htmlex

  Public
  Run pdf2htmlEX in a Docker container.

  Python

  •

  Apache License 2.0

  •10•25•1•0•Updated Jan 30, 2024Jan 30, 2024

• w3act

  Public
  w3act is an annotation and curation tool for building web archive collections

  Java

  •

  Apache License 2.0

  •6•19•65•0•Updated Jan 30, 2024Jan 30, 2024

• docker-superset

  Public
  Dockerized Apache Superset including Solr module

  Shell

  •1•0•1•0•Updated Nov 16, 2023Nov 16, 2023

• kevals

  Public
  Key-values data aggregator

  Python

  •

  Apache License 2.0

  •1•0•0•0•Updated Nov 16, 2023Nov 16, 2023