eventlog-live

⚠️ This library is under active development. Any part of this repository may change without warning and the package versions do not follow the Package Versioning Policy.

This repository contains a collection of libraries and tools for the live profiling of Haskell applications instrumented with eventlog-socket.

Demo

The following is a screenshow of Grafana which shows live heap profiling statistics coming from the oddball example program.

To run this example for yourself, run the following command from the root of the repository, wait until all containers have started, then navigate to Grafana at localhost:3000, log in using username admin and password admin, and open the heap profiling visualisation under ☰ > Dashboards > Browse then General > Eventlog Heap.

docker compose -f dockerfiles/eventlog-live-otelcol/docker-compose.yml up --build

This Docker Compose configuration builds and runs a number of Docker containers:

• oddball

  The oddball example program, which repeatedly generates and sums large quantities of random numbers.

• eventlog-live-otelcol

  The eventlog-live-otelcol program, which streams eventlog data from oddball to the OpenTelemetry Collector.

• otel/opentelemetry-collector-contrib

  The OpenTelemetry Collector, which streams the data to Prometheus.

• prom/prometheus

  The Prometheus metric processor and database, which acts as a datasource for Grafana.

• grafana/grafana-oss

  The Grafana instance, which visualises the eventlog data.

Getting Started

To use the code in this repository to profile your own application, follow these steps.

Add eventlog-socket as a dependency

The eventlog-socket package is not yet published on Hackage, so you must add it to your cabal.project file as a source repository package:

source-repository-package
  type:     git
  location: https://github.com/well-typed/eventlog-socket
  tag:      1acb92ff60f4bbc87815466f904366ea5078ed9a

Then add eventlog-socket to the build-depends for your application:

executable my-app
  ...
  build-depends:
    ...
    , eventlog-socket  >=0.1.0 && <0.2
    ...

Instrument your application

To instrument your application, and allow the eventlog data to be streamed over a socket, all you have to do is call GHC.Eventlog.Socket.start with the path to your eventlog socket.

module Main where

import           Data.Maybe (fromMaybe)
import qualified GHC.Eventlog.Socket
import           System.Environment (lookupEnv)

main :: IO ()
main = do
  putStrLn "Creating eventlog socket..."
  eventlogSocket <-
      fromMaybe "/tmp/ghc_eventlog.sock"
          <$> lookupEnv "GHC_EVENTLOG_SOCKET"
  GHC.Eventlog.Socket.start eventlogSocket
  ...

If you wish for your application to block until the client process connects to the eventlog socket, you can call GHC.Eventlog.Socket.startWait.

Build your application with profiling

To enable runtime configuration of various eventlog options, you must compile your application with the -rtsopts GHC option.

executable my-app
  ...
  ghc-options: -rtsopts
  ...

Since GHC 9.4, eventlog profiling is enabled by default. However, if you are using GHC 9.2 or earlier, you must also enable eventlog profiling at compile time by passing the -eventlog GHC option.

executable my-app
  ...
  if impl(ghc < 9.4)
    ghc-options: -eventlog
  ...

To ensure that the RTS flushes the events queue consistently and every second, you can pass the --eventlog-flush-interval=1 and --no-automatic-heap-samples RTS options at runtime. If you do, you must compile your application with -threaded. See GHC issue #26222 for further details.

executable my-app
  ...
  ghc-options: -threaded
  ...

The program eventlog-live-influxdb requires that your eventlog is written in binary form, which requires the -l RTS option. The heap statistics visualisation in the demo requires that your application is run with heap profiling enabled, which requires the -hT RTS option. You can pass these options at runtime with:

./my-app +RTS -l -hT --eventlog-flush-interval=1 --no-automatic-heap-samples

Alternatively, you can set these options at compile time with:

executable my-app
  ...
  ghc-options: -rtsopts "-with-rtsopts=-l -hT --eventlog-flush-interval=1 --no-automatic-heap-samples"
  ...

Putting it all together

To visualise the profiling data of your instrumented application, you must connect it to the demo system. The Docker Compose configuration in dockerfiles/eventlog-live-otelcol/docker-compose-external.yml sets up the same infrastructure used in the demo without the example program. To use it, follow these steps:

1. Start the containers with the OpenTelemetry Collector, Prometheus, and Grafana:

   docker compose -f dockerfiles/eventlog-live-otelcol/docker-compose-external.yaml up --build -d

2. Set the GHC_EVENTLOG_SOCKET environment variable:

   export GHC_EVENTLOG_SOCKET="/tmp/oddball_eventlog.sock"

3. Start your instrumented application:

   ./my-app +RTS -l -hT --eventlog-flush-interval=1 --no-automatic-heap-samples

4. Start eventlog-live-otelcol:

   eventlog-live-otelcol \
     --eventlog-socket "$GHC_EVENTLOG_SOCKET" \
     -hT \
     --otelcol-host=localhost