Unable to run artillery scenario script with just 1 user

[skerit]

Version info:

Artillery: 2.0.23
Node.js:   v20.13.1
OS:        linux

---

I'm unable to get my artillery script to work properly, and I don't know where it's going wrong.

Here's how I start it:

artillery run --solo --output report-20250630-1528.json myconfig.json

The json config looks like this:

{
    "config": {
        "target": "http://localhost:8049",
        "phases": [
            {
                "name": "Debug tests",
                "arrivalCount": 1,
                "maxDuration": 500
            }
        ],
        "processor": "/home/skerit/project/booking-flow.js",
        "engines": {
            "playwright": {
                "launchOptions": {
                    "headless": true,
                    "timeout": 120000
                }
            }
        },
        "variables": {
            "node_url": "http://localhost:8049",
            "language": "nl",
            "userPrefix": "perfuser",
            "teamSizes": [
                1,
                2,
                3
            ]
        }
    },
    "scenarios": [
        {
            "name": "Booking Flow",
            "engine": "playwright",
            "testFunction": "bookingFlow",
            "weight": 100
        }
    ]
}

The only thing I see in the console is this:

Test run id: ccc
Phase started: unnamed (index: 0, duration: 1s) 15:29:22(+0200)

Phase completed: unnamed (index: 0, duration: 1s) 15:29:23(+0200)
--------------------------------------
Metrics for period to: 15:29:30(+0200) (width: 0s)
--------------------------------------

vusers.created:................................. 1
vusers.created_by_name.Booking Flow:....... 1

I'm trying to debug my playwright script using a single vuser.
So I also tried to run the script without the --solo arg:

artillery run --output report-20250630-1531.json myconfig.json

But now it's somehow creating thousands of vusers (still without running the script!)

Test run id: ccc
Phase started: Debug tests (index: 0, duration: undefined) 15:33:14(+0200)

--------------------------------------
Metrics for period to: 15:33:20(+0200) (width: 4.969s)
--------------------------------------

vusers.created:................................. 1746
vusers.created_by_name.booking_flow:............ 1746


--------------------------------------
Metrics for period to: 15:33:30(+0200) (width: 9.967s)
--------------------------------------

vusers.created:................................. 4279
vusers.created_by_name.booking_flow:............ 4279

That's a lot of playwright instances.
So the only thing that seems to work is to remove the arrivalCount and the --solo argument:

{
	"phases": [
	    {
		"name": "Debug tests",
		"arrivalRate": 1,
		"duration": 100
	    }
	]
}

But now I get 1 new arrival every second.
After a while I then get:

browser.http_requests:.......................... 832
browser.page.CLS.http://localhost:8049/trips:
  min:.......................................... 0
  max:.......................................... 0
  mean:......................................... 0
  median:....................................... 0
  p95:.......................................... 0
  p99:.......................................... 0
browser.page.FCP.http://localhost:8049/bf21:
  min:.......................................... 1228
  max:.......................................... 1632
  mean:......................................... 1436
  median:....................................... 1495.5
  p95:.......................................... 1495.5
  p99:.......................................... 1495.5
browser.page.FCP.http://localhost:8049/login:
  min:.......................................... 436
  max:.......................................... 1660
  mean:......................................... 907.1
  median:....................................... 685.5
  p95:.......................................... 1587.9
  p99:.......................................... 1587.9
browser.page.FID.http://localhost:8049/login:
  min:.......................................... 0.3
  max:.......................................... 1.2
  mean:......................................... 0.7
  median:....................................... 0.6
  p95:.......................................... 0.9
  p99:.......................................... 0.9
browser.page.INP.http://localhost:8049/trips:
  min:.......................................... 56
  max:.......................................... 352
  mean:......................................... 140.8
  median:....................................... 104.6
  p95:.......................................... 120.3
  p99:.......................................... 120.3
browser.page.LCP.http://localhost:8049/login:
  min:.......................................... 528
  max:.......................................... 1756
  mean:......................................... 1035.6
  median:....................................... 889.1
  p95:.......................................... 1755
  p99:.......................................... 1755
browser.page.TTFB.http://localhost:8049/bf21:
  min:.......................................... 114.2
  max:.......................................... 317
  mean:......................................... 241.4
  median:....................................... 267.8
  p95:.......................................... 267.8
  p99:.......................................... 267.8
browser.page.TTFB.http://localhost:8049/login:
  min:.......................................... 35.1
  max:.......................................... 327.4
  mean:......................................... 104.4
  median:....................................... 58.6
  p95:.......................................... 257.3
  p99:.......................................... 257.3
browser.page.codes.200:......................... 803
browser.page.codes.204:......................... 242
browser.page.codes.302:......................... 29
browser.page.codes.403:......................... 14
browser.page.codes.404:......................... 5
vusers.created:................................. 10
vusers.created_by_name.booking_flow:............ 10

I also tried to set the arrivalRate to 1 and the duration to 1, but then the phase immediately ends and once again my script never runs:

Phase started: Debug tests (index: 0, duration: 1s) 15:54:46(+0200)

Phase completed: Debug tests (index: 0, duration: 1s) 15:54:47(+0200)

--------------------------------------
Metrics for period to: 15:54:50(+0200) (width: 0s)
--------------------------------------

vusers.created:................................. 1
vusers.created_by_name.booking_flow:............ 1

So the script works, but somehow I can't test this with only a single vuser!

I saw on another issue that --solo is not documented, so I don't know what its status is (#2706)

I also saw another issue mentioning "Too many vusers created for TypeScript test", but this fix hasn't been released yet (#3546)

[brtak]

Have you tried setting arrivalCount to 1 and set a duration value to 1? (this is how I debug with 1 user in our playwright load tests). Example of my config below. I also change headless to false so I can see what i need to see.

config:
  target: myTargetUrl
  phases:
  - arrivalCount: 1
    duration: 1
    name: Arrival Count Phase 1
  # - arrivalCount: 60
  #   duration: 2m
  #   name: Arrival Count Phase 2
  # - arrivalCount: 120
  #   duration: 2m
  #   name: Arrival Count Phase 3
  engines:
    playwright:
      contextOptions:
        viewport:
          width: 1600
          height: 1200
      trace: true
      aggregateByName: true
      extendedMetrics: true
      launchOptions:
        headless: true
  processor: ../tests/checkout.ts
scenarios:
- engine: playwright
  testFunction: checkout
  name: Artillery_Cart_Checkout