Feat: Add new binary processor #4222

lacroixthomas · 2025-07-20T22:00:12Z

What type of PR is this?
/kind feature

What this PR does / Why we need it:

Added scaffolding for the new binary "processor"

The processor only logs and have a default leader election (under dev feature flag ProcessorAllocator

Which issue(s) this PR fixes:

Part of #4190

Special notes for your reviewer:

agones-bot · 2025-07-20T22:06:34Z

Build Failed 😭

Build Id: 7c2efa8a-b7d7-4113-86da-7d977bdadcb0

Status: FAILURE

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

markmandel · 2025-07-20T22:07:40Z

Just checking before review - should this be in draft? or is it ready for review?

lacroixthomas · 2025-07-20T22:10:20Z

Just checking before review - should this be in draft? or is it ready for review?

Yep, no need to review, still in draft, still need to do some update and test around the deployment.yaml, adding some default values and also checking the service account, either using the one from the controller or adding a new one, it will need permissions for the leases

markmandel

Quick review 😄

cmd/processor/main.go

cloudbuild.yaml

cmd/processor/Dockerfile

install/helm/agones/templates/processor-deployment.yaml

markmandel

Quick review 😄

markmandel · 2025-07-20T22:11:14Z

Sweet - lemme know if you want to kick it through full CI.

lacroixthomas · 2025-07-20T22:12:05Z

Perfect, will let you know, might be all good next week for review
Thanks for the quick review though, will update that 😄

agones-bot · 2025-07-20T22:42:13Z

Build Failed 😭

Build Id: 57b536ec-ee2e-4bec-8f83-89ebbda6538b

Status: FAILURE

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

…r-election

agones-bot · 2025-07-23T19:55:10Z

Build Succeeded 🥳

Build Id: e18d0f4a-c3f4-4f1b-afb2-d6056814652f

The following development artifacts have been built, and will exist for the next 30 days:

image: us-docker.pkg.dev/agones-images/ci/agones-controller:1.51.0-dev-912dd31
image: us-docker.pkg.dev/agones-images/ci/agones-extensions:1.51.0-dev-912dd31
image: us-docker.pkg.dev/agones-images/ci/agones-sdk:1.51.0-dev-912dd31-linux
image: us-docker.pkg.dev/agones-images/ci/agones-ping:1.51.0-dev-912dd31
image: us-docker.pkg.dev/agones-images/ci/agones-allocator:1.51.0-dev-912dd31
image: us-docker.pkg.dev/agones-images/ci/agones-processor:1.51.0-dev-912dd31
Linux C++ SDK (build): agonessdk-1.51.0-dev-912dd31-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.51.0-dev-912dd31.zip

A preview of the website (the last 30 builds are retained):

https://912dd31-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/4222/head:pr_4222 && git checkout pr_4222
helm install agones ./install/helm/agones --namespace agones-system --set agones.image.registry=us-docker.pkg.dev/agones-images/ci --set agones.image.tag=1.51.0-dev-912dd31

lacroixthomas · 2025-07-23T19:56:36Z

@markmandel
That should be good for review, there is some things as the health check part / metrics, that I would prefer to setup when integrating the allocator inside this agones-processor (from a following PR)

This PR is only to setup the scaffolding of this new binary (with leader election)

agones-bot · 2025-07-24T20:27:35Z

Build Succeeded 🥳

Build Id: 07907aef-3246-4002-9738-af3be0c2e06e

The following development artifacts have been built, and will exist for the next 30 days:

image: us-docker.pkg.dev/agones-images/ci/agones-controller:1.51.0-dev-100c1bb
image: us-docker.pkg.dev/agones-images/ci/agones-extensions:1.51.0-dev-100c1bb
image: us-docker.pkg.dev/agones-images/ci/agones-sdk:1.51.0-dev-100c1bb-linux
image: us-docker.pkg.dev/agones-images/ci/agones-ping:1.51.0-dev-100c1bb
image: us-docker.pkg.dev/agones-images/ci/agones-allocator:1.51.0-dev-100c1bb
image: us-docker.pkg.dev/agones-images/ci/agones-processor:1.51.0-dev-100c1bb
Linux C++ SDK (build): agonessdk-1.51.0-dev-100c1bb-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.51.0-dev-100c1bb.zip

A preview of the website (the last 30 builds are retained):

https://100c1bb-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/4222/head:pr_4222 && git checkout pr_4222
helm install agones ./install/helm/agones --namespace agones-system --set agones.image.registry=us-docker.pkg.dev/agones-images/ci --set agones.image.tag=1.51.0-dev-100c1bb

cmd/processor/main.go

lacroixthomas · 2025-07-25T16:30:39Z

/gcbrun

markmandel · 2025-07-29T02:48:16Z

/gcbrun

markmandel

Looks good - just dropped some nits. 👍🏻

markmandel · 2025-07-29T02:55:40Z

install/helm/agones/values.yaml

@@ -282,6 +286,32 @@ agones:
    totalRemoteAllocationTimeout: 30s
    allocationBatchWaitTime: 500ms
    topologySpreadConstraints: []
+  processor:


I am wondering if this should be a child of agones.allocator ? Since the processor is only actually part of allocation.

So it would be agones.allocator.processor - then it's contextual, and processor doesn't really make sense on it's own. WDYT?

Yep totally make sense, will update that

markmandel · 2025-07-29T03:00:12Z

install/helm/agones/templates/processor-deployment.yaml

@@ -0,0 +1,178 @@
+# Copyright 2025 Google LLC All Rights Reserved.


This file could just be processor.yaml - "deployment" is a bit redundant.

…r-election

lacroixthomas · 2025-07-29T20:11:32Z

/gcbrun

agones-bot · 2025-07-30T21:21:20Z

Build Failed 😭

Build Id: 88b1364a-cab9-4bf2-b365-dbbbde1b6180

Status: FAILURE

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2025-07-30T21:21:20Z

Build Failed 😭

Build Id: b722a1a0-3a0a-4c9a-b50f-ae7c6dfa8101

Status: FAILURE

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

lacroixthomas · 2025-07-31T01:31:52Z

/gcbrun

agones-bot · 2025-07-31T01:47:38Z

Build Failed 😭

Build Id: 09504aa4-d0d0-4755-a845-099defdaa053

Status: FAILURE

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

…r-election

agones-bot · 2025-07-31T02:46:08Z

Build Failed 😭

Build Id: b2bfee6f-fbad-422b-9831-9055d50b7a3f

Status: FAILURE

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

…r-election

agones-bot · 2025-07-31T16:43:17Z

Build Failed 😭

Build Id: 943c99fb-2e84-41ca-b7e9-2f99e227e308

Status: FAILURE

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2025-07-31T18:17:22Z

Build Failed 😭

Build Id: 266a0491-3b34-45ec-8cb7-2bb6edc5a97b

Status: FAILURE

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2025-07-31T19:00:00Z

Build Failed 😭

Build Id: fb50fba0-95d8-48b8-bac2-7fa91f6d3a9d

Status: FAILURE

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2025-07-31T21:05:04Z

Build Succeeded 🥳

Build Id: 6b84e59b-2e4f-4864-8e19-08b6ef3157e2

The following development artifacts have been built, and will exist for the next 30 days:

image: us-docker.pkg.dev/agones-images/ci/agones-controller:1.51.0-dev-77233cb
image: us-docker.pkg.dev/agones-images/ci/agones-extensions:1.51.0-dev-77233cb
image: us-docker.pkg.dev/agones-images/ci/agones-sdk:1.51.0-dev-77233cb-linux
image: us-docker.pkg.dev/agones-images/ci/agones-ping:1.51.0-dev-77233cb
image: us-docker.pkg.dev/agones-images/ci/agones-allocator:1.51.0-dev-77233cb
image: us-docker.pkg.dev/agones-images/ci/agones-processor:1.51.0-dev-77233cb
Linux C++ SDK (build): agonessdk-1.51.0-dev-77233cb-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.51.0-dev-77233cb.zip

A preview of the website (the last 30 builds are retained):

https://77233cb-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/4222/head:pr_4222 && git checkout pr_4222
helm install agones ./install/helm/agones --namespace agones-system --set agones.image.registry=us-docker.pkg.dev/agones-images/ci --set agones.image.tag=1.51.0-dev-77233cb

lacroixthomas · 2025-07-31T21:06:13Z

/gcbrun

agones-bot · 2025-07-31T22:04:10Z

Build Succeeded 🥳

Build Id: 0a25737a-6992-458f-b68d-3a31a5aed635

The following development artifacts have been built, and will exist for the next 30 days:

image: us-docker.pkg.dev/agones-images/ci/agones-controller:1.51.0-dev-77233cb
image: us-docker.pkg.dev/agones-images/ci/agones-extensions:1.51.0-dev-77233cb
image: us-docker.pkg.dev/agones-images/ci/agones-sdk:1.51.0-dev-77233cb-linux
image: us-docker.pkg.dev/agones-images/ci/agones-ping:1.51.0-dev-77233cb
image: us-docker.pkg.dev/agones-images/ci/agones-allocator:1.51.0-dev-77233cb
image: us-docker.pkg.dev/agones-images/ci/agones-processor:1.51.0-dev-77233cb
Linux C++ SDK (build): agonessdk-1.51.0-dev-77233cb-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.51.0-dev-77233cb.zip

A preview of the website (the last 30 builds are retained):

https://77233cb-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/4222/head:pr_4222 && git checkout pr_4222
helm install agones ./install/helm/agones --namespace agones-system --set agones.image.registry=us-docker.pkg.dev/agones-images/ci --set agones.image.tag=1.51.0-dev-77233cb

…r-election

lacroixthomas · 2025-08-01T15:45:09Z

Rolled back the fix of the pipeline to keep it on a separate task

agones-bot · 2025-08-01T16:45:01Z

Build Succeeded 🥳

Build Id: 2d2b4880-b3fc-4a23-93cd-073870a74149

The following development artifacts have been built, and will exist for the next 30 days:

image: us-docker.pkg.dev/agones-images/ci/agones-controller:1.52.0-dev-d77c96d
image: us-docker.pkg.dev/agones-images/ci/agones-extensions:1.52.0-dev-d77c96d
image: us-docker.pkg.dev/agones-images/ci/agones-sdk:1.52.0-dev-d77c96d-linux
image: us-docker.pkg.dev/agones-images/ci/agones-ping:1.52.0-dev-d77c96d
image: us-docker.pkg.dev/agones-images/ci/agones-allocator:1.52.0-dev-d77c96d
image: us-docker.pkg.dev/agones-images/ci/agones-processor:1.52.0-dev-d77c96d
Linux C++ SDK (build): agonessdk-1.52.0-dev-d77c96d-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.52.0-dev-d77c96d.zip

A preview of the website (the last 30 builds are retained):

https://d77c96d-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/4222/head:pr_4222 && git checkout pr_4222
helm install agones ./install/helm/agones --namespace agones-system --set agones.image.registry=us-docker.pkg.dev/agones-images/ci --set agones.image.tag=1.52.0-dev-d77c96d

markmandel

Looking good - just some minor questions.

markmandel · 2025-08-03T21:34:53Z

cmd/processor/main.go

+	whenLeader(ctx, cancelCtx, logger, conf, kubeClient, func(ctx context.Context) {
+		logger.Info("Starting processor work as leader")
+
+		// Simulate processor work (to ensure the leader is working)


(non blocking suggestion)

Suggested change

// Simulate processor work (to ensure the leader is working)

// Simulate processor work (to ensure the leader is working)

// TODO: implement processor work

Just to make it extra clear that more work is coming.

Updated: a9b200f

markmandel · 2025-08-03T21:36:47Z

cmd/processor/main.go

+	signals.NewSigTermHandler(func() {
+		logger.Info("Pod shutdown has been requested, failing readiness check")
+		cancelCtx()
+		time.Sleep(1 * time.Second)


Do we need a minute sleep timer before shutdown? Is this to handle any remaining processing items that are mid flight? (do we care? maybe we do).

If so, should it be configurable like it is in the allocator?

I should probably get rid of it from this PR as it's just the scaffolding, it will indeed be configurable on the implementation of the allocator on it 😄

Updated: a9b200f

install/helm/agones/templates/processor.yaml

…r-election

agones-bot · 2025-08-03T22:48:41Z

Build Failed 😭

Build Id: 51355fed-f058-40f3-8f5b-0ee10106f26a

Status: FAILURE

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

lacroixthomas · 2025-08-04T12:49:28Z

/gcbrun

agones-bot · 2025-08-04T13:57:26Z

Build Failed 😭

Build Id: e4fa21c7-cde5-4051-8e92-372140ef949c

Status: FAILURE

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

…r-election

agones-bot · 2025-08-04T15:53:21Z

Build Succeeded 🥳

Build Id: c43466a0-d7b3-4b73-a025-ab46ea534957

The following development artifacts have been built, and will exist for the next 30 days:

image: us-docker.pkg.dev/agones-images/ci/agones-controller:1.52.0-dev-b089ea4
image: us-docker.pkg.dev/agones-images/ci/agones-extensions:1.52.0-dev-b089ea4
image: us-docker.pkg.dev/agones-images/ci/agones-sdk:1.52.0-dev-b089ea4-linux
image: us-docker.pkg.dev/agones-images/ci/agones-ping:1.52.0-dev-b089ea4
image: us-docker.pkg.dev/agones-images/ci/agones-allocator:1.52.0-dev-b089ea4
image: us-docker.pkg.dev/agones-images/ci/agones-processor:1.52.0-dev-b089ea4
Linux C++ SDK (build): agonessdk-1.52.0-dev-b089ea4-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.52.0-dev-b089ea4.zip

A preview of the website (the last 30 builds are retained):

https://b089ea4-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/4222/head:pr_4222 && git checkout pr_4222
helm install agones ./install/helm/agones --namespace agones-system --set agones.image.registry=us-docker.pkg.dev/agones-images/ci --set agones.image.tag=1.52.0-dev-b089ea4

Feat: Add new binary processor

be7edf1

github-actions bot added size/M kind/feature New features for Agones labels Jul 20, 2025

markmandel reviewed Jul 20, 2025

View reviewed changes

lacroixthomas added 2 commits July 20, 2025 18:16

feat: fix apache year and binary name

e5b619a

feat: fix lint issue

8e64144

lacroixthomas added 2 commits July 23, 2025 14:27

feat: update install and leader election

1f017d1

Merge branch 'main' into features/add-processor-scafolding-with-leade…

912dd31

…r-election

lacroixthomas marked this pull request as ready for review July 23, 2025 19:55

feat: add featureGates in log

100c1bb

miai10 reviewed Jul 25, 2025

View reviewed changes

cmd/processor/main.go Show resolved Hide resolved

cmd/processor/main.go Outdated Show resolved Hide resolved

cmd/processor/main.go Outdated Show resolved Hide resolved

feat: add lease config

838ca62

github-actions bot added the size/L label Jul 25, 2025

lacroixthomas force-pushed the features/add-processor-scafolding-with-leader-election branch from a24a274 to 838ca62 Compare July 25, 2025 18:25

markmandel reviewed Jul 29, 2025

View reviewed changes

lacroixthomas added 2 commits July 29, 2025 08:43

feat: update helm values to embed processor in allocator

c234b28

Merge branch 'main' into features/add-processor-scafolding-with-leade…

741834d

…r-election

Merge branch 'main' into features/add-processor-scafolding-with-leade…

71fd8b4

…r-election

Merge branch 'main' into features/add-processor-scafolding-with-leade…

eb00606

…r-election

lacroixthomas force-pushed the features/add-processor-scafolding-with-leader-election branch from 49b1bac to eb00606 Compare July 31, 2025 19:09

lacroixthomas force-pushed the features/add-processor-scafolding-with-leader-election branch from 77233cb to eb00606 Compare August 1, 2025 15:43

Merge branch 'main' into features/add-processor-scafolding-with-leade…

d77c96d

…r-election

markmandel reviewed Aug 3, 2025

View reviewed changes

feat: add comment about todo for next PR

a9b200f

markmandel approved these changes Aug 3, 2025

View reviewed changes

markmandel enabled auto-merge (squash) August 3, 2025 22:35

Merge branch 'main' into features/add-processor-scafolding-with-leade…

013c35a

…r-election

Merge branch 'main' into features/add-processor-scafolding-with-leade…

b089ea4

…r-election

markmandel merged commit 231024b into googleforgames:main Aug 4, 2025
4 checks passed

lacroixthomas mentioned this pull request Aug 13, 2025

Approver request #4245

Closed

5 tasks

		@@ -0,0 +1,178 @@
		# Copyright 2025 Google LLC All Rights Reserved.

	// Simulate processor work (to ensure the leader is working)
	// Simulate processor work (to ensure the leader is working)
	// TODO: implement processor work

Feat: Add new binary processor #4222

Feat: Add new binary processor #4222

Uh oh!

Conversation

lacroixthomas commented Jul 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

agones-bot commented Jul 20, 2025

Uh oh!

markmandel commented Jul 20, 2025

Uh oh!

lacroixthomas commented Jul 20, 2025

Uh oh!

markmandel left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

markmandel left a comment

Choose a reason for hiding this comment

Uh oh!

markmandel commented Jul 20, 2025

Uh oh!

lacroixthomas commented Jul 20, 2025

Uh oh!

agones-bot commented Jul 20, 2025

Uh oh!

agones-bot commented Jul 23, 2025

Uh oh!

lacroixthomas commented Jul 23, 2025

Uh oh!

agones-bot commented Jul 24, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lacroixthomas commented Jul 25, 2025

Uh oh!

markmandel commented Jul 29, 2025

Uh oh!

markmandel left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lacroixthomas commented Jul 29, 2025

Uh oh!

agones-bot commented Jul 30, 2025

Uh oh!

agones-bot commented Jul 30, 2025

Uh oh!

lacroixthomas commented Jul 31, 2025

Uh oh!

agones-bot commented Jul 31, 2025

Uh oh!

agones-bot commented Jul 31, 2025

Uh oh!

agones-bot commented Jul 31, 2025

Uh oh!

agones-bot commented Jul 31, 2025

Uh oh!

agones-bot commented Jul 31, 2025

Uh oh!

agones-bot commented Jul 31, 2025

Uh oh!

lacroixthomas commented Jul 31, 2025

Uh oh!

agones-bot commented Jul 31, 2025

Uh oh!

lacroixthomas commented Aug 1, 2025

Uh oh!

agones-bot commented Aug 1, 2025

Uh oh!

lacroixthomas commented Jul 20, 2025 •

edited

Loading