dmq: node-to-client applications #5173

crocodile-dentist · 2025-07-29T13:28:56Z

Description

-local message submission
-local message notification

todo haddocks, tests

Checklist

Quality

Commit sequence makes sense and have useful messages, see ref.
New tests are added and existing tests are updated.
Self-reviewed the PR.

Maintenance

Linked an issue or added the PR to the current sprint of ouroboros-network project.
Added labels.
Updated changelog files.
The documentation has been properly updated, see ref.

coot

Please add codec tests to this PR. Even if we change the protocols, they won't change that much (just small changes in the generators).

decentralized-message-queue/src/DMQ/Protocol/LocalMsgNotification/Type.hs

coot · 2025-07-30T08:18:20Z

decentralized-message-queue/src/DMQ/Protocol/LocalMsgNotification/Type.hs

+    -- | Termination message, initiated by the server when the client is
+    -- making a blocking call for more messages.
+    --
+    MsgServerDone
+      :: Message (LocalMsgNotification msg) (StBusy StBlocking) StDone


We should discuss whether this message is needed at all.

Yes, I have doubts as well.

This may be used during server shutdown.

decentralized-message-queue/src/DMQ/Protocol/LocalMsgNotification/Type.hs

decentralized-message-queue/src/DMQ/Protocol/LocalMsgNotification/Server.hs

decentralized-message-queue/src/DMQ/Protocol/LocalMsgNotification/Codec.hs

decentralized-message-queue/src/DMQ/Protocol/LocalMsgSubmission/Client.hs

decentralized-message-queue/src/DMQ/Protocol/LocalMsgSubmission/Server.hs

decentralized-message-queue/src/DMQ/Protocol/LocalMsgNotification/Type.hs

coot · 2025-08-04T09:38:02Z

decentralized-message-queue/src/DMQ/Protocol/SigSubmission/Codec.hs

+encodeSigId :: SigId -> CBOR.Encoding
+encodeSigId SigId { getSigId } = CBOR.encodeBytes (getSigHash getSigId)
+
+
+decodeSigId :: forall s. CBOR.Decoder s SigId
+decodeSigId = SigId . SigHash <$> CBOR.decodeBytes
+
+
+encodeSig :: Sig -> CBOR.Encoding
+encodeSig Sig { sigId,
+                sigBody,
+                sigExpiresAt,
+                sigKesSignature,
+                sigOpCertificate
+              }
+   = CBOR.encodeListLen 5
+  <> encodeSigId sigId
+  <> CBOR.encodeBytes (getSigBody sigBody)
+  <> CBOR.encodeWord32 (floor sigExpiresAt)
+  <> CBOR.encodeBytes (getSigKesSignature sigKesSignature)
+  <> CBOR.encodeBytes (getSigOpCertificate sigOpCertificate)
+
+
+decodeSig :: forall s. CBOR.Decoder s Sig
+decodeSig = do
+  a <- CBOR.decodeListLen
+  when (a /= 5) $ fail (printf "codecSigSubmission: unexpected number of parameters %d" a)
+  sigId <- decodeSigId
+  sigBody <- SigBody <$> CBOR.decodeBytes
+  sigExpiresAt <- realToFrac <$> CBOR.decodeWord32
+  sigKesSignature <- SigKesSignature <$> CBOR.decodeBytes
+  sigOpCertificate <- SigOpCertificate <$> CBOR.decodeBytes
+  return Sig {
+      sigId,
+      sigBody,
+      sigExpiresAt,
+      sigKesSignature,
+      sigOpCertificate
+    }


Do we have tests for these?

decentralized-message-queue/test/Test/DMQ/Protocol/LocalMsgNotification.hs

decentralized-message-queue/decentralized-message-queue.cabal

coot · 2025-08-08T08:00:47Z

decentralized-message-queue/decentralized-message-queue.cabal

@@ -74,16 +96,19 @@ library
    ouroboros-network-framework ^>=0.19,
    ouroboros-network-protocols ^>=0.15,
    random ^>=1.2,
+    singletons,
+    serialise,


Can we avoid serialise constraint in the dependency tree of dmq-exe?

decentralized-message-queue/src/DMQ/NodeToClient.hs

decentralized-message-queue/src/DMQ/NtC_Applications/LocalMsgNotification.hs

coot · 2025-08-08T08:27:25Z

decentralized-message-queue/src/DMQ/NtC_Applications/LocalMsgNotification.hs

+              return $ ServerReply (BlockingReply (NonEmpty.fromList msgs))
+                                   DoesNotHaveMore
+                                   (serverIdle undefined lastIdx')


We're sending all at once (the same below in the non-blocking case), but this will likely change once we speak with the mithril team.

I've floated the idea on the slack channel to have the max number of messages a negotiated parameter in the handshake. We can impose some sane hard limit on top in addition of course, and this flexibility would be acceptable for a local protocol I think.

coot · 2025-08-08T08:36:53Z

decentralized-message-queue/src/DMQ/NtC_Applications/LocalMsgSubmission.hs

+        maybe failure success . listToMaybe <$> mempoolAddTxs [msg]
+        -- case addTxRes of
+        --   MempoolTxAdded _tx             -> return (SubmitSuccess, server)
+        --   MempoolTxRejected _tx addTxErr -> return (SubmitFail addTxErr, server)


local msg notif test

crocodile-dentist self-assigned this Jul 29, 2025

crocodile-dentist requested review from a team as code owners July 29, 2025 13:28

crocodile-dentist added the dmq-node Issues / PRs related to DMQ Node label Jul 29, 2025

github-project-automation bot added this to Ouroboros Network Jul 29, 2025

github-project-automation bot moved this to In Progress in Ouroboros Network Jul 29, 2025

crocodile-dentist force-pushed the mwojtowicz/coot/dmq-node-to-node branch 3 times, most recently from 93d68e0 to 86a6d5b Compare July 29, 2025 17:08

coot requested changes Jul 30, 2025

View reviewed changes

crocodile-dentist force-pushed the mwojtowicz/coot/dmq-node-to-node branch 3 times, most recently from ff6004b to fc7da51 Compare July 31, 2025 15:34

crocodile-dentist marked this pull request as draft July 31, 2025 15:35

crocodile-dentist force-pushed the mwojtowicz/coot/dmq-node-to-node branch 2 times, most recently from 05da6c0 to 4c9c0a4 Compare August 1, 2025 08:08

coot reviewed Aug 4, 2025

View reviewed changes

decentralized-message-queue/src/DMQ/Protocol/LocalMsgNotification/Type.hs Outdated Show resolved Hide resolved

coot reviewed Aug 4, 2025

View reviewed changes

crocodile-dentist force-pushed the mwojtowicz/coot/dmq-node-to-node branch from 4c9c0a4 to 7858cd3 Compare August 4, 2025 14:11

coot force-pushed the coot/dmq-node-to-node branch from c7e804a to 880441d Compare August 5, 2025 07:43

coot linked an issue Aug 6, 2025 that may be closed by this pull request

Implement N2C mithril mini-protocols #5178

Open

crocodile-dentist force-pushed the mwojtowicz/coot/dmq-node-to-node branch 6 times, most recently from cf95d94 to dcd0e7a Compare August 7, 2025 13:21

coot force-pushed the coot/dmq-node-to-node branch from 4c1e720 to 0274efd Compare August 7, 2025 14:49

crocodile-dentist force-pushed the mwojtowicz/coot/dmq-node-to-node branch 2 times, most recently from 704c4ad to 357bd29 Compare August 7, 2025 16:17

crocodile-dentist force-pushed the mwojtowicz/coot/dmq-node-to-node branch from 357bd29 to d5067f1 Compare August 7, 2025 16:19

coot reviewed Aug 8, 2025

View reviewed changes

coot force-pushed the coot/dmq-node-to-node branch 5 times, most recently from 76bd86d to f10d698 Compare August 8, 2025 13:06

crocodile-dentist force-pushed the mwojtowicz/coot/dmq-node-to-node branch from 5a069e5 to ecc97d5 Compare August 8, 2025 13:16

crocodile-dentist changed the title ~~DMQ local miniprotocol definitions~~ dmq: node-to-client applications Aug 8, 2025

coot force-pushed the coot/dmq-node-to-node branch 4 times, most recently from cf10ee9 to 6fc67a3 Compare August 8, 2025 15:21

Base automatically changed from coot/dmq-node-to-node to main August 8, 2025 16:27

crocodile-dentist added 6 commits August 19, 2025 16:07

dmq: local message submission

994ca9e

dmq: local message notification

244b9cf

testlib: lift distinctlist

3899f81

dmq: local protocols tests

0839b84

local msg notif test

dmq: NtC applications

080e74f

sig-submission: export

a7e9ca5

crocodile-dentist force-pushed the mwojtowicz/coot/dmq-node-to-node branch from ecc97d5 to a7e9ca5 Compare August 19, 2025 14:09

dmq: node-to-client applications #5173

Are you sure you want to change the base?

dmq: node-to-client applications #5173

Conversation

crocodile-dentist commented Jul 29, 2025

Description

Checklist

Quality

Maintenance

Uh oh!

coot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!