Generate a local UUID for requests with a parent UUID for debug purposes

[pmvilaca]

At the moment, if service A needs to make 2 independent calls to service B, according the best-practices, service A should pass the UUID to service B.

That is useful to identify a global transaction but isn't so useful if we need to identify a single request. We can track the requests looking at the logs, timestamps, ... but that could be a problem if we have alerts based on that field.

[eswdd]

There was a suggestion a year ago that we derive new request uuids from passed in ones, such that they become unique but it is still possible to search on the parent. the intention was to append to the string of the parent uuid, but that could conceivably be considered to be an implementation choice of this request.

[pmvilaca]

I think that it's better to generate a new uuid and log that as a new field
instead of appending something to the original uuid.. That would force us
to apply a regex to extract the original uuid.

On Tuesday, January 7, 2014, Simon Matic Langford wrote:

There was a suggestion a year ago that we derive new request uuids from
passed in ones, such that they become unique but it is still possible to
search on the parent. the intention was to append to the string of the
parent uuid, but that could conceivably be considered to be an
implementation choice of this request.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/60#issuecomment-31778529
.

[eswdd]

Won't you have to use regex or some other configuration to extract the uuid anyway?

If we introduce a seperate uuid, then bear in mind that that will only be logged in one place alongside the parent (likely the request log since it's common to all transports). From the PoV of something like Zipkin which is automagically stitching things together and grabbing all available data this is simple, for a human you will always need to join across this single log to trace something, making scripts more complex.

The appending mechanism is actually quite an elegant solution since it helps in a number of styles of usage:

• Want to trace what requests a service/client made because it was slow: just grep for the parent uuid in all logs
• Want to find the source of a request from the access log: just decompose the uuid and grep in the logs to find the parent request.

It's also not dissimilar to a mechanism we have in tracing events through the system - perhaps we should/could consider joining them. If we change the appending style from what I'd originally considered, namely just appending -1, -2, -3 to the current uuid when making a request, to including the hostname like we do for new uuids, then we know the exact location of the source (at least where it's something sharing the uuid scheme - and if it's not at least it will look different).

[pmvilaca]

Fair enough.. But I don't know how could we append the -1, -2, -... in the client side. Most of the times, there are requests to the same endpoints in parallel.

[eswdd]

And yet they all get passed the same RequestUUID - I see 2 choices here:

1. Add a method to the request uuid to get a child one which uses an internal atomic integer or long
2. Make the code create the sub-Requests prior to passing in to the cougar client - more code complexity but more control maybe?

[pmvilaca]

1. We can do that in the code that sends the requests to other services
   (cougar client), assuming that original context is being used for those
   requests but that isn't valid for requests from the sites (I'm talking
   about Mantis). Those requests are creating ExecutionContext instances based
   using the UUID (already generated). Just to clarify.. With this option, you
   are suggesting that we invoke a method before sending the requests just to
   increment the number of the request (-1, -2, -...), right?
2. I'm not sure about what you are trying to say.. Could you elaborate?

2014/1/8 Simon Matic Langford notifications@github.com

And yet they all get passed the same RequestUUID - I see 2 choices here:

1. Add a method to the request uuid to get a child one which uses an
   internal atomic integer or long
2. Make the code create the sub-Requests prior to passing in to the cougar
   client - more code complexity but more control maybe?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/60#issuecomment-31877938
.

[eswdd]

I think you got the jist of it in 1.. the 2 choices here weren't really choices on re-reading, just different parts of the same thing.

I do think we should follow my suggestion to add the host as well as the -1, -2, but only if the uuid that we're creating from is not generated by this host (because there's wasn't one on the request).

e.g. I'm making a new request on host abc001:

• if i received a request with no uuid on it then the uuid will be of the form abc001-xxxxxxxx-yyyy in which case new uuids should be of the form abc001-xxxxxxxx-yyyy-1, abc001-xxxxxxxx-yyyy-2 etc
• if i received a request with a uuid (e.g. xyz001-xxxxxxx-yyyy), then I should also append my host: xyz001-xxxxxxx-yyyy-abc001-1, xyz001-xxxxxxx-yyyy-abc001-2

[pmvilaca]

Sounds like a plan.. :)
+1

[eswdd]

So I started looking at this this morning and there are some probs..

1. Any change in what we send as uuid to legacy servers is going to hit the regexp limiting uuids to 20-50 characters, which is irritating.
2. If we keep chaining info together, we're going to hit massive uuids within a few hops, and at some point we need to set a sensible limit.

Suggested solutions:
2. We can meet the various requirements by having a triple component uuid, namely "root:parent:mine" which then allows us to set a sensible limit of (say) 150 characters.

1. We can add some settings to our various clients which allow us to disable sending the root and parent component part uuids, which can be set until we know that servers have been upgraded.

[eswdd]

I wonder if the extended uuid should be sent in a new header alongside the legacy one, so that we don't have to have a config option - will have to consider impact on binary client..