You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+27-29Lines changed: 27 additions & 29 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,9 +1,7 @@
1
1
## Development Status
2
-
We are actively working on version 0.9 that will offer several improvements over both the API and module
3
-
implementations. 0.8.x is being used in several projects currently, and we will continue to enhance
4
-
0.8.x for the foreseeable future while 0.9 work takes place.
5
-
6
-
If you want to work on 0.9, feel free to reach out!
2
+
Released 0.9.0! A lot of updates here. Many things are not backward compatible
3
+
with 0.8.x, in particular the configuration. See the `sample` for information
4
+
on usage and configuration changes.
7
5
8
6
### Availability
9
7
We have requests in to deploy to a public repo, that should be available shortly
@@ -14,20 +12,20 @@ written by Michael Bevilacqua-Linn discusses our experiences at Comcast implemen
14
12
15
13
## Money: Distributed Tracing Made Simple
16
14
### Money makes it simple to trace across threads and systems
17
-
Money is a modular distributed tracing platform that can be seamlessly incorporated into modern applications. *It's
15
+
Money is a modular distributed tracing platform that can be seamlessly incorporated into modern applications. *It's
18
16
purpose is to provide a foundation for operational analytics through distributed tracing.*
19
17
20
18
Money is built on Scala and Akka, to be non-blocking from the core. It is purposefully un-opinionated, keeping undesired pull-through dependencies to a minimum.
21
19
22
20
Money modules build on the core, so implementing tracing is a snap. From Apache Http Components to Spring 4, from thread pools to Scala Futures, Money has modules to support a wide range of architectures.
23
21
24
22
## Why is it different?
25
-
Money was inspired by inspired by [Google Dapper](http://research.google.com/pubs/pub36356.html)
23
+
Money was inspired by inspired by [Google Dapper](http://research.google.com/pubs/pub36356.html)
26
24
and [Twitter Zipkin](http://twitter.github.io/zipkin/); however there are some subtle yet fundamental differences
27
25
between those systems and Money, the biggest one being...
28
26
29
27
### Spans start and end within a single process
30
-
In Dapper, a Span can encompass the communication between a client and a server. Let's use an example of an
28
+
In Dapper, a Span can encompass the communication between a client and a server. Let's use an example of an
31
29
*Order System* calling an *Inventory Management System*. With Dapper, you could have the following:
32
30
33
31
-*span-name* - GetInventory
@@ -40,8 +38,8 @@ In Dapper, a Span can encompass the communication between a client and a server.
40
38
41
39
The idea being that everything can be calculated when the data is at rest.
42
40
43
-
In Money, it is theoretically possible to do the same, but by default we always extend a span when the server
44
-
starts processing. We do this because we like to record important notes by default, namely the *span-duration* and
41
+
In Money, it is theoretically possible to do the same, but by default we always extend a span when the server
42
+
starts processing. We do this because we like to record important notes by default, namely the *span-duration* and
45
43
the *span-success*. Using the example above, with Money we would get:
46
44
47
45
-*app-name* - OrderSystem
@@ -74,59 +72,59 @@ more data will be generated
74
72
Here are some advantages:
75
73
76
74
- You can use Money as a basis for operational analytics. Money generates 3 important stats: latency, throughput and error rate.
77
-
We find that these are the most important stats in analyzing system performance and capacity. You could calculate
75
+
We find that these are the most important stats in analyzing system performance and capacity. You could calculate
78
76
these at rest in Dapper; but we believe that these can be understood more quickly to support real-time monitoring
79
77
- You can very quickly integrate with monitoring tools. As opposed to having to calculate metrics at rest in Dapper and Zipkin,
80
78
you can publish span data directly to Graphite (for instance), and create charts immediately. This is a very strong advantage
81
-
to calculating metrics at the source. Money actually comes with a *Graphite Emtitter* out-of-the box. Given that
79
+
to calculating metrics at the source. Money actually comes with a *Graphite Emtitter* out-of-the box. Given that
82
80
money is pluggable, it is quite trivial to integrate with many other systems.
83
-
- In distributed system calls, it is important to know that the server "succeeded" or "failed", allowing you to more quickly
84
-
focus your troubleshooting efforts when things go wrong. For example, if the server succeeded, then we can direct our
81
+
- In distributed system calls, it is important to know that the server "succeeded" or "failed", allowing you to more quickly
82
+
focus your troubleshooting efforts when things go wrong. For example, if the server succeeded, then we can direct our
85
83
efforts toward the client or the network
86
84
- Less calculation is needed on the data - we have basic measures that we can use for calculations, allowing us
87
85
to process data and perform analytics faster
88
86
89
87
### We don't sample...yet
90
88
I can see rocks starting to fly here, and I understand. Money was not built in its present incarnation to support
91
-
systems which generate 10s of millions of events per second. Money was built to use distributed tracing as a foundation
89
+
systems which generate 10s of millions of events per second. Money was built to use distributed tracing as a foundation
92
90
for operational analytics.
93
-
91
+
94
92
We were much more interested in creating a standard set of metrics that we could use as a basis to perform operational analytics.
95
93
As such, for us, every event does matter as we can build aggregates very easy (even success).
96
94
97
-
We have been able to instrument systems that do generate many millions of events per hour with success, but Money did
98
-
not have the same considerations that went into the Dapper design. Being able to process our base metrics gets us
99
-
closer to real-time understanding of processing; distributing the calculations to the origin systems gave us a lot of
95
+
We have been able to instrument systems that do generate many millions of events per hour with success, but Money did
96
+
not have the same considerations that went into the Dapper design. Being able to process our base metrics gets us
97
+
closer to real-time understanding of processing; distributing the calculations to the origin systems gave us a lot of
100
98
flexibility and speed in processing the data at rest.
101
99
102
100
> We are committed to support sampling and are evaluating designs...ideas are welcome. Look for basic sampling to be added shortly
103
101
104
102
### We do not provide a backend like Zipkin
105
103
Zipkin comes with an entire infrastructure built around Scribe and Cassandra that actually allows you to see data. This
106
-
is super cool, and something we aspire to complete. We have looked at Spark Streaming and Akka Cluster Sharding as
104
+
is super cool, and something we aspire to complete. We have looked at Spark Streaming and Akka Cluster Sharding as
107
105
implementation mechanisms (and have some prototype / experimental code to that end), but we have not yet gotten our act
108
106
together.
109
107
110
-
One of the main advantages of Money is that it provides usable operational analytics out-of-the-box; whether
108
+
One of the main advantages of Money is that it provides usable operational analytics out-of-the-box; whether
111
109
reporting to Graphite, exposing data via JMX, and/or aggregating logs in Logstash. As such, we have been able to gain
112
110
key insight into traces and performance using standard open source tools; here are some examples:
113
111
114
-
- Splunk / LogStash - we use log aggregators to perform metrics across systems at rest. The nice thing about log
115
-
aggregators is that you can not only see your metrics, but you can also look at all of the log entries that happend
112
+
- Splunk / LogStash - we use log aggregators to perform metrics across systems at rest. The nice thing about log
113
+
aggregators is that you can not only see your metrics, but you can also look at all of the log entries that happend
116
114
for a single Span
117
115
- Graphite / Prometheus - we use common analytic systems for monitoring and altering on Money data
118
116
119
117
## Should I use Money?
120
118
This depends on the scale of your implementation. Money _tries_ to serve a wide range of implementations.
121
119
122
-
Certainly, if you want to implement an application that is serving 1000s or 10000s of request per second per JVM, Money
120
+
Certainly, if you want to implement an application that is serving 1000s or 10000s of request per second per JVM, Money
123
121
should work for you. You can easily funnel data into your log aggregator or other reporting system and start
124
122
getting the benefits immediately.
125
123
126
-
If your implementation is in the order of 50000+ RPS with lots of spans, then things will get difficult
124
+
If your implementation is in the order of 50000+ RPS with lots of spans, then things will get difficult
127
125
as you will have to manage a lot of data. Spooling span events to disk and sending them as you can is one approach.
128
126
You can use FluentD, Heka, PipeD or something else to eventually get the data off of disk. Theoretically it is possible,
129
-
but without sampling, Money is generating a ton of data. If you are not using that data for analytics, you can
127
+
but without sampling, Money is generating a ton of data. If you are not using that data for analytics, you can
130
128
filter it out (or contribute back a sampling feature); either way, it becomes a challenge.
131
129
132
130
## I don't need no docs, gimme Money!
@@ -136,10 +134,10 @@ Add a dependency as follows for maven:
0 commit comments