Skip to content

feat(observability): micrometer telemetry provider #1089

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions gradle/libs.versions.toml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ otel-version = "1.32.0"
slf4j-version = "2.0.9"
slf4j-v1x-version = "1.7.36"
crt-kotlin-version = "0.8.5"
micrometer-version = "1.12.5"

# codegen
smithy-version = "1.47.0"
Expand Down Expand Up @@ -59,6 +60,7 @@ slf4j-api = { module = "org.slf4j:slf4j-api", version.ref = "slf4j-version" }
slf4j-api-v1x = { module = "org.slf4j:slf4j-api", version.ref = "slf4j-v1x-version" }
slf4j-simple = { module = "org.slf4j:slf4j-simple", version.ref = "slf4j-version" }
crt-kotlin = { module = "aws.sdk.kotlin.crt:aws-crt-kotlin", version.ref = "crt-kotlin-version" }
micrometer-core = { module = "io.micrometer:micrometer-core", version.ref = "micrometer-version" }

smithy-codegen-core = { module = "software.amazon.smithy:smithy-codegen-core", version.ref = "smithy-version" }
smithy-cli = { module = "software.amazon.smithy:smithy-cli", version.ref = "smithy-version" }
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
public final class aws/smithy/kotlin/runtime/telemetry/micrometer/MicrometerTelemetryProvider : aws/smithy/kotlin/runtime/telemetry/TelemetryProvider {
public fun <init> (Lio/micrometer/core/instrument/MeterRegistry;Laws/smithy/kotlin/runtime/telemetry/logging/LoggerProvider;)V
public synthetic fun <init> (Lio/micrometer/core/instrument/MeterRegistry;Laws/smithy/kotlin/runtime/telemetry/logging/LoggerProvider;ILkotlin/jvm/internal/DefaultConstructorMarker;)V
public fun getContextManager ()Laws/smithy/kotlin/runtime/telemetry/context/ContextManager;
public fun getLoggerProvider ()Laws/smithy/kotlin/runtime/telemetry/logging/LoggerProvider;
public fun getMeterProvider ()Laws/smithy/kotlin/runtime/telemetry/metrics/MeterProvider;
public fun getTracerProvider ()Laws/smithy/kotlin/runtime/telemetry/trace/TracerProvider;
}

Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
/*
* Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
* SPDX-License-Identifier: Apache-2.0
*/
description = "Telemetry provider based on MicroMeter"
extra["displayName"] = "Smithy :: Kotlin :: Observability :: MicroMeter Provider"
extra["moduleName"] = "aws.smithy.kotlin.runtime.telemetry.micrometer"

kotlin {
sourceSets {
commonMain {
dependencies {
api(project(":runtime:observability:telemetry-api"))
implementation(project(":runtime:observability:telemetry-defaults"))
}
}

jvmMain {
dependencies {
api(libs.micrometer.core)
}
}
all {
languageSettings.optIn("aws.smithy.kotlin.runtime.InternalApi")
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,211 @@
package aws.smithy.kotlin.runtime.telemetry.micrometer

import aws.smithy.kotlin.runtime.collections.AttributeKey
import aws.smithy.kotlin.runtime.collections.Attributes
import aws.smithy.kotlin.runtime.telemetry.context.Context
import aws.smithy.kotlin.runtime.telemetry.metrics.AsyncMeasurementHandle
import aws.smithy.kotlin.runtime.telemetry.metrics.DoubleAsyncMeasurement
import aws.smithy.kotlin.runtime.telemetry.metrics.DoubleGaugeCallback
import aws.smithy.kotlin.runtime.telemetry.metrics.DoubleHistogram
import aws.smithy.kotlin.runtime.telemetry.metrics.LongAsyncMeasurement
import aws.smithy.kotlin.runtime.telemetry.metrics.LongGaugeCallback
import aws.smithy.kotlin.runtime.telemetry.metrics.LongHistogram
import aws.smithy.kotlin.runtime.telemetry.metrics.LongUpDownCounterCallback
import aws.smithy.kotlin.runtime.telemetry.metrics.Meter
import aws.smithy.kotlin.runtime.telemetry.metrics.MeterProvider
import aws.smithy.kotlin.runtime.telemetry.metrics.MonotonicCounter
import aws.smithy.kotlin.runtime.telemetry.metrics.UpDownCounter
import io.micrometer.core.instrument.DistributionSummary
import io.micrometer.core.instrument.MeterRegistry
import io.micrometer.core.instrument.Tag
import io.micrometer.core.instrument.Tags
import io.micrometer.core.instrument.Counter as MicrometerCounter
import io.micrometer.core.instrument.Gauge as MicrometerGauge

internal class MicrometerMeterProvider(meterRegistry: MeterRegistry) : MeterProvider {
private val meter = MicrometerMeter(meterRegistry)

override fun getOrCreateMeter(scope: String): Meter = meter
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: The scope parameter is intended to delineate which component of the SDK is under observation by specific meters. For instance, meters related to individual calls are given the scope aws.sdk.kotlin.<service-name> whereas meters related to the default HTTP engine are given the scope aws.smithy.kotlin.runtime.http.engine.okhttp. Could the scope be used here, perhaps as tags attached to newly-created meters?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

}

private class MicrometerMeter(
private val meterRegistry: MeterRegistry,
) : Meter {
override fun createUpDownCounter(name: String, units: String?, description: String?): UpDownCounter =
MicrometerUpDownCounter(
meterMetadata = MeterMetadata(name, units, description),
meterRegistry = meterRegistry,
)

override fun createAsyncUpDownCounter(
name: String,
callback: LongUpDownCounterCallback,
units: String?,
description: String?,
): AsyncMeasurementHandle =
MicrometerLongGauge(
callback = callback,
meterMetadata = MeterMetadata(name, units, description),
meterRegistry = meterRegistry,
)

override fun createMonotonicCounter(name: String, units: String?, description: String?): MonotonicCounter =
MicrometerMonotonicCounter(
meterMetadata = MeterMetadata(name, units, description),
meterRegistry = meterRegistry,
)

override fun createLongHistogram(name: String, units: String?, description: String?): LongHistogram =
MicrometerLongHistogram(
MicrometerDoubleHistogram(
meterMetadata = MeterMetadata(name, units, description),
meterRegistry = meterRegistry,
)
)

override fun createDoubleHistogram(name: String, units: String?, description: String?): DoubleHistogram =
MicrometerDoubleHistogram(
meterMetadata = MeterMetadata(name, units, description),
meterRegistry = meterRegistry,
)

override fun createLongGauge(
name: String,
callback: LongGaugeCallback,
units: String?,
description: String?,
): AsyncMeasurementHandle = MicrometerLongGauge(
callback = callback,
meterMetadata = MeterMetadata(name, units, description),
meterRegistry = meterRegistry,
)

override fun createDoubleGauge(
name: String,
callback: DoubleGaugeCallback,
units: String?,
description: String?,
): AsyncMeasurementHandle = MicrometerDoubleGauge(
callback = callback,
meterMetadata = MeterMetadata(name, units, description),
meterRegistry = meterRegistry,
)

}

private data class MeterMetadata(val meterName: String, val units: String?, val description: String?)

private class MicrometerUpDownCounter(
private val meterMetadata: MeterMetadata,
private val meterRegistry: MeterRegistry,
) : UpDownCounter {
override fun add(value: Long, attributes: Attributes, context: Context?) {
meterMetadata
.counter()
.tags(attributes.toTags())
.register(meterRegistry)
.increment(value.toDouble())
}
}
Comment on lines +106 to +117
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correctness: The UpDownCounter type accepts both increments and decrements (e.g., something like queue length which could rise/fall over time). Micrometers' Counter seems to accept only positive increments. What will happen if MicrometerUpDownCounter.add gets called with a negative value? If the underlying Micrometer implementation would throw an exception or otherwise misbehave then it's better to make UpDownCounter a no-op implementation (i.e., createUpDownCounter returns a dummy object that does nothing when add is called).

Copy link
Contributor Author

@monosoul monosoul Jun 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would work fine, you can use this snippet to check it yourself with a Kotlin scratch file in IntelliJ:

import io.micrometer.core.instrument.simple.SimpleMeterRegistry

val registry = SimpleMeterRegistry()
val counter = registry.counter("test.counter")

counter.increment(3.0)
println(counter.count())

counter.increment(-2.0)
println(counter.count())

Will produce:

3.0
1.0


private class MicrometerMonotonicCounter(
private val meterMetadata: MeterMetadata,
private val meterRegistry: MeterRegistry,
) : MonotonicCounter {
override fun add(value: Long, attributes: Attributes, context: Context?) {
meterMetadata
.counter()
.tags(attributes.toTags())
.register(meterRegistry)
.increment(value.toDouble())
}
}

private class MicrometerDoubleHistogram(
private val meterMetadata: MeterMetadata,
private val meterRegistry: MeterRegistry,
) : DoubleHistogram {
override fun record(value: Double, attributes: Attributes, context: Context?) {
DistributionSummary.builder(meterMetadata.meterName)
.baseUnit(meterMetadata.units)
.description(meterMetadata.description)
.tags(attributes.toTags())
.publishPercentileHistogram()
.register(meterRegistry)
}
}

private class MicrometerLongHistogram(
private val doubleHistogram: MicrometerDoubleHistogram,
) : LongHistogram {
override fun record(value: Long, attributes: Attributes, context: Context?) {
doubleHistogram.record(value.toDouble(), attributes, context)
}
}

private class MicrometerGaugeDoubleAsyncMeasurement : DoubleAsyncMeasurement {
private var _value: Double? = null
private var _tags: Tags? = null
val value: Double get() = checkNotNull(_value) { "The value has not yet been measured" }
val tags: Tags get() = checkNotNull(_tags) { "The value has not yet been measured" }

override fun record(value: Double, attributes: Attributes, context: Context?) {
_value = value
_tags = attributes.toTags()
}
}

private class MicrometerDoubleGauge(
private val meterMetadata: MeterMetadata,
private val meterRegistry: MeterRegistry,
private val callback: DoubleGaugeCallback,
) : AsyncMeasurementHandle {

private val gauge = MicrometerGaugeDoubleAsyncMeasurement()
.apply(callback)
.let { measurement ->
MicrometerGauge
.builder(meterMetadata.meterName) { measurement.apply(callback).value }
.baseUnit(meterMetadata.units)
.description(meterMetadata.description)
.tags(measurement.tags)
.strongReference(true)
.register(meterRegistry)
}

override fun stop() {
gauge.close()
meterRegistry.remove(gauge)
}
}

private class MicrometerGaugeLongAsyncMeasurement(
private val doubleAsyncMeasurement: DoubleAsyncMeasurement,
) : LongAsyncMeasurement {
override fun record(value: Long, attributes: Attributes, context: Context?) {
doubleAsyncMeasurement.record(value.toDouble(), attributes, context)
}
}

private class MicrometerLongGauge(
private val callback: LongGaugeCallback,
meterMetadata: MeterMetadata,
meterRegistry: MeterRegistry,
) : AsyncMeasurementHandle {

private val gauge = MicrometerDoubleGauge(meterMetadata, meterRegistry) {
callback.invoke(MicrometerGaugeLongAsyncMeasurement(it))
}

override fun stop() = gauge.stop()
}

private fun MeterMetadata.counter() = MicrometerCounter.builder(meterName)
.baseUnit(units)
.description(description)

@Suppress("UNCHECKED_CAST")
private fun Attributes.toTags() = keys.mapNotNull {
val attributeKey = it as? AttributeKey<Any> ?: return@mapNotNull null
Tag.of(attributeKey.name, getOrNull(attributeKey).toString())
}.let(Tags::of)
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
package aws.smithy.kotlin.runtime.telemetry.micrometer

import aws.smithy.kotlin.runtime.ExperimentalApi
import aws.smithy.kotlin.runtime.telemetry.GlobalTelemetryProvider
import aws.smithy.kotlin.runtime.telemetry.TelemetryProvider
import aws.smithy.kotlin.runtime.telemetry.context.ContextManager
import aws.smithy.kotlin.runtime.telemetry.logging.LoggerProvider
import aws.smithy.kotlin.runtime.telemetry.metrics.MeterProvider
import aws.smithy.kotlin.runtime.telemetry.trace.TracerProvider
import io.micrometer.core.instrument.MeterRegistry

@ExperimentalApi
public class MicrometerTelemetryProvider(
meterRegistry: MeterRegistry,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: Would it make sense for meterRegistry to default to Metrics.globalRegistry?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, why not

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

override val loggerProvider: LoggerProvider = GlobalTelemetryProvider.instance.loggerProvider,
) : TelemetryProvider {
override val meterProvider: MeterProvider = MicrometerMeterProvider(meterRegistry)
override val tracerProvider: TracerProvider = TracerProvider.None
override val contextManager: ContextManager = ContextManager.None
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like Micrometer supports more than just metrics, it also supports tracing (either directly or via bridges to other providers like OTel and Brave) and context propagation—which seem like natural fits for new TracingProvider and ContextManager implementations. Have you looked yet at what it would take to add support for these?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At my org we use another tool for tracing, so atm I'm only interested in metrics

1 change: 1 addition & 0 deletions settings.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ include(":runtime:observability:logging-slf4j2")
include(":runtime:observability:telemetry-api")
include(":runtime:observability:telemetry-defaults")
include(":runtime:observability:telemetry-provider-otel")
include(":runtime:observability:telemetry-provider-micrometer")
include(":runtime:protocol:aws-protocol-core")
include(":runtime:protocol:aws-event-stream")
include(":runtime:protocol:aws-json-protocols")
Expand Down