Skip to content

Commit 42a8048

Browse files
bmwillibmwilli
authored
Merge pull request #31 from IBMStreams/develop
Develop
2 parents 54cb35f + e2d513a commit 42a8048

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

42 files changed

+6737
-4610
lines changed

‎README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,5 +2,5 @@
22

33
| Application | Description |
44
| ----------- | ----------- |
5-
| [streams-metric-exporter](streams-metric-exporter/) | IBM Streams Metric Exporter for Prometheus (more destinations to come). Connects to the IBM Streams JMX Server and pulls metrics for one or more IBM Streams instances and jobs using the very efficient InstanceMXBean.jobSnapshotMetrics() call. The metrics are available to Prometheus for scraping via HTTP or HTTPS. Instructions and supporting files for implementing with Prometheus and Grafana via docker-compose are included. [Grafana Dashboard Example](streams-metric-exporter/images/IBMStreamsDomainDashboard.png)|
5+
| [streams-metric-exporter](streams-metric-exporter/) | IBM Streams Metric Exporter for Prometheus. Connects to the IBM Streams JMX Server and pulls metrics for one or more IBM Streams instances and jobs using the very efficient InstanceMXBean.jobSnapshotMetrics() call. The metrics are available to Prometheus for scraping via HTTP or HTTPS. Instructions and supporting files for implementing with Prometheus and Grafana via docker-compose are included. [Grafana Dashboard Examples](streams-metric-exporter/dashboards/)<br><img src="streams-metric-exporter/images/IBMStreamsDomainDashboard.png" width="45%" border="5"> <img src="streams-metric-exporter/images/IBMStreamsInstanceDashboard.png" width="45%"> <img src="streams-metric-exporter/images/IBMStreamsJobDashboard.png" width="45%"> <img src="streams-metric-exporter/images/IBMStreamsResourceDashboard.png" width="45%">|
66
| [streams-jmx-client](streams-jmx-client/) | IBM Streams Command Line Interface similar to streamtool command. This application uses JMX connection and does not require colocation with an IBM Streams Installation. It is great for running on mac osx, windows, and lightweight docker images. It support username/password as well as PKI certificate authentication. The list of commands supported will grow. Contributions welcome!!|

‎streams-metric-exporter/README.md

Lines changed: 20 additions & 169 deletions
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,14 @@
11
# streams-metric-exporter
22

3-
This application provides an interface to the IBM Streams - Stream Processing System for the purposes of retrieving status and metrics of an IBM Streams domain, instances, jobs, and resources.
3+
Prometheus Metrics Exporter for IBM Streams version 4.x.
44

5-
This version provides 2 HTTP/HTTPS interfaces:
6-
* Prometheus: HTTP/Text endpoint in Prometheus metrics format.
7-
* REST: Multiple endpoints returning json.
5+
IBM Streams provides a JMX Service (with HTTP GET interface for batch metric pulls) that is capable of providing status of the Streams instance, deployed streaming application (jobs), and cluster resources. In addition, metrics are available via the Streams JMX Service.
86

9-
IBM Streams provides a JMX Service (with HTTP GET interface for batch metric pulls) that is capable of providing status of the Streams instance, deployed streaming application (jobs), and cluster resources. In addition, metrics are available via the Streams JMX Server.
10-
11-
The service supports user-defined custom metrics that are found with Streams jobs. These custom metrics do not have to be predefined within the Streams Metric Exporter. They are automatically discovered and made available as Prometheus metrics and in the REST JSON.
7+
The service supports user-defined custom metrics found in Streams analytic jobs. These custom metrics do not have to be predefined within the Streams Metric Exporter. They are automatically discovered and made available as Prometheus metrics.
128

139
The primary use-case for this application is as a Prometheus metrics exporter to provide time series displays using Grafana.<br>
14-
The original use case for this application was as a Streams Application Metrics exporter. It is not meant to monitor the internal system services of IBM Streams. It does, however, provide metrics and status of domains, instances, and resources which do give a operations view of the health of your system.<br>
15-
This application is **optimized** for better performance over per-job metric scraping approaches. The metrics and status are pulled from IBM Streams for all job metrics and status (via the JMX Server HTTP callbacks). The pulls from IBM STreams can be performed whenever a rest endpoint is hit (on-demand) or scheduled to be pulled and cached with a configurable interval. Users can use the REST endpoints (including Prometheus endpoint) to get metrics and status of specific jobs, thus providing granular access to the information without sacrificing performance and impacting the services of IBM Streams.
10+
11+
This application is **optimized** for better performance over per-job metric scraping approaches. The metrics and snapshots are pulled from IBM Streams for all jobs at the same time. The pulls from IBM Streams can be performed whenever the rest endpoint (/metrics) is requested or scheduled to be pulled and cached with a configurable interval (``--refresh``).
1612

1713
The service can be configured with periodic refresh (refresh rate > 0) or on-demand refresh (refresh rate == 0) when the HTTP/HTTPS endpoints are accessed.
1814

@@ -130,7 +126,7 @@ Usage: streams-metric-exporter [options]
130126
-r, --refresh
131127
Refresh rate of metrics in seconds or 0 for no automatic refresh
132128
Environment Variable: STREAMS_EXPORTER_REFRESHRATE
133-
Default: 10
129+
Default: 0
134130
--serverkeystore
135131
Java keystore containing server certificate and key to identify server side of this application
136132
Environment Variable:
@@ -229,15 +225,15 @@ When the Streams domain is running inside of Kubernetes, and the Streams metric
229225
## Endpoint
230226

231227
```bash
232-
/prometheus
228+
/metrics or /prometheus
233229
```
234230
## prometheus.yml
235-
This file configures how prometheus will scrape the streams-metric-exporter. By default, the metrics are cached and retrieved every 10 seconds. If you desire for the exporter to work as a "proper" prometheus exporter, set the refresh rate (--refresh) to 0, which causes refreshes to be manual and performed whenever an end point is accessed (the /prometheus endpoint for example).
231+
This file configures how prometheus will scrape the streams-metric-exporter.
236232

237233
```yml
238234
- job_name: "ibmstreams"
239-
scrape_interval: "15s"
240-
metrics_path: "/prometheus"
235+
scrape_interval: "10s"
236+
metrics_path: "/metrics"
241237
static_configs:
242238
- targets: ['localhost:25500']
243239
```
@@ -321,19 +317,17 @@ streams_operator_ip_nTuplesProcessed{domainname="StreamsDomain",instancename="St
321317
streams_pe_op_connection_congestionFactor{domainname="StreamsDomain",instancename="StreamsInstance",jobname="MultiPEJob",resource="streamsqse",peid="1",index="0",connectionid="o0p1i0p0",} 0.0
322318
```
323319
324-
# Grafana Examples
320+
# Grafana Sample Dashboards
325321
See [dashboards directory](dashboards/README.md) for more information.
326322
327-
![Grafana Example](images/IBMStreamsDomainDashboard.png)
323+
![Grafana Example](images/IBMStreamsInstanceDashboard.png)
328324
329325
# Running with Docker
330326
331327
The easiest way to try out the Streams Metric Exporter is to run it using Docker. Included in this release is a Dockerfile for building the image and a docker-compose.yml file for starting it up with Prometheus and Grafana instances.
332328
333329
The versions of Prometheus and Grafana specified in the docker-compose.yml file are those that were used for testing.
334330
335-
The current version uses Docker 5.0 and its new provisioning capabilities to create the initial datasource pointing to prometheus and initial general purpose dashboards for Streams domains and instances.
336-
337331
## Prerequisites
338332
339333
* Compiled version of Streams Metric Exporter (executable-streams-metric-exporter.jar)
@@ -342,8 +336,8 @@ The current version uses Docker 5.0 and its new provisioning capabilities to cre
342336
* Docker Compose (version 1.9.0-5 with .yml file format 2 used in development)
343337
* Access to Dockerhub or local repository with Images:
344338
* ibmjava:sfj-alpine (or any 1.8 version)
345-
* prom/prometheus (2.1.0 used in development)
346-
* grafana/grafana (5.0.4 used in development)
339+
* prom/prometheus (2.x or higher)
340+
* grafana/grafana (5.x or higher)
347341
348342
## Setup environment
349343
@@ -392,8 +386,10 @@ http://localhost:9090
392386
```
393387

394388

395-
# Cached REST endpoints
396-
The original version of streams-metric-exporter provided REST endpoints that returned json information. The REST endpoints are still available in this version. The reason to use these REST endpoints over those available directly from IBM Streams is that by default the auto-refresh is used, and these endpoints only retrieve from the metrics cached in the streams-metric-exporter. Beware, however, if you turn auto-refresh off (--refresh 0) then each of these access points will cause the metrics to be pulled from the Streams JMX Server.
389+
# REST endpoints
390+
391+
## /metrics (or /prometheus)
392+
Retrieve the prometheus format of the metrics
397393

398394
## /domain
399395
Retrieve information about the domain being monitored
@@ -472,56 +468,6 @@ Retrieves resources and specific metrics about them
472468
}
473469
```
474470

475-
## /instances/{instancename}/joblist
476-
List of job names and ids along with links to details about the job
477-
478-
`curl http://localhost:25500/joblist`
479-
480-
```json
481-
{
482-
"total": 1,
483-
"jobs": [
484-
{
485-
"snapshotnow": "http://localhost:25500/instances/StreamsInstance/jobs/0/snapshotnow",
486-
"name": "MultiPEJob",
487-
"id": 0,
488-
"metrics": "http://localhost:25500/instances/StreamsInstance/jobs/0/metrics",
489-
"jobInfo": "http://localhost:25500/instances/StreamsInstance/jobs/0",
490-
"snapshot": "http://localhost:25500/instances/StreamsInstance/jobs/0/snapshot"
491-
}
492-
]
493-
}
494-
```
495-
496-
## /instances/{instancename}/jobs/{jobid}/status
497-
Status of a single job
498-
499-
`curl http://localhost:25500/instances/StreamsInstance/jobs/0/status`
500-
501-
```json
502-
{
503-
"status": "running"
504-
}
505-
```
506-
507-
## /instances/{instancename}/jobs/{jobid}/health
508-
Health of a single job
509-
510-
511-
## /instances/{instancename}/jobs/{jobid}/metrics
512-
Metrics of a single job
513-
514-
## /instances/{instancename}/jobs/{jobid}/snapshots
515-
Snapshot of a single job
516-
517-
## /instances/{instancename}/jobs
518-
Array of all jobs in the instance along with job information and the metrics for each job
519-
520-
521-
522-
## /instances/{instancename}/jobs/{jobid}
523-
Job information and metrics for a single job
524-
525471
## /instances/{instancename}/metrics
526472
Retrieves all metrics for the selected instance
527473

@@ -535,101 +481,6 @@ The Provides a complete overview of the streams-metric-exporter server. Not rec
535481
Displays JSON view of the configuration values the program is using.<br>
536482
**Note:** passwords are displayed as "(hidden)" if they were specified
537483

484+
## /version
485+
Displays the version of the application
538486

539-
# Passthrough REST Endpoints
540-
These endpoints are not cached, rather, they are passed through directly to the Streams JMX Server API
541-
542-
## instances/{instancename}/jobs/{jobid}/snapshotnow
543-
544-
Calls the snapshot() jmx method on the specified job
545-
546-
### Parameters:
547-
548-
depth : depth of job topology to return (default: 1)
549-
550-
static: include static job attributes in addition to dynamic attributes (default: true)
551-
552-
`curl http://localhost:25500/instances/StreamsInstance/jobs/0/snapshot`
553-
554-
`curl http://localhost:25500/instances/StreamsInstance/jobs/0/snapshot?depth=2&static=false`
555-
556-
`curl http://localhost:25500/instances/StreamsInstande/jobs/0/snapshot?depth=2&statuc=true`
557-
558-
```json
559-
{
560-
561-
"applicationVersion": "4211",
562-
"instance": "StreamsInstance",
563-
"submitParameters": [ ],
564-
"startedBy": "streamsadmin",
565-
"health": "healthy",
566-
"resources": [
567-
{
568-
"id": "streamsqse.localdomain"
569-
}
570-
],
571-
"jobGroup": "default",
572-
"dataPath": "/home/streamsadmin/git/streams/spl/SimpleJob/-a",
573-
"adlFile": "output/SimpleJob.adl",
574-
"submitTime": 1506945170000,
575-
"applicationPath": "toolkits/SimpleJob",
576-
"checkpointPath": "/home/streamsadmin/git/streams/spl/SimpleJob/-a/ckpt",
577-
"outputPath": "output",
578-
"domain": "StreamsDomain",
579-
"name": "SimpleJob_0",
580-
"id": "0",
581-
"applicationName": "SimpleJob",
582-
"status": "running",
583-
"applicationScope": "Default",
584-
"pes": [
585-
{
586-
"launchCount": 1,
587-
"restartable": true,
588-
"resource": "streamsqse.localdomain",
589-
"health": "healthy",
590-
"relocatable": true,
591-
"tracingLevel": "error",
592-
"optionalConnections": "connected",
593-
"resourceTags": [ ],
594-
"pendingTracingLevel": null,
595-
"indexWithinJob": 0,
596-
"statusReason": "none",
597-
"processId": "5951",
598-
"operators": [
599-
{
600-
"indexWithinJob": 2,
601-
"name": "FunctorStream",
602-
"operatorKind": "spl.relational::Functor"
603-
},
604-
{
605-
"indexWithinJob": 0,
606-
"name": "BeaconStream",
607-
"operatorKind": "spl.utility::Beacon"
608-
},
609-
{
610-
"indexWithinJob": 1,
611-
"name": "FilterStream",
612-
"operatorKind": "spl.relational::Filter"
613-
},
614-
{
615-
"indexWithinJob": 3,
616-
"name": "SinkStream",
617-
"operatorKind": "spl.relational::Functor"
618-
},
619-
{
620-
"indexWithinJob": 4,
621-
"name": "Sink",
622-
"operatorKind": "spl.utility::Custom"
623-
}
624-
],
625-
"requiredConnections": "connected",
626-
"outputPorts": [ ],
627-
"inputPorts": [ ],
628-
"id": "0",
629-
"osCapabilities": [ ],
630-
"status": "running"
631-
}
632-
]
633-
634-
}
635-
```

0 commit comments

Comments
 (0)