Skip to content

Documentation Updates #188

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jul 25, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
77 changes: 66 additions & 11 deletions Documentation/docs/benchmarks.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,9 @@ sidebar_position: 90

# Benchmarks

The benchmarks provided in the HiveMQtt GitHub repository are built using BenchmarkDotNet, a .NET library for benchmarking. These benchmarks are designed to measure the performance of various messaging operations against any MQTT broker.
The HiveMQtt GitHub repository provides benchmarks built using BenchmarkDotNet, a .NET library for benchmarking. These benchmarks measure the performance of various messaging operations against any MQTT broker.



## Running Benchmarks

Expand All @@ -17,18 +19,61 @@ dotnet run ClientBenchmarkApp.csproj -c Release

## Results

The benchmarks provide insights into the performance of different messaging methods under various scenarios. Below are the results obtained from running the benchmarks on a local MBP (MacBook Pro) against a HiveMQ v4 broker running in a Docker container over localhost.
The benchmarks provide insights into the performance of different messaging methods under various scenarios. Below are the results obtained from running the benchmarks on a local MacBook Pro against a HiveMQ v4 broker running in a Docker container over localhost.

## Legend

```
Mean : Arithmetic mean of all measurements
Error : Half of 99.9% confidence interval
StdDev : Standard deviation of all measurements
Median : Value separating the higher half of all measurements (50th percentile)
1 us : 1 Microsecond (0.000001 sec)
1 ms : 1,000 Microseconds
```
* **Mean**: Arithmetic mean of all measurements
* **Error**: Half of 99.9% confidence interval
* **StdDev**: Standard deviation of all measurements
* **Median**: Value separating the higher half of all measurements (50th percentile)
* **1 us**: 1 Microsecond (0.000001 sec)
* **1 ms**: 1,000 Microseconds
* <font color="green">Green</font>: Improvement
* <font color="red">Red</font>: Regression

## July 25, 2024

### Summary

The client has had many feature additions and improvements from March 2024 including:

* New Back-pressure support
* Improved QoS reliability
* Improved Async await strategies
* Re-architected internal queues
* Improved event & trigger handling
* All around better error handling

The following table shows the benchmark results compared against previous benchmark runs.

| Method | Alpha | March 2024 | July 2024 |
| ------ | ----- | -------- | -------------- |
| QoS 0 Message | <font color="">4.875 us</font> | <font color="red">9.084 us</font> | <font color="green">5.500 us</font> |
| QoS 1 Message | <font color="">790.645 us</font> | <font color="red">1,357.063 us</font> | <font color="green">982.208 us</font> |
| QoS 2 Message | <font color="">1,653.083 us</font> | <font color="green">1,292.396 us</font> | <font color="green">1,004.854 us</font> |
| 100 QoS 0 @ 256b | No Data | <font color="">79.604 us</font> | <font color="red">100.375 us</font> |
| 100 QoS 1 @ 256b | No Data | <font color="">42,482.520 us</font> | <font color="green">29,661.605 us</font> |
| 100 QoS 2 @ 256b | No Data | <font color="">85,640.167 us</font> | <font color="green">55,474.250 us</font> |
| 100 QoS 0 @ 256k | No Data | <font color="">69.709 us</font> | <font color="red">99.354 us</font> |
| 100 QoS 1 @ 256k | No Data | <font color="">266,506.583 us</font> | <font color="green">215,158.104 us</font> |
| 100 QoS 2 @ 256k | No Data | <font color="">296,254.688 us</font> | <font color="green">227,255.729 us</font> |

### July Benchmark Data

The raw benchmark data from July.

| Method | Mean | Error | StdDev | Median |
|------------------------------------------------- |--------------:|-------------:|--------------:|---------------:|
| 'Publish a QoS 0 message' | 32.59 us | 85.64 us | 252.53 us | 5.500 us |
| 'Publish a QoS 1 message' | 6,323.30 us | 16,671.57 us | 49,156.51 us | 982.208 us |
| 'Publish a QoS 2 message' | 6,554.90 us | 16,998.38 us | 50,120.10 us | 1,004.854 us |
| 'Publish 100 256b length payload QoS 0 messages' | 131.73 us | 92.15 us | 271.71 us | 100.375 us |
| 'Publish 100 256b length payload QoS 1 messages' | 35,405.71 us | 17,183.26 us | 50,665.23 us | 29,661.605 us |
| 'Publish 100 256b length payload QoS 2 messages' | 61,955.04 us | 17,559.19 us | 51,773.66 us | 55,474.250 us |
| 'Publish 100 256k length payload QoS 0 messages' | 129.00 us | 83.24 us | 245.42 us | 99.354 us |
| 'Publish 100 256k length payload QoS 1 messages' | 226,498.28 us | 42,050.59 us | 123,987.13 us | 215,158.104 us |
| 'Publish 100 256k length payload QoS 2 messages' | 244,227.03 us | 47,932.40 us | 141,329.77 us | 227,255.729 us |

## Mar 22, 2024

Expand All @@ -50,7 +95,17 @@ See also: [What is MQTT Quality of Service (QoS) 0,1, & 2? – MQTT Essentials:

## Mar 21, 2024

With release [v0.11.0](https://github.com/hivemq/hivemq-mqtt-client-dotnet/releases/tag/v0.11.0) there was a big performance improvement. All messaging performance was improved but particularly publishing a QoS level 2 message went from ~206ms down to ~1.6ms.
With release [v0.11.0](https://github.com/hivemq/hivemq-mqtt-client-dotnet/releases/tag/v0.11.0) there was a big performance improvement. All messaging performance was improved but particularly publishing a **QoS level 2 message went from ~206ms down to ~1.6ms**.

### Summary

From the early version through two refactors, this table summarizes the performance improvements.

| Method | Original | First Pass Refactor | Final Refactor |
| ------ | -------- | ------------------- | -------------- |
| QoS 0 Message | <font color="red">5.646 us</font> | 9.250 us | <font color="green">4.875 us</font> |
| QoS 1 Message | <font color="red">103,536.375 us</font> | 1,324.251 us | <font color="green">790.645 us</font> |
| QoS 2 Message | <font color="red">206,959.834 us</font> | 2,569.166 us | <font color="green">1,653.083 us</font> |

### Previous Performance

Expand Down
45 changes: 12 additions & 33 deletions Documentation/docs/connecting.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ var client = new HiveMQClient();
var connectResult = await client.ConnectAsync().ConfigureAwait(false);
```

## With Options
## With Specific Options

The `HiveMQClientOptions` class provides a set of options that can be used to configure various aspects of the `HiveMQClient`.

Expand All @@ -35,44 +35,23 @@ var client = new HiveMQClient(options);
var connectResult = await client.ConnectAsync().ConfigureAwait(false);
```

## `HiveMQClientOptionsBuilder` Reference

To illustrate _each and every possible call_ with `HiveMQClientOptionsBuilder`, see the following example:
## With Automatic Reconnect

```csharp
using HiveMQtt.MQTT5.Types; // For QualityOfService enum

var options = new HiveMQClientOptionsBuilder()
.WithBroker("broker.hivemq.com")
.WithPort(1883)
.WithClientId("myClientId")
.WithAllowInvalidBrokerCertificates(true)
.WithUseTls(true)
.WithCleanStart(true)
.WithKeepAlive(60)
.WithAuthenticationMethod("UsernamePassword")
.WithAuthenticationData(Encoding.UTF8.GetBytes("authenticationData"))
.WithUserProperty("property1", "value1")
.WithUserProperties(new Dictionary<string, string> { { "property1", "value1" }, { "property2", "value2" } })
.WithLastWill(new LastWillAndTestament {
Topic = "lwt/topic",
PayloadAsString = "LWT message",
QoS = QualityOfService.AtLeastOnceDelivery,
Retain = true })
.WithMaximumPacketSize(1024)
.WithReceiveMaximum(100)
.WithSessionExpiryInterval(3600)
.WithUserName("myUserName")
.WithPassword("myPassword")
.WithPreferIPv6(true)
.WithTopicAliasMaximum(10)
.WithRequestProblemInformation(true)
.WithRequestResponseInformation(true)
.Build();
var options = new HiveMQClientOptionsBuilder().
WithBroker('candy.x39.eu.hivemq.cloud').
WithAutomaticReconnect(true)
Build();

var client = new HiveMQClient(options);
var connectResult = await client.ConnectAsync().ConfigureAwait(false);
```


## See Also

* [HiveMQClientOptionsBuilder Reference](/docs/reference/client_options_builder)
* [Automatic Reconnect](/docs/reference/automatic_reconnect)
* [How to Set a Last Will & Testament](/docs/how-to/set-lwt)
* [Connect with TLS but allow Invalid TLS Certificates](/docs/how-to/allow-invalid-certs)
* [Securely Connect to a Broker with Basic Authentication Credentials](/docs/how-to/connect-with-auth)
Expand Down
Loading