Skip to content

Update "New Features" page #3026

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: release-5.10
Choose a base branch
from
Open

Update "New Features" page #3026

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
ad780ce
Update "New Features" page
Meggielqk Jun 20, 2025
462bbe8
Update zh file
Meggielqk Jun 26, 2025
eb64128
Add image for cluster linking feature
Meggielqk Jun 27, 2025
81972a2
Editorial updates
Meggielqk Jul 1, 2025
6f1b5b8
Update opentelemetry.md
Meggielqk Jul 2, 2025
1896a9c
Update new-features.md
Meggielqk Jul 18, 2025
27a4592
Update new-features.md
Meggielqk Jul 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added en_US/getting-started/assets/cluster_linking_feature.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
154 changes: 63 additions & 91 deletions en_US/getting-started/new-features.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,141 +1,113 @@
---
description: This section lists the new features introduced in EMQX 5.0 and 5.1.
description: This section lists the new features introduced in EMQX 5.2 and later.
---

# New Features

This section lists the new features introduced in EMQX Enterprise after version 5.0.
This page highlights major new features supported in the current release. Note that it does not list every feature provided by EMQX.

## Core + Replica Cluster Architecture
## Cluster Linking

EMQX 5.0 adopts a new [Mria cluster architecture](../deploy/cluster/mria-introduction.md). With this architecture, one EMQX cluster can support [100 million concurrent MQTT connections](https://www.emqx.com/en/blog/reaching-100m-mqtt-connections-with-emqx-5-0), and more, making it the world’s most scalable open-source MQTT broker.
Cluster Linking enables seamless, secure, and efficient message sharing between geographically distributed EMQX clusters. Unlike traditional MQTT bridges, which forward all messages and often require topic prefixes to prevent feedback loops, Cluster Linking transfers only relevant messages based on active subscriptions, minimizing bandwidth use, reducing latency, and increasing scalability.

<img src="./assets/100m-benckmark.png" alt="100m-benckmark" style="zoom:50%;" />
The configuration and management of Cluster Linking are designed to be simple and flexible. You can create, modify, and monitor cluster links directly from the EMQX Dashboard, configuration file, or through REST APIs. EMQX also provides visual status indicators and link statistics for real-time visibility.

Besides this obvious scalability improvement, the Mria cluster architecture is also capable of reducing the risk of brain-splitting and its effects in large-scale deployments, to empower our clients with more stable and reliable IoT data access services.
To get started with the Cluster Linking, see [Quick Start with Cluster Linking](../cluster-linking/quick-start.md).

To get started, see [Create an EMQX Cluster](../deploy/cluster/create-cluster.md).
<img src="./assets/cluster_linking_feature.png" alt="cluster_linking_feature" style="zoom:80%;" />

## Rolling Upgrades without Downtime
## Namespace

Starting with EMQX 5.1, the system now supports seamless rolling upgrades for the cluster. This enhances the overall system availability and reliability by allowing transitions to newer versions without any service interruption.
Namespaces provide logical multi-tenancy within a single EMQX cluster, enabling you to isolate clients, topics, quotas, and configurations per tenant without creating separate clusters. Each namespace is identified using a special client attribute (`tns`), which can be derived from metadata like username or Server Name Indication (SNI), a flexible method that ensures accurate tenant attribution at connection time.

## MQTT over QUIC
Namespaces support two creation modes:

EMQX 5.0 introduces QUIC support (MQTT over QUIC) as an experimental feature and designs a unique messaging mechanism and management approach. In EMQX 5.1 we added [QUIC multistream](https://www.emqx.com/en/blog/emqx-newsletter-202302) support and from now on consider this feature as "generally available".
- **Explicit**: Administrators define namespaces manually via the Dashboard or REST API.
- **Automatic**: EMQX generates namespaces dynamically by extracting the `tns` attribute from incoming client connections.

As the underlying transport protocol of the next-generation Internet protocol HTTP/3, [QUIC](https://datatracker.ietf.org/doc/html/rfc9000) can provide connectivity for the modern mobile Internet with less connection overhead and message latency compared to TCP/TLS protocols. Therefore, EMQX attempted to replace the transport layer of MQTT with QUIC, which led to the MQTT over QUIC.
Currently, namespaces support tenant-level rate limiting configuration, allowing resource allocation and usage control per tenant. To learn more details about this feature and how to create and configure the namespaces, see the [Namespace](../multi-tenancy/namespace-overview.md) section.

To evaluate MQTT over QUIC and verify how it could improve network connectivity, please read [Use MQTT over QUIC ](../mqtt-over-quic/getting-started.md).
## Smart Data Hub

## File Transfer over MQTT
The [Smart Data Hub](../data-integration/smart-data-hub.md) offers a unified solution for managing schema-based message validation and transformation across MQTT data streams. It simplifies the development of structured and reliable data flows through key components:

EMQX 5.1 introduces File Transfer over MQTT, which supports file transfer using the MQTT protocol.
### Schema Registry

This feature is implemented based on the extended implementation of the standard MQTT protocol, allowing integration with existing clients and applications without modification. Clients can send file segments to specific topics using the MQTT protocol. After the transfer is complete, the server merges the file segments and saves them to the local disk or exports them to object storage compatible with the S3 protocol.
The Schema Registry now supports both internal schemas (like JSON, Avro, and Protobuf) and external schemas via HTTP services. For formats not natively supported, EMQX can delegate schema operations to [external HTTP services](../data-integration/schema-registry-example-external-http.md) via the `schema_encode` and `schema_decode` functions.

Compared to HTTP/FTP protocols, MQTT has the advantages of low bandwidth consumption and minimal resource utilization, enabling fast and efficient file transfer. The unified IoT data channel also simplifies system architecture, reducing application complexity and maintenance costs.
### Schema Validation

Get started with [File Transfer over MQTT](../file-transfer/introduction.md) now.
[Schema validation](../data-integration/schema-validation.md) ensures that only messages conforming to predefined formats are processed or delivered. EMQX supports JSON Schema, Protobuf, Avro, and rule engine SQL syntax for validation. Based on the outcome, users can configure actions such as dropping messages, disconnecting clients, or triggering rule engine events for failed validations.

## Backup and Restore
### Message Transformation

EMQX 5.1 introduces a set of command-line tools for backup and restore, allowing you to export data and configuration files from the built-in database as a compressed package and restore them to a new cluster.
[Message Transformation](../data-integration/message-transformation.md) allows users to define transformation pipelines that decode, modify, and re-encode messages before delivery or further processing. The system supports nested transformations, multiple encoders/decoders, and dynamic value assignments using [Variform expressions](../configuration/configuration.md#variform-expressions).

::: details Usage example
Create a backup:
## LLM-Based MQTT Data Processing

```bash
$ ./bin/emqx ctl data export
...
Data has been successfully exported to data/backup/emqx-export-2023-06-21-14-07-31.592.tar.gz.
```
EMQX 5.10.0 adds support for [LLM-based MQTT data processing](../flow-designer/llm-based-data-processing.md) in Flow Designer. It integrates models like OpenAI’s GPT or Anthropic’s Claude to process MQTT messages using natural language prompts. Processing nodes call the AI models via reusable completion profiles and return results for further actions such as republishing or storage. This feature is ideal for intelligent, contextual workflows in low-throughput scenarios.

Restore a backup:
## Expanded Support in Data Integration

```bash
./bin/emqx ctl data import <File>
```
:::
Recent versions of EMQX have significantly enhanced the data integration capabilities. In addition to expanding support for more data sinks and services, data integration now includes fallback actions, a robust mechanism designed to increase reliability in real-time IoT data processing.

Find more details on [Backup and Restore](../operations/backup-restore.md) page.
New data integrations supported in recent versions include, but are not limited to, the following:

## Redesigned IoT Data Integration
- **[Amazon S3 Tables](../data-integration/s3-tables.md)**: Transforms MQTT data into Iceberg-formatted tables and streams them directly into S3. It eliminates the need for traditional databases while retaining SQL-like querying capabilities.
- **[Apache Doris](../data-integration/apache-doris.md)**: Processes the MQTT messages, maps them into structured data, and writes to Doris via HTTP or JDBC. You can query your IoT data in real time using standard SQL and build live dashboards with BI tools like Grafana.
- **[Snowflake](../data-integration/snowflake.md)**: Writes the processed data to the Snowflake Stage and loads it into a Snowflake table. Safely store IoT data in Snowflake for long-term archival and leverage Snowflake's data warehousing and analytics capabilities to perform real-time or batch analysis.

Besides SQL, EMQX 5.x Rule Engine also supports [jq](https://stedolan.github.io/jq/), so it is capable of handling more complex JSON data formats. See more at the [jq Functions documentation](../data-integration/rule-sql-jq.md).
### Fallback Actions

EMQX supports sending data to WebHook or establishing bidirectional data integration with external MQTT services by default. It supports processing and sending your IoT data to over 40 cloud services and enterprise systems in real-time, or retrieving data from them and sending it to designated MQTT topics after processing. EMQX 5.0 provides the Flows feature for visualizing the data integration process on the Dashboard. Now, you can easily check how the rules engine processes IoT data and how data flows to external data services or devices.
Introduced in EMQX 5.9.0, fallback actions provide a way to handle failures during data delivery. If a primary action fails due to delivery errors, buffer overflow, or request timeouts, the system can automatically trigger one or more fallback actions.

For different data integrations that EMQX supports and how to configure them, see [Data Integration](../data-integration/data-bridges.md).
This feature is critical for minimizing data loss, improving system resilience, and enabling better observability. To learn more about the feature, see [Fallback Actions](../data-integration/data-bridges.md#fallback-actions).

## Flexible Authentication/Authorization
## Enhanced Security

EMQX 5.x offers a built-in client authentication/authorization feature, users only need to do some simple configuration work before integrating with various data sources for user authentication and ensuring data security under various scenarios.
Recent versions bring significant improvements to access control, ensuring that EMQX meets enterprise security standards while remaining flexible and easy to manage. These enhancements help protect data integrity, support regulatory compliance, and prevent unauthorized access across complex IoT environments.

**New features**
In recent versions, EMQX supports more authentication and authorization methods, providing more flexible and fine-grained access control capabilities. Newly supported features include:

- Support using Dashboard for the authentication/authorization configuration on the cluster level;
- Support using Dashboard for configuration, commissioning and management;
- Support adjusting the running order of the authenticators and authorization checkers;
- Achieve complete observability with statistics on execution speed and number of times;
- Support authentication configuration on a listener-level, to provide more flexible access capabilities.
- **[Authenticator Preconditions](../access-control/authn/authn.md#authenticator-preconditions)**: Enable conditional execution of authenticators based on client metadata.
- **[LDAP Integration](../access-control/authn/ldap.md)**: Authenticate users against an external LDAP directory, supporting enterprise-grade user management.
- **[REST API-Based MQTT 5.0 SCRAM Authentication](../access-control/authn/scram_restapi.md)**: Leverage a RESTful API for SCRAM-based authentication in compliance with MQTT 5.0 standards.
- **[Kerberos Authentication](../access-control/authn/kerberos.md)**: Integrate with Kerberos-based SSO systems for secure, centralized user authentication.
- **[Client-Info Authentication](../access-control/authn/cinfo.md)**: Allow flexible access control based on client metadata like IP, device ID, or username.

On how to run authentication/authorization configuration with EMQX Dashboard or configuration files, you can continue to read [Authentication](../access-control/authn/authn.md) and [Authorization](../access-control/authz/authz.md).
EMQX Enterprise 5.9.0 introduces a suite of advanced security features to safeguard your deployments.

## User-Friendly EMQX Dashboard
- **[Multi-Factor Authentication](../multi-factor-authn/multi-factor-authentication.md)**: Adds a layer of login security by requiring verification beyond just username and password.
- **[Account Lockout and Unlock](../dashboard/introduction.md#account-lockout-and-unlock)**: Automatically disables user accounts after multiple failed login attempts, with options for manual or timed unlocking.
- **[Password Expiration](../dashboard/introduction.md#password-expiration)**: Enforces password rotation policies to reduce long-term credential risk and comply with organizational security policies.

In EMQX 5.x, we have redesigned the EMQX Dashboard, enhancing the visual experience and supporting more powerful and user-friendly features.
## OpenTelemetry Integration for Metrics, Logs, and Traces

**New features**
EMQX now supports OpenTelemetry, making it easier to monitor and troubleshoot your MQTT systems.

- New UI/UX design: Great enhancement of real-time observability
- Optimized menu structure: Fast and direct access to contents
- Data monitoring and management: Important data at a glance
- Visualized access control: Out-of-the-box authentication/authorization management
- Visualized data flows: Using [Flows](../flow-designer/introduction.md) users can clearly see how data flows from devices or clients through the rule engine
- Configuration updates during runtime: Hot update that takes effect immediately
**Key Features:**

## Overload Protection, Limiter and Buffer Queue for Bridges
- **Metrics**: Exports real-time metrics to OpenTelemetry Collector, then view them in tools like Prometheus and Grafana.
- **Logs**: Sends structured logs with rich context (like trace IDs) to your log system for easier debugging.
- **Tracing**: Enables distributed tracing of MQTT message flows across EMQX nodes. Useful for finding delays, routing issues, or node-specific performance bottlenecks.
- **End-to-End Tracing Mode**: Tracks full message paths and client actions. Filter by client ID, topic, or QoS. Control sampling and export rate to manage system load.

The new **Limiter** feature enhances connection and messaging rate control by providing more precise and layered rate control options. It ensures that the system operates under the expected workloads by limiting the client behavior at the client, listener, or node levels. The combination of the Overload protection and Limiter features prevents the clients from becoming overwhelmed or receiving excessive request traffic and ensures stable system operation.
OpenTelemetry helps you get full visibility into EMQX performance and message flows using open, standard tools. Refer to [Integrate with OpenTelemetry](../observability/opentelemetry/opentelemetry.md) for details.

A generic buffer queue is also added for all bridges, allowing buffer messages generated under stressful conditions. This buffer can be configured to store messages in either memory or disk cache when external resources are unavailable, such as during network fluctuations or service downtime. Buffered messages will be sent once the service is restored. However, requests in the buffer may expire, which is a big difference compared to version 4. If the amount of buffered data exceeds the limit, it will be discarded following the First In First Out (FIFO) rule.
## NATS Protocol Gateway

## Cloud Native and EMQX Operator
EMQX 5.10.0 introduces a native NATS protocol gateway, enabling bi-directional messaging between NATS and MQTT. This feature allows NATS clients to connect directly to EMQX and exchange messages with MQTT clients using topic-to-subject mapping.

Horizontal expansion and elastic clusters are features that a cloud-native application must support.
**Key Features:**

[EMQX Kubernetes Operator](https://www.emqx.com/en/emqx-kubernetes-operator) lets you take full advantage of the Replicant node of EMQX 5.x. You can deploy a stateless EMQX node with Kubernetes Deployment and then build the EMQX cluster that supports large-scale MQTT connections and message throughput.
- **Full NATS Protocol Support**: Handles core message types like PUB, SUB, PING, and request/reply.
- **MQTT Interoperability**: Converts NATS subjects to MQTT topics, supports wildcards, and shared subscriptions.
- **Flexible Deployment**: Enable via Dashboard, REST API, or config file.
- **Transport Support**: Works over TCP, TLS, WebSocket, and secure WebSocket (WSS).
- **Authentication**: Supports multiple backends, including database, HTTP, JWT, and LDAP.

## New Gateway Framework
With this gateway, EMQX bridges MQTT with modern cloud-native environments where NATS is used, expanding integration possibilities in hybrid messaging systems. Refer to the [NATS Protocol Gateway](../gateway/nats.md) for details.

EMQX 5.1 presents a new extended gateway framework with reconstructed underlying architecture for multi-protocol access and a unified configuration format and management interface:
## More Features

- **Unified statistic and monitoring indicators:** EMQX 5.0 provides the gateway/client-level statistic indicators, for example, number of bytes sent and received, messages, etc.
- **Independent connection and session management:** Different from EMQX 4.x, gateway clients are also managed under the MQTT client list, EMQX 5.0 has created an independent gateway page for each gateway, and one Client ID can be reused across gateways.
- **Independent client authentication**: Different from EMQX 4.x, where the gateway authentication is also managed under the MQTT client, EMQX 5.0 supports configuring a unique authentication mechanism for each gateway.
- **Easy to expand with clear specifications:** The framework provides a set of standard concepts and interfaces to make it easier to customize the gateways.

The new gateway framework further improves EMQX's usability by unifying access and managing multiple protocols. Now clients implementing third-party protocols can also leverage the advantages of EMQX, such as data integration, safe and reliable authentication/authorization, billion-level horizontal expansion capabilities, etc.

## **More Feature Updates**

### Simplified Configuration

The configuration files have been changed to a concise and readable [HOCON](https://github.com/emqx/hocon) format, and contains commonly used configuration items by default, to improve the readability and maintainability.

### Improved REST API

REST API is now compliant with the OpenAPI 3.0 specification, and comes with clear and rich API documentation.

### Rapid Troubleshooting

More diagnostic tools such as slow subscriptions and online tracing are added so users can quickly troubleshoot issues in production.

### Structured Logs

Machine (indexer) friendly structured logs in JSON format are supported. Error logs are consistently tagged with 'msg' tokens to facilitate locating the cause of the problem.

### Flexible Expansion and Customization

A new plugin architecture is developed, with which users can compile, distribute, and install their extension plugins in the form of independent plugin packages to customize and extend the usage of EMQX.
In addition to the highlights covered above, recent EMQX updates include many other new features and enhancements. For a complete list, see the [Release Notes](../changes/changes-ee-v5.md).
Loading