-
Notifications
You must be signed in to change notification settings - Fork 68
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
2009779
commit 67cbc03
Showing
2 changed files
with
107 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,106 @@ | ||
## Overview | ||
|
||
As IoT deployments grow in scale and complexity, managing data quality and compatibility becomes increasingly challenging. Today, we're thrilled to announce the launch of Smart Data Hub on the EMQX Platform. This powerful new module integrates enterprise-grade schema management, data validation, and message transformation capabilities into one unified solution, all designed to help you streamline and enrich your IoT data pipelines in real-time. | ||
|
||
Starting today, Smart Data Hub is available exclusively for our Dedicated and Premium plan users. By centralizing schema management, data validation, and dynamic transformations, it empowers you to ensure data consistency, reduce integration complexity, and accelerate time-to-value for your business applications. | ||
|
||
## Why You Need Smart Data Hub? | ||
|
||
In the modern IoT landscape, ensuring data consistency and compatibility is not just a nice-to-have feature; it is essential for maintaining reliable operations. We've heard from many customers about their struggles with: | ||
|
||
- Maintaining data format consistency across different devices and systems | ||
- Validating incoming data to prevent downstream processing issues | ||
- Transforming messages to meet various backend system requirements | ||
- Managing schemas across their entire IoT infrastructure | ||
|
||
Smart Data Hub addresses these challenges head-on by providing a unified solution for managing your MQTT data pipeline. | ||
|
||
## What's Inside Smart Data Hub? | ||
|
||
Within Smart Data Hub, you’ll find three main functionalities: | ||
|
||
![Smart Data Hub](https://assets.emqx.com/images/acff2927acbb96c28ade471a80fa155c.png) | ||
|
||
1. **Schema Registry** | ||
Centrally manage your data schemas using industry-standard formats including JSON Schema, Protobuf, and Avro. This ensures consistency across your entire IoT data pipeline and provides a single source of truth for your data structures. | ||
2. **Schema Validation** | ||
Automatically validate incoming MQTT messages against your defined schemas. This helps you: | ||
- Catch data format issues early | ||
- Prevent invalid data from reaching your downstream systems | ||
- Monitor validation results in real-time | ||
- Track validation failures through comprehensive logging | ||
3. **Message Transformation** | ||
Transform your MQTT messages on the fly to match your specific needs: | ||
- Convert between different data formats | ||
- Restructure message content | ||
- Modify message properties | ||
- Test transformations in real-time with our built-in testing tool | ||
|
||
## Key Benefits | ||
|
||
Smart Data Hub brings a range of valuable benefits to your IoT deployments: | ||
|
||
- **Data Quality Assurance:** Catch and prevent data issues before they impact your downstream systems. | ||
- **Real-Time Processing:** Apply transformations on the fly, so data is always in the right format the moment it’s published. | ||
- **Reduced Development Overhead:** No need to build and maintain custom validation and transformation solutions. | ||
- **Simplified Operations:** Manage schemas, validation rules, and transformations from a single, intuitive interface | ||
- **Enhanced Visibility:** Monitor data quality and transformation processes with built-in analytics and logging | ||
- **Seamless Integration:** Works natively within your EMQX Platform deployment with no additional infrastructure needed | ||
|
||
## Who Can Benefit? | ||
|
||
Smart Data Hub is currently an add-on module for EMQX Platform Dedicated and Premium plans. It provides the most benefits to organizations managing large-scale IoT data, dealing with complex data formats, or operating compliance-sensitive applications such as healthcare, automotive, and smart manufacturing. | ||
|
||
If you’re on our Dedicated or Premium plan, you can enable Smart Data Hub directly from your deployment settings. If you’re not on one of these plans but would like to explore Smart Data Hub, [please contact us](mailto:[email protected]) to learn about our upgrade options. | ||
|
||
## Getting Started | ||
|
||
You can use all the functions provided by the Smart Data Hub once it is enabled. | ||
|
||
**Enable Smart Data Hub** | ||
In your EMQX Platform console, navigate to your Dedicated or Premium deployment. On the Smart Data Hub page, enable Smart Data Hub and confirm the updated hourly pricing (including your trial, if applicable). | ||
|
||
![Enable Smart Data Hub](https://assets.emqx.com/images/2bfa8a33dcaea11deddba2b0568d1c88.png) | ||
|
||
**Manage Your Schemas** | ||
Use the Schema Registry to add your schemas. | ||
|
||
![Manage Your Schemas](https://assets.emqx.com/images/744159a9b752142102dbdfdd2c7274e8.png) | ||
|
||
**Set Up Validation Rules** | ||
Define the validation rules that suit your use case. Whether you’re working with JSON Schema, Protobuf, or Avro, our interface helps you map exactly how incoming data should look. | ||
|
||
![Set Up Validation Rules](https://assets.emqx.com/images/a7038270b6440518cfb7c44c5e368563.png) | ||
|
||
**Configure Message Transformations** | ||
Specify how your MQTT messages should be transformed—modify fields, rename attributes, convert formats in real-time, and more. | ||
|
||
![Configure Message Transformations](https://assets.emqx.com/images/2b5714c76bd08ac0b75347885ea04fd5.png) | ||
|
||
We’ve prepared a detailed step-by-step guide for each feature in the Smart Data Hub module. For more details, check out our documentation: | ||
|
||
- [Smart Data Hub Overview](https://docs.emqx.com/en/cloud/latest/data_hub/smart_data_hub.html) | ||
- [Schema Registry](https://docs.emqx.com/en/cloud/latest/data_hub/schema_registry.html) | ||
- [Schema Validation](https://docs.emqx.com/en/cloud/latest/data_hub/schema_validation.html) | ||
- [Message Transformation](https://docs.emqx.com/en/cloud/latest/data_hub/message_transformation.html) | ||
|
||
## What’s Next? | ||
|
||
This is just the beginning of our journey with Smart Data Hub. We're already working on additional features and capabilities to make your data processing even more powerful and efficient. Stay tuned for upcoming blog posts where we'll dive deeper into each component of Smart Data Hub and share best practices for common use cases. | ||
|
||
## Conclusion | ||
|
||
Smart Data Hub is more than just a new feature; it’s a leap forward in how the EMQX Platform handles and processes IoT data. By integrating schema management, validation, and transformation in one place, we aim to help you eliminate complexities, save time, and improve the reliability of your data architecture. | ||
|
||
Whether you're looking to enforce strict data contracts for compliance or simply need a smarter way to standardize and transform incoming messages, Smart Data Hub has you covered. We can’t wait to see what you’ll build with it! | ||
|
||
Ready to get started? Log in to your [EMQX Platform console](https://cloud-intl.emqx.com/console), enable Smart Data Hub, and take your IoT data workflows to the next level. | ||
|
||
|
||
|
||
<section class="promotion"> | ||
<div> | ||
Talk to an Expert | ||
</div> | ||
<a href="https://www.emqx.com/en/contact?product=solutions" class="button is-gradient">Contact Us →</a> | ||
</section> |