Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] add seatunnel monitor #2782

Merged
merged 15 commits into from
Oct 29, 2024
+687 −1
Merged

[Feature] add seatunnel monitor #2782

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
59c6d1b
Merge remote-tracking branch 'upstream/master'
zhangshenghang Sep 2, 2024
f5bcd4f
Merge branch 'apache:master' into master
zhangshenghang Sep 11, 2024
af2f531
Merge remote-tracking branch 'upstream/master'
zhangshenghang Sep 23, 2024
615a0c2
Merge remote-tracking branch 'upstream/master' into HEAD
zhangshenghang Oct 23, 2024
7d38628
init
zhangshenghang Oct 22, 2024
9c6a3ee
init
zhangshenghang Oct 23, 2024
1475e12
fix ci
zhangshenghang Oct 23, 2024
dd6eb08
update doc
zhangshenghang Oct 25, 2024
4474098
Merge branch 'master' into feature-add-seatunnel-monitor
zhangshenghang Oct 25, 2024
2b32df3
Merge branch 'master' into feature-add-seatunnel-monitor
yuluo-yx Oct 26, 2024
b9dbfcf
Merge branch 'master' into feature-add-seatunnel-monitor
Aias00 Oct 26, 2024
d97dff8
Merge branch 'master' into feature-add-seatunnel-monitor
Aias00 Oct 26, 2024
51bc336
Merge branch 'master' into feature-add-seatunnel-monitor
yuluo-yx Oct 27, 2024
ba95024
Merge branch 'master' into feature-add-seatunnel-monitor
zuobiao-zhou Oct 28, 2024
fb3f266
Merge branch 'master' into feature-add-seatunnel-monitor
Aias00 Oct 29, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
99 changes: 99 additions & 0 deletions home/docs/help/seatunnel.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
---
id: seatunnel
title: Monitoring： SeaTunnel
sidebar_label: SeaTunnel
keywords: [ Open Source Monitoring System, Monitor SeaTunnel ]
---

> Collect monitoring metrics for SeaTunnel.

### Configuration Parameters

| Parameter Name | Parameter Help Description |
|-------------------|-----------------------------------------------------------------------------------------------------------------|
| Target Host | The monitored endpoint's IPV4, IPV6, or domain name. Note ⚠️ no protocol header (e.g., https://, http://). |
| Task Name | The name that identifies this monitoring task, which needs to be unique. |
| Port | The monitoring port opened by SeaTunnel, default value: 5801. |
| SSL | Whether SSL is enabled for connecting to SeaTunnel. |
| Query Timeout | Set the timeout for unresponsive queries, in milliseconds (ms), default 6000 ms. |
| Collection Interval| The interval time for periodic data collection, in seconds; the minimum interval that can be set is 30 seconds. |
| Binding Tags | Used for categorizing and managing monitoring resources. |
| Description Notes | Additional identification and description notes for this monitoring; users can add notes here. |

### Collected Metrics

#### Metric Set: Cluster Overview

| Metric Name | Metric Unit | Metric Help Description |
|-----------------------|-------------|----------------------------|
| projectVersion | None | Project version |
| gitCommitAbbrev | None | Git commit hash |
| totalSlot | None | Total number of slots |
| unassignedSlot | None | Number of unassigned slots |
| runningJobs | None | Number of running tasks |
| finishedJobs | None | Number of completed tasks |
| failedJobs | None | Number of failed tasks |
| cancelledJobs | None | Number of cancelled tasks |
| workers | None | Number of workers |

#### Metric Set: Thread Information

| Metric Name | Metric Unit | Metric Help Description |
|------------------|-------------|-------------------------|
| threadName | None | Thread name |
| threadId | None | Thread ID |
| threadState | None | Thread state |
| stackTrace | None | Stack trace information |

#### Metric Set: Node Monitoring

| Metric Name | Metric Unit | Metric Help Description |
|---------------------------------|-------------|------------------------------|
| isMaster | None | Whether it is a master node |
| host | None | IP address |
| port | None | Port |
| processors | None | Number of processors |
| physical.memory.total | None | Total physical memory |
| physical.memory.free | None | Available physical memory |
| swap.space.total | None | Total swap space |
| swap.space.free | None | Available swap space |
| heap.memory.used | None | Used heap memory |
| heap.memory.free | None | Available heap memory |
| heap.memory.total | None | Total heap memory |
| heap.memory.max | None | Maximum heap memory |
| heap.memory.used/total | None | Heap memory usage rate |
| heap.memory.used/max | None | Maximum heap memory usage rate|
| minor.gc.count | None | Minor garbage collection count|
| minor.gc.time | None | Minor garbage collection time |
| major.gc.count | None | Major garbage collection count|
| major.gc.time | None | Major garbage collection time |
| load.process | None | Process load |
| load.system | None | System load |
| load.systemAverage | None | Average system load |
| thread.count | None | Number of threads |
| thread.peakCount | None | Peak thread count |
| cluster.timeDiff | None | Cluster time difference |
| event.q.size | None | Event queue size |
| executor.q.async.size | None | Asynchronous execution queue size |
| executor.q.client.size | None | Client execution queue size |
| executor.q.client.query.size | None | Client query queue size |
| executor.q.client.blocking.size | None | Client blocking queue size |
| executor.q.query.size | None | Query queue size |
| executor.q.scheduled.size | None | Scheduled execution queue size |
| executor.q.io.size | None | IO queue size |
| executor.q.system.size | None | System execution queue size |
| executor.q.operations.size | None | Operations queue size |
| executor.q.priorityOperation.size| None | Priority operations queue size |
| operations.completed.count | None | Completed operations count |
| executor.q.mapLoad.size | None | Map load queue size |
| executor.q.mapLoadAllKeys.size | None | Map load all keys size |
| executor.q.cluster.size | None | Cluster execution queue size |
| executor.q.response.size | None | Response queue size |
| operations.running.count | None | Number of running operations |
| operations.pending.invocations.percentage | None | Percentage of pending invocations |
| operations.pending.invocations.count | None | Number of pending invocations |
| proxy.count | None | Number of proxies |
| clientEndpoint.count | None | Number of client endpoints |
| connection.active.count | None | Number of active connections |
| client.connection.count | None | Number of client connections |
| connection.count | None | Total number of connections |
99 changes: 99 additions & 0 deletions home/i18n/zh-cn/docusaurus-plugin-content-docs/current/help/seatunnel.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
---
id: seatunnel
title: 监控：SeaTunnel
sidebar_label: SeaTunnel
keywords: [ 开源监控系统, 监控SeaTunnel ]
---

> 对 SeaTunnel 指标进行采集监控。

### 配置参数

| 参数名称 | 参数帮助描述 |
|---------|------------------------------------------------------|
| 目标Host | 被监控的对端IPV4，IPV6或域名。注意⚠️不带协议头(eg: https://, http://)。 |
| 任务名称 | 标识此监控的名称，名称需要保证唯一性。 |
| 端口 | SeaTunnel开放的监控端口，默认值：5801。 |
| 启用HTTPS | 是否启用HTTPS。 |
| 查询超时时间 | 设置查询未响应数据时的超时时间，单位ms毫秒，默认6000毫秒。 |
| 采集间隔 | 监控周期性采集数据间隔时间，单位秒，可设置的最小间隔为30秒 |
| 绑定标签 | 用于对监控资源进行分类管理。 |
| 描述备注 | 更多标识和描述此监控的备注信息，用户可以在这里备注信息。 |

### 采集指标

#### 指标集合：集群概览

| 指标名称 | 指标单位 | 指标帮助描述 |
|-----------------------|------|----------|
| projectVersion | 无 | 项目版本 |
| gitCommitAbbrev | 无 | Git提交编号 |
| totalSlot | 无 | 总槽位数 |
| unassignedSlot | 无 | 未分配槽位数 |
| runningJobs | 无 | 运行中任务数 |
| finishedJobs | 无 | 已完成任务数 |
| failedJobs | 无 | 失败任务数 |
| cancelledJobs | 无 | 取消任务数 |
| workers | 无 | Worker数 |

#### 指标集合：线程信息

| 指标名称 | 指标单位 | 指标帮助描述 |
|-------------|------|--------|
| threadName | 无 | 线程名称 |
| threadId | 无 | 线程ID |
| threadState | 无 | 线程状态 |
| stackTrace | 无 | 堆栈信息 |

#### 指标集合：节点监控

| 指标名称 | 指标单位 | 指标帮助描述 |
|------------|------|------------|
| isMaster | 无 | 是否主节点 |
| host | 无 | IP地址 |
| port | 无 | 端口 |
| processors | 无 | 处理器数 |
| physical.memory.total | 无 | 物理内存总量 |
| physical.memory.free | 无 | 物理内存可用 |
| swap.space.total | 无 | 交换空间总量 |
| swap.space.free | 无 | 交换空间可用 |
| heap.memory.used | 无 | 堆内存已用 |
| heap.memory.free | 无 | 堆内存可用 |
| heap.memory.total | 无 | 堆内存总量 |
| heap.memory.max | 无 | 堆内存最大 |
| heap.memory.used/total | 无 | 堆内存使用率 |
| heap.memory.used/max | 无 | 堆内存使用率最大 |
| minor.gc.count | 无 | 垃圾回收次数 |
| minor.gc.time | 无 | 垃圾回收时间 |
| major.gc.count | 无 | Major垃圾回收次数 |
| major.gc.time | 无 | Major垃圾回收时间 |
| load.process | 无 | 进程负载 |
| load.system | 无 | 系统负载 |
| load.systemAverage | 无 | 系统平均负载 |
| thread.count | 无 | 线程数 |
| thread.peakCount | 无 | 线程峰值 |
| cluster.timeDiff | 无 | 集群时间差 |
| event.q.size | 无 | 事件队列大小 |
| executor.q.async.size | 无 | 异步执行队列大小 |
| executor.q.client.size | 无 | 客户端执行队列大小 |
| executor.q.client.query.size | 无 | 客户端查询队列大小 |
| executor.q.client.blocking.size | 无 | 客户端阻塞队列大小 |
| executor.q.query.size | 无 | 查询队列大小 |
| executor.q.scheduled.size | 无 | 定时执行队列大小 |
| executor.q.io.size | 无 | IO队列大小 |
| executor.q.system.size | 无 | 系统执行队列大小 |
| executor.q.operations.size | 无 | 操作队列大小 |
| executor.q.priorityOperation.size | 无 | 优先操作队列大小 |
| operations.completed.count | 无 | 完成操作次数 |
| executor.q.mapLoad.size | 无 | 映射加载队列大小 |
| executor.q.mapLoadAllKeys.size | 无 | 映射加载所有键大小 |
| executor.q.cluster.size | 无 | 集群执行队列大小 |
| executor.q.response.size | 无 | 响应队列大小 |
| operations.running.count | 无 | 正在运行的操作数量 |
| operations.pending.invocations.percentage | 无 | 待处理调用百分比 |
| operations.pending.invocations.count | 无 | 待处理调用数量 |
| proxy.count | 无 | 代理数量 |
| clientEndpoint.count | 无 | 客户端端点数量 |
| connection.active.count | 无 | 活动连接数 |
| client.connection.count | 无 | 客户端连接数 |
| connection.count | 无 | 总连接数 |
3 changes: 2 additions & 1 deletion home/sidebars.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -304,7 +304,8 @@
"help/elasticsearch",
"help/influxdb",
"help/presto",
"help/hugegraph"
"help/hugegraph",
"help/seatunnel"
]
},
{
Expand Down
Loading
Loading