Skip to content

Commit

Permalink
docs: add docs for custom tasks
Browse files Browse the repository at this point in the history
  • Loading branch information
MistEO committed Dec 7, 2023
1 parent 356cdfa commit 13839d7
Show file tree
Hide file tree
Showing 2 changed files with 191 additions and 0 deletions.
95 changes: 95 additions & 0 deletions docs/en_us/2.3-CustomTasks.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
# Custom Tasks

In addition to using the pipeline protocol to write JSON for low-code development, MaaFramework also supports various ways to write code for custom tasks. Currently, there are several main approaches:

- [FFI (Foreign Function Interface)](#ffi)

This framework provides a C language API, and other languages can integrate relevant interfaces by calling bindings. In theory, it supports all mainstream languages, although the number of adapted bindings is currently limited and may require you to write your own (feel free to submit a PR after doing so!).
Advantages: Relatively high execution efficiency, more in line with standard programming paradigms.

- [Executor Proxy](#executor-proxy)

In simple terms, developers can write their own executable (exe) and pass the exe path through MaaToolkit interfaces. When MaaFramework reaches the corresponding task, it calls the exe, passing in the current screen's image, task information, and other custom parameters as command-line arguments.
Developers can perform additional actions in their exe and can output commands directly (print / std::cout / ...) following the [protocol](#input-output-protocol). MaaFramework captures these outputs through a pipeline to execute more tasks such as taking screenshots, clicking, sliding, recognition, etc. The results are then passed back through the pipeline, which can be obtained through standard input (input / std::cin / ...).
The exe can be any executable file, including exe, bat, shell, Python scripts, etc. (also supports executable files under Linux / macOS).
Advantages: Simple.

## FFI

Integrate [custom actions](https://github.com/MaaAssistantArknights/MaaFramework/blob/main/include/MaaFramework/Task/MaaCustomAction.h) and [custom recognizers](https://github.com/MaaAssistantArknights/MaaFramework/blob/main/include/MaaFramework/Task/MaaCustomRecognizer.h) using relevant [registration interfaces](https://github.com/MaaAssistantArknights/MaaFramework/blob/main/include/MaaFramework/Instance/MaaInstance.h#L20). The `MaaSyncContextHandle` can be used to invoke more recognition, screenshots, clicks, slides, etc. Please refer to the [Sample](https://github.com/MaaAssistantArknights/MaaFramework/blob/main/sample/cpp/main.cpp#L90).

## Executor Proxy

### Launch Parameters

#### Custom Recognizer

For custom recognizers, the launch parameters are:

```shell
/path/to/my_recognizer.exe custom_arg_1 custom_arg_2 ... sync_context image_path task_name custom_recognition_param
```

- `/path/to/my_recognizer.exe`

The path to the executable file passed through the registration interface. If it is a script like Python, you can also directly pass `Python.exe`.

- `custom_arg_1 custom_arg_2 ...`

Custom parameters passed through the registration interface (multiple parameters). If it is a script like Python, the first parameter is the path to your `.py` file.

- `sync_context`

Information needed when outputting commands to MaaFramework according to the protocol. See [Input-Output Protocol](#input-output-protocol) for details.

- `image_path`

File path of the current screen's screenshot image.

- `task_name`

The name of the task currently being executed.

- `custom_recognition_param`

The value of `custom_recognition_param` defined in the pipeline JSON.

#### Custom Action

For custom actions, the launch parameters are:

```shell
/path/to/custom_action.exe custom_arg_1 custom_arg_2 ... sync_context task_name custom_action_param cur_box cur_rec_detail
```

- `/path/to/custom_action.exe`

The path to the executable file passed through the registration interface. If it is a script like Python, you can also directly pass `Python.exe`.

- `custom_arg_1 custom_arg_2 ...`

Custom parameters passed through the registration interface (multiple parameters). If it is a script like Python, the first parameter is the path to your `.py` file.

- `task_name`

The name of the task currently being executed.

- `sync_context`

Information needed when outputting commands to MaaFramework according to the protocol. See [Input-Output Protocol](#input-output-protocol) for details.

- `custom_action_param`

The value of `custom_action_param` defined in the pipeline JSON.

- `cur_box`

The current target position recognized by the task recognizer. Format is JSON array [x, y, w, h].

- `cur_rec_detail`

Detailed information recognized by the task recognizer. Format is JSON, specific protocol to be added~.

### Input-Output Protocol

TODO
96 changes: 96 additions & 0 deletions docs/zh_cn/2.3-自定义任务.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
# 自定义任务

除使用流水线协议编写 JSON 实现低代码开发外,MaaFramework 也支持通过多种方式自行编写代码来实现自定义任务,目前主要有以下几种方式:

- [FFI(语言交互接口)](#ffi)

本框架提供了 C 语言 API,其他语言可通过调用 Binding 的方式来集成相关接口,理论上支持所有主流语言。
但目前已适配的 Binding 较少,可能需要您自行编写(也欢迎在写完后向我们提交 PR!)。
优点:执行效率相对较高,更符合标准编程范式。

- [执行器代理](#执行器代理)

简单来说,开发者可以编写一个自己的 exe,并通过 MaaToolkit 相关接口传入 exe 路径,MaaFramework 会在执行到对应任务时,调用该 exe 并通过启动参数传入当前画面的图片、任务信息、及其他自定义参数。
开发者可在自己的 exe 中执行自己需要的更多动作,并可以按照 [协议](#输入输出协议) 直接输出(print / std::cout / ...)部分命令,MaaFramework 会通过管道捕获这些输出,去执行更多的截图、点击滑动、识别等操作,然后 MaaFramework 也会通过管道来传递执行结果,可通过标准输入(input / std::cin / ...)来获取。
这里的 exe 可以是任何可执行文件,包括 exe, bat, shell, python 脚本等(同样也支持 Linux / macOS 下的可执行文件)。
优点:简单。

## FFI

可集成 [自定义动作](https://github.com/MaaAssistantArknights/MaaFramework/blob/main/include/MaaFramework/Task/MaaCustomAction.h)[自定义识别器](https://github.com/MaaAssistantArknights/MaaFramework/blob/main/include/MaaFramework/Task/MaaCustomRecognizer.h),并通过相关的 [注册接口](https://github.com/MaaAssistantArknights/MaaFramework/blob/main/include/MaaFramework/Instance/MaaInstance.h#L20) 传入。其中的 `MaaSyncContextHandle` 可通过 [SyncContext 接口](https://github.com/MaaAssistantArknights/MaaFramework/blob/main/include/MaaFramework/Task/MaaSyncContext.h) 调用更多识别、截图、点击滑动等,请参考 。可参考 [Sample](https://github.com/MaaAssistantArknights/MaaFramework/blob/main/sample/cpp/main.cpp#L90)

## 执行器代理

### 启动参数

#### 自定义识别器(CustomRecognizer)

对于自定义识别器,启动参数为:

```shell
/path/to/my_recognizer.exe custom_arg_1 custom_arg_2 ... sync_context image_path task_name custom_recognition_param
```

- `/path/to/my_recognizer.exe`

通过注册接口传入的可执行文件路径。若是 Python 等脚本,也可直接传入 `Python.exe`

- `custom_arg_1 custom_arg_2 ...`

通过注册接口传入的自定义参数(多个参数)。若是 Python 等脚本,其中第一个参数则为你的 `.py` 文件路径。

- `sync_context`

按照协议向 MaaFramework 输出命令时,所需要带上的信息。详见 [输入输出协议](#输入输出协议)

- `image_path`

当前画面截图的图片文件路径。

- `task_name`

当前正在执行的任务名。

- `custom_recognition_param`

在流水线 Json 中定义的 `custom_recognition_param` 值。

#### 自定义动作(CustomAction)

对于自定义动作,启动参数为:

```shell
/path/to/custom_action.exe custom_arg_1 custom_arg_2 ... sync_context task_name custom_action_param cur_box cur_rec_detail
```

- `/path/to/custom_action.exe`

通过注册接口传入的可执行文件路径。若是 Python 等脚本,也可直接传入 `Python.exe`

- `custom_arg_1 custom_arg_2 ...`

通过注册接口传入的自定义参数(多个参数)。若是 Python 等脚本,其中第一个参数则为你的 `.py` 文件路径。

- `task_name`

当前正在执行的任务名。

- `sync_context`

按照协议向 MaaFramework 输出命令时,所需要带上的信息。详见 [输入输出协议](#输入输出协议)

- `custom_action_param`

在流水线 Json 中定义的 `custom_action_param` 值。

- `cur_box`

本任务识别器当前识别到的目标位置。格式为 Json 数组,[x, y, w, h]

- `cur_rec_detail`

本任务识别器识别到的详细信息。格式为 Json,具体协议待补充~

### 输入输出协议

TODO

0 comments on commit 13839d7

Please sign in to comment.