Implement workflow for automatic ECS templates generation

.github/workflows/generate-ecs-mappings.yml

@@ -0,0 +1,43 @@
+name: ECS Generator
+
+on:
+  push:
+    paths:
+      - "ecs/**/*.json"
+      - "ecs/**/*.yml"
+
+jobs:
+  run-ecs-generator:
+    if: github.repository == 'wazuh/wazuh-indexer'
+    runs-on: ubuntu-24.04
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 2
+
+      - name: Extract branch name
+        shell: bash
+        run: echo "branch=${GITHUB_HEAD_REF:-${GITHUB_REF#refs/heads/}}" >> $GITHUB_OUTPUT
+        id: branch-name
+
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v3
+
+      - name: Set up Docker Compose
+        run: sudo apt-get install docker-compose
+
+      - name: Generate PR to wazuh-indexer-plugins
+        env:
+          GITHUB_TOKEN: ${{ secrets.ACTION_TOKEN }}
+        run: |
+          bash ecs/scripts/generate-pr-to-plugins.sh \
+            -b ${{ steps.branch-name.outputs.branch }} \
+            -o ./ecs-templates
+
+      - name: Upload artifact
+        uses: actions/upload-artifact@v4
+        with:
+          name: ecs-templates
+          path: "./ecs-templates"

ecs/README.md

@@ -4,71 +4,22 @@ This script generates the ECS mappings for the Wazuh indices.
 
 ### Requirements
 
-- ECS repository clone. The script is meant to be launched from the root level of that repository.
-- `Python` 3.6 or higher + `venv` module
-- `jq`
+- [Docker Compose](https://docs.docker.com/compose/install/)
 
 ### Folder structure
 
-There is a folder for each module. Inside each folder, there is a `fields` folder with the required
-files to generate the mappings. These are the inputs for the ECS generator.
+There is a folder for each module. Inside each folder, there is a `fields` folder with the required files to generate the mappings. These are the inputs for the ECS generator.
 
 ### Usage
 
-1. Get a copy of the ECS repository at the same level as the `wazuh-indexer` repo:
-
-    ```console
-    git clone [email protected]:elastic/ecs.git
-    ```
-
-2. Install the dependencies:
-
-    ```console
-    cd ecs
-    python3 -m venv env
-    source env/bin/activate
-    pip install -r scripts/requirements.txt
-    ```
-
-2. Copy the `generate.sh` script to the root level of the ECS repository.
-
-    ```console
-    cp generate.sh ../../ecs
-    cd ../../ecs
-    bash generate.sh
-    ```
-
-    Expected output:
-    ```
-    Usage: generate.sh <ECS_VERSION> <INDEXER_SRC> <MODULE> [--upload <URL>]
-      * ECS_VERSION: ECS version to generate mappings for
-      * INDEXER_SRC: Path to the wazuh-indexer repository
-      * MODULE: Module to generate mappings for
-      * --upload <URL>: Upload generated index template to the OpenSearch cluster. Defaults to https://localhost:9200
-    Example: generate.sh v8.11.0 ~/wazuh-indexer states-vulnerabilities --upload https://indexer:9200
-    ```
-
-3. Use the `generate.sh` script to generate the mappings for a module. The script takes 3 arguments,
-plus 2 optional arguments to upload the mappings to the `wazuh-indexer`. Both, composable and legacy mappings
-are generated. For example, to generate the mappings for the `states-vulnerabilities` module using the
-    ECS version `v8.11.0` and assuming that path of this repository is `~/wazuh/wazuh-indexer`:
-
+1. Execute the mapping-generator tool
     ```bash
-    ./generate.sh v8.11.0 ~/wazuh/wazuh-indexer states-vulnerabilities
-    ```
-
-    The tool will output the folder where they have been generated.
-
-    ```console
-    Loading schemas from git ref v8.11.0
-    Running generator. ECS version 8.11.0
-    Mappings saved to ~/wazuh/wazuh-indexer/ecs/states-vulnerabilities/mappings/v8.11.0
+    bash ecs/generator/mapping-generator.sh run <MODULE_NAME>
     ```
-
-4. When you are done. Exit the virtual environment.
-
-    ```console
-    deactivate
+2. (Optional) Run the tool's cleanup
+    > The tool stops the container automatically, but it is recommended to run the down command if the tool is not going to be used anymore.
+    ```bash
+    bash ecs/generator/mapping-generator.sh down
     ```
 
 ### Output
@@ -80,8 +31,8 @@ For our use case, the most important files are under `mappings/<ECS_VERSION>/gen
 - `template.json`: Elasticsearch compatible index template for the module
 - `opensearch-template.json`: OpenSearch compatible index template for the module
 
-The original output is `template.json`, which is not compatible with OpenSearch by default. In order
-to make this template compatible with OpenSearch, the following changes are made:
+The original output is `template.json`, which is not compatible with OpenSearch by default.
+In order to make this template compatible with OpenSearch, the following changes are made:
 
 - The `order` property is renamed to `priority`.
 - The `mappings` and `settings` properties are nested under the `template` property.
@@ -104,64 +55,74 @@ Notes:
 ### Adding new mappings
 
 The easiest way to create mappings for a new module is to take a previous one as a base.
-Copy a folder and rename it to the new module name. Then, edit the `fields` files to
-match the new module fields.
+Copy a folder and rename it to the new module name. Then, edit the `fields` files to match the new module fields.
 
-The name of the folder will be the name of the module to be passed to the script. All 3 files
-are required.
+The name of the folder will be the name of the module to be passed to the script. All 3 files are required.
 
 - `fields/subset.yml`: This file contains the subset of ECS fields to be used for the module.
 - `fields/template-settings-legacy.json`: This file contains the legacy template settings for the module.
 - `fields/template-settings.json`: This file contains the composable template settings for the module.
 
 ### Event generator
 
-For testing purposes, the script `generate_events.py` can be used to generate events for a given module.
-Currently, it is only able to generate events for the `states-vulnerabilities` module. To support other
-modules, please extend of refactor the script.
-
-The script prompts for the required parameters, so it can be launched without arguments:
+Each module contains a Python script to generate events for its module. The script prompts for the required parameters, so it can be launched without arguments:
 
 ```bash
 ./event_generator.py
 ```
 
-The script will generate a JSON file with the events, and will also ask whether to upload them to the
-indexer. If the upload option is selected, the script will ask for the indexer URL and port, credentials,
-and index name.
-
+The script will generate a JSON file with the events, and will also ask whether to upload them to the indexer. If the upload option is selected, the script will ask for the indexer URL and port, credentials, and index name.
 The script uses log file. Check it out for debugging or additional information.
 
+---
+
+### Automatic PR creation tool
+
+The `generate-pr-to-plugins.sh` script found in the `ecs/scripts` folder is a tool that detects modified ECS modules, generates new templates, commits the changes to a target repository, and creates or updates a pull request.
+
+#### Requirements
+
+- Docker Compose
+- GitHub CLI (`gh`)
+
+#### Usage
+
+To use the script, run the following command:
+
+```sh
+./update-ecs-templates.sh -t <GITHUB_TOKEN>
+```
+
+**Options**
+
+- `-b <BRANCH_NAME>`: (Optional) Branch name to create or update the pull request. Default is current branch.
+- `-t <GITHUB_TOKEN>`: (Optional) GitHub token to authenticate with the GitHub API. If not provided, the script will use the `GITHUB_TOKEN` environment variable.
+
+#### Script Workflow
+
+1. **Validate Dependencies**
+   - Checks if the required commands (`docker`, `docker-compose`, and `gh`) are installed.
+
+2. **Detect Modified Modules**
+   - Fetches and extracts modified ECS modules by comparing the current branch with the base branch.
+   - Identifies relevant ECS modules that have been modified.
+
+3. **Run ECS Generator**
+   - Runs the ECS generator script for each relevant module to generate new ECS templates.
+
+4. **Clone Target Repository**
+   - Clones the target repository (`wazuh/wazuh-indexer-plugins`) if it does not already exist.
+   - Configures Git and GitHub CLI with the provided GitHub token.
+
+5. **Commit and Push Changes**
+   - Copies the generated ECS templates to the appropriate directory in the target repository.
+   - Commits and pushes the changes to the specified branch.
+
+6. **Create or Update Pull Request**
+   - Creates a new pull request or updates an existing pull request with the modified ECS templates.
+
 #### References
 
 - [ECS repository](https://github.com/elastic/ecs)
 - [ECS usage](https://github.com/elastic/ecs/blob/main/USAGE.md)
 - [ECS field reference](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html)
-
-### All-in-one script
-
-```bash
-#!/bin/bash
-
-indices=(
-    agent
-    alerts
-    command
-    states-fim
-    states-inventory-hardware
-    states-inventory-hotfixes
-    states-inventory-networks
-    states-inventory-packages
-    states-inventory-ports
-    states-inventory-processes
-    states-inventory-system
-    states-vulnerabilities
-)
-
-ECS="v8.11.0"
-WI_REPO_PATH=~/wazuh/wazuh-indexer
-
-for index in "${indices[@]}"; do
-    bash generate.sh $ECS $WI_REPO_PATH "$index"
-done
-```