Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AMORO-3404] Correct the description about MySQL usage #3416

Merged
merged 2 commits into from
Jan 22, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions docs/admin-guides/deployment-on-kubernetes.md
Original file line number Diff line number Diff line change
Expand Up @@ -156,17 +156,17 @@ ingress:

### Configure the database.

AMS default is to use Derby database for storage. When the pod is destroyed, the data will also disappear.
In production environments, we recommend using MySQL as the storage for system data.
AMS uses embedded [Apache Derby](https://db.apache.org/derby/) as its backend storage by default.
In production environments, we recommend using a RDBMS(Relational Database Management System) with higher availability guarantees as the storage for system data, you can ref to [Database Configuration](/deployment/#configure-system-database) for more detail.

```yaml
amoroConf:
database:
type: mysql
driver: com.mysql.cj.jdbc.Driver
url: <jdbc-uri>
username: <mysql-user>
password: <mysql-password>
type: ${your_database_type}
driver: ${your_database_driver}
url: ${your_jdbc_url}
username: ${your_username}
password: ${your_password}
```


Expand Down Expand Up @@ -291,4 +291,4 @@ Notes:

By default, the `serviceAccount` will be used to create the flink-optimizer.
Therefore, if you need to schedule the flink-optimizer across namespaces,
please create a `cluster-role` or use your own created `serviceAccount`.
please create a `cluster-role` or use your own created `serviceAccount`.
40 changes: 12 additions & 28 deletions docs/admin-guides/deployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,42 +100,26 @@ Make sure the port is not used before configuring it.

### Configure system database

You can use MySQL/PostgreSQL as the system database instead of the default Derby.
AMS uses embedded [Apache Derby](https://db.apache.org/derby/) as the backend storage by default, so you can use `Derby` directly without any additional configuration.

If you would like to use MySQL as the system database, you need to manually download the [MySQL JDBC Connector](https://repo1.maven.org/maven2/com/mysql/mysql-connector-j/8.1.0/mysql-connector-j-8.1.0.jar)
and move it into the `{AMORO_HOME}/lib/` directory. You can use the following command to complete these operations:
```shell
$ cd ${AMORO_HOME}
$ MYSQL_JDBC_DRIVER_VERSION=8.0.30
$ wget https://repo1.maven.org/maven2/mysql/mysql-connector-java/${MYSQL_JDBC_DRIVER_VERSION}/mysql-connector-java-${MYSQL_JDBC_DRIVER_VERSION}.jar
$ mv mysql-connector-java-${MYSQL_JDBC_DRIVER_VERSION}.jar lib
```
You can also configure a relational backend storage as you needed.

Create an empty database in MySQL/PostgreSQL, then AMS will automatically create tables in this MySQL/PostgreSQL database when it first started.
If you want to create tables on yourself, set `ams.database.auto-create-tables` to false.
> If you would like to use MySQL as the system database, you need to manually download the [MySQL JDBC Connector](https://repo1.maven.org/maven2/com/mysql/mysql-connector-j/8.1.0/mysql-connector-j-8.1.0.jar)
and move it into the `${AMORO_HOME}/lib/` directory.

You need to create an empty database in the RDBMS before to start the server, then AMS will automatically create tables in the database when it first started.

One thing you need to do is Adding MySQL/PostgreSQL configuration under `config.yaml` of Ams:
One thing you need to do is adding configuration under `config.yaml` of Ams:

```yaml
# MySQL
ams:
database:
type: mysql
jdbc-driver-class: com.mysql.cj.jdbc.Driver
url: jdbc:mysql://127.0.0.1:3306/amoro?useUnicode=true&characterEncoding=UTF8&autoReconnect=true&useAffectedRows=true&allowPublicKeyRetrieval=true&useSSL=false
username: root
password: root
type: ${database_type} # postgres or mysql
jdbc-driver-class: ${your_driver_name}
url: ${your_jdbc_url}
username: ${your_username}
password: ${your_password}
auto-create-tables: true
# PostgreSQL
#ams:
# database:
# type: postgres
# jdbc-driver-class: org.postgresql.Driver
# url: jdbc:postgresql://127.0.0.1:5432/amoro
# auto-create-tables: false
# username: user
# password: passwd
```

### Configure high availability
Expand Down Expand Up @@ -301,7 +285,7 @@ $ bin/ams.sh stop

### Upgrade system databases

You can find all the upgrade SQL scripts under `{AMORO_HOME}/conf/mysql/` with name pattern `upgrade-a.b.c-to-x.y.z.sql`.
You can find all the upgrade SQL scripts under `${AMORO_HOME}/conf/${db_type}/` with name pattern `upgrade-a.b.c-to-x.y.z.sql`.
Execute the upgrade SQL scripts one by one to your system database based on your starting and target versions.

### Replace all libs and plugins
Expand Down
4 changes: 2 additions & 2 deletions docs/concepts/catalogs.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ the multi-catalog feature can be used to support SQL across data sources, such a

```SQL
SELECT c.ID, c.NAME, c.AGE, o.AMOUNT
FROM MYSQL.ONLINE.CUSTOMERS c JOIN HIVE.OFFLINE.ORDERS o
FROM ${CATALOG_A}.ONLINE.CUSTOMERS c JOIN ${CATALOG_B}.OFFLINE.ORDERS o
ON (c.ID = o.CUSTOMER_ID)
```

Expand Down Expand Up @@ -67,4 +67,4 @@ Through metadata center and SQL-based federated computing of the computing engin
such as DataOps and DataFabric
- Automatic catalog detection: In compute engines like Spark and Flink, it is possible to automatically detect the creation and changes of a
catalog, enabling a one-time configuration for permanent scalability.


Loading