Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multi-tenancy extension documentation #290

Open
wants to merge 53 commits into
base: 4.6
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
dc3ae53
Added description of command line arguments valid for all commands
trimoq Aug 25, 2022
c2314fb
Merge pull request #280 from AxonIQ/feature/cli-option-description
trimoq Aug 25, 2022
a11e79c
Merge branch '4.6'
smcvb Aug 29, 2022
b49020a
Merge branch '4.6'
smcvb Sep 1, 2022
ef9ea1f
Merge branch '4.6'
MGathier Sep 2, 2022
b87c97b
Merge branch '4.6'
MGathier Sep 5, 2022
6cad9ad
Update docs with (expected) changes.
Sep 9, 2022
408fc56
Update extensions/kafka.md
Sep 12, 2022
103f25c
Update docs with (expected) changes.
Sep 12, 2022
ad76572
Deadletter documentation added
Aug 29, 2022
e143023
Added remark about idem potency
Aug 30, 2022
ac0e8b9
Rephrased sentence
Aug 30, 2022
93aa49e
Rephrased sentence
Aug 30, 2022
e99595b
Review comments
Aug 31, 2022
ae06307
Added non Spring configuration example
Sep 1, 2022
1a0a57a
Update axon-framework/events/event-processors/README.md
Sep 5, 2022
b5a3c98
Update axon-framework/events/event-processors/README.md
Sep 5, 2022
aebe42e
Update axon-framework/events/event-processors/README.md
Sep 5, 2022
4ac3e4d
Update axon-framework/events/event-processors/README.md
Sep 5, 2022
a86feb6
Update axon-framework/events/event-processors/README.md
Sep 5, 2022
73f17b9
Update axon-framework/events/event-processors/README.md
Sep 5, 2022
18c0977
Update axon-framework/events/event-processors/README.md
Sep 5, 2022
a30300f
Added a section for deadletter attributes
Sep 5, 2022
c1d4125
Fixed tupos
Sep 5, 2022
1c34858
Revert "Review comments" which introduced indentation adjustments
smcvb Sep 8, 2022
6bb860f
Adjust DLQ intro section
smcvb Sep 9, 2022
17e1387
Adjust DLQ configuration section
smcvb Sep 9, 2022
c8c16e2
Adjust DLQ processing section
smcvb Sep 9, 2022
2ffdecd
Adjust DLQ attributes section
smcvb Sep 9, 2022
43ed042
Adjust DLQ policy section
smcvb Sep 9, 2022
89b0a54
Adjust DLQ idempotency section
smcvb Sep 9, 2022
ad70236
Strengthen point the InMem is not production ready
smcvb Sep 9, 2022
7e9cd11
Update axon-framework/events/event-processors/README.md
smcvb Sep 9, 2022
110b947
Update axon-framework/events/event-processors/README.md
smcvb Sep 9, 2022
171be4a
Update axon-framework/events/event-processors/README.md
smcvb Sep 9, 2022
dca166c
Update axon-framework/events/event-processors/README.md
smcvb Sep 9, 2022
56d667f
Update axon-framework/events/event-processors/README.md
smcvb Sep 9, 2022
145a261
Update axon-framework/events/event-processors/README.md
smcvb Sep 9, 2022
bcbc63b
Update axon-framework/events/event-processors/README.md
smcvb Sep 9, 2022
825ee8a
Update axon-framework/events/event-processors/README.md
smcvb Sep 9, 2022
8fff711
Update axon-framework/events/event-processors/README.md
smcvb Sep 9, 2022
6f548b0
Update axon-framework/events/event-processors/README.md
smcvb Sep 9, 2022
2d2db38
Update axon-framework/events/event-processors/README.md
smcvb Sep 9, 2022
79f16af
Process review comments
smcvb Sep 9, 2022
98a1055
Merge remote-tracking branch 'origin/4.6'
smcvb Sep 19, 2022
3d5f344
Merge branch '4.6'
MGathier Sep 21, 2022
82f9498
Merge branch '4.6'
smcvb Oct 6, 2022
a5c3560
Merge branch '4.6'
smcvb Oct 13, 2022
367dd28
multitenancy documentation
schananas Oct 17, 2022
ae04f69
Apply suggestions from code review
schananas Oct 25, 2022
e9c65e4
multitenancy documentation - pr comments
schananas Oct 25, 2022
c3cc200
multitenancy documentation - pr comments
schananas Oct 25, 2022
dd4a272
multitenancy documentation - pr comments
schananas Oct 25, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -169,6 +169,7 @@
* [Reactor Gateways](extensions/reactor/reactive-gateways/reactive-gateways.md)
* [Spring Cloud](extensions/spring-cloud.md)
* [Tracing](extensions/tracing.md)
* [Multitenancy](extensions/multitenancy.md)

## Appendices

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ A quick summary of the various commands is depicted below. Each command has a sp
<thead>
<tr>
<th style="text-align:left">Area (Server Edition)</th>
<th style="text-align:left">Command-Line Options</th>
<th style="text-align:left">Command name</th>
<th style="text-align:left">Description</th>
</tr>
</thead>
Expand Down Expand Up @@ -254,6 +254,43 @@ axonserver-cli.jar <Command> <command options> -S <server-to-send-command-to> -

The option -S with the url to the Axon Server is optional, if it is omitted it defaults to [http://localhost:8024](http://localhost:8024/).‌ While for Axon Server SE, the URL for the Axon Server SE will be the single running node, for Axon Server EE, the URL should be pointing to any node serving the _\_admin_ context within an Axon Server EE cluster.

The `<command options>` valid for all commands, are: `-S`, `-s`, `-i`, `-o`.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldn't expect this change in your pull request to be honest 🤔

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Granted, the commit log as shown by GitHub strikes me as odd too.

Their effect is described in the table below.

<table>
<thead>
<tr>
<th style="text-align:left">Option - Short</th>
<th style="text-align:left">Option - Long</th>
<th style="text-align:left">Description</th>
</tr>
</thead>
<tbody>
<tr>
<td style="text-align:left">`-S`</td>
<td style="text-align:left">`server`</td>
<td style="text-align:left">Server to send command to (default http://localhost:8024)</td>
</tr>
<tr>
<td style="text-align:left">`-s`</td>
<td style="text-align:left">`https`</td>
<td style="text-align:left">Use HTTPS (SSL,TLS) to connect to the server, rather than HTTP.</td>
</tr>
<tr>
<td style="text-align:left">`-i`</td>
<td style="text-align:left">`insecure-ssl`</td>
<td style="text-align:left">Do not check the certificate when connecting using HTTPS.</td>
</tr>
<tr>
<td style="text-align:left">`-o`</td>
<td style="text-align:left">`output`</td>
<td style="text-align:left">Output format (txt,json)</td>
</tr>
</tbody>
</table>

For options specific to individual commands, see the descriptions of the commands below.

### Access control

When running Axon Server with access control enabled, executing commands remotely requires an access token. This needs to be provided with the -t option. When you run a command on the Axon Server node itself from the directory where Axon Server was started, you don't have to provide a token.‌
Expand Down
22 changes: 13 additions & 9 deletions extensions/kafka.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ public class KafkaEventPublicationConfiguration {
}
```

The second infrastructure component to introduce is the `KafkaPublisher`, which has a hard requirement on the `ProducerFactory`. Additionally, this would be the place to define the Kafka topic upon which Axon event messages will be published. Note that the `KafkaPublisher` needs to be `shutDown` properly, to ensure all `Producer` instances are properly closed.
The second infrastructure component to introduce is the `KafkaPublisher`, which has a hard requirement on the `ProducerFactory`. Additionally, this would be the place to define the Kafka topics upon which Axon event messages will be published. You can set a function from event to `Optional<String>`. You can use this to only publish certain events, or put different events to different topics. Its not uncommon for Kafka topics to only contain one type of message. Note that the `KafkaPublisher` needs to be `shutDown` properly, to ensure all `Producer` instances are properly closed.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldn't expect these changes in your pull request either 🤔


```java
public class KafkaEventPublicationConfiguration {
Expand All @@ -71,7 +71,7 @@ public class KafkaEventPublicationConfiguration {
KafkaMessageConverter<String, byte[]> kafkaMessageConverter,
int publisherAckTimeout) {
return KafkaPublisher.<String, byte[]>builder()
.topic(topic) // Defaults to "Axon.Events"
.topicResolver(m -> Optional.of(topic)) // Defaults to "Axon.Events" for all events
.producerFactory(producerFactory) // Hard requirement
.messageConverter(kafkaMessageConverter) // Defaults to a "DefaultKafkaMessageConverter"
.publisherAckTimeout(publisherAckTimeout) // Defaults to "1000" milliseconds; only used for "WAIT_FOR_ACK" mode
Expand Down Expand Up @@ -250,7 +250,7 @@ public class KafkaEventConsumptionConfiguration {
}
```

Note that as with any tracking event processor, the progress on the event stream is stored in a `TrackingToken`. Using the `StreamableKafkaMessageSource` means a `KafkaTrackingToken` containing topic-partition to offset pairs is stored in the `TokenStore`.
Note that as with any tracking event processor, the progress on the event stream is stored in a `TrackingToken`. Using the `StreamableKafkaMessageSource` means a `KafkaTrackingToken` containing topic-partition to offset pairs is stored in the `TokenStore`. If no other `TokenStore` is provided, and auto-configuration is used, a `KafkaTokenStore` will be set instead of an `InMemoryTokenStore`. The `KafkaTokenStore` by default uses the `__axon_token_store_updates` topic. This should be a compacted topic, which should be created and configured automatically.

## Customizing event message format

Expand Down Expand Up @@ -278,27 +278,31 @@ public class KafkaMessageConversationConfiguration {
BiFunction<String, Object, RecordHeader> headerValueMapper,
EventUpcasterChain upcasterChain) {
return DefaultKafkaMessageConverter.builder()
.serializer(serializer) // Hard requirement
.sequencingPolicy(sequencingPolicy) // Defaults to a "SequentialPerAggregatePolicy"
.upcasterChain(upcasterChain) // Defaults to empty upcaster chain
.serializer(serializer) // Hard requirement
.sequencingPolicy(sequencingPolicy) // Defaults to a "SequentialPerAggregatePolicy"
.upcasterChain(upcasterChain) // Defaults to empty upcaster chain
.headerValueMapper(headerValueMapper) // Defaults to "HeaderUtils#byteMapper()"
.build();
}
// ...
}
```

Make sure to use an identical `KafkaMessageConverter` on both the producing and consuming end, as otherwise exception upon deserialization should be expected.
Make sure to use an identical `KafkaMessageConverter` on both the producing and consuming end, as otherwise exception upon deserialization should be expected. A `CloudEventKafkaMessageConverter` is also available using the [Cloud Events](https://cloudevents.io/) spec.

## Configuration in Spring Boot

This extension can be added as a Spring Boot starter dependency to your project using group id `org.axonframework.extensions.kafka` and artifact id `axon-kafka-spring-boot-starter`. When using the auto configuration, the following components will be created for you automatically:

**Generic Components:**

* A `DefaultKafkaMessageConverter` using the configured `eventSerializer` \(which defaults to `XStreamSerializer`\).
* A `DefaultKafkaMessageConverter` using the configured `eventSerializer` \(which defaults to `XStreamSerializer`\), which is used by default to convert between Axon Event messages and Kafka records.

Uses a `String` for the keys and a `byte[]` for the record's values
Uses a `String` for the keys and a `byte[]` for the record's values.

When the property `axon.kafka.message-converter-mode` is set to `cloud_event` a `CloudEventKafkaMessageConverter` will be used instead. This will use `String` for the keys and `CloudEvent`.

For each the matching Kafka (de)serializers will also be set as default.

**Producer Components:**

Expand Down
148 changes: 148 additions & 0 deletions extensions/multitenancy.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,148 @@
# Multitenancy Extension

The Axon Framework Multitenancy Extension provides your application with the ability to serve multiple tenants (event-stores) at once.
Multi-tenancy is important in cloud computing, as this extension provides the ability to connect tenants dynamically, physical separate tenant-data, and scale tenants independently.

### Requirements

- Currently, It's possible to configure extension using **Axon Framework 4.6+** together with **Spring Framework**.
- Minimal configuration and out-of-the box solution is available only for **Axon Server EE 4.6+** or Axon Cloud (*).
- Any other custom user solutions should implement [own factory beans for components and tenant provider](https://github.com/AxonFramework/extension-multitenancy/blob/main/multitenancy-spring-boot-autoconfigure/src/main/java/org/axonframework/extensions/multitenancy/autoconfig/MultiTenancyAxonServerAutoConfiguration.java)
- If you wish to enable multi-tenancy for your projections and token store, note that only JPA is supported out-of-the box.

> **Axon Cloud and the Multi-Tenancy extension**
>
> Currently, Axon Cloud works only with static tenant configuration.

### Configuration

A minimal configuration is needed to get this extension up and running.
Please choose **either** the static or the dynamic tenant configuration.

#### Static tenants configuration

If you have a predefined list of tenants that your application should connect to, set following property:
`axon.axonserver.contexts=tenant-context-1,tenant-context-2,tenant-context-3`

#### Dynamic tenants configuration

If you plan to create tenants during runtime, you can define a predicate that will tell the application to which tenant-contexts to connect to once they appear:

```java
@Bean
public TenantConnectPredicate tenantFilterPredicate() {
return context -> context.tenantId().startsWith("tenant-");
}
```

### Route Messages to specific tenants

Backbone of multitenancy is ability to route message to specific tenant.
This extension offers you meta-data based routing which is ready to be used with minimal configuration.
Also, one may wish to define stronger contract and include tenant information in message payload, which is also possible by defining custom tenant resolver.

#### Using meta-data

By default, to route any `Message` to a specific tenant, you need to tag the initial message that enters your system with metadata.
This is done with a meta-data helper function, which should add the tenant name with key `TenantConfiguration.TENANT_CORRELATION_KEY`.

```java
message.andMetaData(Collections.singletonMap(TENANT_CORRELATION_KEY, "tenant-context-1")
```

Note that you only need to add metadata to the initial message entering your system.
Any message produced as a consequence of the initial message will have this metadata copied automatically using a `CorrelationDataProvider`.

#### Custom tenant resolver

If you wish to define a custom tenant resolver, set following property:

`axon.multi-tenancy.use-metadata-helper=false`

Then define the custom tenant resolver bean.
The following example can use the message payload to route a message to specific tenant:

```java
@Bean
public TargetTenantResolver<Message<?>> customTargetTenantResolver() {
return (message, tenants) ->
TenantDescriptor.tenantWithId(
((TenantAwareMessage) message.getPayload()).getTenantName()
);
}
```

In example above, all messages should implement custom `TenantAwareMessage` interface that exposes tenant name.
Then we can use this interface to extract tenant name from the payload and define our tenant resolver.

### Multi-tenant projections

If you wish to use distinct tenant-databases to store projections and tokens, please configure the following:

```java
@Bean
public Function<TenantDescriptor, DataSourceProperties> tenantDataSourceResolver() {
return tenant -> {
DataSourceProperties properties = new DataSourceProperties();
properties.setUrl("jdbc:postgresql://localhost:5432/"+tenant.tenantId());
properties.setDriverClassName("org.postgresql.Driver");
properties.setUsername("postgres");
properties.setPassword("postgres");
return properties;
};
}
```

Note that this works by using the JPA multi-tenancy support provided in this extension.
That means that currently only SQL Databases are supported out of the box.

If you wish to implement multi-tenancy for a different type of databases (e.g. NoSQL) make sure that your projection database supports multi-tenancy, too.
When doing so, you can find it which tenants own the transaction by invoking `TenantWrappedTransactionManager.getCurrentTenant()`.

> **Pre-initialized schema**
>
> Schema migration tools like Liquibase or Flyway usually won't be able to initialize schemas for dynamically created data sources.
> Hence, any data source that you use needs to have a the schema pre-initialized.

#### Resetting projections

Resetting projections works a bit different, because there are multiple instances of the "same" event processor.
Namely, one per tenant.

Regard the following sample to reset an Event Processor for a specific tenant:

```java
TrackingEventProcessor trackingEventProcessor = configuration.eventProcessingConfiguration()
.eventProcessor("com.demo.query-ep@tenant-context-1", TrackingEventProcessor.class)
.get();
```

Note that the convention for naming tenant-specific event processor is `{even processor name}@{tenant name}`.

If you need to access all tenant event processors in one go, you can retrieve the `MultiTenantEventProcessor` for a specific processing name.
The `MultiTenantEventProcessor` acts as a proxy event processor referencing all tenant-specific event processors.

### Supported multi-tenant components

Currently, the following infrastructure components support multi-tenancy:

- <span style="color:green">MultiTenantCommandBus</span>
- <span style="color:green">MultiTenantEventProcessor</span>
- <span style="color:green">MultiTenantEventStore</span>
- <span style="color:green">MultiTenantQueryBus</span>
- <span style="color:green">MultiTenantQueryUpdateEmitter</span>
- <span style="color:green">MultiTenantEventProcessorControlService</span>
- <span style="color:green">MultiTenantDataSourceManager</span>

The following components are not yet supported:

- <span style="color:red">MultitenantDeadlineManager</span>
- <span style="color:red">MultitenantEventScheduler</span>


### Disabling this Extension

By default, this extension is enabled if found on class path when utilizing Spring Boot.
If you wish to disable the extension without removing the dependency, you can set the following property to `false`:

`axon.multi-tenancy.enabled=false`