Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Orchestration Grounding convenience #293

Merged
merged 14 commits into from
Jan 27, 2025
46 changes: 23 additions & 23 deletions docs/guides/ORCHESTRATION_CHAT_COMPLETION.md
Original file line number Diff line number Diff line change
Expand Up @@ -206,31 +206,31 @@ In this example, the input will be masked before the call to the LLM and will re
Use the grounding module to provide additional context to the AI model.

```java
var message =
Message.user(
"{{?groundingInput}} Use the following information as additional context: {{?groundingOutput}}");
var prompt =
new OrchestrationPrompt(Map.of("groundingInput", "What does Joule do?"), message);

var filterInner =
DocumentGroundingFilter.create().id("someID").dataRepositoryType(DataRepositoryType.VECTOR);
var groundingConfigConfig =
GroundingModuleConfigConfig.create()
.inputParams(List.of("groundingInput"))
.outputParam("groundingOutput")
.addFiltersItem(filterInner);

var groundingConfig =
GroundingModuleConfig.create()
.type(GroundingModuleConfig.TypeEnum.DOCUMENT_GROUNDING_SERVICE)
.config(groundingConfigConfig);
var configWithGrounding = config.withGroundingConfig(groundingConfig);

var result =
new OrchestrationClient().chatCompletion(prompt, configWithGrounding);
// optional filter for collections
var documentMetadata =
SearchDocumentKeyValueListPair.create()
.key("my-collection")
.value("value")
.addSelectModeItem(SearchSelectOptionEnum.IGNORE_IF_KEY_ABSENT);
// optional filter for document chunks
var databaseFilter =
DocumentGroundingFilter.create()
.id("")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Minor/Comment)

According to spec this identifier shall be unique per request.
Multiple filters are possible per requset.

See spec
GroundingModuleConfig:
  type: object
  required:
    - type
    - config
  additionalProperties: false
  properties:
    type:
      ...
    config:
      type: object
      required:
        - input_params
        - output_param
      additionalProperties: false
      properties:
        filters:
          type: array
          items:
            oneOf:
              - $ref: "#/components/schemas/DocumentGroundingFilter"
          description: Document grounding service filters to be used
        ...

DocumentGroundingFilter:
  type: object
  required:
    - id
    - data_repository_type
  additionalProperties: false
  properties:
    id:
      $ref: "#/components/schemas/GroundingFilterId"

...

GroundingFilterId:
  title: Id
  description: Identifier of this SearchFilter - unique per request.

GroundingFilterSearchConfiguration:
  ...

With the proposed convenience API we are

  • limiting users to define exactly one filter - this is why "id" seems unnecessary.
  • sending a filter no matter what - is this a requirement, that at least one filter must be present?
  • not blocking future changes to cardinality - which is good. Looks like we could easily extend the logic.

Copy link
Contributor

@newtork newtork Jan 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Therefore:
Would allow for setting multiple filter on Grounding then for 1..*?
Or will you keep the cardinality 1?
Is 0 filters an option we have to consider?

Copy link
Contributor Author

@CharlesDuboisSAP CharlesDuboisSAP Jan 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I modified the API to allow 1+ filters.
0 filters is not allowed because we have to specify the repository type

.dataRepositoryType(DataRepositoryType.VECTOR)
.addDocumentMetadataItem(documentMetadata);

var groundingConfig = Grounding.create().filter(databaseFilter);
var prompt = groundingConfig.createGroundingPrompt("What does Joule do?");
var configWithGrounding = config.withGrounding(groundingConfig);

var result = client.chatCompletion(prompt, configWithGrounding);
```

In this example, the AI model is provided with additional context in the form of grounding information. Note, that it is necessary to provide the grounding input via one or more input variables.
In this example, the AI model is provided with additional context in the form of grounding information.

`Grounding.create()` is by default a document grounding service with a vector data repository.

Please find [an example in our Spring Boot application](../../sample-code/spring-app/src/main/java/com/sap/ai/sdk/app/services/OrchestrationService.java).

## Stream chat completion

Expand Down
1 change: 1 addition & 0 deletions docs/release-notes/release_notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@

- New Orchestration features:
- [Spring AI integration](../guides/ORCHESTRATION_CHAT_COMPLETION.md#spring-ai-integration)
- [Add Grounding configuration convenience](../guides/ORCHESTRATION_CHAT_COMPLETION.md#grounding)
- Images are now supported as input in newly introduced `MultiChatMessage`.
- `MultiChatMessage` also allows for multiple content items (text or image) in one object.
- Grounding input can be masked with `DPIConfig`.
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
package com.sap.ai.sdk.orchestration;

import com.google.common.annotations.Beta;
import com.sap.ai.sdk.orchestration.model.DataRepositoryType;
import com.sap.ai.sdk.orchestration.model.DocumentGroundingFilter;
import com.sap.ai.sdk.orchestration.model.GroundingModuleConfig;
import com.sap.ai.sdk.orchestration.model.GroundingModuleConfig.TypeEnum;
import com.sap.ai.sdk.orchestration.model.GroundingModuleConfigConfig;
import com.sap.ai.sdk.orchestration.model.GroundingModuleConfigConfigFiltersInner;
import java.util.List;
import java.util.Map;
import javax.annotation.Nonnull;
import lombok.Setter;
import lombok.experimental.Accessors;
import lombok.val;

/**
* Grounding integrates external, contextually relevant, domain-specific, or real-time data into AI
* processes. This data supplements the natural language processing capabilities of pre-trained
* models, which are trained on general material.
*
* @link <a href="https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/grounding">SAP AI
* Core: Orchestration - Grounding</a>
*/
@Beta
@Accessors(fluent = true)
public class Grounding implements GroundingProvider {

@Nonnull
private List<GroundingModuleConfigConfigFiltersInner> filters =
List.of(
DocumentGroundingFilter.create().id("").dataRepositoryType(DataRepositoryType.VECTOR));

@Setter(onMethod_ = {@Nonnull})
private TypeEnum documentGroundingService = TypeEnum.DOCUMENT_GROUNDING_SERVICE;

/**
* Create a new default grounding provider.
*
* <p>It is by default a document grounding service with a vector data repository.
*
* @return The grounding provider.
*/
@Nonnull
public static Grounding create() {
return new Grounding();
}

/**
* Set filters for grounding.
*
* @param filters List of filters to set.
* @return The modified grounding configuration.
*/
@Nonnull
public Grounding filters(@Nonnull final GroundingModuleConfigConfigFiltersInner... filters) {
if (filters.length != 0) {
this.filters = List.of(filters);
}
return this;
}

/**
* Create a prompt with grounding parameters included in the message.
*
* <p>It uses the inputParams {@code userMessage} for the user message and {@code
* groundingContext} for the grounding context.
*
* @param message The user message.
* @return The prompt with grounding.
*/
@Nonnull
public OrchestrationPrompt createGroundingPrompt(@Nonnull final String message) {
return new OrchestrationPrompt(
Map.of("userMessage", message),
Message.user(
"{{?userMessage}} Use the following information as additional context: {{?groundingContext}}"));
}

@Nonnull
@Override
public GroundingModuleConfig createConfig() {
val groundingConfigConfig =
GroundingModuleConfigConfig.create()
.inputParams(List.of("userMessage"))
.outputParam("groundingContext")
.filters(filters);

return GroundingModuleConfig.create()
.type(documentGroundingService)
.config(groundingConfigConfig);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
package com.sap.ai.sdk.orchestration;

import com.sap.ai.sdk.orchestration.model.GroundingModuleConfig;
import javax.annotation.Nonnull;

/**
* Interface for grounding configurations.
*
* @link <a href="https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/grounding">SAP AI
* Core: Orchestration - Grounding</a>
*/
@FunctionalInterface
public interface GroundingProvider {
CharlesDuboisSAP marked this conversation as resolved.
Show resolved Hide resolved

/**
* Create a grounding configuration.
*
* @return the grounding configuration
*/
@Nonnull
GroundingModuleConfig createConfig();
}
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,9 @@
public interface MaskingProvider {

/**
* Create a masking provider for the configuration.
* Create a masking configuration.
*
* @return the masking provider
* @return the masking configuration
*/
@Nonnull
MaskingProviderConfig createConfig();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -197,4 +197,18 @@ public OrchestrationModuleConfig withOutputFiltering(

return this.withFilteringConfig(newFilteringConfig);
}

/**
* Creates a new configuration with the given grounding configuration.
*
* @link <a href="https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/grounding">SAP
* AI Core: Orchestration - Grounding</a>
* @param groundingProvider The grounding configuration to use.
* @return A new configuration with the given grounding configuration.
*/
@Nonnull
public OrchestrationModuleConfig withGrounding(
@Nonnull final GroundingProvider groundingProvider) {
return this.withGroundingConfig(groundingProvider.createConfig());
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,17 @@
import static com.sap.ai.sdk.orchestration.AzureFilterThreshold.ALLOW_SAFE_LOW_MEDIUM;
import static com.sap.ai.sdk.orchestration.OrchestrationAiModel.GPT_4O;
import static com.sap.ai.sdk.orchestration.OrchestrationAiModel.Parameter.MAX_TOKENS;
import static com.sap.ai.sdk.orchestration.model.DataRepositoryType.VECTOR;
import static com.sap.ai.sdk.orchestration.model.GroundingModuleConfig.TypeEnum.DOCUMENT_GROUNDING_SERVICE;
import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.assertThatThrownBy;

import com.sap.ai.sdk.orchestration.model.DPIConfig;
import com.sap.ai.sdk.orchestration.model.DPIEntities;
import com.sap.ai.sdk.orchestration.model.DocumentGroundingFilter;
import com.sap.ai.sdk.orchestration.model.GroundingModuleConfigConfig;
import com.sap.ai.sdk.orchestration.model.GroundingModuleConfigConfigFiltersInner;
import java.util.List;
import java.util.Map;
import org.junit.jupiter.api.Test;

Expand Down Expand Up @@ -121,4 +127,50 @@ void testLLMConfig() {
.withFailMessage("Static models should be unchanged")
.isEqualTo("latest");
}

@Test
void testGroundingConfig() {
var groundingConfig = Grounding.create();
var config =
new OrchestrationModuleConfig().withLlmConfig(GPT_4O).withGrounding(groundingConfig);

assertThat(config.getGroundingConfig()).isNotNull();
assertThat(config.getGroundingConfig().getType()).isEqualTo(DOCUMENT_GROUNDING_SERVICE);

GroundingModuleConfigConfig configConfig = config.getGroundingConfig().getConfig();
assertThat(configConfig).isNotNull();
assertThat(configConfig.getInputParams()).containsExactly("userMessage");
assertThat(configConfig.getOutputParam()).isEqualTo("groundingContext");

List<GroundingModuleConfigConfigFiltersInner> filters = configConfig.getFilters();
assertThat(filters).hasSize(1);
DocumentGroundingFilter filter = (DocumentGroundingFilter) filters.get(0);
assertThat(filter.getId()).isEqualTo("");
assertThat(filter.getDataRepositoryType()).isEqualTo(VECTOR);
}

@Test
void testGroundingConfigWithFilters() {
var filter1 = DocumentGroundingFilter.create().id("123").dataRepositoryType(VECTOR);
var filter2 = DocumentGroundingFilter.create().id("234").dataRepositoryType(VECTOR);
var groundingConfig = Grounding.create().filters(filter1, filter2);
var config =
new OrchestrationModuleConfig().withLlmConfig(GPT_4O).withGrounding(groundingConfig);

assertThat(config.getGroundingConfig()).isNotNull();
var configConfig = config.getGroundingConfig().getConfig();
assertThat(configConfig).isNotNull();

assertThat(config.getGroundingConfig().getConfig().getFilters()).hasSize(2);
}

@Test
void testGroundingPrompt() {
var prompt = Grounding.create().createGroundingPrompt("Hello, World!");
assertThat(prompt.getMessages()).hasSize(1);
var message = prompt.getMessages().get(0);
assertThat(message.content())
.isEqualTo(
"{{?userMessage}} Use the following information as additional context: {{?groundingContext}}");
}
}
Loading
Loading