Add a config file for experiments

[recitalAI]

Too many parameters are shared between the different generation steps. It would be more convenient to use a configuration file to define all the steps and run them at once.

[GillesMoyse]

In progress (cf. PR from Youssouf)

[GillesMoyse]

The configuration file describes how an Expe will go from one step to another, in which order.
Thus, the configuration file is made of the 3 main blocks :

1. an Expe file
2. a dictionary of Steps
3. a workflow, an ordered list of Steps

Expe file : a string to the Expe file

Each Step is defined as follows:

• 1 TextGenerator
• export parameters (json, html, spreadsheet)

Each TextGenerator comprises:

• a Prompter
• one or several LLMs - LLMs can be defined as classes or names - if defined as names, they are instances of LiteLLM
• a set of parameters (overwrite, start_from, missing_only)

Below is an example of a configuration file with 3 steps, AnswerGen, FactsGen and EvalGen:

• expe_file
• expe_folder
• steps: {
  • "AnswerGen": {
    • generator: "AnsGenerator" # TextGenerator class
    • params: {"overwrite", "start_from", "b_missing"}
    • prompter: "PptrAnsBase"
    • retriever: None
    • llm_names: ["gpt-4o", "mistral-large"]
    • llms: ["Albert"]
    • export: {
      • "json": {"do_export": true}
      • "htrml": {"do_export": true, "template": "basic.jinja"}
      • "spreadsheet": {"do_export": false} }
  • "Facts
    • ...
      }
• workflow: ["AnswerGen", "AsnwerGen", "Facts"]

pipeline = load_pipeline(path_to_pipeline.toml)
pipeline.run()