Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extend support for file, property formats across backends #205

Merged
merged 27 commits into from
Jan 20, 2025

Conversation

ivanperez-keera
Copy link
Member

Extend support for file, property formats across backends, including the "literal" property format for the standalone backend and all the same supported property formats, file formats, and use of external processors for ROS and FPrime, as prescribed in the solution proposed for #204.

Ogma used to support only one kind of input file that used the term
Component Spec for its specification. That is no longer the case, since
now other kinds of formats, in JSON, XML and diagrams are supported.

This commit changes a comment accordingly.
Ogma-test were originally centered around a specific file format but,
with the support for new formats in JSON, XML and diagrams, it's become
more versatile.

This commit renames examples to organize them based on the file formats
they use, as defined by the file format property in the CLI.
Ogma-test were originally centered around a specific file format but,
with the support for new formats in JSON, XML and diagrams, it's become
more versatile.

This commit renames tests and test files to organize them based on
backends and the file formats they use, as defined by the file format
property values supported by ogma-core.
Ogma-test were originally centered around a specific file format but,
with the support for new formats in JSON, XML and diagrams, it's become
more versatile.

This commit renames tests and test files to organize them based on the
file formats they use, as defined by the file format property in the
CLI.
…asa#204.

The diagram backend supports literal Copilot expressions, but that
capability is not yet available in the standalone backend.

This commit adds the capability by introducing a new expression pair or
property parsing/printing handler.
The ROS backend used to be specialized for one file format as input,
which was reflected in the name of the options expected by the backend.

For consistency across backend, and to facilitate the introduction of
support for different kinds of file formats in the future, we rename the
backend's options to be more format-agnostic.
The ROS backend used to be specialized for one file format as input,
which was reflected in the name of the options expected by the backend.

A prior commit has renamed the options expected by Ogma's backend in
ogma-core to make them more format-agnostic, in alignment with the plan
to support different kinds of file formats.

This commit performs the same change for users in the CLI.,
The ROS backend has a large amount of input options, making it
cumbersome to pass them around.

This commit packs those options in a record, making the signature of the
top-level function simpler and more uniform.
The ROS backend has a large amount of input options, making it
cumbersome to pass them around.

A prior commit has modified the backend in ogma-core to packs those
options in a record, making the signature of the top-level function
simpler and more uniform.

This commit adapts the CLI to provide the arguments in a record as
required by the updated backend.
… Refs nasa#204.

The standalone backend supports cocospec, smv and literal Copilot
expressions, but that capability is not yet available in the ROS
backend.

This commit adds that capability by introducing a notion of expression
pairs or expression input file handlers parser can use to parse
expressions or properties contained in input files.
…Refs nasa#204.

The standalone backend supports cocospec, smv and literal Copilot
expressions, but that capability is not yet available in the ROS
backend.

A prior commit has added to ogma-core the the capability of handling
expressions in multiple languages.
…sa#204.

The standalone backend supports JSON and XML, which can be further
confirured in the command line, but that capability is not yet available
in the ROS backend.

This commit adds that capability by introducing the capability of
handling JSON and XML file formats, and reading the specific paths
of the elements needed in properties from the command line.
…a#204.

The standalone backend supports JSON and XML, which can be further
confirured in the command line, but that capability is not yet available
in the ROS backend in the command line interface.

A prior commit has expanded the ROS backend in ogma-core with support
for the aforementioned capability.

This commit exposes the new capability to the user in the CLI.
…ties. Refs nasa#204.

There is a need in Ogma to make the frontend more versatile, and to
extend it to support languages unknown to Ogma without users having to
modify the code of Ogma itself. To that end, we want to empower users
with the ability to use a custom command to transform individual
properties into a format that Ogma understands, and have Ogma call that
external command on demand. This capability is already available in the
standalone backend, but not in other backends.

This commit modifies the ROS backend to take an auxiliary command that
is used as the process name to call to parse individual properties.
… properties. Refs nasa#204.

There is a need in Ogma to make the frontend more versatile, and to
extend it to support languages unknown to Ogma without users having to
modify the code of Ogma itself. To that end, we want to empower users
with the ability to use a custom command to transform individual
properties into a format that Ogma understands, and have Ogma call that
external command on demand.

Prior commits have extended the ROS backend with the capability of
taking an auxiliary command as argument and running it to transform
individual properties.

This commit exposes that new argument to the user in the CLI.
…a#204.

The FPrime backend used to be specialized for one file format as input,
which was reflected in the name of the options expected by the backend.

For consistency across backend, and to facilitate the introduction of
support for different kinds of file formats in the future, we rename the
backend's options to be more format-agnostic.
…#204.

The FPrime backend used to be specialized for one file format as input,
which was reflected in the name of the options expected by the backend.

A prior commit has renamed the options expected by Ogma's backend in
ogma-core to make them more format-agnostic, in alignment with the plan
to support different kinds of file formats.

This commit performs the same change for users in the CLI.,
The FPrime backend has a large amount of input options, making it
cumbersome to pass them around.

This commit packs those options in a record, making the signature of the
top-level function simpler and more uniform.
The FPrime backend has a large amount of input options, making it
cumbersome to pass them around.

A prior commit has modified the backend in ogma-core to packs those
options in a record, making the signature of the top-level function
simpler and more uniform.

This commit adapts the CLI to provide the arguments in a record as
required by the updated backend.
…ts. Refs nasa#204.

The standalone backend supports cocospec, smv and literal Copilot
expressions, but that capability is not yet available in the FPrime
backend.

This commit adds that capability by introducing a notion of expression
pairs or expression input file handlers parser can use to parse
expressions or properties contained in input files.
…s. Refs nasa#204.

The standalone backend supports cocospec, smv and literal Copilot
expressions, but that capability is not yet available in the FPrime
backend.

A prior commit has added to ogma-core the the capability of handling
expressions in multiple languages.
…asa#204.

The standalone backend supports JSON and XML, which can be further
confirured in the command line, but that capability is not yet available
in the FPrime backend.

This commit adds that capability by introducing the capability of
handling JSON and XML file formats, and reading the specific paths
of the elements needed in properties from the command line.
…asa#204.

The standalone backend supports JSON and XML, which can be further
confirured in the command line, but that capability is not yet available
in the FPrime backend in the command line interface.

A prior commit has expanded the FPrime backend in ogma-core with support
for the aforementioned capability.

This commit exposes the new capability to the user in the CLI.
…perties. Refs nasa#204.

There is a need in Ogma to make the frontend more versatile, and to
extend it to support languages unknown to Ogma without users having to
modify the code of Ogma itself. To that end, we want to empower users
with the ability to use a custom command to transform individual
properties into a format that Ogma understands, and have Ogma call that
external command on demand. This capability is already available in the
standalone backend, but not in other backends.

This commit modifies the FPrime backend to take an auxiliary command that
is used as the process name to call to parse individual properties.
…rse properties. Refs nasa#204.

There is a need in Ogma to make the frontend more versatile, and to
extend it to support languages unknown to Ogma without users having to
modify the code of Ogma itself. To that end, we want to empower users
with the ability to use a custom command to transform individual
properties into a format that Ogma understands, and have Ogma call that
external command on demand.

Prior commits have extended the FPrime backend with the capability of
taking an auxiliary command as argument and running it to transform
individual properties.

This commit exposes that new argument to the user in the CLI.
@ivanperez-keera
Copy link
Member Author

Change Manager: Verified that:

  • Solution is implemented:
    • The code proposed compiles and passes all tests. Details:
    • The solution proposed produces the expected result. Details:
      The following dockerfile uses the standalone backend to generate a monitor using a literal Copilot expression gathered from a custom JSON file, and then uses the FPrime backend to produce an FPrime component. The Copilot monitor is further compiled, and building the Dockerfile generated by the FPrime backend completes the compilation process correctly. Note that, due to a standing issue with the FPrime backend (java is needed in newer versions), we need to modify the Dockerfile generated to install java. This can be done in a way that does not interfere with future tests (the modification will succeed even if java is available on the guest image already).
      --- Dockerfile-verify-204
      FROM ubuntu:trusty
      
      ENV DEBIAN_FRONTEND=noninteractive
      
      RUN apt-get update
      
      RUN apt-get install --yes software-properties-common
      RUN add-apt-repository ppa:hvr/ghc
      RUN apt-get update
      
      RUN apt-get install --yes ghc-8.6.5 cabal-install-2.4
      RUN apt-get install --yes libz-dev
      
      ENV PATH=/opt/ghc/8.6.5/bin:/opt/cabal/2.4/bin:$PWD/.cabal-sandbox/bin:$PATH
      
      RUN cabal update
      RUN cabal v1-sandbox init
      RUN cabal v1-install alex happy
      RUN apt-get install --yes git
      
      ADD document.json       /tmp/document.json
      ADD json-format.cfg     /tmp/json-format.cfg
      ADD fprime-variable-dbs /tmp/fprime-variable-dbs
      ADD fprime-handlers     /tmp/fprime-handlers
      
      CMD git clone $REPO && \
          cd $NAME && \
          git checkout $COMMIT && \
          cd .. && \
          cabal v1-sandbox init && \
          cabal v1-install copilot-4.2 $NAME/$PAT**/ --constraint="aeson>=2.0.3.0" && \
          ./.cabal-sandbox/bin/ogma standalone --file-name /tmp/document.json -f /tmp/json-format.cfg -p literal --target-file-name copilot --target-dir fprime_demo && \
          ./.cabal-sandbox/bin/ogma fprime --app-target-dir fprime_demo --handlers-file /tmp/fprime-handlers --input-file /tmp/document.json -f /tmp/json-format.cfg -p literal --variable-db /tmp/fprime-variable-db && \
          cabal v1-exec -- runhaskell fprime_demo/Copilot.hs && \
          sed -i -e '0,/RUN apt-get install/{s/\(RUN apt-get install .*\)/\1 default-jre/}' fprime_demo/Dockerfile && \
          mv copilot* fprime_demo/
      
      --- document.json
      {
        "Example": {
          "internal_variables": [],
          "external_variables": [
            {"name":"input", "type":"Float", "meaning": "Input from the system"}
          ],
          "properties": [
            {
              "id":      "MyProperty",
              "formula": "PTLTL.alwaysBeen (input /= 30.0)",
              "text":    "Input is never 30"
            }
          ]
        }
      }
      
      --- fprime-handlers
      handlerMyProperty
      
      --- fprime-variable-db
      ("pullup", "bool")
      ("input", "float")
      
      --- json-format.cfg
      JSONFormat
         { specInternalVars    = Just "..internal_variables[*]"
         , specInternalVarId   = ".name"
         , specInternalVarExpr = ".meaning"
         , specInternalVarType = Just ".type"
         , specExternalVars    = Just "..external_variables[*]"
         , specExternalVarId   = ".name"
         , specExternalVarType = Just ".type"
         , specRequirements    = "..properties[*]"
         , specRequirementId   = ".id"
         , specRequirementDesc = Just ".text"
         , specRequirementExpr = ".formula"
         }   
      
      Command (substitute variables based on new path after merge):
      $ docker run -v $PWD/fprime_demo:/fprime_demo -e "REPO=https://github.com/ivanperez-keera/ogma" -e "NAME=ogma" -e "PAT=ogma" -e "COMMIT=0a2123826fee7d677abc80228ead521f8dbaca6e" -it ogma-verify-204
      $ cd fprime_demo
      $ docker build -t ogma-verify-204-fprime -f Dockerfile .
      
  • Implementation is documented. Details:
    All new top-level definitions are documented.
  • Change history is clear.
  • Commit messages are clear.
  • Changelogs are updated.
  • Examples are updated. Details:
    This patch changes some examples to make their names conform to the names of the formats supported.
  • Required version bumps are evaluated. Details:
    Bump required (the changes modify the command-line arguments, which is a breaking change to the CLI).

@ivanperez-keera ivanperez-keera merged commit 2814056 into nasa:develop Jan 20, 2025
2 checks passed
@ivanperez-keera ivanperez-keera deleted the develop-versatile branch January 21, 2025 05:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant