Chore/maas merge main #118
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Create a manifest table with fields id, applieddate and manifest data. Applied date will be populated by SQL when the record is inserted. * Add APIs to list all manifests, get manifest by id, delete manifest by id. * Add aditional API to get latest inserted record when manifest id is specied with keyword latest. * TIMESTAMP type in database does not save the time in nano second precision and saved only until seconds precision. So retreiving latest inserted record should get all the records with the latest timestamp and return the last one.
Add python helper functions to create/retreive/ delete manifest from cluster database.
Add manifest option to sunbeam bootstrap. Add step to write the manifest to cluster db.
Add following options manifest and clear-manifest to refresh command. clear-manifest option updates cluster database manifest table with empty manifest.
Add terraform to manifest file. terraform will be a dictionary of terraform plan and local path of the terraform plan.
Add initial support for manifest
Add revision/config to microk8s, microceph, openstack-hypervisor terraform plans.
* Create default manifest * Add terraform plan location to default manifest * Define manifest part and tfvar/manifest map in plugins * Remove configurable tfplan options * Use latest manifest to populate terraform vars in openstack, configure plan * Use latest manifest to populate terraform vars in plugins
* All the commands have logic to copy the teraform plans to snap user common and creation of terraform helper object. Move this logic as part of Manifest class. * All the plans more or less get terraform vars from cluster db, update with generated tfvars based on plan, update with tfvars from manifest, write tfvars to the terraform plan and apply the plan. Move this logic as part of Manifest class
Add unit tests to Manifest class Rename few functions and arguments
Use manifest to update terraform vars
Add bootstrap args from the manifest to the
juju bootstrap command. The bootstrap args
will be appended to the juju bootstrap command.
Manifest for juju bootstrap args looks like:
juju:
bootstrap_args:
- --agent-version=3.2.4
Fix typos in the dns plugin. Create manifest after cluster is joined in join command.
[manifest] miscellaneous fixes
Add manifest override for juju bootstrap args
Add manifest list and manifest show commands.
Add manifest management commands
* Add add_manifest_section to plugin interface so that plugins can add new manifest attributes. * Add caas_config section to manifest as part of caas plugin * Update caas plugin terraform plan to introduce variables that can be overriden by manifest * Update plan for caas configure to read manifest caas_config section and use them as override_tfvars
[manifest] Add caas_config section
terraform vars for the plan are created by merging a. tfvars from cluster db b. From manifest along with defaults c. override_tfvars from the plan step. If any attribute in manifest is removed in subsequent update of manifest, the merged tfvar map still contains the value from cluster db (step a). To avoid this, remove all the tfvar keys that can come from manifest in step a.
[manifest] fix terraformvar updates
…t-manifest Merge main to feat manifest
* Use manifest for refresh intra channel updates * Refresh the apps in openstack and controller only when channel is specified in manifest (including default values) and no revision is specified. This will take care of app refresh for new revisions on the same channel. The logic to check if there is any latest revision on the channel is removed since the juju refresh should take care of the logic. * Add Reapply terraform plan for Control plane. * Add flag refresh to DeployMachineApplication so that the terraform plan can be reapplied when refresh is true. This covers terraform plans for microk8s, microceph. sunbeam-machine. * Use ReapplyHypervisorTerraformPlan for openstack-hypervisor. * Add all the above plans to the list of plans to execute on refresh * For the plugins, do not apply any terraform plan if the plan is part of sunbeam-terraform as the control plane refresh should take care. * Do not apply terraform plans if there is no change in current terraform vars and updated terraform vars.
[manifest] support for intra channel refresh
Use manifest for Upgrade releases. For control plane, upgrade charms in following order - mysql charms, openstack core charms, openstack plugins. In each upgrade, only update the terraform variables corresponding to charms and apply the terraform plan. Followed similar approach for machine charms as well. Split machine charms to individual ones as each machine charm may have pre and post steps during upgrade. Upgrade plugins using terraform plan openstack-plan as part of control plane upgrade.
…-upgrade [manifest] Support for upgrade-release
Add missing client argument to `get_all_external_repos`.
Add manifest option to enable group. Plugins of type EnableDisablePlugin should read the user provided manifest during enable of the plugin.
Fix regression from canonical#91
[manifest] Add option manifest to plugin enable
Expand terraform variables for observability plugin to parameterise channel, revision, config. Update manifest_default and corresponding tfvar map in the plugin. Update plugin to use manifest functions instead of terraform helper functions.
…bility-plugin Add manifest support to observability plugin
Validate the charms in the manifest file are valid charms known by the sunbeam defaults. If not, raise an error.
…ions [manifest] Validate charms section in manifest file
Skip the step AddManifest that writes manifest to cluster db if the content of provided manifest and the latest one in cluster db is same.
…-writes Skip writing manifest to db if there is no change
Add subcommand generate in sunbeam manifest. Write the manifest to user provided file with $HOME/.config/openstack/manifest.yaml as default. Generate manifest with the current configuration by default. If the cluster is not bootstrapped, generate manifest with default values.
Add command to generate manifest
Handle catching ManifestItemNotFound exception in getting latest manfiest from cluster db and return result as completed for AddManifest is_skip.
Donot print default terraform sources in the generated manifest.
Use single file for both manifest and preseed. The file should have deployment as key for preseed entries and software for software config entries. Remove --preseed option from commands and pass the preseed dict from manifest to the plans.
Merge feat manifest to main
Since recent charm-microceph update, replicas size 1 is available under limited conditions, get the number of OSDs configured on bootstrap and resize to update the number of pool replicas for glance and cinder.
…ry-prints Remove unnecessary warnings
Compute number of OSDs to decide number of replicas
libdqlite-dev introduced dependency to libraft-canonical-dev which is more advanced than libraft-dev.
gboutry
force-pushed
the
chore/maas-merge-main
branch
from
9bdbdab to
60d7875
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Merge main to feat maas
Manifest endpoints in microcluster have been set to AllowUntrusted (for the same reasons as the other endpoints, lack of ssl verification / authentication)