Merge pull request #1613 from advancedtelematic/feat/prep-2020.4

Feat/prep 2020.4

CHANGELOG.md

@@ -6,9 +6,28 @@ Our versioning scheme is `YEAR.N` where `N` is incremented whenever a new releas
 
 ## [??? (unreleased)]
 
+
+## [2020.4] - 2020-03-24
+
 ### Added
 
-- Aktualizr secondaries can now reboot automatically after triggering an update: [PR](https://github.com/advancedtelematic/aktualizr/pull/1578)
+- aktualizr-secondary can now reboot automatically after triggering an update: [PR](https://github.com/advancedtelematic/aktualizr/pull/1578)
+- Reports are now stored in the SQL database so they persist through unexpected shutdown: [PR](https://github.com/advancedtelematic/aktualizr/pull/1559)
+
+### Changed
+
+- garage-push now always pushes the OSTree ref to Treehub: [PR](https://github.com/advancedtelematic/aktualizr/pull/1575)
+- Consistently follow the [Uptane standard's style guide](https://github.com/uptane/uptane-standard#style-guide) when using Uptane concepts, including the metadata output options of aktualizr-info: [PR](https://github.com/advancedtelematic/aktualizr/pull/1591)
+- Public contributions now are tested with Github Actions instead of Travis CI: [PR](https://github.com/advancedtelematic/aktualizr/pull/1597)
+- Default/recommended Yocto branch is zeus (3.0): [PR](https://github.com/advancedtelematic/aktualizr/pull/1603)
+- Improved logging for aktualizr-secondary: [PR](https://github.com/advancedtelematic/aktualizr/pull/1609)
+
+### Fixed
+
+- Abort initialization if ECUs are already registered: [PR](https://github.com/advancedtelematic/aktualizr/pull/1579)
+- Always use 64-bit integers for disk space arithmetic: [PR](https://github.com/advancedtelematic/aktualizr/pull/1588)
+- Reject Director Targets metadata with delegations or repeated ECU IDs: [PR](https://github.com/advancedtelematic/aktualizr/pull/1600)
+
 
 ## [2020.3] - 2020-02-27
 

docs/README.adoc

@@ -35,6 +35,7 @@ The link above is for the doxygen docs on master. Doxygen docs for the following
 * https://advancedtelematic.github.io/aktualizr/2020.1/index.html[2020.1]
 * https://advancedtelematic.github.io/aktualizr/2020.2/index.html[2020.2]
 * https://advancedtelematic.github.io/aktualizr/2020.3/index.html[2020.3]
+* https://advancedtelematic.github.io/aktualizr/2020.4/index.html[2020.4]
 ====
 
 == Release process

docs/ota-client-guide/antora.yml

@@ -1,6 +1,6 @@
 name: ota-client
 title: OTA Connect Developer Guide
 version: latest
-display_version: 2020.2 (latest)
+display_version: 2020.3 (latest)
 nav:
 - modules/ROOT/nav.adoc

docs/ota-client-guide/modules/ROOT/nav.adoc

@@ -109,6 +109,7 @@ ifndef::env-github[:pageroot:]
 
 .Troubleshooting
 * xref:{pageroot}troubleshooting.adoc[Troubleshooting]
+* xref:{pageroot}reporting-problems.adoc[Reporting problems]
 
 .For Contributors
 // Dev-authored topics

docs/ota-client-guide/modules/ROOT/pages/_partials/aktualizr-version.adoc

@@ -3,6 +3,6 @@
 // the version being viewed, but when we are referencing aktualizr from
 // the other, non-versioned docs, we want to make sure we're using the
 // latest version.
-:aktualizr-version: 2020.3
+:aktualizr-version: 2020.4
 
-:yocto-version: 2.6
+:yocto-version: 3.0

docs/ota-client-guide/modules/ROOT/pages/account-setup.adoc

@@ -8,12 +8,24 @@ We recommend that you link:https://docs.ota.here.com/ota-client/latest/{docname}
 endif::[]
 
 
-In OTA Connect, all devices and software belong to one *user* login. However, you don't want to mix up test software and production software by creating them all under the same user.
+If you do not want to mix up test and production software, you may use separate user logins or create additional environments.
 
-In a proper production workflow, you'll need separate user logins to manage the different stages:
+== Separate user logins
 
-. A developer user such as "\[email protected]".
-. A QA user such as "\[email protected]".
-. A production user such as "\[email protected]".
+To manage the different stages of production workflow, you can create separate user logins:
 
-These logins provide you with a convenient way of clearly separating your development, QA and production resources.
+* A developer user such as _\[email protected]_
+* A QA user such as _\[email protected]_
+* A production user such as _\[email protected]_
+
+These logins provide you with a convenient way to separate your development, QA, and production resources.
+
+== Additional environments image:img::beta-icon.svg[Beta]
+
+NOTE: This feature is in beta and is only visible to users who are part of our beta program. If you would like to join the beta program, contact us at link:mailto:[email protected][[email protected]] to request access.
+
+Instead of creating multiple accounts for each environment, you can now xref:ota-web::create-environment.adoc[create] additional environments using only one account. For example, you may need to have different environments for development, QA, and production.
+
+After you create an environment, you can xref:ota-web::manage-members.adoc[add] your colleagues to work together on device provisioning, device groups, software versions, software updates, and campaigns.
+
+To get more information on the environments feature, see the xref:ota-web::environments-intro.adoc[*Environments*] section in the OTA Connect User Guide.

docs/ota-client-guide/modules/ROOT/pages/add-ota-functonality-existing-yocto-project.adoc

@@ -7,6 +7,8 @@ We recommend that you link:https://docs.ota.here.com/ota-client/latest/{docname}
 ====
 endif::[]
 
+include::_partials/aktualizr-version.adoc[]
+
 :page-layout: page
 :page-categories: [quickstarts]
 :page-date: 2017-05-23 16:27:58
@@ -15,9 +17,9 @@ endif::[]
 
 If you already have a Yocto-based project, you can start your functional integration with {product-name} by following these four steps:
 
-1.  Clone the https://github.com/advancedtelematic/meta-updater[meta-updater] layer and add it to your https://www.yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#structure-build-conf-bblayers.conf[bblayers.conf].
+1.  Clone the https://github.com/advancedtelematic/meta-updater[meta-updater] layer and add it to your https://www.yoctoproject.org/docs/{yocto-version}/ref-manual/ref-manual.html#structure-build-conf-bblayers.conf[bblayers.conf].
 2.  Clone a BSP integration layer (`meta-updater-$\{PLATFORM}`, e.g. https://github.com/advancedtelematic/meta-updater-raspberrypi[meta-updater-raspberrypi]) and add it to your `conf/bblayers.conf`. If your board isn't supported yet, you could write a BSP integration for it yourself. See xref:supported-boards.adoc#_adding_support_for_your_board[Adding support for your board] for more details.
-3.  Set up your https://www.yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#var-DISTRO[distro]. If you are using "poky", the default distro in Yocto, you can change it in your `conf/local.conf` to `poky-sota` or to `poky-sota-systemd`. Alternatively, if you are using your own or a third-party distro configuration, you can add the following parameters to it, thus combining the capabilities of your distro with meta-updater features.
+3.  Set up your https://www.yoctoproject.org/docs/{yocto-version}/ref-manual/ref-manual.html#var-DISTRO[distro]. If you are using "poky", the default distro in Yocto, you can change it in your `conf/local.conf` to `poky-sota` or to `poky-sota-systemd`. Alternatively, if you are using your own or a third-party distro configuration, you can add the following parameters to it, thus combining the capabilities of your distro with meta-updater features.
 +
 ----
 INHERIT += " sota"

docs/ota-client-guide/modules/ROOT/pages/aktualizr-config-options.adoc

@@ -38,7 +38,7 @@ For examples of configuration files, see the following resources:
 
 * link:{aktualizr-github-url}/config/[Config files used by unit tests]
 * link:{aktualizr-github-url}/tests/config/[Config files used by continuous integration tests]
-* link:https://github.com/advancedtelematic/meta-updater/tree/thud/recipes-sota/config/files[Configuration fragments used in meta-updater recipes].
+* link:https://github.com/advancedtelematic/meta-updater/tree/zeus/recipes-sota/config/files[Configuration fragments used in meta-updater recipes].
 
 All fields are optional, and most have reasonable defaults that should be used unless you have a particular need to do otherwise.
 

docs/ota-client-guide/modules/ROOT/pages/build-agl.adoc

@@ -102,6 +102,6 @@ TIP: You can also write the image using `dd`, but since the wrong kind of typo i
 
 === Run with QEMU
 
-You can now run the image in QEMU using the same method as described for a xref:build-qemu.adoc#_run_the_built_image_with_qemu[regular QEMU build]. However, the exact image you'll need will vary depending on the architecture you're building forfootnote:[For example, building the `agl-image-minimal` target for QEMU creates an image at `build/tmp/deploy/images/raspberrypi3/agl-image-minimal-qemux86-64.ota-ext4`.], but it will be located in the `/tmp/deploy/images` directory under your build directory.
+You can now run the image in QEMU using the same method as described for a xref:build-qemu.adoc#_run_the_built_image_with_qemu[regular QEMU build]. However, the exact image you'll need will vary depending on the architecture you're building forfootnote:[For example, building the `agl-image-minimal` target for QEMU creates an image at `build/tmp/deploy/images/qemux86-64/agl-image-minimal-qemux86-64.ota-ext4`.], but it will be located in the `/tmp/deploy/images` directory under your build directory.
 
 include::partial$recommended-steps.adoc[tags=firstbuild-nextstep]

docs/ota-client-guide/modules/ROOT/pages/build-qemu.adoc

@@ -33,6 +33,8 @@ The build process creates disk images as an artefact. You can then directly run
 
 Both arguments are optional; image name defaults to `core-image-minimal`, and if a mac address isn't specified, a random one is generated.
 
+TIP: Depending on your build, the `meta-updater` directory might be in a slightly different location. For example, if you're building an xref:build-agl.adoc[AGL image], the meta-updater layer would be at `../external/meta-updater`.
+
 .Persistent storage
 ****
 By default, QEMU will run your image in snapshot mode, *discarding any changes you made* to the disk image as soon as it exits. If you want to have a persistent VM, you need to create an link:https://wiki.archlinux.org/index.php/QEMU#Overlay_storage_images[overlay storage image] in qcow2 format. The helper script can also manage this for you, making it easy to create an emulated fleet of devices:

docs/ota-client-guide/modules/ROOT/pages/build-raspberry.adoc

@@ -8,6 +8,7 @@ We recommend that you link:https://docs.ota.here.com/ota-client/latest/{docname}
 ====
 endif::[]
 
+include::_partials/aktualizr-version.adoc[]
 
 :page-layout: page
 :page-categories: [quickstarts]
@@ -30,7 +31,7 @@ endif::[]
 
 You'll need a build machine with the following:
 
-* A x86-64 Linux distro link:https://www.yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#detailed-supported-distros[supported by the Yocto project] with the link:https://www.yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#required-packages-for-the-build-host[required packages] installed.
+* A x86-64 Linux distro link:https://www.yoctoproject.org/docs/{yocto-version}/ref-manual/ref-manual.html#detailed-supported-distros[supported by the Yocto project] with the link:https://www.yoctoproject.org/docs/{yocto-version}/ref-manual/ref-manual.html#required-packages-for-the-build-host[required packages] installed.
 ** On a Debian-based system, you should be able to install all the required packages with the following command:
 +
 ----
@@ -61,7 +62,7 @@ First, clone a manifest file for the quickstart project:
 ----
 mkdir myproject
 cd myproject
-repo init -u https://github.com/advancedtelematic/updater-repo.git -m thud.xml
+repo init -u https://github.com/advancedtelematic/updater-repo.git -m zeus.xml
 repo sync
 ----
 

docs/ota-client-guide/modules/ROOT/pages/customise-targets-metadata.adoc

@@ -7,6 +7,8 @@ We recommend that you link:https://docs.ota.here.com/ota-client/latest/{docname}
 ====
 endif::[]
 
+include::_partials/aktualizr-version.adoc[]
+
 In some cases, you might find it useful to include extra metadata about your software inside the signed Uptane metadata that OTA Connect delivers to your device. Some reasons you might want to do this include:
 
 * to provide installation instructions or scripts for an image that can't or shouldn't be included in the image itself
@@ -84,4 +86,4 @@ You can then upload your customized `targets.json` to OTA Connect as normal:
 garage-sign targets push --repo myimagerepo
 ----
 
-NOTE: You also might want to add custom metadata while bitbaking. You can do this, for example, by modifying the `IMAGE_CMD_garagesign` function in link:https://github.com/advancedtelematic/meta-updater/blob/master/classes/image_types_ostree.bbclass#L217[image_types_ostree.bbclass]. A detailed guide on how to accomplish this is out of our scope, however. Refer to http://www.yoctoproject.org/docs/2.7/dev-manual/dev-manual.html[the Yocto Reference Manual] for further details.
+NOTE: You also might want to add custom metadata while bitbaking. You can do this, for example, by modifying the `IMAGE_CMD_garagesign` function in link:https://github.com/advancedtelematic/meta-updater/blob/master/classes/image_types_ostree.bbclass#L217[image_types_ostree.bbclass]. A detailed guide on how to accomplish this is out of our scope, however. Refer to http://www.yoctoproject.org/docs/{yocto-version}/dev-manual/dev-manual.html[the Yocto Reference Manual] for further details.

docs/ota-client-guide/modules/ROOT/pages/debugging-tips.adoc

@@ -74,7 +74,7 @@ uptane-generator generate <repo_dir>
 
 Then, serve the generated directory using a web server such as the link:{aktualizr-github-url}/tests/fake_http_server/fake_test_server.py[fake test server].
 
-For more information about using uptane-generator, see xref:uptane-generator.adoc[uptane-generator.adoc].
+For more information about using uptane-generator, see the xref:uptane-generator.adoc[uptane-generator article].
 
 Here is an example configuration for nginx:
 
@@ -98,7 +98,7 @@ server {
 
 == Inject faults
 
-See xref:fault-injection.adoc[fault-injection.adoc]
+See the xref:fault-injection.adoc[fault injection article] for more information.
 
 == Developing and debugging with an OpenEmbedded system
 

docs/ota-client-guide/modules/ROOT/pages/ostree-usage.adoc

@@ -13,7 +13,7 @@ endif::[]
 :page-order: 3
 :icons: font
 
-Sometimes, while troubleshooting, it might be helpful to see and manipulate your local OSTree repos. You can do that using the copy of ostree that bitbake builds. For the rest of these commands, we'll assume that you've exported the executable as `$OSTREE`, and the location of your local repo as `$REPO` (for example, like so):
+Sometimes, while troubleshooting, it might be helpful to see and manipulate your local OSTree repos. You can do that using the copy of `ostree` that bitbake builds. For the rest of these commands, we'll assume that you've exported the executable as `$OSTREE`, and the location of your local repo as `$REPO` (for example, like so):
 
     export OSTREE=$(pwd)/tmp/sysroots/x86_64-linux/usr/bin/ostree
     export REPO=$(pwd)/tmp/deploy/images/raspberrypi3/ostree_repo/
@@ -24,7 +24,7 @@ You'll need a branch name for most other commands, so it's often useful to check
 
     $OSTREE refs --repo $REPO
 
-The branch name defaults to \{MACHINE}-ota, so if you were building for raspberrypi3, your branch name would be raspberrypi3-ota by default. However, you can set the branch name for your build in local.conf using the *OSTREE_BRANCHNAME* configuration option, letting you keep your different builds, projects, or branches under different names.
+The branch name defaults to \{MACHINE}, so if you were building for raspberrypi3, your branch name would be raspberrypi3 by default. However, you can set the branch name for your build in `local.conf` using the `OSTREE_BRANCHNAME` configuration option, letting you keep your different builds, projects, or branches under different names. See the xref:build-configuration.adoc[build configuration article] for more information.
 
 == Show the log for a particular branch
 
@@ -42,3 +42,22 @@ The -R option is supported, for recursive file listing.
 
 Refs can be branch names or commit hashes, much like in git. Note that this does not show the contents of the diff; it just shows files added, deleted, and modified.
 
+== Pruning unused/unwanted objects
+
+OSTree normally manages garbage collection of objects on its own, but if your OSTree repo in your build directory is too large, you can manually remove unwanted commits and objects. You can also use the `ostree` tool on your device to remove coommits and objects on your device. This may be useful if you know that a certain commit has a flaw that you do not want to let get deployed.
+
+First, you will need to find which commits and refs are currently not deployed and are no longer needed. You can use these commands to help make that determination:
+
+    ostree admin status
+    ostree refs
+    ostree log <ref>
+
+Then, for each ref that you would like to remove:
+
+    ostree refs --delete <ref>
+
+And then for each commit that you would like to remove:
+
+    ostree prune --delete-commit=<commit>
+
+Note that `ostree admin status` will still show deleted commits, but if you try to deploy a deleted commit, the operation will fail as expected.

docs/ota-client-guide/modules/ROOT/pages/pushing-updates.adoc

@@ -8,6 +8,8 @@ We recommend that you link:https://docs.ota.here.com/ota-client/latest/{docname}
 ====
 endif::[]
 
+include::_partials/aktualizr-version.adoc[]
+
 Every time you bitbake a new image, it is automatically pushed to {product-name}. You can then send the updated image out to any of your devices. In this guide, you learn a few ways to push updated system images from your build machine or workstation to {product-name-short}.
 
 == Add a new package from an existing recipe
@@ -102,7 +104,7 @@ The screen sessions will be named `pianobar` and `patiobar` respectively; use `s
 
 == Make your own recipe or layer
 
-A complete guide to writing Yocto recipes is out of scope here, but http://www.yoctoproject.org/docs/2.6/dev-manual/dev-manual.html#new-recipe-writing-a-new-recipe[the Yocto Reference Manual] is a great resource. You can also take a look at the recipes in https://github.com/advancedtelematic/meta-jukebox[`meta-jukebox`] to use as examples: they're all fairly simple, and there are examples of four different types of recipe:
+A complete guide to writing Yocto recipes is out of scope here, but http://www.yoctoproject.org/docs/{yocto-version}/dev-manual/dev-manual.html#new-recipe-writing-a-new-recipe[the Yocto Reference Manual] is a great resource. You can also take a look at the recipes in https://github.com/advancedtelematic/meta-jukebox[`meta-jukebox`] to use as examples: they're all fairly simple, and there are examples of four different types of recipe:
 
 * https://github.com/advancedtelematic/meta-jukebox/tree/master/recipes-multimedia/pianobar[Pianobar] itself is a fully manual recipe, though it's a pretty simple one; it has specific instructions for the compile and install steps, though they're essentially just `make` and `make install` with a couple of config options.
 * https://github.com/advancedtelematic/meta-jukebox/tree/master/recipes-multimedia/libao[libao] and https://github.com/advancedtelematic/meta-jukebox/tree/master/recipes-multimedia/faad2[faad2] make use of https://en.wikipedia.org/wiki/GNU_Build_System[Autotools] to build. They include the line `inherit autotools` in the recipe, which automatically generates configure, compile, and install instructions based on Autotools.