Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build RPM package for every created or updated PR #6

Open
TimothyAsirJeyasing opened this issue Jan 23, 2018 · 24 comments
Open

Build RPM package for every created or updated PR #6

TimothyAsirJeyasing opened this issue Jan 23, 2018 · 24 comments
Assignees

Comments

@TimothyAsirJeyasing
Copy link

TimothyAsirJeyasing commented Jan 23, 2018

(2018-02-07 updated/rewritten by @dahorak)

We would like to be able to automatically build RPM packages for every created or updated PR.
Build status should be reported back to the respective PR.
Packages should be available in some repository for any additional testing.

The intended workflow is:

  • Every new or updated PR will trigger package build job in CentOS CI Jenkins.
  • The job might be triggered also by comment run build in the respective PR.
  • The job will create rpm repository with packages related to the particular git repo/PR. The repositories will be hosted on http://artifacts.ci.centos.org/.
  • ❔ Prospectively some basic validation (unit tests execution, rpmlint) might be performed.

Functional testing of the PR builds is covered in another issue: #10.

@TimothyAsirJeyasing
Copy link
Author

@mbukatov, @dahorak -> Please review/update the details.
@r0h4n, @nthomas-redhat, @shtripat -> please review.

@shtripat
Copy link
Member

@TimothyAsir do we need PR to be merged in master for feature build? I feel without merge we should be able to cherrypick the PR and do a build. What you think?

@r0h4n comments?

@dahorak
Copy link
Contributor

dahorak commented Jan 30, 2018

@TimothyAsirJeyasing sorry for later response.

Did I understand it correctly, that we want to:

  • monitor for any commit into PR in all the Tendrl repos ->
  • trigger new build (in copr) from the (not yet merged) PR ->
  • store the build in repo (specific for the particular PR) for further usage ->
  • prepare testing cluster using the packages from the PR ->
  • run some testsuite on the cluster and report results?

@TimothyAsirJeyasing
Copy link
Author

@dahorak yes you are right, that's what its required.

@dahorak
Copy link
Contributor

dahorak commented Jan 30, 2018

Triggering package build in CentOS CI for each pull request should be possible[1].

On the repository level we have to:

  1. add the 'centos-ci' user as a collaborator (to each repository)
    !!This gives ci.centos.org commit access in order to set the test status on each pull request!!
  2. add the Github Pull Request Builder webhook as described on the Wiki[1] (section Mode 2. Trigger a build for each Pull Request).

Everything else will be done in the job configuration (I'll create some proposal and send PR later).

[1] https://wiki.centos.org/QaWiki/CI/GithubIntegration

@nthomas-redhat
Copy link

@dahorak , That is a good start.

Also we need to think about:

  • Where to keep the builds?
  • How to prepare the test machines, install and trigger the tests
  • How to handle the PR dependencies; i.e. a PR depends on another PR(otherwise tests might break)

@dahorak
Copy link
Contributor

dahorak commented Jan 30, 2018

Where to keep the builds?

For now, I would keep the builds in copr/tendrl/Features or similar repo (and for the test execution on a temporary test server which will be part of the testing cluster created ad hoc in CentOS CI).

We can also consider usage of CentOS CI Artifacts.

How to prepare the test machines, install and trigger the tests

We already have one testing cluster automated in CentOS CI (by default for packages from release repo). The test machines are prepared using ansible playbooks [1, 2]. So it will be similar (with some additional updates to be able to consume the PR builds.

How to handle the PR dependencies; i.e. a PR depends on another PR(otherwise tests might break)

I have the same question and I'm not sure how to do that...

@dahorak
Copy link
Contributor

dahorak commented Jan 31, 2018

@nthomas-redhat @TimothyAsirJeyasing the source for the PR build should be taken from refs/pull/XYZ/head or refs/pull/XYZ/merge (or both)?
If I understand it correctly, the head points to the last commit on the "to be merged" branch, while the merge points to already merged version (how it will look after the merge).

@TimothyAsirJeyasing
Copy link
Author

TimothyAsirJeyasing commented Jan 31, 2018

@nthomas-redhat Could you please add "centos-ci' as a user to all our tendrl repos so that the centos ci will get access to post the results and other stuff ?

This can be done by the following steps:
In our every tendrl repository visit the Collaborators tab in the repo Settings
Search by username 'centos-ci' and click 'Add Collaborator'

@dahorak
Copy link
Contributor

dahorak commented Jan 31, 2018

@nthomas-redhat Can you add "centos-ci' as a user to all our tendrl repos so that the centos ci will get access to post the results and other stuff ?

For reference see CentOS CI Wiki - section Mode 2: Trigger a build for each Pull Request.

@nthomas-redhat
Copy link

@TimothyAsirJeyasing , @dahorak , Ack

@dahorak
Copy link
Contributor

dahorak commented Jan 31, 2018

@nthomas-redhat Could you please also check, which Webhooks are configured in the repositories? Accordingly to the CentOS CI Wiki, there should be at least Pull Request and Issue Comment in the Let me select individual events section selected.
If it is configured differently, could you please update it (for now at least in the api repo, which I'm using now for testing the jobs)?

@r0h4n
Copy link
Contributor

r0h4n commented Jan 31, 2018

@dahorak Who are the admin users from Tendrl github org that can trigger the tests automatically on centos ci? How are we automating that since it requires a comment on PR from whitelisted admin user everytime you want trigger the tests

@r0h4n
Copy link
Contributor

r0h4n commented Jan 31, 2018

I have added the required centos ci webhooks for PR builds in each repo

@dahorak
Copy link
Contributor

dahorak commented Feb 1, 2018

@r0h4n Thanks for the webhooks!

The intended state is, that every PR from somebody from Tendrl organization will automatically trigger the build/tests.
The comments (like run tests) are there just as additional possibility, for example to retest something or to trigger tests for PR submitted by somebody not in the Tendrl organization (this is because of security reasons).

@r0h4n
Copy link
Contributor

r0h4n commented Feb 1, 2018

Ok sounds good, I have also added "centos-ci" user to the Tendrl organization and to indvidual repos with "write" access as Collaborator.

Let me know if you need anything else

@dahorak
Copy link
Contributor

dahorak commented Feb 1, 2018

Hopefully, there will be not much "spam" from the centos-ci while I'm debugging the jobs :-)...

@dahorak
Copy link
Contributor

dahorak commented Feb 1, 2018

It will definitely need some more tweaking, but now we have one job for each Tendrl repository (CentOS CI tendrl-PR view). The particular job should be triggered by new or updated PR or comment [re]run tests (there might be required some more tweaking of the list of users whose update can trigger the build).
The job will build snapshot package for the particular PR with added prXYZ to the version/release string (where XYZ is the PR number). The package is submitted to tendr/Features Copr repo.
Additionally separated repo with just the packages related to the PR is created and uploaded to artifacts.ci.centos.org (for example: http://artifacts.ci.centos.org/tendrl/pr/tendrl-api/pr345.merge/).

Jobs are configured not to run more concurrent jobs (for one repo), so if there are multiple PR/updates in one repo at once, it should trigger multiple jobs, but they will be executed one by one consecutively (we might consider to allow them run concurrently though).

@TimothyAsirJeyasing
Copy link
Author

There will be an individual jobs created for every tendrl repo to run the functional tests whenever a pr is submitted to a repo
Ex. For review:
https://ci.centos.org/view/tendrl-PR/job/tendrl-build-1-pr-api
Each job does the following things:

  • check if build_server is available
  • prepare remote working directory
  • clone git repository with source code
  • fetch and checkout the PR branch
  • build srpm package
  • prepare copr configuration
  • build package in copr
  • check status of copr build
  • download the packages from the copr
  • prepare rpm repo
  • prepare repofile
  • upload generated repo to artifacts.ci.centos.org
  • cleanup older packages for the particular PR
  • request the machines
  • install the packages and run the script

@dahorak
Copy link
Contributor

dahorak commented Feb 2, 2018

Timothy basically summarize it (the last two steps are not implemented yet).

Here is PR with the job configuration: #8 (still WIP, so please do not merge).

About the missing steps (cluster installation):

The question would be, if we wan't to have everything in one big job (build PR packages, validate the packages, prepare and install testing cluster,...), or if we want to split it to multiple jobs?
Have everything in one big job will allow us easily track and report status for each PR. But it will take quite long time to run one job.

@dahorak
Copy link
Contributor

dahorak commented Feb 2, 2018

@r0h4n there seems to be still some problems with updating the status back to github. Do you know, if it is required to somehow accept the invitation from the centos-ci site? I'm asking because I'm not sure, if the problem is with the job configuration of my job, or if it might be just unaccepted invitation from centos-ci site.

//update: I've send email to CentOS ci-users mailing list with question about this.

@dahorak
Copy link
Contributor

dahorak commented Feb 5, 2018

Ok, the notification back to github seems to be fixed. The status is reported in the Checks section as CentOS CI results.

There seems to be still some remaining issue with whitelisted organization (members of Tendrl organization should be able to trigger the build, which I'm not sure if it works correctly).

@dahorak
Copy link
Contributor

dahorak commented Feb 6, 2018

Accordingly what we discussed yesterday, my intention will be following functionality:

Every new or updated PR will trigger package build job

  • The job might be triggered also by comment run build in the respective PR.
  • The job create rpm repository with packages related to the particular git repo/PR.
  • ❔ Prospectively some basic validation (unit tests execution, rpmlint) will be performed.

Once PR will be ready for merge, somebody will manually trigger test job

  • The job will be triggered via comment run tests in the respective PR
  • ❔ It might be possible to somehow specify dependencies on PRs from other repos (probably in the PR description or the run tests... comment).

@dahorak dahorak changed the title Create framework for functional testing Build RPM package for every created or updated PR Feb 7, 2018
@dahorak
Copy link
Contributor

dahorak commented Feb 7, 2018

I'll keep this issue just for tasks related to building PR packages and I'll create new issue for executing tests suite.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants