Skip to content
This repository has been archived by the owner on Sep 4, 2024. It is now read-only.

Add Argos to report visual differences #341

Merged
merged 1 commit into from
Jul 11, 2024

Conversation

arturowczarek
Copy link
Contributor

@arturowczarek arturowczarek commented Jun 25, 2024

This pull request introduces the first step of visual differences reporting as described in https://docusaurus.io/blog/upgrading-frontend-dependencies-with-confidence-using-visual-regression-testing. I've noticed the used Dinosaurus is obsolete and wanted to update it.

For this pull request to work, we need to sign up to https://app.argos-ci.com/signup and connect it to the repository as described in https://argos-ci.com/docs/github.

This is the first part of the two part change. After merging them both, we'll be able to see the differences like arturowczarek#12 after adding visual-comparison-required label.

pull request

argos comparison

Agros has a free tier of 5000 screenshots per month. One pull request uses around 200 screenshots. There is also an open source tier OpenLineage is eligible for. We have a consent from the Argos creator to have it if we need it.

greg

About changes:

  • I've introduced a new directory argos which is configured as a node workspace in root package.json. It means the run commands from this directory can be executed with workspace argos prefix. See .github/workflows/argos.yml file with yarn commands. For example yarn workspace argos screenshot runs the screenshot script from argos/package.json file.
  • Argos is responsible only for screenshots comparison. The actual screenshots are taken with playwright
  • I had to add a couple of changes to .gitignore to ignore the temporary things playwright and argos/package.json were creating
  • argos/playwright.config.ts is is the default file located by playwright to take screenshots. It goes back to the root directory, serves the documentation and configures chrome to take screenshots
  • argos/screenshot.css contains some css styles as suggested in the original article. It is meant to hide some changes which could make the comparison harder. For example the dynamically generated text published 1 minute ago would change to published 1 day ago when we run it the next day
  • argos/screenshot.spec.ts - in this file we start taking the screenshots. It is modelled as a test for ease of execution. This script takes all the links from sitemap.xml and takes a screenshot

Copy link

netlify bot commented Jun 25, 2024

Deploy Preview for mellifluous-eclair-326dcd ready!

Name Link
🔨 Latest commit 6e73724
🔍 Latest deploy log https://app.netlify.com/sites/mellifluous-eclair-326dcd/deploys/668d9c63f4040f0008bc68d7
😎 Deploy Preview https://deploy-preview-341--mellifluous-eclair-326dcd.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@arturowczarek arturowczarek reopened this Jun 25, 2024
@arturowczarek arturowczarek force-pushed the agros branch 2 times, most recently from 3280e11 to 532465d Compare June 25, 2024 09:30
@arturowczarek arturowczarek changed the title Add Agros to report visual differences introduced by pull requests. Add Argos to report visual differences introduced by pull requests. Jun 25, 2024
@pawel-big-lebowski
Copy link
Collaborator

@arturowczarek This sounds cool. Could you provide some example output of using Argos within the project? Perhaps some dummy change on the branch created on the top of this branch to see how does it work within this project.

@arturowczarek
Copy link
Contributor Author

@pawel-big-lebowski See arturowczarek#3 and ArgosCI bot comment. There you can click on Review to see that one of the pages has changed (this one)

@pawel-big-lebowski
Copy link
Collaborator

When just modifying a markdown, adding a single line causes whole content to go line down and screenshot diff presents entire page red. This tool is cool to verify docs engine configuration changes, but may be an overkill when just editing a content.

My understanding is: there are 5000 free screenshots monthly and each run is about 200 screenshots. This means after 25 commits, project runs out of credits and CI will start failing. This doesn't sound that cool, but perhaps I am missing something.

@arturowczarek arturowczarek changed the title Add Argos to report visual differences introduced by pull requests. Add Argos to report visual differences Jul 2, 2024
@arturowczarek arturowczarek force-pushed the agros branch 2 times, most recently from 205c8a8 to 1d446d5 Compare July 3, 2024 18:25
@arturowczarek
Copy link
Contributor Author

@pawel-big-lebowski Please read the updated pull request description. I've added some extra information. The second part of the change is here #352

@pawel-big-lebowski pawel-big-lebowski merged commit bf2b124 into OpenLineage:main Jul 11, 2024
6 checks passed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants