Skip to content

Commit

Permalink
Make nightly dashboard trend plot interactive (#64)
Browse files Browse the repository at this point in the history
This PR is a draft to address #55. To do this I have ported the report
to become a quarto doc rather than rmarkdown and then written the viz in
javascript for more interactivity.

Because the way this is implemented, it needs to be served up via https
rather than a local html file. So screenshots it is. Here is the default
which sets the x-axes to extend to the last 120 days but we can slide to only look at the last ten days or look at the past 6 months.

I have also updated the build table to include passing runs. Because
this adds a significant amount of rows to the table, I've implemented
some interactivity for the build table. This looks like this:
  • Loading branch information
boshek authored Aug 26, 2023
1 parent 29a9b02 commit 4828ea0
Show file tree
Hide file tree
Showing 13 changed files with 2,988 additions and 342 deletions.
57 changes: 32 additions & 25 deletions .github/workflows/nightly_dashboard.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,13 @@ on:
paths:
- "csv_reports/*.csv"
- ".github/workflows/nightly_dashboard.yml"
- "crossbow-nightly-report.Rmd"
- "crossbow-nightly-report/crossbow-nightly-report.qmd"
push:
branches: main
paths:
- "csv_reports/*.csv"
- ".github/workflows/nightly_dashboard.yml"
- "crossbow-nightly-report.Rmd"
- "crossbow-nightly-report/crossbow-nightly-report.qmd"

jobs:
build:
Expand All @@ -27,51 +27,58 @@ jobs:
with:
repository: apache/arrow
path: arrow

- uses: actions/checkout@v3
with:
path: crossbow

- name: Set up Python
uses: actions/setup-python@v4
with:
cache: pip
python-version: 3.11

- name: Generate List of Crossbow Tasks
run: |
python -m pip install -e arrow/dev/archery[crossbow]
archery crossbow check-config > crossbow/all.yml
- name: Setup Quarto
uses: quarto-dev/quarto-actions/setup@v2
with:
version: '1.3.450'

- uses: r-lib/actions/setup-r@v2
with:
r-version: '4.3.1'
use-public-rspm: true
- uses: r-lib/actions/setup-pandoc@v2
- uses: r-lib/actions/setup-r-dependencies@v2

# Needed due to https://github.com/r-lib/actions/issues/618
- name: Link renv.lock
run: ln -sf 'crossbow/crossbow-nightly-report/renv.lock'

- name: Setup renv
uses: r-lib/actions/setup-renv@v2
env:
# This is set by setup-r
RENV_CONFIG_REPOS_OVERRIDE: ""
with:
dependencies: 'NA'
packages: |
any::rmarkdown
any::readr
any::rlang
any::ggplot2
any::gt
any::dplyr
any::glue
any::purrr
any::tidyr
any::cli
any::lubridate
any::tibble
any::yaml
cache-version: 1
- name: Build Report
working-directory: crossbow
shell: Rscript {0}
run: |
rmarkdown::render("crossbow-nightly-report.Rmd", output_file = "index.html")
uses: quarto-dev/quarto-actions/render@v2
env:
CROSSBOW_LOOKBACK_WINDOW: 120
with:
to: html
path: 'crossbow/crossbow-nightly-report/crossbow-nightly-report.qmd'

- name: Upload Rendered Dashboard
if: github.event_name == 'pull_request'
uses: actions/upload-artifact@v3
with:
name: dashboard
path: |
crossbow/index.html
crossbow/crossbow-nightly-report/crossbow-nightly-report.html
crossbow/all.yml
- name: Upload result
if: github.event_name == 'push'
Expand All @@ -83,4 +90,4 @@ jobs:
AWS_DEFAULT_REGION: ${{ secrets.CROSSBOW_DOCS_S3_BUCKET_REGION }}
BUCKET: ${{ secrets.CROSSBOW_DOCS_S3_BUCKET }}
run: |
aws s3 cp index.html $BUCKET/index.html
aws s3 cp crossbow-nightly-report/crossbow-nightly-report.html $BUCKET/index.html
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,4 @@
performance-release-report.html
crossbow-nightly-report.html
crossbow-nightly-report_cache
all.yml
3 changes: 3 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,6 @@ cron jobs, see the github actions workflows defined in this repository.

For the triggered tasks see the nightly group of crossbow's
[task definition file](https://github.com/apache/arrow/blob/main/dev/tasks/tasks.yml).


Nightly report CSVs are stored on the `csv_reports` directory.
Loading

0 comments on commit 4828ea0

Please sign in to comment.