Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(GDB-10638) Fix sonar stage #1766

Open
wants to merge 50 commits into
base: GDB-10373-workbench-with-single-spa
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
31f4859
Fixed jsdoc for the showCookiePolicyLink flag. Refactored get princip…
svilenvelikov Jun 26, 2024
d93367d
Add root-config directory
svilenvelikov Jun 26, 2024
9ea7690
Add package.json in the project root
svilenvelikov Jun 26, 2024
715f400
Created new project with Angular18 for the new workbench
svilenvelikov Jun 26, 2024
1a9b370
Add root-config directory
svilenvelikov Jun 26, 2024
2247b72
GDB-10373 legacy workbench as single spa (#1451)
svilenvelikov Jun 28, 2024
8370a06
GDB-10373 add shared api service (#1456)
svilenvelikov Jul 4, 2024
d3de249
Removes hardcoded host and port from the legacy workbench's app.js fi…
boyan-tonchev Jul 5, 2024
6217189
GDB-10463 workbench lyaout and navigation (#1458)
svilenvelikov Jul 5, 2024
092d640
GDB-10463 Improve the main layout and styling
svilenvelikov Jul 5, 2024
84a2ee9
GDB-10463 extended workbench layout with more components
svilenvelikov Jul 10, 2024
fbd2b6f
GDB-10463 implemented dynamic navbar with stenciljs (#1468)
svilenvelikov Jul 18, 2024
7ddc4c2
GDB-10512: run the host and legacy workbench on a single server (#1476)
boyan-tonchev Jul 18, 2024
7517a19
GDB-10563: Fix workbench version replacements (#1477)
boyan-tonchev Jul 19, 2024
69d6b4f
GDB-10577: Fix workbench license report functionality (#1478)
boyan-tonchev Jul 19, 2024
74e0111
GDB-10579: Fix resource loading error due to Content Security Policy …
boyan-tonchev Jul 23, 2024
2701710
GDB-10443 Implement i18n in the new workbench (#1485)
boyan-tonchev Jul 29, 2024
6a2394a
GDB-10463 navbar toggle and improvements (#1493)
svilenvelikov Jul 31, 2024
2556c2c
additional stuff moved around after all the cherry-picks
svilenvelikov Nov 14, 2024
202a0cd
package-loc files and webpack configs updated with missing imports
svilenvelikov Nov 14, 2024
9da74a1
Fixed some issues after rebase conflicts: webpack configs, plugin def…
svilenvelikov Nov 15, 2024
3679154
Updated concurrently and the webpack-config-single-spa which also req…
svilenvelikov Nov 15, 2024
667cdfd
Updated html-loader to the latest 5.1.0 version and reconfigured the …
svilenvelikov Nov 15, 2024
ae39dec
Enrich the Translation Service
boyan-tonchev Nov 18, 2024
67d4dd8
Introduce dropdown component
boyan-tonchev Nov 19, 2024
8ae4959
GDB-10673 restore TTYG tests which were lost during rebase conflicts
svilenvelikov Nov 25, 2024
6831e54
GDB-10570: Introduce Language Selector
boyan-tonchev Nov 25, 2024
ca1fc60
Fixes warning messages in browser console
boyan-tonchev Nov 22, 2024
ccd0e9c
GDB-11267: Implementation of Tooltip Component
boyan-tonchev Nov 27, 2024
14e8ae0
feat(GDB-10638) Add Jenkins pipeline
teodossidossev Aug 9, 2024
0dea626
GDB-10569: WB Migration: Implement the repository change selector in …
boyan-tonchev Dec 4, 2024
29cb120
rebased and fixed collisions
svilenvelikov Dec 5, 2024
c65b2af
GDB-10569: WB Migration: implement better deep copy functionality
boyan-tonchev Dec 6, 2024
55066d1
GDB-10569: WB Migration: create MapperProvider
boyan-tonchev Dec 9, 2024
f559a69
fix(GDB-10638) Fix installation process
teodossidossev Dec 9, 2024
cb9692d
Proper separation of test commands in shared components
svilenvelikov Dec 10, 2024
2c48986
GDB-11297: Implement Service for REST API Integration
boyan-tonchev Dec 13, 2024
269f938
`npm update` all dependencies in all packages
svilenvelikov Dec 6, 2024
952786c
Updated dependencies in api module
svilenvelikov Dec 13, 2024
1d40317
GDB-11369: WB Migration: Implement the Repository Selector Tooltip
boyan-tonchev Dec 16, 2024
c2c273a
Refactored and extended some of the tests
svilenvelikov Dec 17, 2024
ba4d290
fix(GDB-10638) Add baseUrl and mount api directory
teodossidossev Dec 18, 2024
e05448a
set cypress 13.17.0; fix sonar env prop
teodossidossev Dec 18, 2024
3cb299f
set higher node version of jenkins
teodossidossev Dec 18, 2024
08bda41
downgrade sonar-cube
teodossidossev Dec 18, 2024
d7464ac
set env for sonar cloud
teodossidossev Dec 18, 2024
ab82faa
set env for sonar cloud
teodossidossev Dec 18, 2024
ac584e6
set env for sonar cloud
teodossidossev Dec 18, 2024
37a3fed
set env for sonar cloud
teodossidossev Dec 18, 2024
2804438
add sonar login
teodossidossev Dec 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
7 changes: 0 additions & 7 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,3 @@ target
node_modules/
dist/
coverage/

# Dev specific
report/
cypress/
test-cypress/logs/
logs/
.downloads
14 changes: 3 additions & 11 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,13 +1,5 @@
FROM nginx:1.17-alpine
FROM node:18-alpine

ENV GRAPHDB_URL=http://graphdb:7200
WORKDIR /app

COPY docker-rootfs/ /

COPY dist/ /usr/share/nginx/html

RUN chmod +x /usr/local/bin/*.sh

EXPOSE 80

CMD ["start-workbench.sh"]
ENTRYPOINT ["npm"]
177 changes: 84 additions & 93 deletions Jenkinsfile
Original file line number Diff line number Diff line change
@@ -1,113 +1,104 @@
pipeline {

agent {
label 'graphdb-jenkins-node'
}

tools {
nodejs 'nodejs-20.11.1'
}

environment {
CI = "true"
NEXUS_CREDENTIALS = credentials('nexus-kim-user')
// Needed for our version of webpack + newer nodejs
NODE_OPTIONS = "--openssl-legacy-provider"
// Tells NPM and co. not to use color output (looks like garbage in Jenkins)
NO_COLOR = "1"
}

stages {

stage('Install') {
steps {
sh "npm install"
}
agent {
label env.AGENT
}

stage('Validate translations') {
steps {
sh 'node scripts/validate-translations.js || exit 1'
}
environment {
REPO_URL = 'https://github.com/Ontotext-AD/graphdb-workbench.git'
}

stage('Print Branch Name') {
steps {
script {
echo "Building branch: ${env.BRANCH_NAME}"
}
}
tools {
nodejs 'nodejs-20.11.1'
}

stage('Build') {
steps {
sh "npm run build"
}
}

stage('Sonar') {
steps {
withSonarQubeEnv('SonarCloud') {
script {
if (env.BRANCH_NAME == 'master') {
sh "node sonar-project.js --branch='${env.BRANCH_NAME}'"
} else {
sh "node sonar-project.js --branch='${env.ghprbSourceBranch}' --target-branch='${env.ghprbTargetBranch}' --pull-request-id='${env.ghprbPullId}'"
stages {
stage('Build Info') {
steps {
script {
echo "Agent: ${env.AGENT}"
echo "Building branch: ${env.BRANCH_NAME}"
}
}
}
}
}
}


stage('Acceptance') {
when {
expression {
return env.BRANCH_NAME != 'master'
stage('Install') {
steps {
catchError(buildResult: 'FAILURE', stageResult: 'FAILURE') {
script {
sh 'docker-compose run --rm npm run install:ci'
}
}
}
}
}
steps {
configFileProvider(
[configFile(fileId: 'ceb7e555-a3d9-47c7-9afe-d008fd9efb14', targetLocation: 'graphdb.license')]) {
sh 'cp graphdb.license ./test-cypress/fixtures/'

stage('Build') {
steps {
catchError(buildResult: 'FAILURE', stageResult: 'FAILURE') {
script {
sh 'docker-compose run --rm npm run build'
}
}
}
}
sh "ls ./test-cypress/fixtures/"
// --no-ansi suppresses color output that shows as garbage in Jenkins
sh "docker-compose --no-ansi build --force-rm --no-cache --parallel"
sh "docker-compose --no-ansi up --abort-on-container-exit --exit-code-from cypress-tests"

// Fix coverage permissions
sh "sudo chown -R \$(id -u):\$(id -g) coverage/"
sh "sudo chown -R \$(id -u):\$(id -g) cypress/"
sh "sudo chown -R \$(id -u):\$(id -g) report/"
stage('Lint') {
steps {
catchError(buildResult: 'FAILURE', stageResult: 'FAILURE') {
script {
sh 'docker-compose run --rm npm run lint'
}
}
}
}

// Move up to be picked by Sonar
// sh "mv test-cypress/coverage/ cypress-coverage/"
// sh "sync"
//
// // Update relative paths to absolute path
// sh "sed -i.backup \"s@^SF:..@SF:\$(pwd)@\" cypress-coverage/lcov.info"
}
}
}
stage('Test') {
steps {
catchError(buildResult: 'FAILURE', stageResult: 'FAILURE') {
script {
sh 'docker-compose run --rm npm run test'
}
}
}
}

post {
always {
// upload failed tests report and artifacts
junit allowEmptyResults: true, testResults: 'cypress/results/**/*.xml'
archiveArtifacts allowEmptyArchive: true, artifacts: 'report/screenshots/**/*.png, report/videos/**/*.mp4, cypress/logs/*.log'
stage('Cypress Test') {
steps {
catchError(buildResult: 'FAILURE', stageResult: 'FAILURE') {
script {
// Fix user rights
sh 'sudo chown -R $(id -u):$(id -g) .'
sh 'npm run cy:run'
}
}
}
}

// --no-ansi suppresses color output that shows as garbage in Jenkins
sh "docker-compose --no-ansi down -v --remove-orphans --rmi=local || true"
// clean root owned resources from docker volumes, just in case
sh "sudo rm -rf ./coverage"
sh "sudo rm -rf ./cypress"
sh "sudo rm -rf ./report"
stage('Sonar') {
steps {
catchError(buildResult: 'FAILURE', stageResult: 'FAILURE') {
script {
withSonarQubeEnv('New-sonar') {
withEnv(["BRANCH_NAME=${env.BRANCH_NAME}"]) {
sh 'npm run sonar'
}
}
}
}
}
}
}

failure {
archiveArtifacts artifacts: 'translation-validation-result.json', onlyIfSuccessful: false
post {
failure {
wrap([$class: 'BuildUser']) {
emailext(
to: env.BUILD_USER_EMAIL,
from: "Jenkins <[email protected]>",
subject: '''[Jenkins] $PROJECT_NAME - Build #$BUILD_NUMBER - $BUILD_STATUS!''',
mimeType: 'text/html',
body: '''${SCRIPT, template="groovy-html.template"}'''
)
}
}
}
}
}

84 changes: 35 additions & 49 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@ The web application for GraphDB APIs
[![Build Status](https://jenkins.ontotext.com/buildStatus/icon?job=graphdb-workbench-pipeline)]
### Setup Environment

* Checkout or clone the project.
* Enter the project directory and execute `npm install` in order to install all
needed dependencies locally.
* Clone the project or check it out from version control.
* Open a terminal, navigate to the project's root directory, and run `npm install:ci` to install all required dependencies.
The `api` project is built automatically.

## Development

Expand All @@ -16,80 +16,45 @@ https://ontotext-ad.github.io/graphdb-workbench/developers-guide

### Dev server

Running `npm run start` will bundle application and vendor code in memory and start a webpack
development server and proxy requests to `localhost:7200` (default).
The workbench can be run in a development mode by executing `npm run start`. This will start each of the child applications
in watch mode and will proxy requests to a GraphDB instance running on `localhost:7200` (default).

## Testing

### Acceptance/functional tests

Cypress is used as a framework for writing functional tests which cover concrete UI components as
well as whole acceptance scenarios. The tests are executed against a GraphDB version as defined in
`package.json#versions.graphdb` which is run in a docker container.

> Some tests may require a GraphDB Enterprise Edition license file. The file must be named graphdb.license
and put in the `test-cypress/fixtures` folder. In case you don't have such license, you can ignore the
failing tests.


There are two options for running the tests. One is a headless execution and the second is through
the Cypress's dashboard application. Follow the steps described below:
* Ensure a GraphDB instance is running on `localhost:7200`. One can be run by executing
`docker-compose up` in the `graphdb-workbench/test-cypress` folder. Check path directory which is set with **-Dgraphdb.workbench.importDirectory** property when server is started. The folder with this path have to contains all files from "fixture/graphdb-import"

* In `graphdb-workbench` folder execute `npm run start` to build and run the workbench web
application. In result, it is published and served by webpack's web dev server.
* In terminal, go in `graphdb-workbench/test-cypress` folder and choose one of the options below:
* Execute `npm run test` - this will run the test suite in a headless mode and the outcome log
will be seen in the terminal.
* Execute `npm run start` or the equivalent `npx cypress open` - this will open the Cypress's
dashboard application through which the tests can be run one by one or altogether and observing the
outcome in the dashboard.

> There is a separate acceptance tests suite containing some flaky tests which often fail on CI.
> That's why these are moved in the `integration-flaky` suite which is doesn't run on CI and is
> responsibility of the developer to run them manually before the commit.
>
> The flaky test suite can be run similarly to the normal test suite but using following commands:
> `npm run start:flaky` for browser mode or `npm run test:flaky` for a headless mode.

## Release and publish

The workbench is regularly published as a package in the NPM registry.

### Build

Application can be built by executing the `npm run build` command. In result, the application is
bundled, less files are processed and the code is minified. The result of the build command is
emitted in the `/dist` folder. When the workbench is published, only the `/dist` folder gets
published in the NPM registry. This is configured in `package.json#files` property.

## Deploying

### Standalone (Docker)

The repo includes sample Dockerfile that configures NGiNX for serving the workbench and proxying
requests to a GraphDB endpoint. This is configurable via the `GRAPHDB_URL` environment variable.
Example:
### Standalone (Docker)

`docker run -d -p 8080:80 -e GRAPHDB_URL=10.131.2.176:7200 graphdb-workbench`

#### Local development

For ease of use in local development with a locally running GraphDB at localhost:7200, there is also a
Docker compose that can be built and started with `docker-compose up --build`. The compose requires
to have `.env` file in the root directory of the project where the `HOST_IP` environment variable
For ease of use in local development with a locally running GraphDB at localhost:7200, there is also a
Docker compose that can be built and started with `docker-compose up --build`. The docker-compose requires
to have `.env` file in the root directory of the project where the `HOST_IP` environment variable
must be specified, e.g. `HOST_IP=10.131.2.176`. This is needed to proxy requests to locally running GraphDB.

### Using GraphDB distribution

GraphDB exposes a configuration param `-Dgraphdb.workbench.home` for overriding the bundled workbench.
GraphDB exposes a configuration param `-Dgraphdb.workbench.home` for overriding the bundled workbench.
This allows to easily point it to the `dist/` folder of the workbench after it has been bundled
with `npm run build`.

Note: Wrongly configuring the parameter will result in GraphDb responding with HTTP 404.

#### GraphDB Docker distribution
The Docker distribution of GraphDB can also be configured to serve custom workbench, the only difference

The Docker distribution of GraphDB can also be configured to serve custom workbench, the only difference
is that the workbench must be mounted, example:

```
Expand All @@ -99,8 +64,29 @@ docker run -d \
-Dgraphdb.workbench.home=/workbench
```

Note: Instead of mounting the workbench, this can be done in a custom Docker image using the
Note: Instead of mounting the workbench, this can be done in a custom Docker image using the
GraphDB one as a base and then copy the custom workbench.

## CI

The CI pipeline is managed using a Jenkinsfile, which defines the stages for building, testing, and analyzing the project.

The CI pipeline uses a Docker Compose utility setup to ensure independence from the Node.js version installed on the Jenkins agent.

The **Jenkinsfile** defines the CI pipeline for building, testing, and analyzing the **GraphDB Workbench** project. Below is an overview of the workflow:

| **Stage** | **Purpose** | **Tools/Commands** |
|------------------|------------------------------------------------------|----------------------------------------|
| **Build Info** | Logs environment details for traceability | |
| **Install** | Installs dependencies and builds packages | `npm run install:ci`, `docker-compose` |
| **Build** | Builds the project | `npm run buid`, `docker-compose` |
| **Lint** | Performs lint checks | `npm run lint`, `docker-compose` |
| **Test** | Runs unit tests | `npm run test`, `docker-compose` |
| **Cypress Test** | Executes end-to-end tests | `npm run cy:run`, |
| **Sonar** | Performs static code analysis to ensure code quality | `npm run sonar` |

Note: If any stage fails, the pipeline is marked as failed, but proceeds to the next stage.

## License

[LICENSE](licenses/LICENSE)
9 changes: 9 additions & 0 deletions docker-compose.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
version: "3"
services:
npm:
build: ./
stdin_open: true
tty: true
volumes:
- ./:/app

Loading