Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

6.0.0 MVU upgrade docs update #946

Open
wants to merge 13 commits into
base: main
Choose a base branch
from

Conversation

DaedalusG
Copy link
Contributor

@DaedalusG DaedalusG commented Feb 3, 2025

This PR updates the upgrade docs embellishing instructions for the different deployment methods to include better coverage and handling of the Postgres 16 database upgrade when crossing over Sourcegraph 5.10.0 using migrator 6.0.0 or greater.

The notable change here is the inclusion of deployment specific builtin Postgres upgrade instructions, as well as changes to the MVU upgrade docs suggesting that users bring down all services and merge upstream changes from their target version into their release branch before performing their migrator upgrade. Due to migrators depends_on clause this will ensure that Postgres upgrades occur before migrator attempts connection to a PG12 database. This gets around the connect bouncer added in 6.0.0 which blocks services connection to our dbs

Testing

  • docker-compose
    • autoupgrade
    • MVU (tested 5.9 -> 5.11 with migrator 6.0)
  • kustomize
    • autoupgrade
    • MVU (tested 5.9 -> 5.11 with migrator 6.0)
  • Helm
    • autoupgrade
    • MVU

DaedalusG and others added 6 commits January 31, 2025 11:52
They should be "copy-pasteable" now. I put it in quotes because they
have to paste it very specifically, but I think they are sufficiently
idiot proof
Copy link

vercel bot commented Feb 3, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
sourcegraph-docs ✅ Ready (Inspect) Visit Preview 💬 Add feedback Feb 4, 2025 0:07am

@DaedalusG
Copy link
Contributor Author

This is a separate thing, but autoupgrade doesn't really work very well with k8s deployments

{"SeverityText":"WARN","Timestamp":1738577648562528772,"InstrumentationScope":"frontend","Caller":"cli/autoupgrade.go:282","Function":"github.com/sourcegraph/sourcegraph/cmd/frontend/internal/cli.claimAutoUpgradeLock","Body":"named postgres connections found, waiting for them to shutdown, manually shutdown any unexpected ones","Resource":{"service.name":"frontend","service.version":"6.0.0","service.instance.id":"sourcegraph-frontend-bc5dc56f5-t4ns2"},"Attributes":{"applications":["repo-updater"]}}
➜  deploy-sourcegraph-k8s git:(release) ✗ k get pods
NAME                                         READY   STATUS             RESTARTS        AGE
blobstore-776f6ff9d4-tbjk9                   1/1     Running            0               8m36s
codeinsights-db-0                            2/2     Running            0               8m36s
codeintel-db-0                               2/2     Running            0               8m36s
gitserver-0                                  0/1     CrashLoopBackOff   6 (16s ago)     8m36s
grafana-0                                    1/1     Running            0               8m36s
indexed-search-0                             2/2     Running            0               8m6s
node-exporter-8ch2b                          1/1     Running            0               8m37s
pgsql-0                                      2/2     Running            0               8m36s
precise-code-intel-worker-6b4469fcb8-h5bfs   0/1     CrashLoopBackOff   6 (2m17s ago)   8m37s
prometheus-564889b58c-qtd7h                  1/1     Running            0               8m36s
redis-cache-766c488dd8-4hw5n                 2/2     Running            0               8m36s
redis-store-868dc9bd4f-hmwvj                 2/2     Running            0               8m36s
repo-updater-866ddd54df-pkjq2                1/1     Running            6 (11m ago)     13m
repo-updater-df44f556f-fdt4h                 0/1     CrashLoopBackOff   6 (111s ago)    8m37s
searcher-0                                   1/1     Running            0               8m36s
sourcegraph-frontend-bc5dc56f5-hp8x9         1/1     Running            0               8m37s
sourcegraph-frontend-bc5dc56f5-t4ns2         1/1     Running            0               8m37s
symbols-0                                    0/1     Running            6 (2m55s ago)   8m36s
syntect-server-9ddff8b8d-225r2               1/1     Running            0               8m37s
worker-64c947c995-mb7fr                      0/1     CrashLoopBackOff   6 (116s ago)    8m37s

To fix this you'd need to go in and scale down the services that are continually trying to reach the DB

@DaedalusG
Copy link
Contributor Author

DaedalusG commented Feb 3, 2025

Calling it for now but I suspect we'll have to special case Helm and add a note to the kubernetes upgrade notes saying that MVU for helm specifically can't got over the v5.10 boundary. I want to investigate packaging an override file that updates the images in the repo though

https://github.com/sourcegraph/deploy-sourcegraph-helm/blob/main/charts/sourcegraph/examples/basic/override.yaml

Copy link
Collaborator

@MaedahBatool MaedahBatool left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left very minor comments.

docs/admin/deploy/docker-compose/upgrade.mdx Outdated Show resolved Hide resolved
docs/admin/deploy/kubernetes/index.mdx Outdated Show resolved Hide resolved

The following doc contains detailed instructions for upgrading the built-in PostgreSQL databases. Via our `postgresql-16` and `postgresql-16-codeinsights` image entrypoint script. This doc assumes an admin is attempting to upgrade to Sourcegraph `6.0.0` from an older version (usually pre `5.10.0`) using one of our "deploy" repos. For more general info see [Upgrading PostgreSQL](/admin/postgres#upgrading-postgresql).

> WARNING: Upgrading the PostgreSQL database requires stopping your Sourcegraph deployment which will result in **downtime**.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning calllout

@DaedalusG DaedalusG requested a review from a team February 3, 2025 21:38
```yaml
migrator:
container_name: migrator
image: 'index.docker.io/sourcegraph/migrator:5.0.4'
image: 'index.docker.io/sourcegraph/migrator:6.0.0'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we may want to use CURRENT_VERSION_STRING stuff here?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants