-
-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make ClickhouseRepo task timeouts configurable in CE #4494
Conversation
50c54d4
to
034fe8c
Compare
034fe8c
to
eec47cb
Compare
lib/plausible/clickhouse_repo.ex
Outdated
@task_timeout | ||
else | ||
ch_timeout = Keyword.fetch!(config(), :timeout) | ||
max(ch_timeout * 2, @task_timeout) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm doubling the timeout value from the repo just to make sure we don't timeout the task before db_connection does. It could be max(ch_timeout + 1, @task_timeout)
but I thought that * 2
is safer since maybe some tasks do more than one DB request. Maybe it should be * 4
since this seems to be the default relation, (@task_timeout / default_db_connection_timeout = 60_000 / 15_000 = 4
)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, let's adjust it to 4 👍
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ruslandoga Perhaps starting a repo process without name and using https://hexdocs.pm/ecto/Ecto.Repo.html#c:put_dynamic_repo/1 would enable testing it (the original repo pid/name would have to be put back at the end of the test though). |
I tried adding the test: And now I think it is too complicated for what it is testing... |
ca02738
to
d32160d
Compare
Changes
This PR makes CE's ClickhouseRepo tasks reuse timeouts defined on
CLICKHOUSE_DATABASE_URL
For example, setting
CLICKHOUSE_DATABASE_URL=http://plausible_events_db:8123/plausible_events_db?timeout=120000
would make the task timeout value equal eight minutes (480000=url_timeout*4). It has already been used to solve some timeout issues before: #3311 - so it makes sense to standardize on this approach rather than introduce extra ENV vars.Related issues and discussions, where increasing the timeout value for tasks might help:
Some examples for how the timeout param in URL affects Repo config:
Tests
config_test.exs
can be updated, but for a custom repo like in Run migrations in order across repos #4466Changelog
CLICKHOUSE_DATABASE_URL
Documentation
Dark mode