Elasticsearch docker test #1827
Conversation
…t up a fresh testindex on each run (and delete old testindex if it exists)
lib/console/logs/provider/elastic.ex
Outdated
| @@ -92,6 +92,13 @@ defmodule Console.Logs.Provider.Elastic do | |||
| defp add_facets(q, _), do: q | |||
|
|
|||
| defp facets(resp) do | |||
| # make sure that kubernetes.node has at least an empty map | |||
There was a problem hiding this comment.
this todo might be able to be removed
|
|
||
| # We want a fresh index so delete the index if it already exists | ||
| # Just in case test fails we want to make sure next run doesn't reuse old index | ||
| with true <- ES.index_exists?(@elastic_cluster_url, @elastic_index_name) do |
There was a problem hiding this comment.
the delete and recreate should be moved to a mix task and added to the mix test alias in mix.exs
| ES.create_index(@elastic_cluster_url, @elastic_index_name) | ||
|
|
||
| ES.index_documents(@elastic_cluster_url, @elastic_index_name, [ | ||
| %{ |
There was a problem hiding this comment.
let's move these documents to helper function in elastic utils, eg something like:
def log_document(%Service{}, line) do
...map data for log
endAnd that way you can just index by doing something like:
log_document(svc, "valid log") |> index()
log_document(svc, "valid log2") |> index()
log_document(svc2, "valid log") |> index()
refresh()There was a problem hiding this comment.
you should also just import the elastic helper so it's quicker to call the functions I think but that's more aesthetics
kinjalh
force-pushed
the
elasticsearch-docker-test
branch
from
e06827d to
82c1fcb
Compare
| @@ -8,6 +8,16 @@ services: | |||
| POSTGRES_PASSWORD: postgres | |||
| volumes: | |||
| - database_data:/var/lib/postgresql/data | |||
| es: | |||
There was a problem hiding this comment.
i think this might need a volume to really behave nicely
lib/mix/tasks/elasticsearch_setup.ex
Outdated
| @@ -0,0 +1,32 @@ | |||
| # lib/mix/tasks/elasticsearch_setup.ex | |||
| # the index gets set up using a mix task before the test is run, so we can index directly | ||
| log_document(svc, "valid log message") |> index_doc(@elastic_cluster_url, @elastic_index_name) | ||
| log_document(svc, "another valid log message") |> index_doc(@elastic_cluster_url, @elastic_index_name) | ||
| refresh(@elastic_cluster_url, @elastic_index_name) |
There was a problem hiding this comment.
I'd move the cluster url into your elasticsearch helper module as a default arg (so users don't need to specify it w/ every call to index_doc or refresh
There was a problem hiding this comment.
use default args in refresh here too
…volume to elasticsearch docker container
| # the index gets set up using a mix task before the test is run, so we can index directly | ||
| log_document(svc, "valid log message") |> index_doc(@elastic_cluster_url, @elastic_index_name) | ||
| log_document(svc, "another valid log message") |> index_doc(@elastic_cluster_url, @elastic_index_name) | ||
| refresh(@elastic_cluster_url, @elastic_index_name) |
There was a problem hiding this comment.
use default args in refresh here too
Co-authored-by: kinjalhaldar <[email protected]>
Co-authored-by: kinjalhaldar <[email protected]>
Add an unit test that uses a local elasticsearch index to check if it is able to retrieve logs from.
This requires spinning up an elasticsearch container, creating an index on it, adding log entries to the index, querying the index for the logs, and then deleting the index.
Test Plan
mix test test/console/graphql/queries/observability_queries_test.exsChecklist