Use a docker registry for E2E tests

[nicolaslara]

Background

You can't cache docker builds on CI and building it on every run makes it much slower.

Suggested Design

We can modify CI to use an osmosis docker registry so that the builds get posted there and get pulled if there are no changes.

Acceptance Criteria

E2E tests run faster

[ValarDragon]

We build a docker image for main on most commits. Cref: #2264 (comment)

Its a bit unclear to me though, looking at recent PRs, seems like a lot don't have this build step that we want to make faster. But some still do at 2.5 minutes (#2264 (comment)) (EDIT: And some at 4.5 minutes (https://github.com/osmosis-labs/osmosis/actions/runs/3716362935/jobs/6302602855))

So we want to speed that up.

The idea of this issue, is:

1. Understand why we have to rebuild only some of the time
2. Understand if we can get the incremental go build time, if we leveraged docker hub (or is it impossible to get the go CI cache there)

Unassigning myself though, would love if someone can work on it though!

[nicolaslara]

@niccoloraspa I don't think this is high priority, but mentioning you here since you probably are interested in this particular issue

[p0mvn]

What exactly should be done in this issue for it to be addressed? If it's undefined, can we close it?

The description and Dev's comment seem to talk about different things.

Unfortunately, we cannot post the image to the docker registry because we must make sure that we are running an e2e test for the current commit version of Osmosis.

The only possible mitigation strategies that I'm aware of are the following:

1. Reduce the frequency of builds by making sure that we only rebuild when the relevant Go files change

• There is filtering that exists already here

2. Increase the likelihood of Docker cache being reused by avoiding the copying of misc files in the Dockerfile

• This is done by the .dockerignore

So, as far as I understand, we already apply all possible mitigation strategies

[czarcas7ic]

Closing this in favor of other speed up ideas