Challenging contribution: runbot_docker

[bwrsandman]

A new module which changes the main test functions of original runbot.

Overload all functions dealing with runbot/static/* with the exception of the logs.

An option for a base DockerFile image to simplify the build should be added to the runbot.repo.

All builds should use the base DockerFile and clone and merge pull requests. In the job_00.

The job_10 should be based off the job_00 and job_20 should depend on job_10.

The cleanup mechanism should be transposed to clean docker images.

Find a way to optimize repo.branch docker files. For example, a PR on 8.0 should start with the 8.0 job_00 image and add a git fetch and merge from the PR source.

Running the docker images should forward the same port inside the instance, but map it the same way it does now (in a range in 2000).

[lepistone]

Thanks for writing down the proposal @bwrsandman!

The original runbot is a very specific tool for the needs of Odoo SA. They use it to run the tests, which I don't care about because travis & co already exist. On the other hand, it builds instances that can be tested online, which is what we do for OCA.

With runbot_build_instructions and #51, we can have a generic build phase, ideally the same we use on develop, staging and production.

If we add handling of docker images like you suggest here, runbot could in theory become completely independent of odoo: whatever build system like buildout of whatever could produce a docker image, that we deploy on runbot as a multiple staging server of sorts. Then I'd like to be able to deploy the same docker images to normal servers. The different ports can be exposed via docker instead of passing manually options to odoo. At this point we'd have a nice web interface to deploy any multiple docker instances, exposing ports, at every commit/pull request. This is nice, but makes me wonder whether such a thing exists already, and if not, then it would make sense to make the effort independently of odoo.

Does that make any sense? Thanks for your time!

[bwrsandman]

Technically, with runbot_build_instructions you can test things that aren't odoo.
Are you talking about making runbot independent of Odoo as the server and not the product being tested?
If that's the case, that would require a major rewrite of the logic (mainly the ORM). I think a lot could be gained since Odoo is already quite heavy, has many outdated dependencies and limitations.

It would of course mean a fork and that these addons would no longer benefit from Odoo's improvements to the base runbot.

Forking is costly, and would only work if there were a lot of contributors and maintainers. I know base runbot isn't the top priority for development at Odoo, but they do have a lot more resources than the OCA has.

[moylop260]

FYI You can re-use the .travis.yml to auto-generate the Dockerfile.
You can check this case
video-using-t2d
You can see t2d (ugly script yet) here: travisfile2dockerfile.py

[lepistone]

@bwrsandman I was still talking about the product under test. You're right, with build_instructions you somewhat can, but it's still somewhat coupled to odoo because of ports, database names, options being passed on and so on. But it is close enough, I agree.

On the other hand, thinking of the implementation, if we pull from a git repository docker images and run them exposing different ports, with a webapp that allows to login into each of them and see the logs, we're getting quite far from the original runbot. I agree on that, too.

[lepistone]

@moylop260 I've seen the script, thanks!

Personally I'd prefer not to base my builds on the travis file, but on some other system like buildout + erppeek. Then both travis and runbot (or docker or...) can just wrap the existing system with minimal overhead.

[moylop260]

@lepistone
I'm agreed with your point.
However, I need to add my point if you allow me.

Currently, we have ten diferent sides where we have that create a configuration.

• .travis.yml
  • Add git clone's
  • Add pip install
  • Add apt-get install
• runbot
  • Add runbot.repo
  • Configurate repo depends.
  • Install all dependencies that could needs it.
• external_dependencies
  • Add to __openerp__.py the extra bin packages and extra python packages.
• README
  • Explain how to install depends packages.
• (Future) Dockerfile
  • Add git clone's
  • Add pip install
  • Add apt-get install
• buildout
  • ...
• erppekk
  • ...

Then, by each tool we need to create a additional configuration.
The idea of create docker based on .travis.yml is from: shippable

But I'm agreed with you of don't depends directly from .travis.yml file then
IMHO we need to merge all good ideas from all projects to auto-generate instance and integrate it in MT to use it in

• Runbot with a smaller build_instruccions:
  • $: pip install oca-mt && oca-future_script_create_instance oca/PROJECT-X
• Travis with a smaller .travis.yml configuration:
  • $: pip install oca-mt && oca-future_script_create_instance oca/PROJECT-X
• Docker with a smaller Dockerfile configuration:
  • $: pip install oca-mt && oca-future_script_create_instance oca/PROJECT-X
• Buildout
  • I don't know as work.
• erppekk
  • I don't know as work.

oca_dependencies.txt is a good approach to this focus.
Maybe, we will need a oca_packages_dependencies based on...
I don't know, maybe a root file of requirements.txt with pip install packages and requirements.sh with apt-get install packages or extracted.
Or maybe a script to extract external_dependencies from __openerp__.py of modules cloned.
Or maybe ... I don't know...
Similar discussion here

Thanks for getting here, I wanted to express myself.

What do you think?

[moylop260]

FYI
Now travis support docker commands.
http://blog.travis-ci.com/2015-08-19-using-docker-on-travis-ci/

[moylop260]

FYI I'm working in a alpha module of runbot_travis2docker
Now is alpha and I had that change vauxoo/MQT
When I have a stable version... I will create PR to oca/mqt and oca/runbot-addons

However, your feedback is welcome in fork projects.

[moylop260]

FYI this issue finish since that @gurneyalex told us that now oca runbot server has installed runbot_travis2docker

(Same comment from OCA/maintainer-quality-tools#313 (comment))

[gurneyalex]

thanks