Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create ARM client image #503

Closed
jeremyprime opened this issue Sep 22, 2022 · 1 comment
Closed

Create ARM client image #503

jeremyprime opened this issue Sep 22, 2022 · 1 comment
Assignees
Labels
enhancement New feature or request High Priority
Milestone

Comments

@jeremyprime
Copy link
Collaborator

jeremyprime commented Sep 22, 2022

Is your feature request related to a problem? Please describe.

When running our Docker environment on ARM it is very slow (unusable) for some workflows. Currently our Docker containers are only available and only built for x86_64, which causes significant performance issues when run on an ARM machine. This is likely due to the I/O performance when running sbt commands.

Describe the solution you'd like

Create a new client image based on ARM that includes the following:

  • JDK 11
  • Spark 3.3
  • sbt 1.5.2 (1.5.x should be fine)

Then test that the client image works by building the Spark Connector, running examples, and running the functional tests. Also need to do the same for the Kerberos version of the client and test that.

We should then integrate the changes into our Docker files, giving users an option to deploy the ARM-based client image.

Describe alternatives you've considered

An alternative would be to test and document the process of running the sbt commands (build, examples, functional tests) outside the Docker containers. Although this may be difficult as all of our scripts and config assume running in Docker.

@jeremyprime jeremyprime added enhancement New feature or request High Priority labels Sep 22, 2022
@jeremyprime jeremyprime added this to the 3.3.4 milestone Sep 27, 2022
@jeremyprime jeremyprime changed the title Support ARM machines Create ARM client image Sep 27, 2022
@jeremyprime
Copy link
Collaborator Author

Creating an ARM-based client image only resulted in different errors as the versions of the JDK, Spark, and Hadoop were not aligning.

Until there is better ARM support for Vertica and the other images we use, we can use the x86_64-based images on ARM machines, even if they are slower. See #506 for that future ticket.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request High Priority
Projects
None yet
Development

No branches or pull requests

2 participants