Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add tachyon back as dependency #28

Open
dcrankshaw opened this issue Mar 24, 2015 · 1 comment
Open

Add tachyon back as dependency #28

dcrankshaw opened this issue Mar 24, 2015 · 1 comment
Assignees
Labels

Comments

@dcrankshaw
Copy link
Contributor

PR #27 removed tachyon as a dependency to get binary compatibility between hadoop-client and spark-core. Even though we can't use the Tachyon KV store right now, we can still use Tachyon as a faster version of HDFS to transfer training data and models between Spark and Velox. We should add tachyon back as a storage option once we can figure out class compatibility between Spark, Hadoop, and Tachyon.

@dcrankshaw dcrankshaw self-assigned this Mar 24, 2015
@tomerk
Copy link
Contributor

tomerk commented Apr 14, 2015

As of pull request #44 this may be as simple as changing the spark data location config to be in the tachyon filesystem

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants