Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Rados reads in CLS #44

Open
jlefevre opened this issue Jun 11, 2020 · 0 comments
Open

Implement Rados reads in CLS #44

jlefevre opened this issue Jun 11, 2020 · 0 comments

Comments

@jlefevre
Copy link
Member

Based on feedback from RedHat, in lieu of a specialized copy_from() (our original approach) instead implement the ability to invoke rados read() from within the CLS context. This is more general, and has some support already within RedHat. This will be a significant coding effort, and it should be driven with good use-cases in mind.

Idea: from within a cls context, read from a remote object - with the ability to invoke a method directly on the remote object (e.g., any registered cls method such as transform data), returning the data output from the method to the caller object.

Considerations: this must be asynchronous (due to PG blocking), and must work for all object types to include replicated objects (more straightforward) as well as erasure coded objects (more complicated).

Motivation
Skyhook (1) : Data originally written to an object in row format in one object is transposed to column format and written to another object. While normally this could be done outside Ceph in a client, here we prefer to avoid sending data back and forth between client/storage and rather do this directly within storage.
Skyhook (2): A step in a scatter-gather of data from multiple src objects into a single target object, all within the storage layer.

General (1): De/Encrypt data from src to target object.
General (2): Transcode data from src to target object, such downscaling resolution for video data.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant