Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Version Control System (VCS) #4

Closed
hsk81 opened this issue Jun 19, 2013 · 3 comments
Closed

Version Control System (VCS) #4

hsk81 opened this issue Jun 19, 2013 · 3 comments
Assignees
Labels

Comments

@hsk81
Copy link
Owner

hsk81 commented Jun 19, 2013

It should be possible to version the project/documents (using either GIT or - more likely - something much simpler).

@hsk81
Copy link
Owner Author

hsk81 commented Jul 15, 2013

The AcidFS library (available at http://acidfs.readthedocs.org/en/latest) seems very promising: Basically, it's a Python integration of GIT repositories. It could be used to replace NoTex.ch's current storage back-end,which is a combination of metadata being stored in an SQL database, and data as a flat list on the hard disk, with a GIT repository for data with search indices/metainfo in an SQL database.

@hsk81
Copy link
Owner Author

hsk81 commented Jul 15, 2013

The basic problem is that a VCS (like GIT) collides with client side encryption (see #9); figure out a reasonable solution that ensures that the GIT repositories are encrypted without compromising basic VCS features:

  • It could simply mean that the GIT repository is encrypted with a shared key between user and server: But this would also imply that we have to give up on client side encryption;
  • Another possibility could be to use homomorphic encryption: Although that field seems to be still in research, my impression is that it might already be practically usable. If there is already a GIT library with integrated HE the better, but I fear such a thing is still not available.
  • Re-implement GIT with HE, and replace current storage engine with it: But how do we create search indices then? This would imply the implementation of a DB engine with HE (which might be a little challenging though).
  • A compromise might be the usage of plain text metadata, and encrypted data: This would allow us to create search indices with a regular SQL DB, while giving us a smaller challenge of implementing a VCS with HE support.

hsk81 pushed a commit that referenced this issue Aug 16, 2013
…on AcidFS (GIT); have not completely removed the direct filesystem based approach yet though.
hsk81 pushed a commit that referenced this issue Aug 16, 2013
hsk81 pushed a commit that referenced this issue Aug 16, 2013
hsk81 pushed a commit that referenced this issue Aug 16, 2013
hsk81 pushed a commit that referenced this issue Aug 16, 2013
hsk81 pushed a commit that referenced this issue Aug 16, 2013
The *underlying* problem was the fact the path renaming was not correctly implemented.
hsk81 pushed a commit that referenced this issue Aug 16, 2013
The `version-key` for `value-key` used to play the role of a *reference counter* for a particular piece of `value` (data); this is not required anymore due to the storage backend switch to AcidFS. But we *still* need to keep reference counting for the data `size` since that is still tracked directly in the database.
hsk81 pushed a commit that referenced this issue Aug 16, 2013
hsk81 pushed a commit that referenced this issue Aug 16, 2013
hsk81 pushed a commit that referenced this issue Aug 16, 2013
hsk81 pushed a commit that referenced this issue Aug 16, 2013
hsk81 pushed a commit that referenced this issue Aug 17, 2013
In theory DB commits and AcidFS commits should be in sync: Based on this assumption shifting `transaction.commit` from `property.py` to `ext/db.py` should work. Actually it *does* work, but somehow AcidFS only commits the most recent `write` .. seems like a bug of AcidFS to me, but due to the complexity of this "test" case it could also be a flawed conclusion on my part.
hsk81 pushed a commit that referenced this issue Aug 17, 2013
Renaming a path should properly be reflected in the meta information (especially when a directory is involved)
hsk81 pushed a commit that referenced this issue Aug 17, 2013
Using `master` for AcidFS to have access to most recent bug fixed.
@ghost ghost assigned hsk81 Aug 18, 2013
hsk81 pushed a commit that referenced this issue Aug 19, 2013
Have added `description` and generic `FS_ACID_REPO_CFG` possibility.
hsk81 pushed a commit that referenced this issue Aug 19, 2013
Ensured that `post-update` hook gets executed after a commit; otherwise `git` is not able to clone the repository over `http`.
hsk81 pushed a commit that referenced this issue Aug 19, 2013
hsk81 pushed a commit that referenced this issue Aug 19, 2013
hsk81 pushed a commit that referenced this issue Aug 19, 2013
There is a *reason* why the `transaction` package is decoupled from `acidfs`: Therefore making coupling **explicit** by using `VcsTransactionMixin` with only one `transact` method does not seem exaggerated, and seems to make sense.
hsk81 pushed a commit that referenced this issue Aug 20, 2013
Ensured that GIT uses correct *author* and *committer* from the very beginning.
hsk81 pushed a commit that referenced this issue Aug 20, 2013
hsk81 pushed a commit that referenced this issue Aug 20, 2013
hsk81 pushed a commit that referenced this issue Aug 21, 2013
hsk81 pushed a commit that referenced this issue Aug 21, 2013
@hsk81
Copy link
Owner Author

hsk81 commented Aug 21, 2013

Have reset application to have a clean state between DB and acid sub-directory. VCS read-only implementation done; read-write might be implemented only upon popular request; client side encryption will be reviewed as soon as git itself supports it.

@hsk81 hsk81 closed this as completed Aug 21, 2013
hsk81 pushed a commit that referenced this issue Aug 22, 2013
hsk81 pushed a commit that referenced this issue Aug 23, 2013
hsk81 pushed a commit that referenced this issue Aug 23, 2013
hsk81 pushed a commit that referenced this issue Aug 23, 2013
hsk81 pushed a commit that referenced this issue Aug 23, 2013
hsk81 pushed a commit that referenced this issue Aug 23, 2013
hsk81 pushed a commit that referenced this issue Aug 23, 2013
hsk81 pushed a commit that referenced this issue Aug 23, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant