Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Google Drive integration for backup #134576

Open
wants to merge 16 commits into
base: dev
Choose a base branch
from

Conversation

tronikos
Copy link
Member

@tronikos tronikos commented Jan 3, 2025

Proposed change

Add Google Drive integration. Currently it only provides a backup agent that works with the Home Assistant backup solution introduced in Home Assistant 2025.1.
image

Type of change

  • Dependency upgrade
  • Bugfix (non-breaking change which fixes an issue)
  • New integration (thank you!)
  • New feature (which adds functionality to an existing integration)
  • Deprecation (breaking change to happen in the future)
  • Breaking change (fix/feature causing existing functionality to break)
  • Code quality improvements to existing code or addition of tests

Additional information

Checklist

  • The code change is tested and works locally.
  • Local tests pass. Your PR cannot be merged unless tests pass
  • There is no commented out code in this PR.
  • I have followed the development checklist
  • I have followed the perfect PR recommendations
  • The code has been formatted using Ruff (ruff format homeassistant tests)
  • Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

If the code communicates with devices, web services, or third-party tools:

  • The manifest file has all fields filled out correctly.
    Updated and included derived files by running: python3 -m script.hassfest.
  • New or updated dependencies have been added to requirements_all.txt.
    Updated by running python3 -m script.gen_requirements_all.
  • For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.

To help with the load of incoming pull requests:

@tronikos tronikos marked this pull request as ready for review January 4, 2025 11:25
Copy link
Contributor

@rokam rokam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works as expected and LGTM. Missing test coverage should be a priority on follow-up PR.

@pantherale0
Copy link

LGTM - as this uses the folder ID, I'm assuming there could be a chance that someone might "accidentally" delete the folder? Does this impact integration startup / backup service? Could a backup upload "silently" fail for example? Very minor in the grand scheme of things, but I know Google's storage manager likes to recommend things to delete.

@tronikos
Copy link
Member Author

tronikos commented Jan 5, 2025

LGTM - as this uses the folder ID, I'm assuming there could be a chance that someone might "accidentally" delete the folder? Does this impact integration startup / backup service? Could a backup upload "silently" fail for example? Very minor in the grand scheme of things, but I know Google's storage manager likes to recommend things to delete.

Done. If the folder is deleted the setup fails with:
"Google Drive folder not found. Remove the Google Drive integration and re-add it for a new 'Home Assistant' folder to be created in Google Drive."

@rokam
Copy link
Contributor

rokam commented Jan 5, 2025

LGTM - as this uses the folder ID, I'm assuming there could be a chance that someone might "accidentally" delete the folder? Does this impact integration startup / backup service? Could a backup upload "silently" fail for example? Very minor in the grand scheme of things, but I know Google's storage manager likes to recommend things to delete.

Done. If the folder is deleted the setup fails with: "Google Drive folder not found. Remove the Google Drive integration and re-add it for a new 'Home Assistant' folder to be created in Google Drive."

Later this could be changed to use repair instead of reconfigure flow.

@tronikos
Copy link
Member Author

tronikos commented Jan 6, 2025

LGTM - as this uses the folder ID, I'm assuming there could be a chance that someone might "accidentally" delete the folder? Does this impact integration startup / backup service? Could a backup upload "silently" fail for example? Very minor in the grand scheme of things, but I know Google's storage manager likes to recommend things to delete.

Done. If the folder is deleted the setup fails with: "Google Drive folder not found. Remove the Google Drive integration and re-add it for a new 'Home Assistant' folder to be created in Google Drive."

Later this could be changed to use repair instead of reconfigure flow.

That's a good idea. Thanks! For now I changed the quality scale to repair-issues: todo

@hmmbob
Copy link
Contributor

hmmbob commented Jan 6, 2025

Lovely!

Would it make sense to name the folder Home Assistant Backups instead of just Home Assistant? I feel this better describes the purpose of the folder.

@sergeykad
Copy link

A plugin already uses the "Home Assistant Backups" folder in Google Drive. Will not using the same folder cause issues?

See https://github.com/sabeechen/hassio-google-drive-backup

@rokam
Copy link
Contributor

rokam commented Jan 6, 2025

A plugin already uses the "Home Assistant Backups" folder in Google Drive. Will not using the same folder cause issues?

See https://github.com/sabeechen/hassio-google-drive-backup

I don't think so. Google Drive allows the use of the same name.

@sergeykad
Copy link

@rokam But do you want backups created by two different plugins in the same folder?

Moreover, if the file names will also be identical what will happen if they try reading each others files?

@pantherale0
Copy link

It can be renamed after it's been created to whatever you like

Potentially having 2 folders called "Home Assistant Backup" in the root of your Google Drive could be confusing if they are also using the addon.

That being said, perhaps it could be named "Home Assistant Auto Backups" or similar by default?

@hmmbob
Copy link
Contributor

hmmbob commented Jan 6, 2025

A plugin already uses the "Home Assistant Backups" folder in Google Drive. Will not using the same folder cause issues?

Ah, I was unaware as I don't use add-ons myself (running container). If there are too many potential overlaps/issues, I'd just stick with the current naming.

@rokam
Copy link
Contributor

rokam commented Jan 6, 2025

@rokam But do you want backups created by two different plugins in the same folder?

Moreover, if the file names will also be identical what will happen if they try reading each others files?

This code uses id to reference the folder, not name:
https://github.com/home-assistant/core/blob/eb3a3f169b5ccaf2d19dc0fba9cd6e92f571022f/homeassistant/components/google_drive/config_flow.py#L84C1-L102C1

And also the addon:
https://github.com/sabeechen/hassio-google-drive-backup/blob/3d3f94057abe32fac97b7241ae9d8a4ec90efc01/hassio-google-drive-backup/backup/drive/folderfinder.py#L192C1-L204C32

@hmmbob
Copy link
Contributor

hmmbob commented Jan 6, 2025

@rokam But do you want backups created by two different plugins in the same folder?
Moreover, if the file names will also be identical what will happen if they try reading each others files?

This code uses id to reference the folder, not name: https://github.com/home-assistant/core/blob/eb3a3f169b5ccaf2d19dc0fba9cd6e92f571022f/homeassistant/components/google_drive/config_flow.py#L84C1-L102C1

And also the addon:
https://github.com/sabeechen/hassio-google-drive-backup/blob/3d3f94057abe32fac97b7241ae9d8a4ec90efc01/hassio-google-drive-backup/backup/drive/folderfinder.py#L192C1-L204C32

So that means that Google Drive will see the difference, it just might be confusing for the end user when logging into their own Google Drive and seeing 2 folders with the same name - IF (and only IF) one is using the Google Drive add-on already

@sergeykad
Copy link

Maybe just use "Home Assistant/Backups"?
This will prevent potential confusion and allow additional subfolders to be added in the future.

@home-assistant home-assistant deleted a comment from rokam Jan 6, 2025
@tronikos
Copy link
Member Author

tronikos commented Jan 6, 2025

The name is intentionally just Home Assistant. In the future this integration could expose an action to upload any file to Google Drive, e.g. to upload a camera snapshot or recording, similar to the service in the Google Photos integration.

@tronikos tronikos marked this pull request as draft January 7, 2025 05:57
Comment on lines 129 to 135
resp = await async_get_clientsession(self._hass).get(
DRIVE_API_FILES,
params={
"q": f"'{self._folder_id}' in parents and trashed=false",
"fields": "files(properties)",
},
headers=headers,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

normally REST calls directly from HA (without being wrapped in a library) is not permitted in core

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wrapping the REST calls in a library won't provide any value. In fact in an earlier version I was using aiogoogle and moved away from it in 4d3674c due to:

  1. non OSI license of one of its dependencies
  2. inject-websession and strict-typing (requirements for the platinum quality scale) wouldn't be possible with aiogoogle
  3. aiogoogle was slow because it was making unnecessary REST calls for discovery
  4. aiogoogle's support for streaming downloads wasn't compatible with HA. From my first commit I was bypassing aiogoogle and I was making a regular REST call for downloads

If you notice the diff of that commit, it's not like the code became longer after I stopped using the library. backup.py changed from 272 to 273 lines. So I don't see what would the benefits be.

I could wrap the REST calls in my own library. I've done it for other integrations I own for which this made sense, e.g. Opower, Android TV Remote, Google Assistant SDK. In fact for Google Assistant SDK I previously got feedback that the library doesn't need to exist and its code could just live in Home Assistant since it just wraps a simple grpc.

Wrapping the REST calls here is pointless. I'd just have a few library functions that just have 2 lines each calling the REST endpoints.

@tronikos
Copy link
Member Author

tronikos commented Jan 8, 2025

I tried this in my prod instance yesterday and it didn't work. The addons property was hitting the 124 bytes per property limit imposed by Google Drive. I just changed the code to use a backups.json file in Google Drive that is updated with a lock from this integration when a new backup is uploaded or an existing one is deleted.

@tronikos tronikos marked this pull request as ready for review January 8, 2025 07:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants