Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Google Drive integration for backup #134576

Open
wants to merge 23 commits into
base: dev
Choose a base branch
from

Conversation

tronikos
Copy link
Member

@tronikos tronikos commented Jan 3, 2025

Proposed change

Add Google Drive integration. Currently it only provides a backup agent that works with the Home Assistant backup solution introduced in Home Assistant 2025.1.
image

Type of change

  • Dependency upgrade
  • Bugfix (non-breaking change which fixes an issue)
  • New integration (thank you!)
  • New feature (which adds functionality to an existing integration)
  • Deprecation (breaking change to happen in the future)
  • Breaking change (fix/feature causing existing functionality to break)
  • Code quality improvements to existing code or addition of tests

Additional information

Checklist

  • The code change is tested and works locally.
  • Local tests pass. Your PR cannot be merged unless tests pass
  • There is no commented out code in this PR.
  • I have followed the development checklist
  • I have followed the perfect PR recommendations
  • The code has been formatted using Ruff (ruff format homeassistant tests)
  • Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

If the code communicates with devices, web services, or third-party tools:

  • The manifest file has all fields filled out correctly.
    Updated and included derived files by running: python3 -m script.hassfest.
  • New or updated dependencies have been added to requirements_all.txt.
    Updated by running python3 -m script.gen_requirements_all.
  • For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.

To help with the load of incoming pull requests:

@tronikos tronikos marked this pull request as ready for review January 4, 2025 11:25
Copy link
Contributor

@rokam rokam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works as expected and LGTM. Missing test coverage should be a priority on follow-up PR.

@pantherale0
Copy link

LGTM - as this uses the folder ID, I'm assuming there could be a chance that someone might "accidentally" delete the folder? Does this impact integration startup / backup service? Could a backup upload "silently" fail for example? Very minor in the grand scheme of things, but I know Google's storage manager likes to recommend things to delete.

@tronikos
Copy link
Member Author

tronikos commented Jan 5, 2025

LGTM - as this uses the folder ID, I'm assuming there could be a chance that someone might "accidentally" delete the folder? Does this impact integration startup / backup service? Could a backup upload "silently" fail for example? Very minor in the grand scheme of things, but I know Google's storage manager likes to recommend things to delete.

Done. If the folder is deleted the setup fails with:
"Google Drive folder not found. Remove the Google Drive integration and re-add it for a new 'Home Assistant' folder to be created in Google Drive."

@rokam
Copy link
Contributor

rokam commented Jan 5, 2025

LGTM - as this uses the folder ID, I'm assuming there could be a chance that someone might "accidentally" delete the folder? Does this impact integration startup / backup service? Could a backup upload "silently" fail for example? Very minor in the grand scheme of things, but I know Google's storage manager likes to recommend things to delete.

Done. If the folder is deleted the setup fails with: "Google Drive folder not found. Remove the Google Drive integration and re-add it for a new 'Home Assistant' folder to be created in Google Drive."

Later this could be changed to use repair instead of reconfigure flow.

@hmmbob
Copy link
Contributor

hmmbob commented Jan 6, 2025

@rokam But do you want backups created by two different plugins in the same folder?
Moreover, if the file names will also be identical what will happen if they try reading each others files?

This code uses id to reference the folder, not name: https://github.com/home-assistant/core/blob/eb3a3f169b5ccaf2d19dc0fba9cd6e92f571022f/homeassistant/components/google_drive/config_flow.py#L84C1-L102C1

And also the addon:
https://github.com/sabeechen/hassio-google-drive-backup/blob/3d3f94057abe32fac97b7241ae9d8a4ec90efc01/hassio-google-drive-backup/backup/drive/folderfinder.py#L192C1-L204C32

So that means that Google Drive will see the difference, it just might be confusing for the end user when logging into their own Google Drive and seeing 2 folders with the same name - IF (and only IF) one is using the Google Drive add-on already

@sergeykad
Copy link

Maybe just use "Home Assistant/Backups"?
This will prevent potential confusion and allow additional subfolders to be added in the future.

@home-assistant home-assistant deleted a comment from rokam Jan 6, 2025
@tronikos
Copy link
Member Author

tronikos commented Jan 6, 2025

The name is intentionally just Home Assistant. In the future this integration could expose an action to upload any file to Google Drive, e.g. to upload a camera snapshot or recording, similar to the service in the Google Photos integration.

@tronikos tronikos marked this pull request as draft January 7, 2025 05:57
@tronikos
Copy link
Member Author

tronikos commented Jan 8, 2025

I tried this in my prod instance yesterday and it didn't work. The addons property was hitting the 124 bytes per property limit imposed by Google Drive. I just changed the code to use a backups.json file in Google Drive that is updated with a lock from this integration when a new backup is uploaded or an existing one is deleted.

@tronikos tronikos marked this pull request as ready for review January 8, 2025 07:24
homeassistant/components/google_drive/config_flow.py Outdated Show resolved Hide resolved
homeassistant/components/google_drive/strings.json Outdated Show resolved Hide resolved
homeassistant/components/google_drive/strings.json Outdated Show resolved Hide resolved
tests/components/google_drive/test_config_flow.py Outdated Show resolved Hide resolved
tests/components/google_drive/test_config_flow.py Outdated Show resolved Hide resolved
tests/components/google_drive/test_init.py Outdated Show resolved Hide resolved
@home-assistant home-assistant bot marked this pull request as draft January 8, 2025 20:57
@home-assistant
Copy link

home-assistant bot commented Jan 8, 2025

Please take a look at the requested changes, and use the Ready for review button when you are done, thanks 👍

Learn more about our pull request process.

@tronikos tronikos marked this pull request as ready for review January 8, 2025 23:51
@home-assistant home-assistant bot requested a review from joostlek January 8, 2025 23:51
@joostlek
Copy link
Member

Okay So I discussed this in the core team yesterday, and we stand by the rule that communication specific details are stored in a library hosted on Pypi. Even for simple integrations we require a library. There is also the possibility to add more functionality to the integration in the future, like storage percentage. Hence the integration being a backup agent in its current form doesn't really form an exception.

And as always, if you need help with this, feel free to reach out. I would love to see this PR succeed!

@tronikos
Copy link
Member Author

Okay So I discussed this in the core team yesterday, and we stand by the rule that communication specific details are stored in a library hosted on Pypi. Even for simple integrations we require a library. There is also the possibility to add more functionality to the integration in the future, like storage percentage. Hence the integration being a backup agent in its current form doesn't really form an exception.

It should be made a bronze requirement then.

Do you recommend putting the whole DriveClient class in api.py in a library? If yes, that would make it very specific to HA and not comply with the recommendations in https://developers.home-assistant.io/docs/api_lib_data_models/ specifically: "API libraries should try to do as little as possible."

The alternative is to add simple wrappers that just make the REST calls. That would be similar to if I was using google-api-python-client (e.g. used by Google Mail) or aiogoogle (used in an earlier commit here). E.g.

    async def async_drive_about(self, access_token: str, fields: str) -> dict[str, Any]:
        resp = await self._session.get(
            "https://www.googleapis.com/drive/v3/about",
            params={"fields": fields},
            headers={"Authorization": f"Bearer {access_token}"},
            raise_for_status=True,
        )
        return await resp.json()

@joostlek
Copy link
Member

In theory everything that is specific for the service. The way you implement that is your choice.

If I imagine a library for this, it would be abstract. I want to put in an access token (or put in a function which can get me an up to date token) and I want to have methods like

async def create_folder(self, name: str, properties: dict[str, str]) -> None:

And that is a bit of beauty of this abstraction is that I don't really care how it creates that folder, as long as it does it.

This does still leave the logic on how the integration interacts with Google Drive in the integration, but it puts the code on how to do that technically away.

I personally really like a typed library, so I would probably use mashumaro to create dataclasses for the return types.

Copy link
Member

@zweckj zweckj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, the issue when the folder is missing (after config flow) was pretty nice imo. If that folder went missing that definitely warrants an issue the user needs to take a look at (we can still fix it automatically in a RepairFlow later)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants