Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automated load and data wrangling #24

Open
ThomasHSimm opened this issue Jan 9, 2023 · 0 comments
Open

Automated load and data wrangling #24

ThomasHSimm opened this issue Jan 9, 2023 · 0 comments

Comments

@ThomasHSimm
Copy link
Owner

This should be possible by github actions. I had a look into it but was a bit beyond me at the moment.

The idea is github action would run every month:

  • imports all the data from the ONS website using the beautiful soup code on the initial commit
  • saves it on the virtual machine
  • commits it to the github repo
  • runs the data loads and cleaning
  • then commits these to a csv as a dataframe

As a start I'll do a branch that does all the above (should be just the bs part now as rest is dealt with in another commit?) and then manually run it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant