Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restoring files with Git LFS #200

Merged
merged 5 commits into from
Nov 25, 2024
Merged

Restoring files with Git LFS #200

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
895b5b2
Adding git lfs as dependency and making git lfs pull when make local
rafaspadilha Oct 21, 2024
f8305af
Updating makefile, setup script and quickstart doc
rafaspadilha Oct 21, 2024
bc91525
Removing comment
rafaspadilha Oct 21, 2024
d7bb670
Update Makefile
rafaspadilha Oct 21, 2024
90d1992
Addressing PR comments
rafaspadilha Oct 22, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,12 +43,15 @@ help: ## Shows this help message
@echo -e This is the farmvibes.ai makefile. Supported targets are:\\n
@grep -E -h '\s##\s' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-20s\033[0m %s\n", $$1, $$2}'

local: cluster local-rest-api local-cache local-worker local-orchestrator local-data-ops ## Builds all images locally and deploys them into the local farmvibes.ai cluster
local: cluster restore-git-lfs local-rest-api local-cache local-worker local-orchestrator local-data-ops ## Builds all images locally and deploys them into the local farmvibes.ai cluster
[ -z $(WAIT_AT_THE_END) ] || kubectl delete pods -l backend=terravibes && \
kubectl wait --for=condition=Available deployment --timeout=300s -l backend=terravibes

revert: cluster revert-rest-api revert-cache revert-worker revert-orchestrator ## Reverts all images to the official version

restore-git-lfs:
git lfs pull || echo "git lfs was not found. Please see https://git-lfs.com/ to install it." && exit 1

services-base: resources/docker/Dockerfile-services-base
@docker manifest inspect `$(subst FILE,$<,$(base_image_name))` || \
az acr login -n `$(subst FILE,$<,$(base_image_name)) | cut -d / -f 1 | sed 's|.azurecr.io||g'` || \
Expand Down Expand Up @@ -122,7 +125,7 @@ revert-data-ops: cluster repo-$(DATA_OPS_REPO) delete-$(DATA_OPS_DEPLOYMENT) ##
DEPLOYMENT=$(DATA_OPS_DEPLOYMENT) IMAGE_FULL_REFERENCE=$(CONTAINER_REGISTRY_BASE)/$(DATA_OPS_REPO):$(FARMVIBES_AI_IMAGE_TAG) $(MAKE) set-registry-image
DEPLOYMENT=$(DATA_OPS_DEPLOYMENT) REPLICAS=$(CURRENT_DATA_OPS_REPLICAS) make scale

local-worker: cluster local-worker-repo delete-$(WORKER_DEPLOYMENT) ## Builds and deploys a local WORKER image (enabling debug)
local-worker: cluster restore-git-lfs local-worker-repo delete-$(WORKER_DEPLOYMENT) ## Builds and deploys a local WORKER image (enabling debug)
DEPLOYMENT=$(WORKER_DEPLOYMENT) IMAGE_FULL_REFERENCE=$(WORKER_REPO):$(TAG) $(MAKE) -C . set-registry-image
DEPLOYMENT=$(WORKER_DEPLOYMENT) $(MAKE) -C . disable-frozen-modules
DEPLOYMENT=$(WORKER_DEPLOYMENT) REPLICAS=$(CURRENT_WORKER_REPLICAS) make scale
Expand Down
17 changes: 17 additions & 0 deletions docs/source/docfiles/markdown/QUICKSTART.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,9 @@ In order to run FarmVibes.AI cluster, you need the following:
the repository. If you already have access to the source code, then Git is
not required.

* [Git LFS](https://git-lfs.com/) to restore some of the large files in the
repository (e.g., model weights).

* [Docker](https://docs.docker.com/engine/install/ubuntu/). Make sure you can
run the docker client without running `sudo` by adding your user account to
the `docker` group (which might require a logout/login when adding oneself
Expand Down Expand Up @@ -60,6 +63,20 @@ bash ./resources/vm/setup_farmvibes_ai_vm.sh

You might needed to restart your shell session once the script finishes.

## Restore files with Git LFS

In case you did not have Git LFS installed when cloning the repository, you will need to do so
to restore the large files in the repository. Note that the last step
["Installing software dependencies](#optional-installing-software-dependencies) already installs
Git LFS.

To restore the missing files, you can run the following command in the root of the repository:

```shell
git lfs install
git lfs pull
```

rafaspadilha marked this conversation as resolved.
Show resolved Hide resolved
## Install the FarmVibes.AI cluster

With python3.8+ and pip installed on your machine, please install
Expand Down
9 changes: 9 additions & 0 deletions docs/source/docfiles/markdown/TROUBLESHOOTING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,15 @@ that are currently being addressed by the development team.

</details>

<details>
<summary> Updating cluster in the `dev` branch after pulling files with Git LFS </summary>

If you did not have Git LFS installed when cloning the repository and checking out to `dev`,
you will be missing some of the large files in the repository (e.g., ONNX models). Make sure
to install and setup Git LFS as described in the [Quickstart guide](QUICKSTART.md#restore-files-with-git-lfs).
You will also need to update your cluster with `make local`.
</details>

<br>

- **Composing and running workflows:**
Expand Down
5 changes: 5 additions & 0 deletions resources/vm/setup_farmvibes_ai_vm.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,3 +48,8 @@ fi

# Run docker without sudo
sudo usermod -aG docker $DOCKER_USER

# Run git-lfs install to restore large files
sudo apt install git-lfs -y
git lfs install
git lfs pull
Loading