Merge pull request #40 from Cannon07/upload_book_workflow

Upload book workflow

.github/workflows/build_pdf_book.yml

@@ -0,0 +1,42 @@
+name: Build latest version of PDF Book
+
+on:
+  workflow_dispatch:
+  repository_dispatch:
+    types: [rebuild-book]
+
+permissions:
+  contents: write
+
+jobs:
+  build_pdf_book:
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v3
+
+      - name: install requirements
+        run: |
+          cd new-website
+          cd utils
+          pip install -r requirements.txt
+          sudo apt-get install -y poppler-utils
+          sudo apt-get install -y wkhtmltopdf
+
+      - name: fetch latest version of tutorials
+        run: |
+          sudo apt-get install jq
+          cd new-website
+          cd utils/tutorials
+          python3 fetch_tutorials.py
+
+      - name: build pdf book
+        env:
+          AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
+          AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
+        run: |
+          cd new-website
+          cd utils/tutorials
+          python3 build_pdf_book.py
+
+          

.github/workflows/test.yml

@@ -24,6 +24,8 @@ jobs:
           cd new-website
           cd utils
           pip install -r requirements.txt
+          sudo apt-get install -y poppler-utils
+          sudo apt-get install -y wkhtmltopdf
 
       - name: Test tutorial fetching and export
         run: |
@@ -37,4 +39,10 @@ jobs:
           cd utils/tutorials
           python3 test_utils.py
 
+      - name: Test tutorials build pdf book functions
+        run: |
+          cd new-website
+          cd utils/tutorials
+          python3 test_build_pdf_book.py
+
       

new-website/README.md

@@ -93,6 +93,7 @@ A detailed description of the working of the scripts is given below.
   - The script then merges these PDFs and creates the file `merged.pdf`.
   - The `merged.pdf` file is then uploaded to the S3 bucket.
     - Please note, pdfunite package is required to be installed for merging. `apt install poppler-utils`
+
 
 
 ## Deployment
@@ -107,12 +108,22 @@ A detailed description of the working of the scripts is given below.
 
 ## Workflow script
 
-- The `deploy_gh_pages.yml` workflow script in `.github/workflows` is triggered on updates to the main branch.
-- The workflow runs a single job comprising of 3 steps
-  - Fetch version data: This step fetches the latest deepchem release version from the github [api endpoint](https://api.github.com/repos/deepchem/deepchem/releases) and updates the terminal commands in `deepchem/data/home/terminal-commands.json`
-  - Install and build: This step checks out the repository, installs the required dependencies using npm i, runs the linting process with npm run lint, and generates the static website with npm run export.
-  - Deploy: This step deploys the website to the gh-pages branch using the [JamesIves/github-pages-deploy-action](https://github.com/JamesIves/github-pages-deploy-action). The website files are copied from the deepchem/out directory, and any files listed in the clean-exclude parameter are excluded from the cleaning process.
+- ### `deploy_gh_pages.yml`
+
+  - The `deploy_gh_pages.yml` workflow script in `.github/workflows` is triggered on updates to the main branch.
+  - The workflow runs a single job comprising of 3 steps
+    - Fetch version data: This step fetches the latest deepchem release version from the github [api endpoint](https://api.github.com/repos/deepchem/deepchem/releases) and updates the terminal commands in `deepchem/data/home/terminal-commands.json`
+    - Install and build: This step checks out the repository, installs the required dependencies using npm i, runs the linting process with npm run lint, and generates the static website with npm run export.
+    - Deploy: This step deploys the website to the gh-pages branch using the [JamesIves/github-pages-deploy-action](https://github.com/JamesIves/github-pages-deploy-action). The website files are copied from the deepchem/out directory, and any files listed in the clean-exclude parameter are excluded from the cleaning process.
+
+  - ### `build_pdf_book.yml`
+
+  - The `build_pdf_book.yml` workflow script in `.github/workflows` is triggered on updates to the `deepchem/examples/tutorials` directory in `deepchem` repository.
+  - The workflow runs a single job comprising of 3 steps
+    - Install requirements: This step installs the dependencies specified in the `requirements.txt` file in `new-website/utils`. It also installs poppler-utils and wkhtmltopdf packages.
+    - Fetch latest version of tutorials: It installs the jq package and then runs the `fetch_tutorials.py` script.
+    - Build pdf book: This step runs the `build_pdf_book.py` script. 
 
 ## Workflow overview
 
-![](./public/assets/workflow.png)
+![](./public/assets/workflow.png)

new-website/utils/tutorials/build_pdf_book.py

@@ -106,20 +106,38 @@ def upload_file(file_name, bucket, object_name=None):
         True if file was uploaded, else False
 
     """
+    s3_client = boto3.client('s3')
+    try:
+        response = s3_client.head_object(Bucket=bucket, Key=object_name)
+    except ClientError as e:
+        logging.error(e)
+    else:
+        last_modified_datetime = response['LastModified']
+        format = '%Y-%m-%d_%H:%M:%S'
+        formatted_time = last_modified_datetime.strftime(format)
+        old_key = object_name
+        new_key = f'{object_name[:-4]}_{formatted_time}.pdf'
+
+        s3_client.copy_object(
+            Bucket=bucket,
+            CopySource={'Bucket': bucket, 'Key': old_key},
+            Key=new_key
+        )
+
+        s3_client.delete_object(
+            Bucket=bucket,
+            Key=old_key
+        )
 
-    # If S3 object_name was not specified, use file_name
     if object_name is None:
         object_name = os.path.basename(file_name)
-
-    # Upload the file
-    s3_client = boto3.client('s3')
     try:
         response = s3_client.upload_file(file_name, bucket, object_name)
     except ClientError as e:
         logging.error(e)
         return False
     return True 
-
+    
 
 def merge_pdf(info_path=INFO_PATH, pdf_path=PDF_PATH):
     """
@@ -182,6 +200,6 @@ def compile_information_pages():
     html_to_pdf()
     merge_pdf()
     compile_information_pages()
-    merge_pdf_pages(['storage/title.pdf', 'storage/acknowledgement.pdf', 'storage/contents.pdf', 'storage/merged.pdf'])
-    upload_file('storage/full_pdf.pdf', 'deepchemtutorials', 'TutorialsBook.pdf')
+    merge_pdf_pages(['cover.pdf', 'storage/title.pdf', 'storage/acknowledgement.pdf', 'storage/contents.pdf', 'storage/merged.pdf'])
+    upload_file('storage/full_pdf.pdf', 'deepchemdata', 'book/TutorialsBook.pdf')