Merge pull request #310 from open-craft/agrendalath/maintainership_pilot_2

build: update CI workflows and add pip-tools

Author: Agrendalath

.github/pull_request_template.md

@@ -0,0 +1,44 @@
+<!--
+Please give your pull request a short but descriptive title.
+Use conventional commits to separate and summarize commits logically:
+https://open-edx-proposals.readthedocs.io/en/latest/oep-0051-bp-conventional-commits.html
+
+Use this template as a guide. Omit sections that don't apply.
+You may link to information rather than copy it, but only if the link is publicly
+readable. If your linked information must be private (because it has secrets),
+clearly label the link as private.
+-->
+
+## Description
+
+Describe what this pull request changes, and why. Include implications for people using this change.
+Design decisions and their rationales should be documented in the repo (docstring / ADR), per
+[OEP-19](https://open-edx-proposals.readthedocs.io/en/latest/oep-0019-bp-developer-documentation.html), and can be
+linked here.
+
+Useful information to include:
+- Which edX user roles will this change impact? Common user roles are "Learner", "Course Author",
+"Developer", and "Operator".
+- Include screenshots for changes to the UI (ideally, both "before" and "after" screenshots, if applicable).
+- Provide links to the description of corresponding configuration changes. Remember to correctly annotate these
+changes.
+
+## Supporting information
+
+Link to other information about the change, such as Jira issues, GitHub issues, or Discourse discussions.
+Be sure to check they are publicly readable, or if not, repeat the information here.
+
+## Testing instructions
+
+Please provide detailed step-by-step instructions for testing this change.
+
+## Deadline
+
+"None" if there's no rush, or provide a specific date or event (and reason) if there is one.
+
+## Other information
+
+Include anything else that will help reviewers and consumers understand the change.
+- Does this change depend on other changes elsewhere?
+- Any special concerns or limitations? For example: deprecations, migrations, security, or accessibility.
+- If your [database migration](https://openedx.atlassian.net/wiki/spaces/AC/pages/23003228/Everything+About+Database+Migrations) can't be rolled back easily.

.github/workflows/ci.yml

@@ -6,14 +6,29 @@ on:
   pull_request:
     branches:
       - '**'
+  workflow_dispatch:
+
+concurrency:
+  group: "${{ github.workflow }}-${{ github.ref }}"
+  cancel-in-progress: true
 
 jobs:
   tests:
     runs-on: ${{ matrix.os }}
+    services:
+      # Using SQLite3 for integration tests throws `django.db.utils.OperationalError: database table is locked: workbench_xblockstate`.
+      mysql:
+        image: mysql:8
+        env:
+          MYSQL_ROOT_PASSWORD: rootpw
+        ports:
+          - 3307:3306
     strategy:
+      fail-fast: false
       matrix:
         os: [ubuntu-20.04]
         python-version: [3.8]
+        toxenv: [py38, integration, quality, translations]
 
     steps:
     - name: checkout repo
@@ -28,18 +43,21 @@ jobs:
 
     # `libgtk2.0-0` and `libxt6` are required by an older version of Firefox.
     - name: Install Required System Packages
+      if: matrix.toxenv == 'integration'
+      run: |
+        sudo apt-get update
+        sudo apt-get install -y libxmlsec1-dev ubuntu-restricted-extras xvfb libxml2-dev libxslt-dev libevent-dev libgtk2.0-0 libxt6
+
+    - name: Install translations dependencies
+      if: matrix.toxenv == 'translations'
       run: |
         sudo apt-get update
-        sudo apt-get install -y libxmlsec1-dev ubuntu-restricted-extras xvfb libxml2-dev libxslt-dev python3-dev libevent-dev python3-pip python3-venv gettext libgtk2.0-0 libxt6
+        sudo apt-get install -y gettext
 
     - name: Install Dependencies
       run: make requirements
 
-    - name: Check if translations are up-to-date
-      run: make check_translations_up_to_date
-
-    - name: Run quality checks
-      run: make test.quality
-
-    - name: Run unit tests
-      run: make test.unit
+    - name: Run Tests
+      env:
+        TOXENV: ${{ matrix.toxenv }}
+      run: tox

.github/workflows/pypi-publish.yml

@@ -1,33 +1,30 @@
-name: pypi-publish
+name: Publish package to PyPI
 
 on:
   release:
     types: [published]
 
 jobs:
-  publish-package:
+  push:
     runs-on: ubuntu-20.04
 
     steps:
       - name: Checkout
         uses: actions/checkout@v3
 
       - name: setup python
-        uses: actions/setup-python@v3
+        uses: actions/setup-python@v4
         with:
           python-version: 3.8
 
-      - name: Install pip
-        run: pip install pip
-
       - name: Install Dependencies
-        run: pip install setuptools wheel
+        run: pip install -r requirements/pip.txt
 
       - name: Build package
         run: python setup.py sdist bdist_wheel
 
       - name: Publish to PyPi
-        uses: pypa/gh-action-pypi-publish@master
+        uses: pypa/gh-action-pypi-publish@release/v1
         with:
           user: __token__
           password: ${{ secrets.PYPI_UPLOAD_TOKEN }}

.github/workflows/upgrade-python-requirements.yml

@@ -6,18 +6,17 @@ on:
   workflow_dispatch:
     inputs:
       branch:
-        description: "Target branch against which to create requirements PR"
+        description: Target branch against which to create requirements PR
         required: true
-        # If copying this template manually, you must provide your default branch name
-        # in quotes, such as 'master'
-        default: "master"
+        default: master
 
 jobs:
   call-upgrade-python-requirements-workflow:
     uses: openedx/.github/.github/workflows/upgrade-python-requirements.yml@master
+    # Do not run on forks
+    if: github.repository_owner == 'openedx'
     with:
-    # If copying manually, also provide your default branch name in quotes here
-      branch: ${{ github.event.inputs.branch || "master" }}
+      branch: ${{ github.event.inputs.branch || 'master' }}
       # optional parameters below; fill in if you'd like github or email notifications
       # user_reviewers: ""
       # team_reviewers: ""

.gitmodules

@@ -0,0 +1,5 @@
+include Changelog.md
+include LICENSE
+include README.md
+include requirements/base.in
+include requirements/constraints.txt

MANIFEST.in

@@ -1,5 +1,6 @@
-.PHONY: clean compile_translations dummy_translations \
-	extract_translations fake_translations help
+.PHONY: clean help compile_translations dummy_translations extract_translations detect_changed_source_translations \
+		build_dummy_translations validate_translations pull_translations push_translations check_translations_up_to_date \
+		install_firefox requirements selfcheck test test.python test.unit test.quality upgrade mysql
 
 .DEFAULT_GOAL := help
 
@@ -10,6 +11,8 @@ EXTRACTED_DJANGO := $(EXTRACT_DIR)/django-partial.po
 EXTRACTED_DJANGOJS := $(EXTRACT_DIR)/djangojs-partial.po
 EXTRACTED_TEXT := $(EXTRACT_DIR)/text.po
 
+FIREFOX_VERSION := "43.0"
+
 help: ## display this help message
 	@echo "Please use \`make <target>' where <target> is one of"
 	@perl -nle'print $& if m{^[a-zA-Z_-]+:.*?## .*$$}' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m  %-25s\033[0m %s\n", $$1, $$2}'
@@ -56,21 +59,57 @@ push_translations: ## push translations to transifex
 check_translations_up_to_date: extract_translations compile_translations dummy_translations detect_changed_source_translations ## extract, compile, and check if translation files are up-to-date
 
 install_firefox:
-	mkdir -p test_helpers
-	cd test_helpers && wget "https://ftp.mozilla.org/pub/firefox/releases/43.0/linux-x86_64/en-US/firefox-43.0.tar.bz2" && tar -xjf firefox-43.0.tar.bz2
+	@mkdir -p test_helpers
+	@test -f ./test_helpers/firefox/firefox && echo "Firefox already installed." || \
+	(cd test_helpers && \
+	wget -N "https://archive.mozilla.org/pub/firefox/releases/$(FIREFOX_VERSION)/linux-x86_64/en-US/firefox-$(FIREFOX_VERSION).tar.bz2" && \
+	tar -xjf firefox-$(FIREFOX_VERSION).tar.bz2)
+
+piptools: ## install pinned version of pip-compile and pip-sync
+	pip install -r requirements/pip.txt
+	pip install -r requirements/pip-tools.txt
+
+requirements: piptools  ## install test requirements locally
+	pip-sync requirements/ci.txt
 
-requirements: install_firefox
-	pip install wheel
-	pip install -r xblock-sdk/requirements/base.txt -r xblock-sdk/requirements/test.txt
-	pip install -r requirements.txt
-	pip uninstall -y transifex-client
+requirements_python: install_firefox piptools  ## install all requirements locally
+	pip-sync requirements/dev.txt requirements/private.*
 
-test.quality: ## run quality checkers on the codebase
-	pycodestyle drag_and_drop_v2 tests --max-line-length=120
-	pylint drag_and_drop_v2
-	pylint tests --rcfile=tests/pylintrc
+test.quality: selfcheck ## run quality checkers on the codebase
+	tox -e quality
 
-test.unit: ## run python unit and integration tests
+test.python: ## run python unit and integration tests
 	PATH=test_helpers/firefox:$$PATH xvfb-run python run_tests.py $(TEST)
 
-test: test.quality test.unit ## Run all tests
+test.unit: ## run all unit tests
+	tox -- $(TEST)
+
+test.integration: ## run all integration tests
+	tox -e integration -- $(TEST)
+
+test: test.unit test.integration test.quality ## Run all tests
+	tox -e translations
+
+# Define PIP_COMPILE_OPTS=-v to get more information during make upgrade.
+PIP_COMPILE = pip-compile --upgrade --resolver=backtracking $(PIP_COMPILE_OPTS)
+
+upgrade: export CUSTOM_COMPILE_COMMAND=make upgrade
+upgrade: ## update the requirements/*.txt files with the latest packages satisfying requirements/*.in
+	pip install -qr requirements/pip-tools.txt
+	# Make sure to compile files after any other files they include!
+	$(PIP_COMPILE) --allow-unsafe -o requirements/pip.txt requirements/pip.in
+	$(PIP_COMPILE) -o requirements/pip-tools.txt requirements/pip-tools.in
+	pip install -qr requirements/pip.txt
+	pip install -qr requirements/pip-tools.txt
+	$(PIP_COMPILE) -o requirements/base.txt requirements/base.in
+	$(PIP_COMPILE) -o requirements/test.txt requirements/test.in
+	$(PIP_COMPILE) -o requirements/quality.txt requirements/quality.in
+	$(PIP_COMPILE) -o requirements/workbench.txt requirements/workbench.in
+	$(PIP_COMPILE) -o requirements/ci.txt requirements/ci.in
+	$(PIP_COMPILE) -o requirements/dev.txt requirements/dev.in
+
+mysql: ## run mysql database for integration tests
+	docker run --rm -it --name mysql -p 3307:3306 -e MYSQL_ROOT_PASSWORD=rootpw -e MYSQL_DATABASE=db mysql:8
+
+selfcheck: ## check that the Makefile is well-formed
+	@echo "The Makefile is well-formed."

Makefile

@@ -455,39 +455,56 @@ Real event example (taken from a devstack):
 ```
 
 
-Testing
--------
+Testing with tox
+----------------
 
 Inside a fresh virtualenv, `cd` into the root folder of this repository
-(`xblock-drag-and-drop-v2`) and run
+(`xblock-drag-and-drop-v2`) and run:
 
 ```bash
-$ git submodule update --init --recursive
 $ make requirements
 ```
 
-You can then run the entire test suite via
+To run integration tests, you need to start MySQL first:
+```bash
+$ make mysql
+```
+
+You can then run the entire test suite via:
 
 ```bash
 $ make test
 ```
 
-To only run the quality checks, do
+To run specific test groups, use one of the following commands:
 
 ```bash
+$ make test.unit
+$ make test.integration
 $ make test.quality
+$ make test.translations
 ```
 
-Similarly, you can run unit and integration test suite via
+To run individual unit tests, use:
 
 ```bash
-$ make test.unit
+$ make test.unit TEST=tests/unit/test_basics.py::BasicTests::test_student_view_data
 ```
+To run individual integration tests, use:
+
+```bash
+$ make test.integration TEST=tests.integration.test_studio.TestStudio.test_custom_image
+```
+
+Manual testing (without tox)
+----------------------------
 
-You can run specific tests via
+To run tests without tox, use:
 
 ```bash
-$ make test.unit TEST=tests.unit.test_basics.BasicTests.test_student_view_data
+$ make mysql
+$ make requirements_python
+$ make test.python TEST=tests.unit.test_basics.BasicTests.test_student_view_data
 ```
 
 

README.md

@@ -1,2 +1,4 @@
 """ Drag and Drop v2 XBlock """
 from .drag_and_drop_v2 import DragAndDropBlock
+
+__version__ = "3.1.1"

drag_and_drop_v2/__init__.py

@@ -654,7 +654,7 @@ def show_answer(self, data, suffix=''):
 
         answer = self._get_correct_state()
 
-        if explanation := self.data.get('explanation', '').strip():
+        if explanation := (self.data.get('explanation') or '').strip():
             if replace_urls_service := self.runtime.service(self, 'replace_urls'):
                 explanation = replace_urls_service.replace_urls(explanation)