Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix mysqldump error #92

Draft
wants to merge 9 commits into
base: main
Choose a base branch
from
Draft

fix mysqldump error #92

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
a607edd
feat: 🎸 remove workaround
kibitan Sep 24, 2023
510e820
Merge branch 'main' into fix_mysqldump_error
kibitan Sep 24, 2023
1d60f12
refactor: 💡 output dump to file for debug
kibitan Nov 19, 2023
dd7d1c7
refactor: 💡 add DUMPFILE_DEBUG option
kibitan Nov 19, 2023
1552734
ci: 🎡 add enviorment variable
kibitan Nov 19, 2023
31ca1e6
Merge branch 'main' into fix_mysqldump_error
kibitan Nov 19, 2023
1e7165e
docs: ✏️ add debug option explaination
kibitan Nov 19, 2023
daca67c
refactor: 💡 add debug version option
kibitan Nov 19, 2023
211dd3d
ping ci
kibitan Nov 19, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 12 additions & 3 deletions .github/workflows/acceptance_test_mariadb.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,6 @@ on:
- main
pull_request:

env:
DEBUG: ${{ vars.DEBUG }}

jobs:
mariadb111:
runs-on: ubuntu-latest
Expand All @@ -19,6 +16,7 @@ jobs:
run: docker-compose/acceptance_test.sh mariadb111 docker-compose-ci.yml
env:
TRACE: ${{ vars.TRACE }}
DUMPFILE_DEBUG: ${{ vars.DUMPFILE_DEBUG }}
mariadb110:
runs-on: ubuntu-latest
steps:
Expand All @@ -29,6 +27,7 @@ jobs:
run: docker-compose/acceptance_test.sh mariadb110 docker-compose-ci.yml
env:
TRACE: ${{ vars.TRACE }}
DUMPFILE_DEBUG: ${{ vars.DUMPFILE_DEBUG }}
mariadb1011:
runs-on: ubuntu-latest
steps:
Expand All @@ -39,6 +38,7 @@ jobs:
run: docker-compose/acceptance_test.sh mariadb1011 docker-compose-ci.yml
env:
TRACE: ${{ vars.TRACE }}
DUMPFILE_DEBUG: ${{ vars.DUMPFILE_DEBUG }}
mariadb1010:
runs-on: ubuntu-latest
steps:
Expand All @@ -49,6 +49,7 @@ jobs:
run: docker-compose/acceptance_test.sh mariadb1010 docker-compose-ci.yml
env:
TRACE: ${{ vars.TRACE }}
DUMPFILE_DEBUG: ${{ vars.DUMPFILE_DEBUG }}
mariadb109:
runs-on: ubuntu-latest
steps:
Expand All @@ -59,6 +60,7 @@ jobs:
run: docker-compose/acceptance_test.sh mariadb109 docker-compose-ci.yml
env:
TRACE: ${{ vars.TRACE }}
DUMPFILE_DEBUG: ${{ vars.DUMPFILE_DEBUG }}
mariadb108:
runs-on: ubuntu-latest
steps:
Expand All @@ -69,6 +71,7 @@ jobs:
run: docker-compose/acceptance_test.sh mariadb108 docker-compose-ci.yml
env:
TRACE: ${{ vars.TRACE }}
DUMPFILE_DEBUG: ${{ vars.DUMPFILE_DEBUG }}
mariadb107:
runs-on: ubuntu-latest
steps:
Expand All @@ -79,6 +82,7 @@ jobs:
run: docker-compose/acceptance_test.sh mariadb107 docker-compose-ci.yml
env:
TRACE: ${{ vars.TRACE }}
DUMPFILE_DEBUG: ${{ vars.DUMPFILE_DEBUG }}
mariadb106:
runs-on: ubuntu-latest
steps:
Expand All @@ -89,6 +93,7 @@ jobs:
run: docker-compose/acceptance_test.sh mariadb106 docker-compose-ci.yml
env:
TRACE: ${{ vars.TRACE }}
DUMPFILE_DEBUG: ${{ vars.DUMPFILE_DEBUG }}
mariadb105:
runs-on: ubuntu-latest
steps:
Expand All @@ -99,6 +104,7 @@ jobs:
run: docker-compose/acceptance_test.sh mariadb105 docker-compose-ci.yml
env:
TRACE: ${{ vars.TRACE }}
DUMPFILE_DEBUG: ${{ vars.DUMPFILE_DEBUG }}
mariadb104:
runs-on: ubuntu-latest
steps:
Expand All @@ -109,6 +115,7 @@ jobs:
run: docker-compose/acceptance_test.sh mariadb104 docker-compose-ci.yml
env:
TRACE: ${{ vars.TRACE }}
DUMPFILE_DEBUG: ${{ vars.DUMPFILE_DEBUG }}
mariadb103:
runs-on: ubuntu-latest
steps:
Expand All @@ -119,6 +126,7 @@ jobs:
run: docker-compose/acceptance_test.sh mariadb103 docker-compose-ci.yml
env:
TRACE: ${{ vars.TRACE }}
DUMPFILE_DEBUG: ${{ vars.DUMPFILE_DEBUG }}
mariadb102:
runs-on: ubuntu-latest
steps:
Expand All @@ -129,3 +137,4 @@ jobs:
run: docker-compose/acceptance_test.sh mariadb102 docker-compose-ci.yml
env:
TRACE: ${{ vars.TRACE }}
DUMPFILE_DEBUG: ${{ vars.DUMPFILE_DEBUG }}
3 changes: 3 additions & 0 deletions .github/workflows/acceptance_test_mysql.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ jobs:
run: docker-compose/acceptance_test.sh mysql81 docker-compose-ci.yml
env:
TRACE: ${{ vars.TRACE }}
DUMPFILE_DEBUG: ${{ vars.DUMPFILE_DEBUG }}
mysql80:
runs-on: ubuntu-latest
steps:
Expand All @@ -26,6 +27,7 @@ jobs:
run: docker-compose/acceptance_test.sh mysql80 docker-compose-ci.yml
env:
TRACE: ${{ vars.TRACE }}
DUMPFILE_DEBUG: ${{ vars.DUMPFILE_DEBUG }}
mysql57:
runs-on: ubuntu-latest
steps:
Expand All @@ -36,3 +38,4 @@ jobs:
run: docker-compose/acceptance_test.sh mysql57 docker-compose-ci.yml
env:
TRACE: ${{ vars.TRACE }}
DUMPFILE_DEBUG: ${{ vars.DUMPFILE_DEBUG }}
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -158,7 +158,7 @@ available option via environment variable:
* `MYSQL_PASSWORD`: password for user(default: `password`)
* `MYSQL_DBNAME`: database name(default: `mydb`)

NOTE: run with `TRACE=1` will show debug print. for the CI, `TRACE` environment variable on [setting field in the repository](https://github.com/kibitan/masking/settings/variables/actions/TRACE)
NOTE: run with `TRACE=1` will show debug print. for the CI, `TRACE` environment variable on [setting field in the repository](https://github.com/kibitan/masking/settings/variables/actions/TRACE) and run with `DUMPFILE_DEBUG=1` will show dumpfile to stdout, [setting field in the repository](https://github.com/kibitan/masking/settings/variables/actions/DUMPFILE_DEBUG)

##### with docker-compose

Expand Down
16 changes: 13 additions & 3 deletions acceptance/run_test.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ set -C # Prevent output redirection using ‘>’, ‘>&’, and ‘<>’ from o
if [[ "${TRACE-0}" == "1" ]]; then
set -vx
fi
DUMPFILE_DEBUG="${DUMPFILE_DEBUG:-0}"

MYSQL_HOST=${MYSQL_HOST:-localhost}
MYSQL_USER=${MYSQL_USER:-root}
Expand All @@ -22,10 +23,19 @@ main() {
# import database
mysql -h "$MYSQL_HOST" -u "$MYSQL_USER" -p"$MYSQL_PASSWORD" "$MYSQL_DBNAME" < "$FILEDIR/import_dumpfile.sql"

echo "mysql --version : $(mysql --version)"
echo "mysqldump --version: $(mysqldump --version)"
echo

# masking & restore
## TODO: temporary add `--skip-extended-insert` as not working now
# mysqldump -h "$MYSQL_HOST" -u "$MYSQL_USER" -p"$MYSQL_PASSWORD" "$MYSQL_DBNAME" --complete-insert | exe/masking -c "$FILEDIR/masking.yml" > "$FILEDIR/tmp/masking_dumpfile.sql"
mysqldump -h "$MYSQL_HOST" -u "$MYSQL_USER" -p"$MYSQL_PASSWORD" "$MYSQL_DBNAME" --complete-insert --skip-extended-insert | exe/masking -c "$FILEDIR/masking.yml" > "$FILEDIR/tmp/masking_dumpfile.sql"
mysqldump -h "$MYSQL_HOST" -u "$MYSQL_USER" -p"$MYSQL_PASSWORD" "$MYSQL_DBNAME" --complete-insert > "$FILEDIR/tmp/mysqldump_output.sql"
if [[ "${DUMPFILE_DEBUG}" == "1" ]]; then
echo "--- mysqldump output ---"
cat "$FILEDIR/tmp/mysqldump_output.sql"
echo "--- mysqldump output end ---"
echo ""
fi
cat "$FILEDIR/tmp/mysqldump_output.sql" | exe/masking -c "$FILEDIR/masking.yml" > "$FILEDIR/tmp/masking_dumpfile.sql"

mysql -h "$MYSQL_HOST" -u "$MYSQL_USER" -p"$MYSQL_PASSWORD" -e "CREATE DATABASE $MYSQL_ANONYMIZED_DBNAME;"
mysql -h "$MYSQL_HOST" -u "$MYSQL_USER" -p"$MYSQL_PASSWORD" "$MYSQL_ANONYMIZED_DBNAME" < "$FILEDIR/tmp/masking_dumpfile.sql"
Expand Down
3 changes: 2 additions & 1 deletion docker-compose/acceptance_test.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,10 @@ cd "$(dirname "$0")"
MYSQL_VERSION=${1:-mysql80}
DOCKER_COMPOSE_FILE=${2:-docker-compose.yml}
TRACE=${TRACE:-0}
DUMPFILE_DEBUG=${DUMPFILE_DEBUG:-0}

main() {
docker-compose -f "../$DOCKER_COMPOSE_FILE" -f "./$MYSQL_VERSION.yml" run -e "MYSQL_HOST=$MYSQL_VERSION" -e "TRACE=$TRACE" app acceptance/run_test.sh
docker-compose -f "../$DOCKER_COMPOSE_FILE" -f "./$MYSQL_VERSION.yml" run -e "MYSQL_HOST=$MYSQL_VERSION" -e "TRACE=$TRACE" -e "DUMPFILE_DEBUG=$DUMPFILE_DEBUG" app acceptance/run_test.sh
}

main "$@"
Loading