Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rewrite scripts in TypeScript #8

Merged
merged 1 commit into from
Jun 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 8 additions & 14 deletions .github/workflows/CI.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -40,25 +40,19 @@ jobs:
- name: Check no lint warnings
run: yamllint -s assets/yaml/*.yaml

python-test:
name: Test Python scripts
deno-test:
name: Test Deno scripts
runs-on: ubuntu-22.04
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v5
- name: Setup Deno
uses: denoland/setup-deno@v1.1.4
with:
python-version: "3.12"
cache: "pip"
- name: Install dependencies
run: |
pip install PyYAML mypy ruff tomli-w types-PyYAML
ruff version
mypy -V
deno-version: v1.x
- name: Check code formatted
run: ruff format --check .
run: deno fmt --check scripts/*.ts
- name: Check no lint warnings
run: ruff check .
run: deno lint scripts/*.ts
- name: Type-check
run: mypy --allow-redefinition .
run: deno check scripts/*.ts
23 changes: 4 additions & 19 deletions README.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,7 @@
:enwp-article-url: {enwp-url}/wiki/Japanese_era_name
:assets-dir: assets
:yaml-dir: {assets-dir}/yaml
:pyyaml-url: {github-url}/yaml/pyyaml
:tomli-w-url: {github-url}/hukkin/tomli-w
:deno-url: https://deno.com/
:jawp-url: https://ja.wikipedia.org
:jawp-article-url: {jawp-url}/wiki/%E5%85%83%E5%8F%B7%E4%B8%80%E8%A6%A7_(%E6%97%A5%E6%9C%AC)
:reuse-spec-url: https://reuse.software/spec/
Expand Down Expand Up @@ -125,31 +124,17 @@ link:{yaml-dir}/modern.yaml[]::

=== Convert to other formats

This project contains Python scripts for converting YAML files to JSON files or
TOML files.

IMPORTANT: {pyyaml-url}[_PyYAML_] is required for conversion.
This project contains {deno-url}[Deno] scripts for converting YAML files to
JSON files.

.To convert to JSON files
[source,sh]
----
./scripts/yaml2json.py
./scripts/yaml2json.ts
----

This will output JSON files in the `{assets-dir}/json` directory.

.To convert to TOML files
[source,sh]
----
./scripts/yaml2toml.py
----

This will output TOML files in the `{assets-dir}/toml` directory.

IMPORTANT: {tomli-w-url}[_Tomli-W_] is required for conversion.

NOTE: Keys with `null` values ​​are removed.

== Acknowledgment

.This dataset was created using the following:
Expand Down
21 changes: 3 additions & 18 deletions README.ja.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,7 @@
:jawp-article-url: {jawp-url}/wiki/%E5%85%83%E5%8F%B7%E4%B8%80%E8%A6%A7_(%E6%97%A5%E6%9C%AC)
:assets-dir: assets
:yaml-dir: {assets-dir}/yaml
:pyyaml-url: {github-url}/yaml/pyyaml
:tomli-w-url: {github-url}/hukkin/tomli-w
:deno-url: https://deno.com/
:enwp-url: https://en.wikipedia.org
:enwp-article-url: {enwp-url}/wiki/Japanese_era_name
:reuse-spec-url: https://reuse.software/spec/
Expand Down Expand Up @@ -125,30 +124,16 @@ link:{yaml-dir}/modern.yaml[]::

=== 他のフォーマットへの変換

このプロジェクトにはYAMLファイルをJSONファイルやTOMLファイルに変換するためのPythonスクリプトが含まれています。

IMPORTANT: 変換にはlink:{pyyaml-url}[_PyYAML_]が必要です。
このプロジェクトにはYAMLファイルをJSONファイルに変換するためのlink:{deno-url}[Deno]スクリプトが含まれています。

.JSONファイルへ変換
[source,sh]
----
./scripts/yaml2json.py
./scripts/yaml2json.ts
----

JSONファイルは``{assets-dir}/json``ディレクトリに出力されます。

.TOMLファイルへ変換
[source,sh]
----
./scripts/yaml2toml.py
----

TOMLファイルは``{assets-dir}/toml``ディレクトリに出力されます。

IMPORTANT: 変換にはlink:{tomli-w-url}[_Tomli-W_]が必要です。

NOTE: 値が``null``のキーは取り除かれます。

== 謝辞

.このデータセットは以下を参照して作成しました:
Expand Down
Empty file removed assets/toml/.gitkeep
Empty file.
7 changes: 7 additions & 0 deletions deno.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{
"imports": {
"@std/path": "jsr:@std/path@^0.225.2",
"@std/yaml": "jsr:@std/yaml@^0.224.1"
},
"lock": false
}
3 changes: 3 additions & 0 deletions deno.json.license
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
SPDX-FileCopyrightText: None

SPDX-License-Identifier: CC0-1.0
53 changes: 12 additions & 41 deletions justfile
Original file line number Diff line number Diff line change
Expand Up @@ -12,53 +12,24 @@ default: fmt
npx prettier -w assets/yaml/*.yaml

# Run the linter
lint:
#!/usr/bin/env bash
source venv/bin/activate
@lint:
yamllint -s assets/yaml/*.yaml

# Configure a development environment for the Python scripts
setup-python:
#!/usr/bin/env bash
python3 -m venv venv
source venv/bin/activate
pip3 install PyYAML mypy ruff tomli-w types-PyYAML yamllint
# Run `deno fmt`
@deno-fmt:
deno fmt scripts/*.ts

# Run the formatter for the Python scripts
python-fmt:
#!/usr/bin/env bash
source venv/bin/activate
ruff format .
# Run `deno lint`
@deno-lint:
deno lint scripts/*.ts

# Run the linter for the Python scripts
python-lint:
#!/usr/bin/env bash
source venv/bin/activate
ruff check .

# Apply lint suggestions for the Python scripts
python-lint-fix:
#!/usr/bin/env bash
source venv/bin/activate
ruff check --fix .

# Run `mypy`
python-type-check:
#!/usr/bin/env bash
source venv/bin/activate
mypy --allow-redefinition .
# Run `deno check`
@deno-type-check:
deno check scripts/*.ts

# Generate the JSON files
generate-json:
#!/usr/bin/env bash
source venv/bin/activate
python3 scripts/yaml2json.py

# Generate the TOML files
generate-toml:
#!/usr/bin/env bash
source venv/bin/activate
python3 scripts/yaml2toml.py
@generate-json:
deno run --allow-read --allow-write scripts/yaml2json.ts

# Run the linter for GitHub Actions workflow files
@lint-github-actions:
Expand Down
23 changes: 0 additions & 23 deletions scripts/yaml2json.py

This file was deleted.

25 changes: 25 additions & 0 deletions scripts/yaml2json.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
#!/usr/bin/env -S deno run --allow-read --allow-write

// SPDX-FileCopyrightText: None
//
// SPDX-License-Identifier: CC0-1.0

import * as path from "@std/path";
import * as yaml from "@std/yaml";

const assetsDir = path.join(
path.dirname(path.dirname(path.fromFileUrl(import.meta.url))),
"assets",
);
const yamlDir = path.join(assetsDir, "yaml");
const jsonDir = path.join(assetsDir, "json");

for await (const dirEntry of Deno.readDir(yamlDir)) {
const yamlFile = path.join(yamlDir, dirEntry.name);
const inputText = await Deno.readTextFile(yamlFile);
const data = yaml.parse(inputText);

const jsonFile = path.join(jsonDir, path.parse(yamlFile).name + ".json");
const jsonString = JSON.stringify(data, null, 2);
await Deno.writeTextFile(jsonFile, jsonString);
}
32 changes: 0 additions & 32 deletions scripts/yaml2toml.py

This file was deleted.