Skip to content

Commit

Permalink
README/rss: update
Browse files Browse the repository at this point in the history
  • Loading branch information
@thiagokokada @github-actions
thiagokokada authored and github-actions[bot] committed Jul 31, 2024
1 parent ab0bbb5 commit 5cd7006
Show file tree
Hide file tree
Showing 2 changed files with 8 additions and 0 deletions.
1 change: 1 addition & 0 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ Mirror of my blog in https://kokada.capivaras.dev/.

[![RSS](https://img.shields.io/badge/RSS-FFA562?style=for-the-badge&logo=rss&logoColor=white)](https://raw.githubusercontent.com/thiagokokada/blog/main/rss.xml)

- [Generating YAML files with Nix](2024-07-31/01-generating-yaml-files-with-nix.md) - 2024-07-31
- [First impressions: FPGBC](2024-07-30/01-first-impressions-fpgbc.md) - 2024-07-30
- [Go, a reasonable good language](2024-07-29/02-go-a-reasonable-good-language.md) - 2024-07-29
- [Quick bits: why you should automate everything](2024-07-29/01-quick-bits-why-you-should-automate-everything.md) - 2024-07-29
Expand Down
7 changes: 7 additions & 0 deletions rss.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,13 @@
<title>kokada&#39;s blog</title>
<link></link>
<description># dd if=/dev/urandom of=/dev/brain0</description>
<item>
<title>Generating YAML files with Nix</title>
<link>https://github.com/thiagokokada/blog/blob/main/2024-07-31/01-generating-yaml-files-with-nix.md</link>
<description>&lt;p&gt;I hate YAML. Instead of writing an essay on why I hate YAML, I can just link to&#xA;&lt;a href=&#34;https://noyaml.com/&#34;&gt;noyaml.com&lt;/a&gt;. In my personal projects I will never use it,&#xA;preferring either JSON, &lt;a href=&#34;https://toml.io/en/&#34;&gt;TOML&lt;/a&gt; or even plain old&#xA;&lt;a href=&#34;https://en.wikipedia.org/wiki/INI_file&#34;&gt;INI&lt;/a&gt; files depending on the use case.&#xA;However the ship has sailed already, there are tons of projects everywhere that&#xA;uses YAML: from most CI systems (&lt;a href=&#34;https://docs.github.com/en/actions/writing-workflows/workflow-syntax-for-github-actions&#34;&gt;GitHub&#xA;Actions&lt;/a&gt;,&#xA;&lt;a href=&#34;https://circleci.com/docs/introduction-to-yaml-configurations/&#34;&gt;CircleCI&lt;/a&gt;,&#xA;&lt;a href=&#34;https://docs.travis-ci.com/user/build-config-yaml&#34;&gt;Travis&lt;/a&gt;, et tu&#xA;&lt;a href=&#34;https://man.sr.ht/builds.sr.ht/&#34;&gt;builds.sr.ht&lt;/a&gt;, to&#xA;&lt;a href=&#34;https://kubernetes.io/docs/concepts/overview/working-with-objects/&#34;&gt;Kubernetes&lt;/a&gt;,&#xA;or in almost every&#xA;&lt;a href=&#34;https://guides.rubyonrails.org/configuring.html#configuring-a-database&#34;&gt;Rails&lt;/a&gt;&#xA;application.&lt;/p&gt;&#xA;&lt;p&gt;One way to avoid at least some issues with the language is to write YAML in&#xA;another language. I will show my solution in one of my &lt;a href=&#34;https://github.com/thiagokokada/nix-configs/&#34;&gt;personal&#xA;repositories&lt;/a&gt;, writing Nix to&#xA;generate GitHub Actions configuration files. Bonus points for validating the&#xA;result against the schema of GitHub Actions, so the famous &amp;quot;this is supposed to&#xA;be string instead of a list of strings&amp;quot; is gone.&lt;/p&gt;&#xA;&lt;p&gt;Let&#39;s start with the basics: YAML is supposed to be a &lt;a href=&#34;https://stackoverflow.com/a/1729545&#34;&gt;superset of&#xA;JSON&lt;/a&gt;. What that means is that a JSON file&#xA;&lt;a href=&#34;https://yaml.org/spec/1.2-old/spec.html#id2759572&#34;&gt;can be parsed&lt;/a&gt; by a YAML&#xA;parser. And Nix itself generates JSON natively, after all, Nix can be imagined&#xA;as &lt;a href=&#34;https://nix.dev/tutorials/nix-language.html&#34;&gt;&amp;quot;JSON with functions&amp;quot;&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;To make things easier, I will assume that you have the &lt;code&gt;nix-commands&lt;/code&gt; and&#xA;&lt;code&gt;flakes&lt;/code&gt; enabled as &lt;code&gt;experimental-features&lt;/code&gt; in your Nix configuration. If not,&#xA;go &lt;a href=&#34;https://wiki.nixos.org/wiki/Flakes&#34;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Using the &lt;code&gt;nix eval&lt;/code&gt; command, we can generate a JSON expression from Nix by:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code class=&#34;language-console&#34;&gt;$ nix eval --expr &#39;{ foo = &amp;quot;bar&amp;quot;; }&#39; --json&#xA;{&amp;quot;foo&amp;quot;:&amp;quot;bar&amp;quot;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, typing long excerpts of Nix code inside the console would be&#xA;impractical. We can write the following code inside a &lt;code&gt;foo.nix&lt;/code&gt; file instead:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code class=&#34;language-nix&#34;&gt;{&#xA; foo = &amp;quot;bar&amp;quot;;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code class=&#34;language-console&#34;&gt;$ nix eval --file foo.nix --json&#xA;{&amp;quot;foo&amp;quot;:&amp;quot;bar&amp;quot;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;While you can use a JSON output as an input for YAML parsers, it is probably&#xA;not the &lt;a href=&#34;https://metacpan.org/pod/JSON::XS#JSON-and-YAML&#34;&gt;best idea&lt;/a&gt;. Sadly (or&#xA;maybe not), Nix has no native functionality to export data to YAML. However,&#xA;since we are using Nix, it is trivial to use &lt;code&gt;nixpkgs&lt;/code&gt; to use some program to&#xA;convert from JSON to YAML.&lt;/p&gt;&#xA;&lt;p&gt;To start, let&#39;s create a new directory, move our &lt;code&gt;foo.nix&lt;/code&gt; file to it, create a&#xA;new &lt;code&gt;flake.nix&lt;/code&gt; file and put the following contents:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code class=&#34;language-nix&#34;&gt;{&#xA; description = &amp;quot;Generate YAML files with Nix&amp;quot;;&#xA;&#xA; inputs = {&#xA; nixpkgs.url = &amp;quot;github:NixOS/nixpkgs/nixos-unstable&amp;quot;;&#xA; };&#xA;&#xA; outputs = { nixpkgs, ... }:&#xA; {&#xA; packages.x86_64-linux =&#xA; let&#xA; inherit (nixpkgs) lib;&#xA; pkgs = import nixpkgs { system = &amp;quot;x86_64-linux&amp;quot;; };&#xA; in&#xA; {&#xA; toYAML =&#xA; let&#xA; file = import ./foo.nix;&#xA; json = builtins.toJSON file;&#xA; in&#xA; pkgs.runCommand &amp;quot;toYAML&amp;quot; {&#xA; buildInputs = with pkgs; [ yj ];&#xA; } &#39;&#39;&#xA; mkdir -p $out&#xA; echo ${lib.escapeShellArg json} | yj -jy &amp;gt; $out/foo.yaml&#xA; &#39;&#39;;&#xA; };&#xA; };&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;We are loading the &lt;code&gt;./foo.nix&lt;/code&gt; as a Nix file, converting it to JSON with&#xA;&lt;code&gt;builtins.toJSON&lt;/code&gt; function, and finally, using &lt;code&gt;pkgs.runCommand&lt;/code&gt; to pipe the&#xA;output of the JSON file to &lt;a href=&#34;https://github.com/sclevine/yj&#34;&gt;yj&lt;/a&gt;, that allow&#xA;convertion between serialisation formats. &lt;code&gt;-jy&lt;/code&gt; flag means &amp;quot;JSON to YAML&amp;quot;. The&#xA;reason I choose &lt;code&gt;yj&lt;/code&gt; is mostly because it is a single binary Go program, but&#xA;you can use whatever you prefer.&lt;/p&gt;&#xA;&lt;p&gt;By the way, there is a&#xA;&lt;a href=&#34;https://github.com/NixOS/nixpkgs/blob/9f918d616c5321ad374ae6cb5ea89c9e04bf3e58/lib/generators.nix#L805&#34;&gt;&lt;code&gt;lib.generators.toYAML&lt;/code&gt;&lt;/a&gt;&#xA;inside &lt;code&gt;nixpkgs.lib&lt;/code&gt;, but as of 2024-07-31 it only calls &lt;code&gt;lib.strings.toJSON&lt;/code&gt;&#xA;(that in turn, calls &lt;code&gt;builtins.toJSON&lt;/code&gt;). So it doesn&#39;t really help here.&lt;/p&gt;&#xA;&lt;p&gt;If we run the following commands, we can see the result:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code class=&#34;language-console&#34;&gt;$ nix build .#packages.x86_64-linux.toYAML&#xA;$ cat result/foo.yaml&#xA;foo: bar&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;That is the basic idea. To have a more realistic example, let&#39;s convert the&#xA;&lt;a href=&#34;https://github.com/thiagokokada/blog/blob/4e3f25485c6682f3e066b219df2290934bc0d256/.github/workflows/go.yml&#34;&gt;&lt;code&gt;go.yml&lt;/code&gt;&lt;/a&gt;,&#xA;that builds this blog, to Nix:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code class=&#34;language-nix&#34;&gt;{&#xA; name = &amp;quot;Go&amp;quot;;&#xA; on.push.branches = [ &amp;quot;main&amp;quot; ];&#xA;&#xA; jobs = {&#xA; build = {&#xA; runs-on = &amp;quot;ubuntu-latest&amp;quot;;&#xA; permissions.contents = &amp;quot;write&amp;quot;;&#xA; steps = [&#xA; { uses = &amp;quot;actions/checkout@v4&amp;quot;; }&#xA; {&#xA; name = &amp;quot;Set up Go&amp;quot;;&#xA; uses = &amp;quot;actions/checkout@v4&amp;quot;;&#xA; &amp;quot;with&amp;quot;.go-version = &amp;quot;1.21&amp;quot;;&#xA; }&#xA; {&#xA; name = &amp;quot;Update&amp;quot;;&#xA; run = &amp;quot;make&amp;quot;;&#xA; }&#xA; {&#xA; name = &amp;quot;Publish&amp;quot;;&#xA; run = &amp;quot;make publish&amp;quot;;&#xA; env.MATAROA_TOKEN = &#39;&#39;&#39;&#39;${{ secrets.MATAROA_TOKEN }}&#39;&#39;;&#xA; }&#xA; {&#xA; name = &amp;quot;Commit&amp;quot;;&#xA; uses = &amp;quot;stefanzweifel/git-auto-commit-action@v5&amp;quot;;&#xA; &amp;quot;with&amp;quot;.commit_message = &amp;quot;README/rss:update&amp;quot;;&#xA; }&#xA; ];&#xA; };&#xA; };&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Some interesting things to highlight: &lt;code&gt;with&lt;/code&gt; is a reserved word in Nix, so we&#xA;need to quote it. Not a problem, but something to be aware. And the template&#xA;string in GitHub Actions uses the same &lt;code&gt;${}&lt;/code&gt; that Nix uses, so we need to&#xA;escape.&lt;/p&gt;&#xA;&lt;p&gt;And after running the following commands:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ nix build .#packages.x86_64-linux.toYAML&#xA;$ cat result/go.yaml&#xA;jobs:&#xA; build:&#xA; permissions:&#xA; contents: write&#xA; runs-on: ubuntu-latest&#xA; steps:&#xA; - uses: actions/checkout@v4&#xA; - name: Set up Go&#xA; uses: actions/checkout@v4&#xA; with:&#xA; go-version: &amp;quot;1.21&amp;quot;&#xA; - name: Update&#xA; run: make&#xA; - env:&#xA; MATAROA_TOKEN: ${{ secrets.MATAROA_TOKEN }}&#xA; name: Publish&#xA; run: make publish&#xA; - name: Commit&#xA; uses: stefanzweifel/git-auto-commit-action@v5&#xA; with:&#xA; commit_message: README/rss:update&#xA;name: Go&#xA;&amp;quot;on&amp;quot;:&#xA; push:&#xA; branches:&#xA; - main&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Yes, the keys are not in the same order as we defined, since Nix, like most&#xA;programming languages (with the exception of&#xA;&lt;a href=&#34;https://mail.python.org/pipermail/python-dev/2017-December/151283.html&#34;&gt;Python&lt;/a&gt;),&#xA;do not guarantee the insertion order in maps/dicts/attrsets/whatever. But I&#xA;really hope whatever is consuming your YAML is not relying in the order the&#xA;keys are defined (this would be more cursed than YAML already is).&lt;/p&gt;&#xA;&lt;p&gt;So that is basically it. For the bonus points that I talked at the start of the&#xA;post, we can modify &lt;code&gt;pkgs.runCommand&lt;/code&gt; to run some kind of validator. I use&#xA;&lt;a href=&#34;https://github.com/mpalmer/action-validator&#34;&gt;&lt;code&gt;action-validator&lt;/code&gt;&lt;/a&gt;, one that I&#xA;particularly packaged in&#xA;&lt;a href=&#34;https://github.com/NixOS/nixpkgs/pull/260217&#34;&gt;nixpkgs&lt;/a&gt; to use in those cases.&#xA;But you could use e.g.: a validator of Kubernetes YAML. Or a generic YAML lint&#xA;like this &lt;a href=&#34;https://github.com/adrienverge/yamllint&#34;&gt;one&lt;/a&gt;. The possibilities are&#xA;endless.&lt;/p&gt;&#xA;&lt;p&gt;Let&#39;s modify our &lt;code&gt;flake.nix&lt;/code&gt; to add the validation:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code class=&#34;language-nix&#34;&gt;{&#xA; # ...&#xA; outputs = { nixpkgs, ... }:&#xA; {&#xA; packages.x86_64-linux =&#xA; let&#xA; inherit (nixpkgs) lib;&#xA; pkgs = import nixpkgs { system = &amp;quot;x86_64-linux&amp;quot;; };&#xA; in&#xA; {&#xA; toYAML =&#xA; let&#xA; file = import ./go.nix;&#xA; json = builtins.toJSON file;&#xA; in&#xA; pkgs.runCommand &amp;quot;toYAML&amp;quot; {&#xA; buildInputs = with pkgs; [ action-validator yj ];&#xA; } &#39;&#39;&#xA; mkdir -p $out&#xA; echo ${lib.escapeShellArg json} | yj -jy &amp;gt; $out/go.yaml&#xA; action-validator -v $out/go.yaml&#xA; &#39;&#39;;&#xA; };&#xA; };&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And let&#39;s add an error in our &lt;code&gt;go.nix&lt;/code&gt; file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code class=&#34;language-diff&#34;&gt;diff --git a/go.nix b/go.nix&#xA;index 25e0596..8c00033 100644&#xA;--- a/go.nix&#xA;+++ b/go.nix&#xA;@@ -5,7 +5,7 @@&#xA; jobs = {&#xA; build = {&#xA; runs-on = &amp;quot;ubuntu-latest&amp;quot;;&#xA;- permissions.contents = &amp;quot;write&amp;quot;;&#xA;+ permissions.contents = [ &amp;quot;write&amp;quot; ];&#xA; steps = [&#xA; { uses = &amp;quot;actions/checkout@v4&amp;quot;; }&#xA; {&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Finally, let&#39;s try to build our YAML file again:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ nix build .#packages.x86_64-linux.toYAML&#xA;error: builder for &#39;/nix/store/j8wr6j1pvyf986sf74hqw8k31lvlzac5-toYAML.drv&#39; failed with exit code 1;&#xA; last 25 log lines:&#xA; &amp;gt; &amp;quot;Additional property &#39;runs-on&#39; is not allowed&amp;quot;,&#xA; &amp;gt; ),&#xA; &amp;gt; path: &amp;quot;/jobs/build&amp;quot;,&#xA; &amp;gt; title: &amp;quot;Property conditions are not met&amp;quot;,&#xA; &amp;gt; },&#xA; &amp;gt; Properties {&#xA; &amp;gt; code: &amp;quot;properties&amp;quot;,&#xA; &amp;gt; detail: Some(&#xA; &amp;gt; &amp;quot;Additional property &#39;steps&#39; is not allowed&amp;quot;,&#xA; &amp;gt; ),&#xA; &amp;gt; path: &amp;quot;/jobs/build&amp;quot;,&#xA; &amp;gt; title: &amp;quot;Property conditions are not met&amp;quot;,&#xA; &amp;gt; },&#xA; &amp;gt; Required {&#xA; &amp;gt; code: &amp;quot;required&amp;quot;,&#xA; &amp;gt; detail: None,&#xA; &amp;gt; path: &amp;quot;/jobs/build/uses&amp;quot;,&#xA; &amp;gt; title: &amp;quot;This property is required&amp;quot;,&#xA; &amp;gt; },&#xA; &amp;gt; ],&#xA; &amp;gt; },&#xA; &amp;gt; ],&#xA; &amp;gt; },&#xA; &amp;gt; ],&#xA; &amp;gt; }&#xA; For full logs, run &#39;nix log /nix/store/j8wr6j1pvyf986sf74hqw8k31lvlzac5-toYAML.drv&#39;.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Yes, the output of &lt;code&gt;action-validator&lt;/code&gt; is awfully verbose, but it is still&#xA;better than making &lt;a href=&#34;https://x.com/eric_sink/status/1430954572848287744&#34;&gt;&amp;quot;8 commits/push in one&#xA;hour&amp;quot;&lt;/a&gt;.&lt;/p&gt;&#xA;</description>
<guid>https://github.com/thiagokokada/blog/blob/main/2024-07-31/01-generating-yaml-files-with-nix.md</guid>
<pubDate>Wed, 31 Jul 2024 00:00:00 +0000</pubDate>
</item>
<item>
<title>First impressions: FPGBC</title>
<link>https://github.com/thiagokokada/blog/blob/main/2024-07-30/01-first-impressions-fpgbc.md</link>
Expand Down

0 comments on commit 5cd7006

Please sign in to comment.