-
Notifications
You must be signed in to change notification settings - Fork 0
/
rss.xml
196 lines (196 loc) · 510 KB
/
rss.xml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
<?xml version="1.0" encoding="UTF-8"?><rss version="2.0" xmlns:content="http://purl.org/rss/1.0/modules/content/">
<channel>
<title>kokada's blog</title>
<link></link>
<description># dd if=/dev/urandom of=/dev/brain0</description>
<item>
<title>Quick bits: basic flake.nix template</title>
<link>https://github.com/thiagokokada/blog/blob/main/posts/2024-12-19/01-quick-bits-basic-flake-nix-template.md</link>
<description><p>Sometimes I want a really basic <code>flake.nix</code> that has no dependencies except for
<code>nixpkgs</code> itself, e.g.: I want to avoid
<a href="https://github.com/numtide/flake-utils">flake-utils</a> or any other dependency.
So, here you go:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>{
</span></span><span style="display:flex;"><span> description <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;Description&#34;</span>;
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> inputs <span style="color:#f92672">=</span> {
</span></span><span style="display:flex;"><span> nixpkgs<span style="color:#f92672">.</span>url <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;github:NixOS/nixpkgs/nixpkgs-unstable&#34;</span>;
</span></span><span style="display:flex;"><span> };
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> outputs <span style="color:#f92672">=</span>
</span></span><span style="display:flex;"><span> { self<span style="color:#f92672">,</span> nixpkgs<span style="color:#f92672">,</span> <span style="color:#f92672">...</span> }:
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">let</span>
</span></span><span style="display:flex;"><span> supportedSystems <span style="color:#f92672">=</span> [
</span></span><span style="display:flex;"><span> <span style="color:#e6db74">&#34;aarch64-linux&#34;</span>
</span></span><span style="display:flex;"><span> <span style="color:#e6db74">&#34;x86_64-linux&#34;</span>
</span></span><span style="display:flex;"><span> <span style="color:#e6db74">&#34;aarch64-darwin&#34;</span>
</span></span><span style="display:flex;"><span> <span style="color:#e6db74">&#34;x86_64-linux&#34;</span>
</span></span><span style="display:flex;"><span> ];
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#75715e"># Helper function to generate an attrset &#39;{ x86_64-linux = f &#34;x86_64-linux&#34;; ... }&#39;.</span>
</span></span><span style="display:flex;"><span> forAllSystems <span style="color:#f92672">=</span> nixpkgs<span style="color:#f92672">.</span>lib<span style="color:#f92672">.</span>genAttrs supportedSystems;
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#75715e"># Nixpkgs instantiated for supported system types.</span>
</span></span><span style="display:flex;"><span> nixpkgsFor <span style="color:#f92672">=</span> forAllSystems (system: <span style="color:#f92672">import</span> nixpkgs { <span style="color:#66d9ef">inherit</span> system; });
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">in</span>
</span></span><span style="display:flex;"><span> {
</span></span><span style="display:flex;"><span> packages <span style="color:#f92672">=</span> forAllSystems (
</span></span><span style="display:flex;"><span> system:
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">let</span>
</span></span><span style="display:flex;"><span> pkgs <span style="color:#f92672">=</span> nixpkgsFor<span style="color:#f92672">.</span><span style="color:#e6db74">${</span>system<span style="color:#e6db74">}</span>;
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">in</span>
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">rec</span> {
</span></span><span style="display:flex;"><span> default <span style="color:#f92672">=</span> pkgs<span style="color:#f92672">.</span>hello;
</span></span><span style="display:flex;"><span> }
</span></span><span style="display:flex;"><span> );
</span></span><span style="display:flex;"><span> };
</span></span><span style="display:flex;"><span>}
</span></span></code></pre>
</body>
</html>
<p>Not sure where I grabbed the definition for <code>forAllSystems</code> and <code>nixpkgsFor</code>. I
have the impression it was in a <a href="https://jvns.ca/">Julia Evans blog post</a>, but
I can't find it.</p>
<p>Anyway, it is here for me to remember, and it may help someone else.</p>
</description>
<guid>https://github.com/thiagokokada/blog/blob/main/posts/2024-12-19/01-quick-bits-basic-flake-nix-template.md</guid>
<pubDate>Thu, 19 Dec 2024 00:00:00 +0000</pubDate>
</item>
<item>
<title>Praise to scdoc to generate man pages</title>
<link>https://github.com/thiagokokada/blog/blob/main/posts/2024-12-04/01-praise-to-scdoc-to-generate-man-pages.md</link>
<description><p>Hey, its been a long time since my <a href="posts/2024-10-07/01-enabling-le-audio-lc3-in-wf-1000xm5.md">last blog
post</a>. It is mostly
because I ran out of things to write, but I expected this. This is probably
more likely how I am actually going to post from now. At least, it shows that
my plan to have a <a href="https://github.com/thiagokokada/blog/blob/main/posts/2024-08-24/01-making-a-blog-for-the-next-10-years.md">blog for a long
time</a>, that is
easy to go back when I wanted is working fine, but I digress.</p>
<p>Going back to the theme of the today blog post, I needed to write a <a href="https://en.wikipedia.org/wiki/Man_page">man
page</a> for the first time in years. I
hate <a href="https://en.wikipedia.org/wiki/Troff">troff</a>, the typesetting system used
for man pages (similar to <a href="https://en.wikipedia.org/wiki/LaTeX">LaTeX</a> for
documents). It is one of the weirdest languages that I ever saw, and even the
example in Wikipedia shows that:</p>
<pre><code class="language-troff">.ND &quot;January 10, 1993&quot;
.AU &quot;Ms. Jane Smith&quot;
.AT &quot;Upcoming appointment&quot;
.MT 5
.DS
Reference #A12345
.sp 4
Mr. Samuel Jones
Field director, Bureau of Inspections
1010 Government Plaza
Capitoltown, ST
.sp 3
Dear Mr. Jones,
.sp 2
.P
Making reference to the noted obligation to submit for state inspection our newly created production process, we request that you consider the possible inappropriateness of subjecting the innovative technologies of tomorrow to the largely antiquated requirements of yesterday. If our great state is to prosper in the twenty-first century, we must take steps
.B now ,
in
.I this
year of
.I this
decade, to prepare our industrial base for the interstate and international competition that is sure to appear. Our new process does precisely that. Please do not let it be undone by a regulatory environment that is no longer apt.
.P
Thank you for your consideration of our position.
.FC Sincerely
.SG
</code></pre>
<p>Keep in mind that the break lines are necessary every time you introduce a
macro, like <code>.I this</code> (that I <em>think</em> it is for italics). Yes, this format is
as illegible as hell, and it is worse that the format lacks good tooling (or at
least I didn't find any good ones).</p>
<p>Most people when they need to write a man page nowadays ends up using some
other format that generates a man page. For example, in the past I used
<a href="https://pandoc.org/">Pandoc</a> to convert Markdown to a man page, but even if
Pandoc is a great project the result is sub-optimal at best: Markdowns are, at
the end, designed for generating HTML (and a subset of it), and not man pages,
so you basically ends up fighting the format for it to do what you want.
Also, Pandoc is a big project, with a ~200MB binary (at least it is the default
Pandoc binary in Nix).</p>
<p>For this specific project I needed something small. I am trying to replace one
of the most essential pieces inside NixOS, <code>nixos-rebuild</code>, written in Bash,
with a <a href="https://discourse.nixos.org/t/nixos-rebuild-ng-a-nixos-rebuild-rewrite/55606/">full rewritten in
Python</a>
(sorry Rust zealots!), called <code>nixos-rebuild-ng</code>.</p>
<p>Since this project will eventually (if successful) be in the critical path for
NixOS, I want to reduce the number of dependencies as much as possible, so
something as big as Pandoc is out. I could use
<a href="https://asciidoc.org/">AsciiDoc</a>, but it is a big complicated Python project
(this may seem ironic, but <code>nixos-rebuild-ng</code> has only one runtime dependency,
that is optional). And I also hated the last time I tried to use it to generate
man pages: it more flexible than Markdown, but still far from optimal.</p>
<p>Thanks to Drew DeVault (creator of <a href="https://swaywm.org/">SwayWM</a>) that seems it
had the same issues in the past and created
<a href="https://drewdevault.com/2018/05/13/scdoc.html"><code>scdoc</code></a>, a very simple man
page generator using a DSL inspired in Markdown, but specific to generate man
pages. The binary is written in C (and advantage in this case since it means it
is easier to bootstrap), is small (~1 Kloc) and has no dependencies, so it
fits the requirement.</p>
<p>While the language suffers from being a niche project for a niche segment, the
<a href="https://man.archlinux.org/man/scdoc.5.en">man page</a> for it is actually really
nice. It is terse though and lacks examples, and this is what this blog post
will try to accomplish.</p>
<p>To start, let's have a quick summary of the syntax, written in <code>scdoc</code> as
comments:</p>
<pre><code class="language-scdoc">; quick summary:
; # new section
; comments starts with ;
; - this is a list
; 	- sub-list
; - *bold*: _underline_, force a line break++
; - [tables], \[ can be used to force an actual [
; . numbered list
; please configure your editor to use hard tabs
; see `man 5 scdoc` for more information about syntax
; or https://man.archlinux.org/man/scdoc.5.en
</code></pre>
<p>I actually added this summary in the <code>.scd</code> (the <code>scdoc</code> extension) files that
I wrote, so it is easy for someone that never saw the format to start
collaborating.</p>
<p>And here an example of a (summarised) man page in <code>.scd</code> format:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>nixos-rebuild-ng(8)
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span># NAME
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>nixos-rebuild - reconfigure a NixOS machine
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span># SYNOPSIS
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="font-style:italic">_nixos-rebuild_</span> \[--upgrade] [--upgrade-all]++
</span></span><span style="display:flex;"><span>		\[{switch,boot}]
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span># DESCRIPTION
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>This command has one required argument, which specifies the desired operation.
</span></span><span style="display:flex;"><span>It must be one of the following:
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="font-style:italic">*switch*</span>
</span></span><span style="display:flex;"><span>	Build and activate the new configuration, and make it the boot default.
</span></span><span style="display:flex;"><span>	That is, the configuration is added to the GRUB boot menu as the
</span></span><span style="display:flex;"><span>	default menu entry, so that subsequent reboots will boot the system
</span></span><span style="display:flex;"><span>	into the new configuration. Previous configurations activated with
</span></span><span style="display:flex;"><span>	nixos-rebuild switch or nixos-rebuild boot remain available in the GRUB
</span></span><span style="display:flex;"><span>	menu.
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="font-style:italic">*boot*</span>
</span></span><span style="display:flex;"><span>	Build the new configuration and make it the boot default (as with
</span></span><span style="display:flex;"><span>	<span style="font-style:italic">*nixos-rebuild switch*</span>), but do not activate it. That is, the system
</span></span><span style="display:flex;"><span>	continues to run the previous configuration until the next reboot.
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span># OPTIONS
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="font-style:italic">*--upgrade, --upgrade-all*</span>
</span></span><span style="display:flex;"><span>	Update the root user&#39;s channel named &#39;nixos&#39; before rebuilding the
</span></span><span style="display:flex;"><span>	system.
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>	In addition to the &#39;nixos&#39; channel, the root user&#39;s channels which have
</span></span><span style="display:flex;"><span>	a file named &#39;.update-on-nixos-rebuild&#39; in their base directory will
</span></span><span style="display:flex;"><span>	also be updated.
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>	Passing <span style="font-style:italic">*--upgrade-all*</span> updates all of the root user&#39;s channels.
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>See the Nix manual, <span style="font-style:italic">*nix flake lock --help*</span> or <span style="font-style:italic">*nix-build --help*</span> for details.
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span># ENVIRONMENT
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>NIXOS_CONFIG
</span></span><span style="display:flex;"><span>	Path to the main NixOS configuration module. Defaults to
</span></span><span style="display:flex;"><span>	<span style="font-style:italic">_/etc/nixos/configuration.nix_</span>.
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span># FILES
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>/etc/nixos/flake.nix
</span></span><span style="display:flex;"><span>	If this file exists, then <span style="font-style:italic">*nixos-rebuild*</span> will use it as if the
</span></span><span style="display:flex;"><span>	<span style="font-style:italic">*--flake*</span> option was given. This file may be a symlink to a
</span></span><span style="display:flex;"><span>	flake.nix in an actual flake; thus <span style="font-style:italic">_/etc/nixos_</span> need not be a
</span></span><span style="display:flex;"><span>	flake.
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span># AUTHORS
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>Nixpkgs/NixOS contributors
</span></span></code></pre>
</body>
</html>
<p>And here is a screenshot of the result:</p>
<p><a href="https://github.com/thiagokokada/blog/raw/main/posts/2024-12-04/2024-12-04-230955_hyprshot.png"><img src="https://github.com/thiagokokada/blog/raw/main/posts/2024-12-04/2024-12-04-230955_hyprshot.png" alt="Man page rendered from scd
file"></a></p>
<p>One of nice things that I found is how looking at the plain text looks kind
like the man page result already. And if you know Markdown, you can basically
understand most things that is happening. There are a few differences, like
<code>*bold*</code> instead of <code>**bold**</code>, and while they're unfortunate they're not the
end of the world.</p>
<p>Now, the format has its quirks. The first line being the name of the program
and section in parenthesis is required, but this makes sense, since you need
this information for the corners. But for one, it requires the usage of hard
tabs to create indentation, and the error messages are awful, in a situation
that kind remembers me of <code>Makefile</code>. Also the choice of <code>[</code> to start a table
means that the traditional <code>app [command]</code> needs in many cases to be escaped as
<code>app \[command]</code>. I found this a strange choice since this is supposed to be a
format that is only used for man pages, and using <code>[command]</code> to indicate an
optional is common, but at least it is easy to escape.</p>
<p>In the end, I think all that matters is the result. And for the first time for
all those years trying to write a man page, I am satisfied with the result. The
man page looks exactly as I wanted once rendered, and the <code>.scd</code> file looks
reasonable good that it can work as a documentation for someone that for one
reason or another can't use the man page (can't say the same for the troff
version). Also, it is really easy for someone to just go there and update the
man page, even without experience in the format (except for maybe the
requirement of tabs). So all in all, I really liked the format, and will use it
again if I need to write another man page in the future.</p>
</description>
<guid>https://github.com/thiagokokada/blog/blob/main/posts/2024-12-04/01-praise-to-scdoc-to-generate-man-pages.md</guid>
<pubDate>Wed, 04 Dec 2024 00:00:00 +0000</pubDate>
</item>
<item>
<title>Enabling LE Audio/LC3 in WF-1000XM5</title>
<link>https://github.com/thiagokokada/blog/blob/main/posts/2024-10-07/01-enabling-le-audio-lc3-in-wf-1000xm5.md</link>
<description><p>One of things that I hate the most about the fact that we are all using
wireless earbuds instead of wired earphones is the latency: it is bad, getting
up to seconds(!) depending on your particular combination of OS/earbuds/device.</p>
<p>There is a solution though: Bluetooth LE Audio, that is supposed to fix
multiple issues with the original design for Bluetooth Classic Audio, including
a much lower latency, improved efficiency (e.g.: less battery power) and even
multiple streams of audio. LE Audio also includes a new default codec for
improved audio quality, <a href="https://en.wikipedia.org/wiki/LC3_(codec)">LC3</a>, that
replaces the venerable <a href="https://en.wikipedia.org/wiki/SBC_(codec)">SBC</a> codec
for audio.</p>
<p>However, the standard is a mess right now: a few wireless headphones already
support it, but they're generally disabled by default and it is pretty messy to
enable. And even after enabling it, getting it to work can be a pain.</p>
<p>I have pretty much the best setup to use LE Audio right now: a recently
released Pixel 9 Pro with Sony's
<a href="https://www.sony.ie/headphones/products/wf-1000xm5">WF-1000XM5</a> earbuds, and
after lots of tries I got it to work. You can see below the versions of
everything I am using:</p>
<ul>
<li>Android: 14</li>
<li><a href="https://play.google.com/store/apps/details?id=com.sony.songpal.mdr">Sound
Connect</a>:
11.0.1</li>
<li>WM-1000XM5: 4.0.2</li>
</ul>
<p>The first thing you need to do is enable in &quot;Sound Connect&quot; app &quot;LE Audio
Priority&quot; in &quot;Device Settings -&gt; System&quot;:</p>
<p><a href="https://github.com/thiagokokada/blog/raw/main/posts/2024-10-07/photo_4909454744305642922_y.jpg"><img src="https://github.com/thiagokokada/blog/raw/main/posts/2024-10-07/photo_4909454744305642922_y.jpg" alt="LE Audio option inside Sound
Connect"></a></p>
<p>After this, you will need to pair your headset with the device again. You can
do this as same as always: press and hold the button in case for a few seconds
until a blue light starts to blink. However, this is where things starts to get
janky: I couldn't get the headset to pair with Android again.</p>
<p>A few of the things that I needed to do (in no specific order):</p>
<ul>
<li>Remove the previous paired headset</li>
<li>Restart the Android</li>
<li>Clean-up &quot;Sound Connect&quot; storage (Long press the app icon -&gt; &quot;App info&quot; -&gt;
&quot;Storage and Cache&quot; -&gt; &quot;Clear storage&quot;)</li>
</ul>
<p>If you can get the headset to connect, go to the &quot;Bluetooth&quot; settings in
Android, click in the gear icon for the headset and enable &quot;LE Audio&quot; option:</p>
<p><a href="https://github.com/thiagokokada/blog/raw/main/posts/2024-10-07/photo_4909454744305642937_y.jpg"><img src="https://github.com/thiagokokada/blog/raw/main/posts/2024-10-07/photo_4909454744305642937_y.jpg" alt="LE Audio option Bluetooth
Settings"></a></p>
<p>If you can't, you may want to <a href="https://helpguide.sony.net/mdr/2963/v1/en/contents/TP1000783925.html">restore the headset to factory
settings</a>
and try again from the start (that means pairing your device with &quot;Sound
Connect&quot; again, and you may want to try to clear the storage before doing so).</p>
<p>Yes, the process is extremely janky, but I think this is why both &quot;Sound
Connect&quot; and Android marks this feature as beta/experimental. And I still need
to test the latency, but from my initial testing there are some glitches when
the audio is only used for a short period of time (e.g.: Duolingo only enables
the audio when the character is speaking). So I only recommend this if you want
to test how LE Audio will behave, since it is clear that this needs more
polish.</p>
</description>
<guid>https://github.com/thiagokokada/blog/blob/main/posts/2024-10-07/01-enabling-le-audio-lc3-in-wf-1000xm5.md</guid>
<pubDate>Mon, 07 Oct 2024 00:00:00 +0000</pubDate>
</item>
<item>
<title>Quick bits: where does Nix store Flake's trusted settings?</title>
<link>https://github.com/thiagokokada/blog/blob/main/posts/2024-09-30/01-quick-bits-where-does-nix-store-flakes-trusted-settings.md</link>
<description><p>Have you ever run a <code>nix build</code> command and had this prompt?</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>$ nix run .#darwinActivations/Sekai-MacBook-Pro
</span></span><span style="display:flex;"><span>do you want to allow configuration setting &#39;extra-substituters&#39; to be set to &#39;https://nix-community.cachix.org https://thiagokokada-nix-configs.cachix.org&#39; (y/N)? y
</span></span><span style="display:flex;"><span>do you want to permanently mark this value as trusted (y/N)? y
</span></span></code></pre>
</body>
</html>
<p>And realise that you did/didn't want to mark this value as trusted? But where
is this stored? Well, I had to look at the Nix source code to find the answer,
since I can't find this anywhere in the web or in ChatGPT (but I am sure that
now that I posted this it will eventually be searchable), and the answer can be
found
<a href="https://github.com/NixOS/nix/blob/c116030605bf7fecd232d0ff3b6fe066f23e4620/src/libflake/flake/config.cc#L13-L16">here</a>:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>Path <span style="color:#a6e22e">trustedListPath</span>()
</span></span><span style="display:flex;"><span>{
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">return</span> getDataDir() <span style="color:#f92672">+</span> <span style="color:#e6db74">&#34;/trusted-settings.json&#34;</span>;
</span></span><span style="display:flex;"><span>}
</span></span></code></pre>
</body>
</html>
<p>Where is <code>getDataDir()</code> though? I found the answer
<a href="https://github.com/NixOS/nix/blob/c116030605bf7fecd232d0ff3b6fe066f23e4620/src/libutil/users.cc#L52-L65">here</a>:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>Path <span style="color:#a6e22e">getDataDir</span>()
</span></span><span style="display:flex;"><span>{
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">auto</span> dir <span style="color:#f92672">=</span> getEnv(<span style="color:#e6db74">&#34;NIX_DATA_HOME&#34;</span>);
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">if</span> (dir) {
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">return</span> <span style="color:#f92672">*</span>dir;
</span></span><span style="display:flex;"><span> } <span style="color:#66d9ef">else</span> {
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">auto</span> xdgDir <span style="color:#f92672">=</span> getEnv(<span style="color:#e6db74">&#34;XDG_DATA_HOME&#34;</span>);
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">if</span> (xdgDir) {
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">return</span> <span style="color:#f92672">*</span>xdgDir <span style="color:#f92672">+</span> <span style="color:#e6db74">&#34;/nix&#34;</span>;
</span></span><span style="display:flex;"><span> } <span style="color:#66d9ef">else</span> {
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">return</span> getHome() <span style="color:#f92672">+</span> <span style="color:#e6db74">&#34;/.local/share/nix&#34;</span>;
</span></span><span style="display:flex;"><span> }
</span></span><span style="display:flex;"><span> }
</span></span><span style="display:flex;"><span>}
</span></span></code></pre>
</body>
</html>
<p>So we solved the mystery:</p>
<ul>
<li>If <code>NIX_DATA_HOME</code> is set, the file will be in
<code>$NIX_DATA_HOME/trusted-settings.json</code></li>
<li>If <code>XDG_DATA_HOME</code> is set, the file will be in
<code>$XDG_DATA_HOME/nix/trusted-settings.json</code></li>
<li>Otherwise Nix will fallback to <code>$HOME/.local/share/nix/trusted-settings.json</code></li>
</ul>
<p>By the way, if you don't know why you got this prompt, if <code>flake.nix</code> has a
<code>nixConfig</code> attribute inside <code>outputs</code> <strong>and</strong> this <code>nixConfig</code> is an unsafe
attribute (like <code>extra-substituters</code>) you will get this prompt, unless you set
<a href="https://nix.dev/manual/nix/2.23/command-ref/conf-file#conf-accept-flake-config"><code>accept-flake-config = true</code></a>
in your Nix configuration (please <strong>do not do this</strong>, it is dangerous because
it may allow running possible unsafe options without asking you first).</p>
<p>You can inspect the JSON file or delete it and Nix will prompt you again the
next time you run a <code>nix</code> command. And yes, saving this preference is
controversial considering this is Nix, but having the power to set <code>nixConfig</code>
attributes is really powerful, and with great powers comes great
responsibilities.</p>
</description>
<guid>https://github.com/thiagokokada/blog/blob/main/posts/2024-09-30/01-quick-bits-where-does-nix-store-flakes-trusted-settings.md</guid>
<pubDate>Mon, 30 Sep 2024 00:00:00 +0000</pubDate>
</item>
<item>
<title>My coding experience with LLM</title>
<link>https://github.com/thiagokokada/blog/blob/main/posts/2024-09-02/01-my-coding-experience-with-llm.md</link>
<description><p>I don't want to talk about AI. AI fatigue is real, with (garbage) AI content
everywhere you go. However, I like to talk about tools, and AIs, or more
specifically, <a href="https://en.wikipedia.org/wiki/Large_language_model">Large Language
Models</a> (LLM), can be
really useful tools. Maybe someday we will have our AI overlords dominating
everything (including my job), but right now I don't feel particularly
threatened given my experience with AIs.</p>
<p>Given that <a href="https://mashable.com/article/google-search-low-quality-research">Google Search is getting
worse</a>, I am
slowly relying in LLMs more and more. This is the focus of this post, basically
<a href="https://www.rottentomatoes.com/m/the_good_the_bad_and_the_ugly">the good, the bad and the
ugly</a> (by the
way, never saw this film) of LLMs, especially focused for solving coding
problems. I will take a look at the my chat history in
<a href="https://chatgpt.com/">ChatGPT</a> and see where it was a good tool, where it
shined, and where it just completely wasted my time.</p>
<p>A disclaimer, I never bothered to buy a ChatGPT subscription because I don't
feel that the tool is useful enough (at least yet), so most of the examples
here are either with <a href="https://openai.com/index/chatgpt/">GPT-3.5</a> or <a href="https://openai.com/index/gpt-4o-mini-advancing-cost-efficient-intelligence/">GPT-4o
mini</a>,
since those where the models offered by free in ChatGPT.</p>
<p>An addendum, I have some experience with <a href="https://github.com/features/copilot">GitHub
Copilot</a> (another LLM, but this one runs
inside your IDE or favorite text editor) in <code>$PREVIOUS_JOB</code>. But my experience
with it is even more underwhelming than what I am going to describe in this
post: it was basically really good for generating fixture for tests and between
ok-ish to plain bad for everything else. Yes, that's would fit in a tweet (or
xeet?).</p>
<h2>The good</h2>
<p><a href="https://chatgpt.com/share/c58a1432-2761-4954-acac-cade29942c34">Chat history</a>.</p>
<blockquote>
<p>Is there any way that I can pass a context for a net.conn to cancel its
Read()?</p>
</blockquote>
<p>This is one of the good cases where ChatGPT felt to me that I was talking with
an expert in Go. To start, I didn't even mention in my initial prompt that I
was talking about Go, but it correctly inferred so. Also, the solution overview
was great:</p>
<blockquote>
<ol>
<li><strong>Use a separate goroutine</strong> to handle the context cancellation.</li>
<li><strong>Set a deadline</strong> on the connection using <code>SetReadDeadline()</code> to unblock
the <code>Read()</code> when the context is canceled.</li>
</ol>
</blockquote>
<p>Before asking ChatGPT about this issue I had already coded my own naive
solution that was something similar to this:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span><span style="color:#66d9ef">func</span> <span style="color:#a6e22e">foo</span>(<span style="color:#a6e22e">ctx</span> <span style="color:#a6e22e">context</span>.<span style="color:#a6e22e">Context</span>) {
</span></span><span style="display:flex;"><span> <span style="color:#75715e">// ...
</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> <span style="color:#a6e22e">done</span> <span style="color:#f92672">:=</span> make(<span style="color:#66d9ef">chan</span> <span style="color:#66d9ef">struct</span>{})
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">var</span> <span style="color:#a6e22e">n</span> <span style="color:#66d9ef">int</span>
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">var</span> <span style="color:#a6e22e">err</span> <span style="color:#66d9ef">error</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">go</span> <span style="color:#66d9ef">func</span>() {
</span></span><span style="display:flex;"><span> <span style="color:#a6e22e">n</span>, <span style="color:#a6e22e">err</span> = <span style="color:#a6e22e">conn</span>.<span style="color:#a6e22e">Read</span>(<span style="color:#a6e22e">buf</span>)
</span></span><span style="display:flex;"><span> <span style="color:#a6e22e">done</span> <span style="color:#f92672">&lt;-</span><span style="color:#66d9ef">struct</span>{}{}
</span></span><span style="display:flex;"><span> }()
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">select</span> {
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">case</span> <span style="color:#f92672">&lt;-</span><span style="color:#a6e22e">ctx</span>.<span style="color:#a6e22e">Done</span>():
</span></span><span style="display:flex;"><span> <span style="color:#a6e22e">conn</span>.<span style="color:#a6e22e">Close</span>()
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">return</span> <span style="color:#ae81ff">0</span>, <span style="color:#a6e22e">ctx</span>.<span style="color:#a6e22e">Err</span>()
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">case</span> <span style="color:#f92672">&lt;-</span><span style="color:#a6e22e">done</span>:
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">return</span> <span style="color:#a6e22e">n</span>, <span style="color:#a6e22e">err</span>
</span></span><span style="display:flex;"><span> }
</span></span><span style="display:flex;"><span> <span style="color:#75715e">// ...
</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>}
</span></span></code></pre>
</body>
</html>
<p>Compared this solution with the one from ChatGPT:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span><span style="color:#66d9ef">func</span> <span style="color:#a6e22e">readWithContext</span>(<span style="color:#a6e22e">ctx</span> <span style="color:#a6e22e">context</span>.<span style="color:#a6e22e">Context</span>, <span style="color:#a6e22e">conn</span> <span style="color:#a6e22e">net</span>.<span style="color:#a6e22e">Conn</span>, <span style="color:#a6e22e">buf</span> []<span style="color:#66d9ef">byte</span>) (<span style="color:#66d9ef">int</span>, <span style="color:#66d9ef">error</span>) {
</span></span><span style="display:flex;"><span> <span style="color:#a6e22e">done</span> <span style="color:#f92672">:=</span> make(<span style="color:#66d9ef">chan</span> <span style="color:#66d9ef">struct</span>{})
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">var</span> <span style="color:#a6e22e">n</span> <span style="color:#66d9ef">int</span>
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">var</span> <span style="color:#a6e22e">err</span> <span style="color:#66d9ef">error</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#75715e">// Start a goroutine to perform the read
</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> <span style="color:#66d9ef">go</span> <span style="color:#66d9ef">func</span>() {
</span></span><span style="display:flex;"><span> <span style="color:#a6e22e">n</span>, <span style="color:#a6e22e">err</span> = <span style="color:#a6e22e">conn</span>.<span style="color:#a6e22e">Read</span>(<span style="color:#a6e22e">buf</span>)
</span></span><span style="display:flex;"><span> close(<span style="color:#a6e22e">done</span>)
</span></span><span style="display:flex;"><span> }()
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">select</span> {
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">case</span> <span style="color:#f92672">&lt;-</span><span style="color:#a6e22e">ctx</span>.<span style="color:#a6e22e">Done</span>(): <span style="color:#75715e">// Context was canceled or timed out
</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> <span style="color:#75715e">// Set a short deadline to unblock the Read()
</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> <span style="color:#a6e22e">conn</span>.<span style="color:#a6e22e">SetReadDeadline</span>(<span style="color:#a6e22e">time</span>.<span style="color:#a6e22e">Now</span>())
</span></span><span style="display:flex;"><span> <span style="color:#f92672">&lt;-</span><span style="color:#a6e22e">done</span> <span style="color:#75715e">// Wait for the read to finish
</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> <span style="color:#66d9ef">return</span> <span style="color:#ae81ff">0</span>, <span style="color:#a6e22e">ctx</span>.<span style="color:#a6e22e">Err</span>()
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">case</span> <span style="color:#f92672">&lt;-</span><span style="color:#a6e22e">done</span>: <span style="color:#75715e">// Read finished successfully
</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> <span style="color:#66d9ef">return</span> <span style="color:#a6e22e">n</span>, <span style="color:#a6e22e">err</span>
</span></span><span style="display:flex;"><span> }
</span></span><span style="display:flex;"><span>}
</span></span></code></pre>
</body>
</html>
<p>The ChatGPT version is much better because:</p>
<ol>
<li>It doesn't close the connection socket allowing it to be reused</li>
<li>It waits until the goroutine is finished to avoid leaks</li>
<li>It closes the channel after it finishes</li>
</ol>
<p>Now, I don't claim to be a Go expert (far from it), but this was one of those
cases where I felt that I was being guided by someone that is coding in Go for
years.</p>
<p>But it didn't end there, I asked:</p>
<blockquote>
<p>If I want to reuse the connection, do I need to reset the SetReadDeadline?</p>
</blockquote>
<p>Since in this particular case I wanted the underlying connection to be reused,
and ChatGPT correctly amended the code:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span><span style="color:#66d9ef">func</span> <span style="color:#a6e22e">readWithContext</span>(<span style="color:#a6e22e">ctx</span> <span style="color:#a6e22e">context</span>.<span style="color:#a6e22e">Context</span>, <span style="color:#a6e22e">conn</span> <span style="color:#a6e22e">net</span>.<span style="color:#a6e22e">Conn</span>, <span style="color:#a6e22e">buf</span> []<span style="color:#66d9ef">byte</span>) (<span style="color:#66d9ef">int</span>, <span style="color:#66d9ef">error</span>) {
</span></span><span style="display:flex;"><span> <span style="color:#75715e">// Same as before
</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> <span style="color:#66d9ef">select</span> {
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">case</span> <span style="color:#f92672">&lt;-</span><span style="color:#a6e22e">ctx</span>.<span style="color:#a6e22e">Done</span>(): <span style="color:#75715e">// Context was canceled or timed out
</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> <span style="color:#75715e">// Set a short deadline to unblock the Read()
</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> <span style="color:#a6e22e">conn</span>.<span style="color:#a6e22e">SetReadDeadline</span>(<span style="color:#a6e22e">time</span>.<span style="color:#a6e22e">Now</span>())
</span></span><span style="display:flex;"><span> <span style="color:#f92672">&lt;-</span><span style="color:#a6e22e">done</span> <span style="color:#75715e">// Wait for the read to finish
</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>
</span></span><span style="display:flex;"><span> <span style="color:#75715e">// Reset the deadline so the connection can be reused
</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> <span style="color:#a6e22e">conn</span>.<span style="color:#a6e22e">SetReadDeadline</span>(<span style="color:#a6e22e">time</span>.<span style="color:#a6e22e">Time</span>{})
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">return</span> <span style="color:#ae81ff">0</span>, <span style="color:#a6e22e">ctx</span>.<span style="color:#a6e22e">Err</span>()
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">case</span> <span style="color:#f92672">&lt;-</span><span style="color:#a6e22e">done</span>: <span style="color:#75715e">// Read finished successfully
</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> <span style="color:#66d9ef">return</span> <span style="color:#a6e22e">n</span>, <span style="color:#a6e22e">err</span>
</span></span><span style="display:flex;"><span> }
</span></span><span style="display:flex;"><span>}
</span></span></code></pre>
</body>
</html>
<p>Also the explanation on why this happens was good:</p>
<blockquote>
<p>When you set a read deadline with <code>conn.SetReadDeadline(time.Now())</code> to unblock
the <code>Read()</code> call, this deadline remains in effect for future <code>Read()</code>
operations. This means that if you attempt another read operation after this,
it could immediately fail with a timeout error if the deadline is still in
effect.
To reuse the connection, you'll need to remove the deadline or set a new one.
You can do this by calling <code>conn.SetReadDeadline(time.Time{})</code>, which clears
any existing deadline.</p>
</blockquote>
<p>I had one last question that I kind already knew the answer:</p>
<blockquote>
<p>Do we need to defer the clearing?</p>
</blockquote>
<p>And ChatGPT correctly returned:</p>
<blockquote>
<p>Whether you need to defer the clearing of the read deadline depends on how
your code is structured. If you want to ensure that the read deadline is
always cleared after a context cancellation, even if the code panics or exits
early, then deferring the <code>conn.SetReadDeadline(time.Time{})</code> call is a good
idea.</p>
</blockquote>
<p>This whole interaction was one of the best ones I ever had in ChatGPT, one that
it gave me an actual functional code with good explanation on what was
happening. But still, it didn't talked at all about the gotcha of setting the
read deadline and reusing the underlying connection, needing another prompt to
give me this information.</p>
<p>Also, there is one issue in ChatGPT solution though: we are discarding the
values of the buffer once the context is done by returning <code>n = 0</code> instead of
the number of bytes read. In this particular case this is what I wanted, but
this is a possible issue that ChatGPT didn't talk.</p>
<p>For comparison,
<a href="https://stackoverflow.com/questions/61637359/how-to-use-context-context-with-tcp-connection-read">this</a>
is the best that I got searching in both DuckDuckGo and Google, that is where I
kind based my first solution on. I still not sure where ChatGPT could give me
such a good answer for this question, because I just couldn't find anything
remotely similar during my searches.</p>
<h2>The amazing</h2>
<p><a href="https://chatgpt.com/share/0c9a148f-4cfd-4b4e-aa84-7486e185125b">Chat history</a>.</p>
<blockquote>
<p>I want to write an extension to <a href="https://github.com/yuin/goldmark/">https://github.com/yuin/goldmark/</a> that allows
me to rewrite <code>[link](../2024-07-30/post.md)</code> to
<code>[link](https://kokada.capivaras.dev/slug)</code></p>
</blockquote>
<p>Here is the generated code:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span><span style="color:#f92672">package</span> <span style="color:#a6e22e">linkrewriter</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#f92672">import</span> (
</span></span><span style="display:flex;"><span>	<span style="color:#e6db74">&#34;fmt&#34;</span>
</span></span><span style="display:flex;"><span>	<span style="color:#e6db74">&#34;path&#34;</span>
</span></span><span style="display:flex;"><span>	<span style="color:#e6db74">&#34;strings&#34;</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>	<span style="color:#e6db74">&#34;github.com/yuin/goldmark&#34;</span>
</span></span><span style="display:flex;"><span>	<span style="color:#e6db74">&#34;github.com/yuin/goldmark/ast&#34;</span>
</span></span><span style="display:flex;"><span>	<span style="color:#e6db74">&#34;github.com/yuin/goldmark/parser&#34;</span>
</span></span><span style="display:flex;"><span>	<span style="color:#e6db74">&#34;github.com/yuin/goldmark/renderer&#34;</span>
</span></span><span style="display:flex;"><span>	<span style="color:#e6db74">&#34;github.com/yuin/goldmark/util&#34;</span>
</span></span><span style="display:flex;"><span>)
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#75715e">// LinkRewriter is the main struct for your extension
</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span><span style="color:#66d9ef">type</span> <span style="color:#a6e22e">LinkRewriter</span> <span style="color:#66d9ef">struct</span>{}
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#75715e">// NewLinkRewriter returns a new instance of LinkRewriter
</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span><span style="color:#66d9ef">func</span> <span style="color:#a6e22e">NewLinkRewriter</span>() <span style="color:#f92672">*</span><span style="color:#a6e22e">LinkRewriter</span> {
</span></span><span style="display:flex;"><span>	<span style="color:#66d9ef">return</span> <span style="color:#f92672">&amp;</span><span style="color:#a6e22e">LinkRewriter</span>{}
</span></span><span style="display:flex;"><span>}
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#75715e">// Extend will be called by Goldmark to add your extension
</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span><span style="color:#66d9ef">func</span> (<span style="color:#a6e22e">e</span> <span style="color:#f92672">*</span><span style="color:#a6e22e">LinkRewriter</span>) <span style="color:#a6e22e">Extend</span>(<span style="color:#a6e22e">m</span> <span style="color:#a6e22e">goldmark</span>.<span style="color:#a6e22e">Markdown</span>) {
</span></span><span style="display:flex;"><span>	<span style="color:#a6e22e">m</span>.<span style="color:#a6e22e">Parser</span>().<span style="color:#a6e22e">AddOptions</span>(<span style="color:#a6e22e">parser</span>.<span style="color:#a6e22e">WithASTTransformers</span>(<span style="color:#a6e22e">util</span>.<span style="color:#a6e22e">Prioritized</span>(<span style="color:#a6e22e">e</span>, <span style="color:#ae81ff">0</span>)))
</span></span><span style="display:flex;"><span>}
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#75715e">// Transform is the method that modifies the AST
</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span><span style="color:#66d9ef">func</span> (<span style="color:#a6e22e">e</span> <span style="color:#f92672">*</span><span style="color:#a6e22e">LinkRewriter</span>) <span style="color:#a6e22e">Transform</span>(<span style="color:#a6e22e">node</span> <span style="color:#f92672">*</span><span style="color:#a6e22e">ast</span>.<span style="color:#a6e22e">Document</span>, <span style="color:#a6e22e">reader</span> <span style="color:#a6e22e">text</span>.<span style="color:#a6e22e">Reader</span>, <span style="color:#a6e22e">pc</span> <span style="color:#a6e22e">parser</span>.<span style="color:#a6e22e">Context</span>) {
</span></span><span style="display:flex;"><span>	<span style="color:#a6e22e">ast</span>.<span style="color:#a6e22e">Walk</span>(<span style="color:#a6e22e">node</span>, <span style="color:#66d9ef">func</span>(<span style="color:#a6e22e">n</span> <span style="color:#a6e22e">ast</span>.<span style="color:#a6e22e">Node</span>, <span style="color:#a6e22e">entering</span> <span style="color:#66d9ef">bool</span>) (<span style="color:#a6e22e">ast</span>.<span style="color:#a6e22e">WalkStatus</span>, <span style="color:#66d9ef">error</span>) {
</span></span><span style="display:flex;"><span>		<span style="color:#66d9ef">if</span> !<span style="color:#a6e22e">entering</span> {
</span></span><span style="display:flex;"><span>			<span style="color:#66d9ef">return</span> <span style="color:#a6e22e">ast</span>.<span style="color:#a6e22e">WalkContinue</span>, <span style="color:#66d9ef">nil</span>
</span></span><span style="display:flex;"><span>		}
</span></span><span style="display:flex;"><span>		<span style="color:#66d9ef">if</span> <span style="color:#a6e22e">link</span>, <span style="color:#a6e22e">ok</span> <span style="color:#f92672">:=</span> <span style="color:#a6e22e">n</span>.(<span style="color:#f92672">*</span><span style="color:#a6e22e">ast</span>.<span style="color:#a6e22e">Link</span>); <span style="color:#a6e22e">ok</span> {
</span></span><span style="display:flex;"><span>			<span style="color:#a6e22e">rewriteLink</span>(<span style="color:#a6e22e">link</span>)
</span></span><span style="display:flex;"><span>		}
</span></span><span style="display:flex;"><span>		<span style="color:#66d9ef">return</span> <span style="color:#a6e22e">ast</span>.<span style="color:#a6e22e">WalkContinue</span>, <span style="color:#66d9ef">nil</span>
</span></span><span style="display:flex;"><span>	})
</span></span><span style="display:flex;"><span>}
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#75715e">// rewriteLink modifies the link URL
</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span><span style="color:#66d9ef">func</span> <span style="color:#a6e22e">rewriteLink</span>(<span style="color:#a6e22e">link</span> <span style="color:#f92672">*</span><span style="color:#a6e22e">ast</span>.<span style="color:#a6e22e">Link</span>) {
</span></span><span style="display:flex;"><span>	<span style="color:#a6e22e">url</span> <span style="color:#f92672">:=</span> string(<span style="color:#a6e22e">link</span>.<span style="color:#a6e22e">Destination</span>)
</span></span><span style="display:flex;"><span>	<span style="color:#66d9ef">if</span> <span style="color:#a6e22e">strings</span>.<span style="color:#a6e22e">HasPrefix</span>(<span style="color:#a6e22e">url</span>, <span style="color:#e6db74">&#34;../&#34;</span>) {
</span></span><span style="display:flex;"><span>		<span style="color:#75715e">// Modify this part according to your slug generation logic
</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>		<span style="color:#a6e22e">slug</span> <span style="color:#f92672">:=</span> <span style="color:#a6e22e">generateSlug</span>(<span style="color:#a6e22e">url</span>)
</span></span><span style="display:flex;"><span>		<span style="color:#a6e22e">link</span>.<span style="color:#a6e22e">Destination</span> = []byte(<span style="color:#a6e22e">fmt</span>.<span style="color:#a6e22e">Sprintf</span>(<span style="color:#e6db74">&#34;https://kokada.capivaras.dev/%s&#34;</span>, <span style="color:#a6e22e">slug</span>))
</span></span><span style="display:flex;"><span>	}
</span></span><span style="display:flex;"><span>}
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#75715e">// generateSlug generates a slug from the given URL
</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span><span style="color:#66d9ef">func</span> <span style="color:#a6e22e">generateSlug</span>(<span style="color:#a6e22e">url</span> <span style="color:#66d9ef">string</span>) <span style="color:#66d9ef">string</span> {
</span></span><span style="display:flex;"><span>	<span style="color:#75715e">// Assuming the slug is the last part of the URL without the extension
</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>	<span style="color:#66d9ef">return</span> <span style="color:#a6e22e">strings</span>.<span style="color:#a6e22e">TrimSuffix</span>(<span style="color:#a6e22e">path</span>.<span style="color:#a6e22e">Base</span>(<span style="color:#a6e22e">url</span>), <span style="color:#a6e22e">path</span>.<span style="color:#a6e22e">Ext</span>(<span style="color:#a6e22e">url</span>))
</span></span><span style="display:flex;"><span>}
</span></span></code></pre>
</body>
</html>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span><span style="color:#f92672">package</span> <span style="color:#a6e22e">main</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#f92672">import</span> (
</span></span><span style="display:flex;"><span>	<span style="color:#e6db74">&#34;github.com/yuin/goldmark&#34;</span>
</span></span><span style="display:flex;"><span>	<span style="color:#e6db74">&#34;github.com/yuin/goldmark/renderer/html&#34;</span>
</span></span><span style="display:flex;"><span>	<span style="color:#a6e22e">linkrewriter</span> <span style="color:#e6db74">&#34;path/to/your/linkrewriter&#34;</span>
</span></span><span style="display:flex;"><span>)
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">func</span> <span style="color:#a6e22e">main</span>() {
</span></span><span style="display:flex;"><span>	<span style="color:#a6e22e">md</span> <span style="color:#f92672">:=</span> <span style="color:#a6e22e">goldmark</span>.<span style="color:#a6e22e">New</span>(
</span></span><span style="display:flex;"><span>		<span style="color:#a6e22e">goldmark</span>.<span style="color:#a6e22e">WithExtensions</span>(
</span></span><span style="display:flex;"><span>			<span style="color:#a6e22e">linkrewriter</span>.<span style="color:#a6e22e">NewLinkRewriter</span>(),
</span></span><span style="display:flex;"><span>		),
</span></span><span style="display:flex;"><span>		<span style="color:#a6e22e">goldmark</span>.<span style="color:#a6e22e">WithRendererOptions</span>(
</span></span><span style="display:flex;"><span>			<span style="color:#a6e22e">html</span>.<span style="color:#a6e22e">WithHardWraps</span>(),
</span></span><span style="display:flex;"><span>		),
</span></span><span style="display:flex;"><span>	)
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>	<span style="color:#75715e">// Use `md.Convert` to convert your markdown to HTML as usual
</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>}
</span></span></code></pre>
</body>
</html>
<p>This one will need some context. <a href="https://github.com/yuin/goldmark/">Goldmark</a>,
the Markdown renderer that <a href="https://github.com/thiagokokada/blog/blob/main/posts/2024-08-24/01-making-a-blog-for-the-next-10-years.md">powers this
blog</a>, does not
have good examples or documentation (at least I didn't found neither).</p>
<p>So I was not really hopeful I would get any answers in ChatGPT, but not only it
put me in the correct path, the code was almost correct. It did had some unused
and missing imports, and there were some compilation errors that were easily
fixed. And when it registered the extension, it also added a
<code>html.WithHardWraps()</code> renderer for no reason at all.</p>
<p>But this is one of the experiences in LLM that most impressed me. I had no idea
at all on how to solve this problem, and ChatGPT basically gave me the
solution. A <a href="https://github.com/thiagokokada/blog/blob/3a2ea58a0638d1005617d25eabeae0626933db46/link_rewriter.go">modified version of the
code</a>
from this chat is still used to power this blog to this day, that also
demonstrates that the code is not garbage (since now I understand what it does
and I can even extend it).</p>
<p>I am still not sure how the license situation of this code is though. Does
licenses even make sense in AI generated code?</p>
<h2>The bad</h2>
<p><a href="https://chatgpt.com/share/95fcd777-b03c-4cfe-8d1d-12e53f5a4dfe">Chat history</a>.</p>
<blockquote>
<p>I want to configure VM options inside IntelliJ by project</p>
</blockquote>
<blockquote>
<p>My project uses sbt, and I want to set this in a way that I can share with my
colleagues</p>
</blockquote>
<blockquote>
<p>I am using a build.sbt file, but IntelliJ is not loading it's JVM options and
I need to set this up manually. I want to automate this</p>
</blockquote>
<p>This is an example of my most common interaction with ChatGPT: it generally
gives some options that could work, but not in my specific case. So I start to
ask for more information until it eventually starts to
<a href="https://en.wikipedia.org/wiki/Hallucination_(artificial_intelligence)">hallucinate</a>.</p>
<p>In this particular case, the hallucination was in:</p>
<blockquote>
<p>The sbt-idea-plugin can generate IntelliJ IDEA project files based on your
SBT configuration, including JVM options. This allows you to ensure that
IntelliJ loads the correct settings from the SBT configuration.</p>
<p>Steps:</p>
<ol>
<li>Add the <code>sbt-idea-plugin</code> to your <code>project/plugins.sbt</code> file:</li>
</ol>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>addSbtPlugin<span style="color:#f92672">(</span><span style="color:#e6db74">&#34;com.github.mwz&#34;</span> <span style="color:#f92672">%</span> <span style="color:#e6db74">&#34;sbt-idea&#34;</span> <span style="color:#f92672">%</span> <span style="color:#e6db74">&#34;2.4.0&#34;</span><span style="color:#f92672">)</span>
</span></span></code></pre>
</body>
</html>
<ol start="2">
<li>Run the following command in your terminal to regenerate IntelliJ IDEA
project files:</li>
</ol>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>sbt gen-idea
</span></span></code></pre>
</body>
</html>
</blockquote>
<p>Yes, there is a
<a href="https://github.com/JetBrains/sbt-idea-plugin"><code>sbt-idea-plugin</code></a>, but no, it
doesn't generate project files based in my SBT configuration. It is not even
from the repository that it is supposed to be pointing too.</p>
<p>I will just not say that this interaction was a waste of time because some of
the other ideas can be applied, but this was the kind of case where I ended up
just being disappointed because the thing that seemed to be a possible solution
for my problem ended up not existing at all.</p>
<h2>The ugly</h2>
<p>There are lots of examples where ChatGPT just plain disappointed me. Here is a
few:</p>
<blockquote>
<p>What is the language that is a fork of Go that has extra features like map?</p>
</blockquote>
<p><a href="https://chatgpt.com/share/2b854bbd-d376-4049-97fe-b47321c1ef07">Chat history</a>.</p>
<p>This is one of those cases where I saw something once and was trying to
remember. ChatGPT just gave me lots of answers that were not a fork of Go
(<a href="https://vlang.io/">V</a>, <a href="https://github.com/gluon-lang/gluon">Gluon</a>,
<a href="https://www.ponylang.io">Pony</a>), hallucinated a programming language (Gomini),
and said that <a href="https://ziglang.org/">Zig</a> is focused in STEM.</p>
<p>By the way, I eventually remember what the language was:
<a href="https://goplus.org/">Go+</a>. To be fair with ChatGPT, I misremembered that the
Go+ had map (as the <a href="https://en.wikipedia.org/wiki/Map_(higher-order_function)">high-order
function</a>, not a
hashmap that both Go and Go+ have). But still, Go+ explicit announces that it
is a language for STEM everywhere in their website, and even them ChatGPT
couldn't answer.</p>
<blockquote>
<p>In IntelliJ, I need to pass some VM options to the run configuration for my
application to run tests, e.g.:</p>
<pre><code>--add-exports=java.base/sun.security.x509=ALL-UNNAMED
--add-opens=java.base/sun.security.ssl=ALL-UNNAMED
-Dconfig.resource=api.test.conf
</code></pre>
<p>However, the <code>api.test.conf</code> may be <code>jobs.api.conf</code> or something else
depending on the subproject I am running inside Idea. Can I parametrize this
in some way?</p>
</blockquote>
<p><a href="https://chatgpt.com/share/aa9bd051-65de-49d9-8d56-0e28de94722c">Chat history</a></p>
<p>So quick answer: you can't. IntelliJ macros are not powerful enough for this.
ChatGPT itself said so when I asked about the IntelliJ macros
<a href="https://chatgpt.com/share/7a057bf5-bcec-458b-9432-98e3f551ad28">directly</a>.</p>
<p>But in this particular case, ChatGPT happily said &quot;yes, you can parameterize VM
options in IntelliJ IDEA to make them more flexible depending on the subproject
or other context&quot;. Thanks for wasting my time, I guess.</p>
<h2>Conclusion</h2>
<p>I do think that LLMs, in particular ChatGPT since this is the one I have more
experience with, are improving. But still, if there is anything that I
concluded after putting all those experiences in a post is that you still need
someone that knows at least as much about whatever you're asking ChatGPT to
actually validate the answer.</p>
<p>Not only because LLMs hallucinates a lot, but also because even in the &quot;good&quot;
example the LLM needed extra input to figure out about the possible drawback of
the solution. The only case where I felt that ChatGPT outsmarted me was the
&quot;amazing&quot; case, but this was literally the only example that I could find in
all my chat history with ChatGPT that was like this.</p>
<p>So yes, at least for now I still don't think AI will be our overlords. Maybe
someday.</p>
</description>
<guid>https://github.com/thiagokokada/blog/blob/main/posts/2024-09-02/01-my-coding-experience-with-llm.md</guid>
<pubDate>Mon, 02 Sep 2024 00:00:00 +0000</pubDate>
</item>
<item>
<title>Making a blog for the next 10 years</title>
<link>https://github.com/thiagokokada/blog/blob/main/posts/2024-08-24/01-making-a-blog-for-the-next-10-years.md</link>
<description><p>So one thing that I realise after starting writing this blog is that I care
more about it than some of my other projects. For some reason or another I want
to make sure that this blog will continue with me for a long time. This is one
of the reasons why I use <a href="https://github.com/thiagokokada/blog/blob/main/posts/2024-07-26/02-using-github-as-a-bad-blog-platform.md">GitHub as mirror
blog</a> and why I
created a <a href="https://github.com/thiagokokada/blog/blob/main/posts/2024-07-29/01-quick-bits-why-you-should-automate-everything.md">bunch of
automation</a>
to make sure I never forget how to maintain this blog.</p>
<p>Still, there are a bunch of dependencies that I need to make sure they're
working so I can publish blog posts:</p>
<ul>
<li>Markdown</li>
<li>A text editor</li>
<li>Hosting and blog platform</li>
<li>Shell script and
<a href="https://github.com/thiagokokada/blog/blob/main/posts/2024-08-13/01-abusing-makefiles-for-fun-and-profit.md">Makefile</a></li>
<li>Go toolchain and modules</li>
</ul>
<p>Let's start with the most important one: the texts themselves, they're all
written in <a href="https://www.markdownguide.org/">Markdown</a>. The nice thing about
Markdown is that it is just plain text files with some special notation that
allow you to format text, but the text itself is still legible even if you have
no support to render Markdowns. So it means as long as I can read a plain text
file, I can still read the posts, no issue there. Closely related issue here is
the text editor, but the nice thing about Markdown is that if tomorrow I decide
to change from <code>neovim</code> to, say, <a href="https://zed.dev/">Zed</a>, I can still write it
Markdown files without issue. I also use a standardised Markdown implementation
called <a href="https://commonmark.org/">CommonMark</a>, that fixes a bunch of ambiguities
of the original Markdown spec.</p>
<p>The next point is the host (<a href="capivaras.dev">capivaras.dev</a>) and the blog
platform (<a href="https://github.com/mataroa-blog/mataroa">Mataroa</a>). One of the nice
things about Mataroa is that it avoids platform lock-in by having multiple ways
to <a href="https://mataroa.blog/export/">export your posts</a>. I could export everything
to <a href="https://gohugo.io/">Hugo</a>, for example, and serve the blog from <a href="https://pages.github.com/">GitHub
Pages</a>.</p>
<p>This is especially nice consider how small <a href="capivaras.dev">capivaras.dev</a> is,
so it is good to know that if I eventually have issues I could move to
somewhere else. I also have an e-mail backup every month with all posts made
(another <a href="https://hey.mataroa.blog/blog/monthly-auto-exports-via-email/">Mataroa
feature</a>), and
of course I have a <a href="https://github.com/thiagokokada/blog">Git repo</a> that also
acts as a <a href="https://github.com/thiagokokada/blog/blob/main/posts/2024-07-26/02-using-github-as-a-bad-blog-platform.md">mirror of this
blog</a>. So I would
say the chance of losing access to the content is slim.</p>
<p>One other issue is the URL of the posts that are indexed in many different
places, like <a href="https://news.ycombinator.com/">Hacker News</a>,
<a href="https://lobste.rs/">Lobte.rs</a>, etc. This is why I finally decide to bite the
bullet and purchase a proper domain, and this website should now be available
in <a href="kokada.dev">kokada.dev</a>. This means that in my eventual exit from
<a href="capivaras.dev">capivaras.dev</a>, I can just point my new blog location to my own
domain if needed (it is not as easy since I also need to preserve the post
URLs, but shouldn't be difficult to fix this if I ever need to do so).</p>
<p>Now for the tools that I use to publish from the original Markdown files to
everything else. Let's start with shell script(s) and Makefile: I decided that
they're less of an issue if they eventually stop working: they're only used to
make my life easier, but I can still publish files manually if needed. Still, I
tried to rewrite both the
<a href="https://github.com/thiagokokada/blog/commit/a0d421ca90f3da059998295c5e3c6c7a6a3f0688">shell</a>
and
<a href="https://github.com/thiagokokada/blog/commit/074580065b21fbdaf930aa51968e69f015d49505">Makefile</a>
to avoid GNUisms, so in the eventual case that I decide to e.g.: stop using a
GNU/Linux system like NixOS and use a *BSD system instead, I am covered.</p>
<p>In other projects I would probably add a <code>shell.nix</code> file and add the GNU
versions of the tools that I am using, but I am trying to avoid Nix while
building this blog as much as I can, mainly because it is not available
everywhere (e.g.: *BSD systems).</p>
<p>Go is the more important part: the tooling used to <a href="https://github.com/thiagokokada/blog/blob/main/posts/2024-07-29/01-quick-bits-why-you-should-automate-everything.md">publish this blog is
written in
Go</a>. Go
is a good language when you want to ensure that things will work for a long
time because of its <a href="https://go.dev/blog/compat">backwards compatibility
guarantee</a>. Also I don't expect Google dropping Go
development soon, but even if this happen (<a href="https://killedbygoogle.com/">&quot;killed by
Google&quot;</a> is a thing after all), it is very likely
some other group or company would adopt its development quickly, considering
<a href="https://www.tiobe.com/tiobe-index/go/">how popular the language</a> is.</p>
<p>However, the <a href="https://github.com/thiagokokada/blog/blob/main/go.mod">Go
modules</a> that I depend
are another story:</p>
<ul>
<li><a href="https://github.com/elliotchance/orderedmap/">elliotchance/orderedmap</a>: an
ordered map implementation that I use until Go adds it in the <a href="https://github.com/thiagokokada/blog/blob/main/posts/2024-08-17/01-an-unordered-list-of-things-i-miss-in-go.md">standard
library</a></li>
<li><a href="https://github.com/gorilla/feeds">gorilla/feeds</a>: a RSS generator library</li>
<li><a href="https://github.com/gosimple/slug">gosimple/slug</a>: a
<a href="https://developer.mozilla.org/en-US/docs/Glossary/Slug">slug</a> generator
library</li>
<li><a href="https://github.com/yuin/goldmark">yuin/goldmark</a>: a CommonMark parser and
renderer</li>
<li><a href="https://github.com/teekennedy/goldmark-markdown">teekennedy/goldmark-markdown</a>:
a renderer for Goldmark to render back to Markdown (since Goldmark itself
doesn't have this capacity)</li>
</ul>
<p>In common for all those modules are that they're clearly small projects
maintained mostly by one developer. They're all very good, don't get me wrong,
but they're still an reliability issue in the future. There is no guarantee
those repositories will not be deleted tomorrow, for example.</p>
<p>Yes, <a href="https://proxy.golang.org/">Go Proxy</a> exist, but from what I understood
reading its page is that while it caches modules contents, this is not
guarantee:</p>
<blockquote>
<p>proxy.golang.org does not save all modules forever. There are a number of
reasons for this, but one reason is if proxy.golang.org is not able to detect
a suitable license. In this case, only a temporarily cached copy of the
module will be made available, and may become unavailable if it is removed
from the original source and becomes outdated. The checksums will still
remain in the checksum database regardless of whether or not they have become
unavailable in the mirror.</p>
</blockquote>
<p>This is why this is the first project that made sense to me to use <a href="https://go.dev/ref/mod#go-mod-vendor"><code>go mod vendor</code></a>. Now I have a copy of the source
code of all modules inside the
<a href="https://github.com/thiagokokada/blog/tree/0b97630d6b30551ffe05b5d8124305b1065f729d/vendor">vendor</a>
directory in the repository, avoiding the risks I commented above. This allows
me to ensure that this blog will still be publishable in the future, as long as
I have a working Go toolchain (and Go toolchain makes this
<a href="https://github.com/thiagokokada/blog/blob/main/posts/2024-08-19/01-quick-bits-go-automatically-downloads-a-newer-toolchain-if-needed.md">easy</a>).</p>
<p>There are a few other things that can bitrot this blog, for example links going
nowhere. I always try to use
<a href="https://en.wikipedia.org/wiki/Permalink">permalinks</a> when it makes sense, but
the only actual way to ensure those links would work in the future would be to
point them to <a href="https://archive.org/">archive.org</a> (but even archive.org may not
exist forever). Maybe something to fix in the future, hope not in the far
future, before things start to break.</p>
</description>
<guid>https://github.com/thiagokokada/blog/blob/main/posts/2024-08-24/01-making-a-blog-for-the-next-10-years.md</guid>
<pubDate>Sat, 24 Aug 2024 00:00:00 +0000</pubDate>
</item>
<item>
<title>An unordered list of hidden gems inside NixOS</title>
<link>https://github.com/thiagokokada/blog/blob/main/posts/2024-08-21/01-an-unordered-list-of-hidden-gems-inside-nixos.md</link>
<description><p>After using <a href="https://nixos.org/">NixOS</a> for the last 5+ years as my main OS, I
end up with a <a href="https://github.com/thiagokokada/nix-configs/">configuration</a>
with many things that are interesting for one reason or another, but it is not
listed anywhere (well, except if you are the kind of person that reads <code>man 5 configuration.nix</code> or the release notes in every release).</p>
<p>So kind in the same spirit as my <a href="https://github.com/thiagokokada/blog/blob/main/posts/2024-08-17/01-an-unordered-list-of-things-i-miss-in-go.md">list of things that I miss in
Go</a>, here is
a list of modules that I find neat in NixOS and are not default already. Again,
the list is unordered since this makes it easier to update in the future if I
find something else, but also I don't want to think too hard about an order
here.</p>
<p>With all above, let's start.</p>
<h2><a href="https://github.com/NixOS/nixpkgs/blob/6afb255d976f85f3359e4929abd6f5149c323a02/nixos/modules/services/networking/nftables.nix"><code>networking.nftables</code></a></h2>
<p><a href="https://www.nftables.org/">nftables</a> is, accordingly to Wikipedia:</p>
<blockquote>
<p>nftables is a subsystem of the Linux kernel providing filtering and
classification of network packets/datagrams/frames.</p>
</blockquote>
<p>It is basically a replacement of the venerable
<a href="https://en.wikipedia.org/wiki/Iptables">iptables</a>, that still exist and is the
default program to configure the famous <code>networking.firewall</code>, the declarative
<a href="https://wiki.nixos.org/wiki/Firewall">Firewall</a> that NixOS enable by default.</p>
<p>To enable, it is simple, just add to your configuration:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>{
</span></span><span style="display:flex;"><span> networking<span style="color:#f92672">.</span>nftables<span style="color:#f92672">.</span>enable <span style="color:#f92672">=</span> <span style="color:#66d9ef">true</span>;
</span></span><span style="display:flex;"><span>}
</span></span></code></pre>
</body>
</html>
<p>And thanks to the fact that NixOS's Firewall is declarative, everything should
still work as expect: any service that you set <code>openFirewall = true</code> will still
have its ports open, if you set <code>networking.firewall.allowPing = false</code> it will
still disable pings like before, etc.</p>
<p>If you look at the documentation of the above option, you will find the
following warning:</p>
<blockquote>
<p>Note that if you have Docker enabled you will not be able to use nftables
without intervention. Docker uses iptables internally to setup NAT for
containers. This module disables the ip_tables kernel module, however Docker
automatically loads the module. Please see
<a href="https://github.com/NixOS/nixpkgs/issues/24318#issuecomment-289216273">https://github.com/NixOS/nixpkgs/issues/24318#issuecomment-289216273</a> for
more information.</p>
</blockquote>
<p>I don't use Docker (switched to Podman instead for quite a long time), so I
don't know how bad the situation is. Also keep in mind that <code>nftables</code> does
offer <code>iptables-compat</code> for compatibility with old iptables scripts, so it is
most likely Docker doing something weird here.</p>
<p>Now, the actual advantage from the user here is not clear: the main advantage
from my point of view (and the reason I used to use in other distros like Arch)
is the improved syntax, however if you are using the declarative NixOS's
Firewall you are not interacting with either <code>iptables</code> or <code>nftables</code> directly
anyway. <code>nftables</code> is supposed to be more efficient, but not sure most users
will care about this.</p>
<p>However if you are the kind of person that needs custom rules, switching to
<code>nftables</code> does bring a few benefits, including
<code>networking.nftables.checkRuleset</code> (enabled by default), that checks if your
ruleset has syntax errors during build time. Really valuable to avoid issues
only after switch.</p>
<p>Anyway, this is one of those options that I think it should be the default for
a long time, since most of the new development in NixOS firewall seems to be
focusing <code>nftables</code> for a while.</p>
<h2><a href="https://github.com/NixOS/nixpkgs/blob/877d19523edcac81b167e8fd716ad2658da2adca/nixos/modules/system/activation/switchable-system.nix#L30-L38"><code>system.switch.enableNg</code></a></h2>
<p><a href="https://github.com/NixOS/nixpkgs/pull/308801">This one</a> I just discovered
today, but it has been available for a while (~2 months if you're using
<code>nixos-unstable</code>). Finally someone is rewriting
<a href="https://github.com/NixOS/nixpkgs/blob/b1eff03c35aa7c90ab3a4d9f6ef297dae5fba37b/nixos/modules/system/activation/switch-to-configuration.pl"><code>switch-to-configuration.pl</code></a>,
the Perl script that is called everytime you run <code>nixos-rebuild switch</code>.</p>
<p>Now, I am not one of those &quot;rewrite in Rust&quot; zealots, but in this case this is
definitely worth it: <code>switch-to-configuration.pl</code> is one of those pieces of
code in NixOS that most people avoid touching at the fear of breaking
something. There is a reason why
<a href="https://github.com/NixOS/nixpkgs/commit/eeb2588a59c938042b74183ce1da7052a6ef7e59"><code>nixos-rebuild</code></a>
is as convoluted as it is, because even if it is a messy shell script, most
people preferred to workaround issues from the <code>switch-to-configuration.pl</code>
inside it than trying to understand the mess that <code>switch-to-configuration.pl</code>
is.</p>
<p>Trying this one is easy:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>{
</span></span><span style="display:flex;"><span> system<span style="color:#f92672">.</span>switch <span style="color:#f92672">=</span> {
</span></span><span style="display:flex;"><span> enable <span style="color:#f92672">=</span> <span style="color:#66d9ef">false</span>;
</span></span><span style="display:flex;"><span> enableNg <span style="color:#f92672">=</span> <span style="color:#66d9ef">true</span>;
</span></span><span style="display:flex;"><span> };
</span></span><span style="display:flex;"><span>}
</span></span></code></pre>
</body>
</html>
<p>Yes, you need to explicit set <code>system.switch.enable = false</code>, since the default
is <code>true</code>.</p>
<p>By the way, what is the reason you would want to set <code>system.switch.enable = false</code> before the <code>enableNg</code> appeared you ask? For systems that are immutable
and updated by e.g.: image upgrades instead of modifying root.</p>
<p>Enabling <code>switch-to-configuration-ng</code> right now is mostly for testing purposes,
but one of the advantages that I saw is that system switches are (slightly)
faster:</p>
<pre><code>$ hyperfine &quot;sudo nixos-rebuild switch&quot; # switch-to-configuration.pl
Benchmark 1: sudo nixos-rebuild switch
 Time (mean ± σ): 3.576 s ± 0.035 s [User: 0.004 s, System: 0.014 s]
 Range (min … max): 3.522 s … 3.645 s 10 runs

$ hyperfine &quot;sudo nixos-rebuild switch&quot; # switch-to-configuration-ng
Benchmark 1: sudo nixos-rebuild switch
 Time (mean ± σ): 3.394 s ± 0.080 s [User: 0.004 s, System: 0.013 s]
 Range (min … max): 3.325 s … 3.608 s 10 runs
</code></pre>
<p>But yes, the difference is not enough to make a significant impact, and it is
not the objective anyway. The real reason for the rewrite is to make it easier
to collaborate. I hope one day we also have someone brave enough to rewrite the
<code>nixos-rebuild</code> script in something saner.</p>
<h2><a href="https://github.com/NixOS/nixpkgs/blob/cce9aef6fd8f010d288d685b9d2a38f3b6ac47e9/nixos/modules/system/boot/systemd/initrd.nix"><code>boot.initrd.systemd</code></a></h2>
<p>A quick recap on how a modern Linux distro generally boots: the first thing
that the bootloader (say <a href="https://www.gnu.org/software/grub/">GRUB</a> or
<a href="https://systemd.io/BOOT/">systemd-boot</a>) loads is <code>initrd</code> (<em>initial
ramdisk</em>), a small image that runs from RAM and includes the Linux kernel and
some utilities that are responsible for setting up the main system. For
example, one of the responsibilities of the <code>initrd</code> is to mount the disks and
start init system (<code>systemd</code>).</p>
<p>It may surprising that this <code>initrd</code> image does <strong>not</strong> generally include
<code>systemd</code>. Traditionally <code>initrd</code> is composed by a bunch of shell scripts and a
minimal runtime (e.g.: <a href="https://www.busybox.net/">busybox</a>), however <code>systemd</code>
can also do this job since a long time ago. It is just the role of the distros
to integrate <code>systemd</code> inside the <code>initrd</code>.</p>
<p>This is what <code>boot.initrd.systemd</code> does: enable <code>systemd</code> inside the <code>initrd</code>.
It make a few subtle changes:</p>
<ul>
<li>If you are using <a href="https://wiki.nixos.org/wiki/Full_Disk_Encryption">Full Disk Encryption via
LUKS</a>, you will get a
different password prompt at login</li>
<li>You will get <code>initrd</code> time information if using <code>systemd-analyze</code> to measure
boot time
<ul>
<li>You can get even more information (bootloader) if you also use
<code>systemd-boot</code></li>
</ul>
</li>
<li>You will also get <code>systemd</code> style status about services during <code>initrd</code> (not
only afterwards)</li>
</ul>
<p>But I think the main reason is that since <code>systemd</code> is event-driven, it should
make boot more reliable, especially in challenging situations (like booting
from network). I can't say that I have any system like this to test if it is
actually more reliable or not, but I don't remember having any issues since I
set <code>boot.initrd.systemd.enable = true</code>, so there is that.</p>
<h2><a href="https://github.com/NixOS/nixpkgs/blob/b4a09f1f9d1599478afadffa782a02690550447c/pkgs/development/libraries/pipewire/default.nix"><code>services.pipewire</code></a></h2>
<p>If there is something in that list that has a good chance that you're using
already, it is this one, especially if you're using
<a href="https://wayland.freedesktop.org/">Wayland</a>. Still, I think it is interesting
to include in this list since <a href="https://www.pipewire.org/">PipeWire</a> is great.</p>
<p>The experience with PipeWire until now for me was seamless: I never had any
issues with it, all my applications still work exactly as it always worked. I
also didn't had any issues with
<a href="https://www.freedesktop.org/wiki/Software/PulseAudio/">PulseAudio</a> for a
while, but I still remember when I first tried PulseAudio during the 0.x in
Fedora and having tons of issues. So bonus points for PipeWire developers for
polishing the experience of enough that most people will feel no difference.</p>
<p>To enable PipeWire, I would recommend:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>{
</span></span><span style="display:flex;"><span> services<span style="color:#f92672">.</span>pipewire <span style="color:#f92672">=</span> {
</span></span><span style="display:flex;"><span> enable <span style="color:#f92672">=</span> <span style="color:#66d9ef">true</span>;
</span></span><span style="display:flex;"><span> alsa<span style="color:#f92672">.</span>enable <span style="color:#f92672">=</span> <span style="color:#66d9ef">true</span>;
</span></span><span style="display:flex;"><span> pulse<span style="color:#f92672">.</span>enable <span style="color:#f92672">=</span> <span style="color:#66d9ef">true</span>;
</span></span><span style="display:flex;"><span> <span style="color:#75715e"># jack.enable = true;</span>
</span></span><span style="display:flex;"><span> };
</span></span><span style="display:flex;"><span> security<span style="color:#f92672">.</span>rtkit<span style="color:#f92672">.</span>enable <span style="color:#f92672">=</span> <span style="color:#66d9ef">true</span>;
</span></span><span style="display:flex;"><span>}
</span></span></code></pre>
</body>
</html>
<p>This enables both ALSA and PulseAudio emulation support in PipeWire for maximum
compatibility with desktop applications (you can also enable
<a href="https://jackaudio.org/"><code>jack</code></a> if you use professional audio applications).
It also enables <a href="https://github.com/heftig/rtkit"><code>rtkit</code></a>, allowing PipeWire
to get (soft) realtime, helping avoiding cracks during high CPU load.</p>
<p>I also recommend taking a look at the <a href="https://wiki.nixos.org/wiki/PipeWire">Wiki
article</a>, that has multiple interesting
configurations that can be added for low-latency setups or improved codecs for
Bluetooth devices.</p>
<h2><a href="https://github.com/NixOS/nixpkgs/blob/c9ec8289781a3c4ac4dd5c42c8d50dd65360e79c/nixos/modules/services/networking/networkmanager.nix#L264-L271"><code>networking.networkmanager.wifi.backend = &quot;iwd&quot;</code></a></h2>
<p>There is a good change that you're using
<a href="https://www.networkmanager.dev/"><code>NetworkManager</code></a> to manage network,
especially for Wi-Fi. And if that is the case, I can't recommend enough
changing the backend from the default <code>wpa_supplicant</code> to
<a href="https://iwd.wiki.kernel.org/"><code>iwd</code></a>.</p>
<p>If you think that your Wi-Fi takes a long time to connect/re-connect, it may be
because <code>wpa_supplicant</code>. <code>iwd</code> seems much more optimised in this regard, and
since switching to it I never felt that my Wi-Fi was worse than other OSes (and
generally slightly better than Windows, but keep in mind that this is a
non-scientific comparison).</p>
<p>Not saying that I never had Wi-Fi issues since switching to <code>iwd</code>, however
switching back to <code>wpa_supplicant</code> in those cases never fixed the issue (it was
the same or worse), so I assume either bad hardware or drivers in those cases.</p>
<h2><a href="https://github.com/NixOS/nixpkgs/blob/d5badef1e0416160298aad6dd8604eaf8b6e7e52/nixos/modules/system/boot/tmp.nix#L25-L32"><code>boot.tmp.useTmpfs</code></a></h2>
<p><em>Added in 2024-08-22</em></p>
<p>Most Linux distro nowadays mount <code>/tmp</code> inside
<a href="https://en.wikipedia.org/wiki/Tmpfs">tmpfs</a>, storing the files inside RAM (and
making them actually temporary, e.g.: does not persist between reboots). There
is a <a href="https://lwn.net/Articles/499410/">long discussion</a> whether this makes
sense or not, but it is a fact that using <code>/tmp</code> as an storage for small files
generally makes sense.</p>
<p>However, NixOS still stores <code>/tmp</code> inside <code>/</code> by default. The main reason for
this is because Nix, by default, still builds everything in <code>TMPDIR</code> (that
defaults to <code>/tmp</code> if not set). To fix the situation you can do:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>{
</span></span><span style="display:flex;"><span> boot<span style="color:#f92672">.</span>tmp<span style="color:#f92672">.</span>useTmpfs <span style="color:#f92672">=</span> <span style="color:#66d9ef">true</span>;
</span></span><span style="display:flex;"><span> systemd<span style="color:#f92672">.</span>services<span style="color:#f92672">.</span>nix-daemon <span style="color:#f92672">=</span> {
</span></span><span style="display:flex;"><span> environment<span style="color:#f92672">.</span>TMPDIR <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;/var/tmp&#34;</span>;
</span></span><span style="display:flex;"><span> };
</span></span><span style="display:flex;"><span>}
</span></span></code></pre>
</body>
</html>
<p>This ensures that Nix will build in <code>/var/tmp</code>, that is the temporary directory
for large files.</p>
<p>If you don't want to use <code>/tmp</code> inside tmpfs for some reason, I recommend at
least setting <code>boot.tmp.cleanOnBoot = true</code>. This is to avoid issues with e.g.:
mistakenly writing a program that depends in <code>/tmp</code> being persistent, because
this is the behavior that most other distros expects.</p>
<h2><a href="https://github.com/NixOS/nixpkgs/blob/68142254d20256e617f596a5d50f9950f98fa36a/nixos/modules/config/zram.nix"><code>zramSwap</code></a></h2>
<p><em>Added in 2024-08-22</em></p>
<p>Have a system with small amounts of RAM? <del>You can download more RAM for free,
just click this <a href="https://downloadmoreram.com/">link</a>.</del> Well, just kidding, but
you can actually improve RAM usage by compressing it. This is what
<a href="https://en.wikipedia.org/wiki/Zram"><code>zram</code></a> does: it will create a block
device using part of your RAM that will be compressed.</p>
<p>While this means that whatever is in the compressed part can't be used as-is
(it needs to be decompressed first), it means you can store a lot more
information. And since RAM is much faster than disk, this generally is a good
trade-off. It is still recommended that you have some swap space though, e.g.:
to swap pages that are unused.</p>
<p>To enable it, you can do:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>{
</span></span><span style="display:flex;"><span> zramSwap <span style="color:#f92672">=</span> {
</span></span><span style="display:flex;"><span> enable <span style="color:#f92672">=</span> <span style="color:#66d9ef">true</span>;
</span></span><span style="display:flex;"><span> algorithm <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;zstd&#34;</span>;
</span></span><span style="display:flex;"><span> };
</span></span><span style="display:flex;"><span>}
</span></span></code></pre>
</body>
</html>
<p>You can play a little with the <code>zramSwap.algorithm</code> parameter, but I recommend
either <code>lz4</code> or <code>zstd</code>. It is important for the compression algorithm to be
fast, since you probably want to minimise CPU usage. Well, unless you have a
very good CPU and small amounts of RAM, them it may make sense to use an
algorithm that compress better but uses more CPU.</p>
<h2><a href="https://github.com/NixOS/nixpkgs/blob/5356420466c4d7901b63acc5e337c5bf30573f8a/nixos/modules/services/misc/fstrim.nix"><code>services.fstrim</code></a></h2>
<p><em>Added in 2024-08-22</em></p>
<p>Do you have a SSD? If so, you probably want to
<a href="https://en.wikipedia.org/wiki/Trim_(computing)">trim</a> it regularly to keep a
good performance. This should be as easy as:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>{
</span></span><span style="display:flex;"><span> services<span style="color:#f92672">.</span>fstrim<span style="color:#f92672">.</span>enable <span style="color:#f92672">=</span> <span style="color:#66d9ef">true</span>;
</span></span><span style="display:flex;"><span>}
</span></span></code></pre>
</body>
</html>
<p>One caveat though: if you are using Full Disk Encryption with LUKS, you also
need to enable <code>boot.initrd.luks.devices.&lt;name&gt;.allowDiscards</code>, otherwise
<code>fstrim</code> will not work. The reason that this is not enabled by default is
because there are some <a href="https://asalor.blogspot.com/2011/08/trim-dm-crypt-problems.html">security
implications</a>.
It shouldn't be an issue for most people, but if you are paranoid please assess
your risks first.</p>
<h2><a href="https://github.com/NixOS/nixpkgs/blob/e83fc891118e5e5660ccca000040e33b0a0a3080/nixos/modules/system/boot/binfmt.nix"><code>boot.binfmt.emulatedSystems</code></a></h2>
<p><em>Added in 2024-08-22</em></p>
<p>This is one of my favorites. Do you want to have the power to run binaries from
other architectures like
<a href="https://en.wikipedia.org/wiki/ARM_architecture_family">ARM</a> or
<a href="https://en.wikipedia.org/wiki/RISC-V">RISC-V</a>? In NixOS it is easy:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>{
</span></span><span style="display:flex;"><span> boot<span style="color:#f92672">.</span>binfmt<span style="color:#f92672">.</span>emulatedSystems <span style="color:#f92672">=</span> [ <span style="color:#e6db74">&#34;aarch64-linux&#34;</span> <span style="color:#e6db74">&#34;riscv64-linux&#34;</span> ];
</span></span><span style="display:flex;"><span>}
</span></span></code></pre>
</body>
</html>
<p>This will install a QEMU emulator for the target architectures, and configure
<a href="https://en.wikipedia.org/wiki/Binfmt_misc"><code>binfmt</code></a> so that the kernel will
automatically detect when a binary from another architecture is run,
running it with the QEMU emulator instead.</p>
<p>Now, keep in mind that you still need e.g.: libraries for the target
architecture. However this is where the Nix magic comes in, you can easily
compile something to another architecture as:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>$ uname -a
</span></span><span style="display:flex;"><span>Linux sankyuu-nixos 6.10.3 #1-NixOS SMP PREEMPT_DYNAMIC Sat Aug 3 07:01:09 UTC 2024 x86_64 GNU/Linux
</span></span><span style="display:flex;"><span>$ cd nixpkgs
</span></span><span style="display:flex;"><span>$ nix build .#hello --system aarch64-linux
</span></span><span style="display:flex;"><span>$ file ./result/bin/hello
</span></span><span style="display:flex;"><span>./result/bin/hello: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /nix/store/kz7xglxzhad64v667wwpn8vrxhjwcbna-glibc-2.39-52/lib/ld-linux-aarch64.so.1, for GNU/Linux 3.10.0, not stripped
</span></span><span style="display:flex;"><span>$ ./result/bin/hello
</span></span><span style="display:flex;"><span>Hello, world!
</span></span></code></pre>
</body>
</html>
<p>And since the binary will be linked with the libraries from the target
architecture, everything works as expect.</p>
<p>You also need to temper your expectations: QEMU emulation is slow. It is
sufficient fast to build small programs, but if you need to build something
bigger, I would recommend you to do it via <a href="https://github.com/thiagokokada/blog/blob/main/posts/2024-08-11/01-building-static-binaries-in-nix.md">cross-compilation
instead</a>.</p>
<h2><a href="https://github.com/NixOS/nixpkgs/blob/d9cf6ea6f809b862b73766705bdadb1236eeeff8/nixos/modules/services/system/dbus.nix#L40-L50"><code>services.dbus.implementation = &quot;broker&quot;</code></a></h2>
<p><em>Added in 2024-08-22</em></p>
<p>Someone in
<a href="https://lobste.rs/s/ymmale/unordered_list_hidden_gems_inside_nixos#c_dz1hck">Lobste.rs</a>
remembered me of this gem: <a href="https://github.com/bus1/dbus-broker">dbus-broker</a>,
a high performance implementation of
<a href="https://en.wikipedia.org/wiki/D-Bus">D-Bus</a>.</p>
<p>For those who doesn't know, D-Bus is the <a href="https://en.wikipedia.org/wiki/Inter-process_communication">Inter-Process
Communication</a>
protocol used mostly in Linux desktop. And distros like
<a href="https://archlinux.org/news/making-dbus-broker-our-default-d-bus-daemon/">Arch</a>
and
<a href="https://fedoraproject.org/wiki/Changes/DbusBrokerAsTheDefaultDbusImplementation">Fedora</a>
already use it to default, so there aren't lots of reason to not use it.</p>
<h2><a href="https://github.com/NixOS/nixpkgs/blob/6afb255d976f85f3359e4929abd6f5149c323a02/nixos/modules/services/hardware/irqbalance.nix"><code>services.irqbalance</code></a></h2>
<p><em>Added in 2024-08-23</em></p>
<p>Here is one that I have no experience but got curious after this comment in
<a href="https://lobste.rs/s/ymmale/unordered_list_hidden_gems_inside_nixos#c_aghpwy">Lobte.rs</a>:</p>
<blockquote>
<p>One more option to add is maybe <code>services.irqbalance.enable</code>. For my old
laptop it’s a life savior: no more freezes of the DE or video playback during
heavy operations like e.g. compilation.</p>
</blockquote>
<p>Never tried it before but will give a try!</p>
</description>
<guid>https://github.com/thiagokokada/blog/blob/main/posts/2024-08-21/01-an-unordered-list-of-hidden-gems-inside-nixos.md</guid>
<pubDate>Wed, 21 Aug 2024 00:00:00 +0000</pubDate>
</item>
<item>
<title>Quick bits: using forks with Go modules</title>
<link>https://github.com/thiagokokada/blog/blob/main/posts/2024-08-20/01-quick-bits-using-forks-with-go-modules.md</link>
<description><p>There are 2 types of blog posts: the ones you write for others, and the ones
that you write for yourself. This blog post is the latter kind. What I am going
to talk here is probably something most people know, but I didn't, and the
documentation is all scattered in the internet. So I am writing mostly to
remember myself, in future, if I have the same kind of issue what I need to do.</p>
<p>The context: <a href="https://mataroa.blog/">Mataroa</a>, the blog platform
<a href="https://capivaras.dev">capivaras.dev</a> is hosted on, relies in
<a href="https://python-markdown.github.io/">python-markdown</a>, predates
<a href="https://commonmark.org/">CommonMark</a> specification and as such, has some
incompatibilities. One of those incompatibilities with CommonMark is the way
sub lists are handled. From the documentation:</p>
<blockquote>
<p>The syntax rules clearly state that when a list item consists of multiple
paragraphs, “each subsequent paragraph in a list item must be indented by
either 4 spaces or one tab” (emphasis added). However, many implementations
do not enforce this rule and allow less than 4 spaces of indentation. The
implementers of Python-Markdown consider it a bug to not enforce this rule.</p>
</blockquote>
<p>CommonMark <a href="https://spec.commonmark.org/0.31.2/#lists">relax those
restrictions</a>, allowing a sublist to
be defined with just 2 spaces of indentation.</p>
<p>So I have automated all posts from this blog <a href="https://github.com/thiagokokada/blog/blob/main/posts/2024-07-29/01-quick-bits-why-you-should-automate-everything.md">using
Go</a> and
a CommonMark renderer called <a href="https://github.com/yuin/goldmark/">Goldmark</a>. I
them re-render the Markdown to Mataroa using a <a href="https://github.com/teekennedy/goldmark-markdown">Markdown
renderer</a> before publising to
<a href="https://capivaras.dev">capivaras.dev</a>, because this allow me to do some
transformations in the original Markdown. It mostly works fine except for sub
lists, thanks to the fact that the Markdown renderer I am using renders sub
lists with 2 spaces.</p>
<p>The only reason sub lists are working right now is because
<a href="https://gluer.org/">@ratsclub</a> fixed this issue in the fork that
<a href="https://capivaras.dev">capivaras.dev</a> runs. But I want to be compatible with
the official instance if I ever need to migrate.</p>
<p>The solution? Let's fix this in a
<a href="https://github.com/teekennedy/goldmark-markdown/pull/21">PR</a>. However now that
I have code to fix the issue, how can I use it without waiting upstream to
merge my code?</p>
<p>If you are using Go modules it is easy, you just need to use the <a href="https://go.dev/ref/mod#go-mod-file-replace"><code>replace</code>
directive</a>:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span><span style="color:#a6e22e">module</span> <span style="color:#a6e22e">github</span>.<span style="color:#a6e22e">com</span><span style="color:#f92672">/</span><span style="color:#a6e22e">thiagokokada</span><span style="color:#f92672">/</span><span style="color:#a6e22e">blog</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">go</span> <span style="color:#ae81ff">1.23</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#a6e22e">require</span> (
</span></span><span style="display:flex;"><span>	<span style="color:#a6e22e">github</span>.<span style="color:#a6e22e">com</span><span style="color:#f92672">/</span><span style="color:#a6e22e">elliotchance</span><span style="color:#f92672">/</span><span style="color:#a6e22e">orderedmap</span><span style="color:#f92672">/</span><span style="color:#a6e22e">v2</span> <span style="color:#a6e22e">v2</span><span style="color:#ae81ff">.4.0</span>
</span></span><span style="display:flex;"><span>	<span style="color:#a6e22e">github</span>.<span style="color:#a6e22e">com</span><span style="color:#f92672">/</span><span style="color:#a6e22e">gorilla</span><span style="color:#f92672">/</span><span style="color:#a6e22e">feeds</span> <span style="color:#a6e22e">v1</span><span style="color:#ae81ff">.2.0</span>
</span></span><span style="display:flex;"><span>	<span style="color:#a6e22e">github</span>.<span style="color:#a6e22e">com</span><span style="color:#f92672">/</span><span style="color:#a6e22e">gosimple</span><span style="color:#f92672">/</span><span style="color:#a6e22e">slug</span> <span style="color:#a6e22e">v1</span><span style="color:#ae81ff">.14.0</span>
</span></span><span style="display:flex;"><span>	<span style="color:#a6e22e">github</span>.<span style="color:#a6e22e">com</span><span style="color:#f92672">/</span><span style="color:#a6e22e">teekennedy</span><span style="color:#f92672">/</span><span style="color:#a6e22e">goldmark</span><span style="color:#f92672">-</span><span style="color:#a6e22e">markdown</span> <span style="color:#a6e22e">v0</span><span style="color:#ae81ff">.3.0</span>
</span></span><span style="display:flex;"><span>	<span style="color:#a6e22e">github</span>.<span style="color:#a6e22e">com</span><span style="color:#f92672">/</span><span style="color:#a6e22e">yuin</span><span style="color:#f92672">/</span><span style="color:#a6e22e">goldmark</span> <span style="color:#a6e22e">v1</span><span style="color:#ae81ff">.7.4</span>
</span></span><span style="display:flex;"><span>	<span style="color:#a6e22e">github</span>.<span style="color:#a6e22e">com</span><span style="color:#f92672">/</span><span style="color:#a6e22e">yuin</span><span style="color:#f92672">/</span><span style="color:#a6e22e">goldmark</span><span style="color:#f92672">-</span><span style="color:#a6e22e">highlighting</span> <span style="color:#a6e22e">v0</span><span style="color:#ae81ff">.0.0</span><span style="color:#f92672">-</span><span style="color:#ae81ff">20220208100518</span><span style="color:#f92672">-</span><span style="color:#ae81ff">594</span><span style="color:#a6e22e">be1970594</span>
</span></span><span style="display:flex;"><span>)
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#a6e22e">require</span> (
</span></span><span style="display:flex;"><span>	<span style="color:#a6e22e">github</span>.<span style="color:#a6e22e">com</span><span style="color:#f92672">/</span><span style="color:#a6e22e">alecthomas</span><span style="color:#f92672">/</span><span style="color:#a6e22e">chroma</span> <span style="color:#a6e22e">v0</span><span style="color:#ae81ff">.10.0</span> <span style="color:#75715e">// indirect
</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>	<span style="color:#a6e22e">github</span>.<span style="color:#a6e22e">com</span><span style="color:#f92672">/</span><span style="color:#a6e22e">dlclark</span><span style="color:#f92672">/</span><span style="color:#a6e22e">regexp2</span> <span style="color:#a6e22e">v1</span><span style="color:#ae81ff">.11.4</span> <span style="color:#75715e">// indirect
</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>	<span style="color:#a6e22e">github</span>.<span style="color:#a6e22e">com</span><span style="color:#f92672">/</span><span style="color:#a6e22e">gosimple</span><span style="color:#f92672">/</span><span style="color:#a6e22e">unidecode</span> <span style="color:#a6e22e">v1</span><span style="color:#ae81ff">.0.1</span> <span style="color:#75715e">// indirect
</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>)
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#a6e22e">replace</span> <span style="color:#a6e22e">github</span>.<span style="color:#a6e22e">com</span><span style="color:#f92672">/</span><span style="color:#a6e22e">teekennedy</span><span style="color:#f92672">/</span><span style="color:#a6e22e">goldmark</span><span style="color:#f92672">-</span><span style="color:#a6e22e">markdown</span> =&gt; <span style="color:#a6e22e">github</span>.<span style="color:#a6e22e">com</span><span style="color:#f92672">/</span><span style="color:#a6e22e">thiagokokada</span><span style="color:#f92672">/</span><span style="color:#a6e22e">goldmark</span><span style="color:#f92672">-</span><span style="color:#a6e22e">markdown</span> <span style="color:#a6e22e">v0</span><span style="color:#ae81ff">.0.0</span><span style="color:#f92672">-</span><span style="color:#ae81ff">20240820111219</span><span style="color:#f92672">-</span><span style="color:#a6e22e">f30775d8ed15</span>
</span></span></code></pre>
</body>
</html>
<p>This will replace all usages of <code>github.com/teekennedy/goldmark-markdown</code> to my
fork in <code>github.com/thiagokokada/goldmark-markdown</code>. You even get all the
reproducibility of modules since Go automatically pins the commit.</p>
<p>Since the Go format for versions is quite unique, you can just set to the
desired branch (e.g.: instead of <code>v0.0.0-20240820111219-f30775d8ed15</code>, you can
use <code>add-sublist-length-opt</code> that is the branch name) and run <code>go mod tidy</code> to
fix the format.</p>
</description>
<guid>https://github.com/thiagokokada/blog/blob/main/posts/2024-08-20/01-quick-bits-using-forks-with-go-modules.md</guid>
<pubDate>Tue, 20 Aug 2024 00:00:00 +0000</pubDate>
</item>
<item>
<title>Quick bits: Go automatically downloads a newer toolchain if needed</title>
<link>https://github.com/thiagokokada/blog/blob/main/posts/2024-08-19/01-quick-bits-go-automatically-downloads-a-newer-toolchain-if-needed.md</link>
<description><p>I am using
<a href="https://github.com/elliotchance/orderedmap/">elliotchance/orderedmap</a> as my
choice of ordered maps (since Go <a href="https://github.com/thiagokokada/blog/blob/main/posts/2024-08-17/01-an-unordered-list-of-things-i-miss-in-go.md">doesn't have
one</a> in
standard library yet). I recently did a
<a href="https://github.com/elliotchance/orderedmap/pull/41">PR</a> to implement Go 1.23
iterators, because I find them neat, however I was avoiding to use it in the
code that generates this <a href="https://github.com/thiagokokada/blog">blog</a> since Go
1.23 was just released and is still not the default Go in
<a href="https://github.com/NixOS/nixpkgs">nixpkgs</a>.</p>
<p>I decided that I would create a
<a href="https://github.com/thiagokokada/blog/pull/2">branch</a> and leave there for a few
months, until I decided to try to run the code locally and got this:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>$ go build
</span></span><span style="display:flex;"><span>go: downloading go1.23.0 (darwin/arm64)
</span></span></code></pre>
</body>
</html>
<p>Nice. And before you ask, yes, the compiled binary works perfectly:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>$ make
</span></span><span style="display:flex;"><span>./blog &gt; README.md
</span></span><span style="display:flex;"><span>./blog -rss &gt; rss.xml
</span></span></code></pre>
</body>
</html>
<p>So how does this work? Take a look at the documentation in the official <a href="https://tip.golang.org/doc/toolchain">Golang
page</a>:</p>
<blockquote>
<p>Starting in Go 1.21, the Go distribution consists of a go command and a
bundled Go toolchain, which is the standard library as well as the compiler,
assembler, and other tools. The go command can use its bundled Go toolchain
as well as other versions that it finds in the local PATH or downloads as
needed.</p>
</blockquote>
<p>There are a bunch of rules here that I am not going to enter in detail (I
recommend you to read the official documentation), but a quick summary:</p>
<ul>
<li>
<p>Go will download a toolchain when either <code>go</code> or <code>toolchain</code> lines <code>go.mod</code>
is set to a Go version higher than your current <code>go</code> binary</p>
<ul>
<li>But only if your <code>go</code> binary is at least version 1.21, since this is the
version that introduces this behavior</li>
</ul>
</li>
<li>
<p>You can force a specific toolchain with <code>GOTOOLCHAIN</code> environment setting,
e.g.: <code>GOTOOLCHAIN=1.23</code></p>
<ul>
<li>The default value for <code>GOTOOLCHAIN</code> is <code>auto</code>, that basically has the
behavior described in this post</li>
<li>You can also set to <code>local</code> to always use the current <code>go</code> binary, or the
previous behaviour pre-1.21 Go</li>
<li>There is also <code>&lt;name&gt;+auto</code> and <code>path</code> options, that can be seen in the
docs</li>
</ul>
</li>
<li>
<p>The downloaded toolchains go to whatever your <code>GOPATH</code> is, inside
<code>golang.org/toolchain</code> module, and version <code>v0.0.1-goVERSION.GOOS-GOARCH</code>, for
example:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>$ ls -lah $GOPATH/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.darwin-arm64
</span></span><span style="display:flex;"><span>total 64
</span></span><span style="display:flex;"><span>dr-xr-xr-x 14 user staff 448B 19 Aug 12:01 .
</span></span><span style="display:flex;"><span>drwxr-xr-x 4 user staff 128B 19 Aug 12:01 ..
</span></span><span style="display:flex;"><span>-r--r--r-- 1 user staff 1.3K 19 Aug 12:01 CONTRIBUTING.md
</span></span><span style="display:flex;"><span>-r--r--r-- 1 user staff 1.4K 19 Aug 12:01 LICENSE
</span></span><span style="display:flex;"><span>-r--r--r-- 1 user staff 1.3K 19 Aug 12:01 PATENTS
</span></span><span style="display:flex;"><span>-r--r--r-- 1 user staff 1.4K 19 Aug 12:01 README.md
</span></span><span style="display:flex;"><span>-r--r--r-- 1 user staff 426B 19 Aug 12:01 SECURITY.md
</span></span><span style="display:flex;"><span>-r--r--r-- 1 user staff 35B 19 Aug 12:01 VERSION
</span></span><span style="display:flex;"><span>dr-xr-xr-x 4 user staff 128B 19 Aug 12:01 bin
</span></span><span style="display:flex;"><span>-r--r--r-- 1 user staff 52B 19 Aug 12:01 codereview.cfg
</span></span><span style="display:flex;"><span>-r--r--r-- 1 user staff 505B 19 Aug 12:01 go.env
</span></span><span style="display:flex;"><span>dr-xr-xr-x 3 user staff 96B 19 Aug 12:01 lib
</span></span><span style="display:flex;"><span>dr-xr-xr-x 4 user staff 128B 19 Aug 12:01 pkg
</span></span><span style="display:flex;"><span>dr-xr-xr-x 77 user staff 2.4K 19 Aug 12:02 src
</span></span></code></pre>
</body>
</html>
</li>
</ul>
<p>By the way, this only works well because Go toolchain binaries are static.</p>
<p>While I don't like a program downloading random binaries from the internet, I
like what Go is doing here. It makes the whole bootstrapping process for a Go
project much easier: as long as you have a reasonable up-to-date <code>go</code> binary in
your <code>PATH</code>, you should be ready to go (pun intended). And Go modules are
already reasonable secure, ensuring that each module have a proper checksum. As
long as nobody else can publish modules in <code>golang.org/toolchain</code> namespace I
can't see much of a security issue here, but I am not a security expert.</p>
<p>But if you don't like this behavior, you can always disable it by setting
<code>GOTOOLCHAIN=local</code>. And just do not forget to set this in your
<a href="https://brandur.org/fragments/go-version-matrix">CI</a>, unless you don't care
about Go versions.</p>
</description>
<guid>https://github.com/thiagokokada/blog/blob/main/posts/2024-08-19/01-quick-bits-go-automatically-downloads-a-newer-toolchain-if-needed.md</guid>
<pubDate>Mon, 19 Aug 2024 00:00:00 +0000</pubDate>
</item>
<item>
<title>An unordered list of things I miss in Go</title>
<link>https://github.com/thiagokokada/blog/blob/main/posts/2024-08-17/01-an-unordered-list-of-things-i-miss-in-go.md</link>
<description><p>I like Go. I think it is a <a href="https://github.com/thiagokokada/blog/blob/main/posts/2024-07-29/02-go-a-reasonable-good-language.md">reasonable good
language</a>, and has some
good qualities that makes up for its flaws. However, this doesn't mean I think
the language couldn't be better, far from it.</p>
<p>This blog post is a list of things that I miss from Go from other languages.
Some of the things here could probably be implemented soon, some other would
probably need a major revision of the language. The list is unordered, because
this makes it easier for me to update in the future if I found something else,
but also because I don't want to think too hard about giving each point here a
rank.</p>
<p>With all above, let's start.</p>
<h2>Ordered maps in standard library</h2>
<p>When I first learned about
<a href="https://docs.python.org/3/library/stdtypes.html#typesmapping">dictionaries</a> in
Python it quickly became one of my favorite data structures ever. They're
extremely versatile, and most modern programming languages have something
similar in its standard library. Go isn't different, it has
<a href="https://go.dev/blog/maps"><code>map</code></a>, that is Go implementation of a <a href="https://en.wikipedia.org/wiki/Hash_table">hash
table</a>. However <code>map</code> in Go are
quirky, for example:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span><span style="color:#f92672">package</span> <span style="color:#a6e22e">main</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">func</span> <span style="color:#a6e22e">main</span>() {
</span></span><span style="display:flex;"><span>	<span style="color:#a6e22e">m</span> <span style="color:#f92672">:=</span> <span style="color:#66d9ef">map</span>[<span style="color:#66d9ef">string</span>]<span style="color:#66d9ef">bool</span>{<span style="color:#e6db74">&#34;foo&#34;</span>: <span style="color:#66d9ef">true</span>, <span style="color:#e6db74">&#34;bar&#34;</span>: <span style="color:#66d9ef">false</span>, <span style="color:#e6db74">&#34;baz&#34;</span>: <span style="color:#66d9ef">true</span>, <span style="color:#e6db74">&#34;qux&#34;</span>: <span style="color:#66d9ef">false</span>, <span style="color:#e6db74">&#34;quux&#34;</span>: <span style="color:#66d9ef">true</span>}
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>	<span style="color:#66d9ef">for</span> <span style="color:#a6e22e">k</span> <span style="color:#f92672">:=</span> <span style="color:#66d9ef">range</span> <span style="color:#a6e22e">m</span> {
</span></span><span style="display:flex;"><span>		println(<span style="color:#a6e22e">k</span>)
</span></span><span style="display:flex;"><span>	}
</span></span><span style="display:flex;"><span>}
</span></span></code></pre>
</body>
</html>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>$ go run ./test.go
</span></span><span style="display:flex;"><span>bar
</span></span><span style="display:flex;"><span>baz
</span></span><span style="display:flex;"><span>qux
</span></span><span style="display:flex;"><span>quux
</span></span><span style="display:flex;"><span>foo
</span></span><span style="display:flex;"><span><span style="color:#960050;background-color:#1e0010">
</span></span></span><span style="display:flex;"><span><span style="color:#960050;background-color:#1e0010"></span>$ go run ./test.go
</span></span><span style="display:flex;"><span>foo
</span></span><span style="display:flex;"><span>bar
</span></span><span style="display:flex;"><span>baz
</span></span><span style="display:flex;"><span>qux
</span></span><span style="display:flex;"><span>quux
</span></span><span style="display:flex;"><span><span style="color:#960050;background-color:#1e0010">
</span></span></span><span style="display:flex;"><span><span style="color:#960050;background-color:#1e0010"></span>$ go run ./test.go
</span></span><span style="display:flex;"><span>qux
</span></span><span style="display:flex;"><span>quux
</span></span><span style="display:flex;"><span>foo
</span></span><span style="display:flex;"><span>bar
</span></span><span style="display:flex;"><span>baz
</span></span></code></pre>
</body>
</html>
<p>Now, I don't expect any hash table implementation to keep the order of the
elements, but Go actually <a href="https://victoriametrics.com/blog/go-map/">randomise each map
instance</a>:</p>
<blockquote>
<p>But here’s the deal, while the hash function used for maps in Go is
consistent across all maps with <strong>the same key type</strong>, the <code>seed</code> used by
that hash function is different for each map instance. So, when you create a
new map, Go generates a random seed just for that map.</p>
</blockquote>
<p>While I understand the reason for this (i.e.: to avoid developers relying in a
specific iteration order), I still find it weird, and I think this is something
unique for Go. This decision means that even if you don't care about a specific
order, you will still need to sort the map before doing something else if you
want reproducibility, something that I care a lot.</p>
<p>The fix for this? Go could offer an ordered map implementation inside the
standard library. An ordered map ensure that the iteration order of the map is
the same as the insertion order (that is, by the way, a powerful property that
allow maps to be used in other contexts, not just my pet peeve above).</p>
<p>Python actually does this for any dictionaries since <a href="https://stackoverflow.com/a/39980744">Python
3.6</a>, but it offered an
<a href="https://docs.python.org/3/library/collections.html#collections.OrderedDict">OrderedDict</a>
before it (and <code>OrderedDict</code> still has some methods that normal <code>dict</code> doesn't,
that maybe useful in specific cases).</p>
<p>Before generics it would be impossible to have a type-safe API for such data
structure without introducing a new data type in the language (like <code>slices</code>),
but now Go has generics so it is not an issue anymore. The other issue is that
you would be forced to iterate manually in this new data structure, but thanks
to the new <a href="https://tip.golang.org/doc/go1.23#language"><code>range-over-func</code></a> in
Go 1.23, it means we can iterate in an ordered map as a library almost exactly
like we can do as a <code>map</code>:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span><span style="color:#f92672">import</span> <span style="color:#e6db74">&#34;orderedmap&#34;</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">func</span> <span style="color:#a6e22e">main</span>() {
</span></span><span style="display:flex;"><span> <span style="color:#a6e22e">m</span> <span style="color:#f92672">:=</span> <span style="color:#a6e22e">orderedmap</span>.<span style="color:#a6e22e">New</span>[<span style="color:#66d9ef">string</span>, <span style="color:#66d9ef">bool</span>]()
</span></span><span style="display:flex;"><span> <span style="color:#a6e22e">m</span>.<span style="color:#a6e22e">Set</span>(<span style="color:#e6db74">&#34;foo&#34;</span>, <span style="color:#66d9ef">true</span>)
</span></span><span style="display:flex;"><span> <span style="color:#a6e22e">m</span>.<span style="color:#a6e22e">Set</span>(<span style="color:#e6db74">&#34;bar&#34;</span>, <span style="color:#66d9ef">false</span>)
</span></span><span style="display:flex;"><span> <span style="color:#a6e22e">m</span>.<span style="color:#a6e22e">Set</span>(<span style="color:#e6db74">&#34;baz&#34;</span>, <span style="color:#66d9ef">true</span>)
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">for</span> <span style="color:#a6e22e">k</span> <span style="color:#f92672">:=</span> <span style="color:#66d9ef">range</span> <span style="color:#a6e22e">m</span>.<span style="color:#a6e22e">Iterator</span>() {
</span></span><span style="display:flex;"><span> println(<span style="color:#a6e22e">k</span>) <span style="color:#75715e">// Order always will be: foo, bar, baz
</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> }
</span></span><span style="display:flex;"><span>}
</span></span></code></pre>
</body>
</html>
<p>Now, of course the lack of Ordered Map in the standard library can be filled
with third party implementations, e.g.: I am using this
<a href="https://github.com/elliotchance/orderedmap">one</a> in one of my projects. But
being in standard library reduces the friction: if there was some
implementation in standard library, I would generally prefer it unless I have
some specific needs. However when the standard library doesn't offer what I
need, I need to find it myself a suitable library, and this ends up taking time
since generally there are lots of alternatives.</p>
<h2>Keyword and default arguments for functions</h2>
<p>Something that comes straight from Python that I miss sometimes in Go is that
you can do things like this when declaring a function:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span><span style="color:#66d9ef">def</span> <span style="color:#a6e22e">hello</span>(name<span style="color:#f92672">=</span><span style="color:#e6db74">&#34;World&#34;</span>):
</span></span><span style="display:flex;"><span> print(<span style="color:#e6db74">f</span><span style="color:#e6db74">&#34;Hello, </span><span style="color:#e6db74">{</span>name<span style="color:#e6db74">}</span><span style="color:#e6db74">&#34;</span>)
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>hello(<span style="color:#e6db74">&#34;Foo&#34;</span>) <span style="color:#75715e"># &#34;normal&#34; function call</span>
</span></span><span style="display:flex;"><span>hello(name<span style="color:#f92672">=</span><span style="color:#e6db74">&#34;Bar&#34;</span>) <span style="color:#75715e"># calling with keyword arguments</span>
</span></span><span style="display:flex;"><span>hello() <span style="color:#75715e"># calling with default arguments</span>
</span></span></code></pre>
</body>
</html>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>$ python hello.py
</span></span><span style="display:flex;"><span>Hello, Foo
</span></span><span style="display:flex;"><span>Hello, Bar
</span></span><span style="display:flex;"><span>Hello, World
</span></span></code></pre>
</body>
</html>
<p>The lack of default arguments especially affects even some of the API decisions
for Go standard library, for example, <code>string.Replace</code>:</p>
<blockquote>
<p><code>func Replace(s, old, new string, n int) string</code></p>
<p>Replace returns a copy of the string s with the first n non-overlapping
instances of old replaced by new. If old is empty, it matches at the
beginning of the string and after each UTF-8 sequence, yielding up to k+1
replacements for a k-rune string. If n &lt; 0, there is no limit on the number
of replacements.</p>
</blockquote>
<p>If Go had default arguments, <code>Replace</code> could have e.g.: <code>func Replace(s, old, new string, n int = -1)</code> signature, that would mean we don't need
<code>strings.ReplaceAll</code> (since this is basically what this function does, call
<code>strings.Replace(s, old, new, -1)</code>).</p>
<h2>Nullability (or nillability)</h2>
<p>I talked I little about this in <a href="https://github.com/thiagokokada/blog/blob/main/posts/2024-07-29/02-go-a-reasonable-good-language.md">my previous post about
Go</a>, but I want to
expand here.</p>
<p>First, I don't think the language needs to support the generic solution for
nullability, that would be either having proper Union or Sum types. Kotlin
AFAIK doesn't support neither, but my 2 years experience with Kotlin showed
that just having nullable types already helped a lot in ensuring type safety.</p>
<p>Second, I do feel that Go has less issues with <code>nil</code> values, than say, Java,
because its decision of using zero values instead of <code>nil</code> in many cases. So
for example, a string can never be <code>nil</code>, however a string pointer can be. This
means that this is fine:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span><span style="color:#66d9ef">func</span>(<span style="color:#a6e22e">s</span> <span style="color:#66d9ef">string</span>) {
</span></span><span style="display:flex;"><span> <span style="color:#75715e">// do something with s
</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>}
</span></span></code></pre>
</body>
</html>
<p>However:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span><span style="color:#66d9ef">func</span>(<span style="color:#a6e22e">s</span> <span style="color:#f92672">*</span><span style="color:#66d9ef">string</span>) {
</span></span><span style="display:flex;"><span> <span style="color:#75715e">// s maybe nil here, better check first
</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>}
</span></span></code></pre>
</body>
</html>
<p>Still, I get more <code>panic</code> for <code>nil</code> pointer deference than I get in other
languages that offer nullables (heck, even Python with
<a href="https://www.mypy-lang.org/"><code>mypy</code></a> is safer).</p>
<p>Sadly this is the change in this post that is more likely to need a completely
new revision of the language.
<a href="https://github.com/golang/go/issues/49202">nillability</a> was proposed before,
but it is really unlikely it can be done without breaking backwards
compatibility.</p>
<p>It could be done the Java way by adding a <code>nullable</code> type to the standard
library (<a href="https://jcp.org/en/jsr/detail?id=305">JSR305</a>), but the fact that
<a href="https://stackoverflow.com/questions/2289694/what-is-the-status-of-jsr-305">JSR305 is considerd
dead</a>
by many shows how difficult it is to do something like this without a major
change in the language. Dart is the only language that I know that <a href="https://dart.dev/null-safety/understanding-null-safety">did this
successfully</a>, but
definitely it was not without its pains. And the fact that most people that
program in Dart probably does because of Flutter (that eventually required
newer versions with null-safety) is not a good sign.</p>
<h2>Lambdas</h2>
<p><em>Added in 2024-08-18</em></p>
<p>Go is a surprising good language for some functional code, thanks to having
first class functions and closures. Sadly the syntax doesn't help, since the
only way you can use anonymous functions in Go is using <code>func</code>. Especially if
the types are complex, this can result in some convoluted code. Take the
example from the <a href="https://go.dev/wiki/RangefuncExperiment"><code>range-over-func</code>
experiment</a>:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span><span style="color:#f92672">package</span> <span style="color:#a6e22e">slices</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">func</span> <span style="color:#a6e22e">Backward</span>[<span style="color:#a6e22e">E</span> <span style="color:#a6e22e">any</span>](<span style="color:#a6e22e">s</span> []<span style="color:#a6e22e">E</span>) <span style="color:#66d9ef">func</span>(<span style="color:#66d9ef">func</span>(<span style="color:#66d9ef">int</span>, <span style="color:#a6e22e">E</span>) <span style="color:#66d9ef">bool</span>) {
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">return</span> <span style="color:#66d9ef">func</span>(<span style="color:#a6e22e">yield</span> <span style="color:#66d9ef">func</span>(<span style="color:#66d9ef">int</span>, <span style="color:#a6e22e">E</span>) <span style="color:#66d9ef">bool</span>) {
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">for</span> <span style="color:#a6e22e">i</span> <span style="color:#f92672">:=</span> len(<span style="color:#a6e22e">s</span>)<span style="color:#f92672">-</span><span style="color:#ae81ff">1</span>; <span style="color:#a6e22e">i</span> <span style="color:#f92672">&gt;=</span> <span style="color:#ae81ff">0</span>; <span style="color:#a6e22e">i</span><span style="color:#f92672">--</span> {
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">if</span> !<span style="color:#a6e22e">yield</span>(<span style="color:#a6e22e">i</span>, <span style="color:#a6e22e">s</span>[<span style="color:#a6e22e">i</span>]) {
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">return</span>
</span></span><span style="display:flex;"><span> }
</span></span><span style="display:flex;"><span> }
</span></span><span style="display:flex;"><span> }
</span></span><span style="display:flex;"><span>}
</span></span></code></pre>
</body>
</html>
<p>If Go had a syntax for lambdas, especially if we could elide the types, this
could be simplified a lot:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span><span style="color:#f92672">package</span> <span style="color:#a6e22e">slices</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">func</span> <span style="color:#a6e22e">Backward</span>[<span style="color:#a6e22e">E</span> <span style="color:#a6e22e">any</span>](<span style="color:#a6e22e">s</span> []<span style="color:#a6e22e">E</span>) <span style="color:#66d9ef">func</span>(<span style="color:#66d9ef">func</span>(<span style="color:#66d9ef">int</span>, <span style="color:#a6e22e">E</span>) <span style="color:#66d9ef">bool</span>) {
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">return</span> (<span style="color:#a6e22e">yield</span>) =&gt; {
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">for</span> <span style="color:#a6e22e">i</span> <span style="color:#f92672">:=</span> len(<span style="color:#a6e22e">s</span>)<span style="color:#f92672">-</span><span style="color:#ae81ff">1</span>; <span style="color:#a6e22e">i</span> <span style="color:#f92672">&gt;=</span> <span style="color:#ae81ff">0</span>; <span style="color:#a6e22e">i</span><span style="color:#f92672">--</span> {
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">if</span> !<span style="color:#a6e22e">yield</span>(<span style="color:#a6e22e">i</span>, <span style="color:#a6e22e">s</span>[<span style="color:#a6e22e">i</span>]) {
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">return</span>
</span></span><span style="display:flex;"><span> }
</span></span><span style="display:flex;"><span> }
</span></span><span style="display:flex;"><span> }
</span></span><span style="display:flex;"><span>}
</span></span></code></pre>
</body>
</html>
<p>Or even something like this would already help, no special syntax but allowing
the types to be elided in an unnamed function:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span><span style="color:#f92672">package</span> <span style="color:#a6e22e">slices</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">func</span> <span style="color:#a6e22e">Backward</span>[<span style="color:#a6e22e">E</span> <span style="color:#a6e22e">any</span>](<span style="color:#a6e22e">s</span> []<span style="color:#a6e22e">E</span>) <span style="color:#66d9ef">func</span>(<span style="color:#66d9ef">func</span>(<span style="color:#66d9ef">int</span>, <span style="color:#a6e22e">E</span>) <span style="color:#66d9ef">bool</span>) {
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">return</span> <span style="color:#66d9ef">func</span>(<span style="color:#a6e22e">yield</span>) {
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">for</span> <span style="color:#a6e22e">i</span> <span style="color:#f92672">:=</span> len(<span style="color:#a6e22e">s</span>)<span style="color:#f92672">-</span><span style="color:#ae81ff">1</span>; <span style="color:#a6e22e">i</span> <span style="color:#f92672">&gt;=</span> <span style="color:#ae81ff">0</span>; <span style="color:#a6e22e">i</span><span style="color:#f92672">--</span> {
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">if</span> !<span style="color:#a6e22e">yield</span>(<span style="color:#a6e22e">i</span>, <span style="color:#a6e22e">s</span>[<span style="color:#a6e22e">i</span>]) {
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">return</span>
</span></span><span style="display:flex;"><span> }
</span></span><span style="display:flex;"><span> }
</span></span><span style="display:flex;"><span> }
</span></span><span style="display:flex;"><span>}
</span></span></code></pre>
</body>
</html>
<p>This feature I am still somewhat hopeful that may become a reality in some
future version of the language, since they didn't close the
<a href="https://github.com/golang/go/issues/21498">issue</a> yet, and the discussion
about the possibility of this feature is still ongoing.</p>
<h2>Error on unused return values</h2>
<p><em>Added in 2024-09-12</em></p>
<p>In a <a href="https://github.com/thiagokokada/blog/blob/main/posts/2024-09-02/01-my-coding-experience-with-llm.md">recent post about
LLMs</a>, I talked about
this function that I asked ChatGPT to generate to create a socket that accepted
a <a href="https://pkg.go.dev/context">context</a>:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span><span style="color:#66d9ef">func</span> <span style="color:#a6e22e">readWithContext</span>(<span style="color:#a6e22e">ctx</span> <span style="color:#a6e22e">context</span>.<span style="color:#a6e22e">Context</span>, <span style="color:#a6e22e">conn</span> <span style="color:#a6e22e">net</span>.<span style="color:#a6e22e">Conn</span>, <span style="color:#a6e22e">buf</span> []<span style="color:#66d9ef">byte</span>) (<span style="color:#66d9ef">int</span>, <span style="color:#66d9ef">error</span>) {
</span></span><span style="display:flex;"><span> <span style="color:#a6e22e">done</span> <span style="color:#f92672">:=</span> make(<span style="color:#66d9ef">chan</span> <span style="color:#66d9ef">struct</span>{})
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">var</span> <span style="color:#a6e22e">n</span> <span style="color:#66d9ef">int</span>
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">var</span> <span style="color:#a6e22e">err</span> <span style="color:#66d9ef">error</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#75715e">// Start a goroutine to perform the read
</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> <span style="color:#66d9ef">go</span> <span style="color:#66d9ef">func</span>() {
</span></span><span style="display:flex;"><span> <span style="color:#a6e22e">n</span>, <span style="color:#a6e22e">err</span> = <span style="color:#a6e22e">conn</span>.<span style="color:#a6e22e">Read</span>(<span style="color:#a6e22e">buf</span>)
</span></span><span style="display:flex;"><span> close(<span style="color:#a6e22e">done</span>)
</span></span><span style="display:flex;"><span> }()
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">select</span> {
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">case</span> <span style="color:#f92672">&lt;-</span><span style="color:#a6e22e">ctx</span>.<span style="color:#a6e22e">Done</span>(): <span style="color:#75715e">// Context was canceled or timed out
</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> <span style="color:#75715e">// Set a short deadline to unblock the Read()
</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> <span style="color:#a6e22e">conn</span>.<span style="color:#a6e22e">SetReadDeadline</span>(<span style="color:#a6e22e">time</span>.<span style="color:#a6e22e">Now</span>())
</span></span><span style="display:flex;"><span> <span style="color:#f92672">&lt;-</span><span style="color:#a6e22e">done</span> <span style="color:#75715e">// Wait for the read to finish
</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> <span style="color:#66d9ef">return</span> <span style="color:#ae81ff">0</span>, <span style="color:#a6e22e">ctx</span>.<span style="color:#a6e22e">Err</span>()
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">case</span> <span style="color:#f92672">&lt;-</span><span style="color:#a6e22e">done</span>: <span style="color:#75715e">// Read finished successfully
</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> <span style="color:#66d9ef">return</span> <span style="color:#a6e22e">n</span>, <span style="color:#a6e22e">err</span>
</span></span><span style="display:flex;"><span> }
</span></span><span style="display:flex;"><span>}
</span></span></code></pre>
</body>
</html>
<p>However while reading <a href="https://joeduffyblog.com/2016/02/07/the-error-model/">this blog
post</a> about &quot;The Error
Model&quot;, I realised that this function is lacking error check during
<code>conn.SetReadDeadline()</code> calls:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span><span style="color:#66d9ef">func</span> <span style="color:#a6e22e">readWithContext</span>(<span style="color:#a6e22e">ctx</span> <span style="color:#a6e22e">context</span>.<span style="color:#a6e22e">Context</span>, <span style="color:#a6e22e">conn</span> <span style="color:#a6e22e">net</span>.<span style="color:#a6e22e">Conn</span>, <span style="color:#a6e22e">buf</span> []<span style="color:#66d9ef">byte</span>) (<span style="color:#a6e22e">n</span> <span style="color:#66d9ef">int</span>, <span style="color:#a6e22e">err</span> <span style="color:#66d9ef">error</span>) {
</span></span><span style="display:flex;"><span>	<span style="color:#a6e22e">done</span> <span style="color:#f92672">:=</span> make(<span style="color:#66d9ef">chan</span> <span style="color:#66d9ef">struct</span>{})
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>	<span style="color:#75715e">// Start a goroutine to perform the read
</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>	<span style="color:#66d9ef">go</span> <span style="color:#66d9ef">func</span>() {
</span></span><span style="display:flex;"><span>		<span style="color:#a6e22e">n</span>, <span style="color:#a6e22e">err</span> = <span style="color:#a6e22e">conn</span>.<span style="color:#a6e22e">Read</span>(<span style="color:#a6e22e">buf</span>)
</span></span><span style="display:flex;"><span>		close(<span style="color:#a6e22e">done</span>)
</span></span><span style="display:flex;"><span>	}()
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>	<span style="color:#66d9ef">select</span> {
</span></span><span style="display:flex;"><span>	<span style="color:#66d9ef">case</span> <span style="color:#f92672">&lt;-</span><span style="color:#a6e22e">done</span>:
</span></span><span style="display:flex;"><span>		<span style="color:#66d9ef">return</span> <span style="color:#a6e22e">n</span>, <span style="color:#a6e22e">err</span>
</span></span><span style="display:flex;"><span>	<span style="color:#66d9ef">case</span> <span style="color:#f92672">&lt;-</span><span style="color:#a6e22e">ctx</span>.<span style="color:#a6e22e">Done</span>():
</span></span><span style="display:flex;"><span>		<span style="color:#75715e">// Set a short deadline to unblock the Read()
</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>		<span style="color:#a6e22e">err</span> = <span style="color:#a6e22e">conn</span>.<span style="color:#a6e22e">SetReadDeadline</span>(<span style="color:#a6e22e">time</span>.<span style="color:#a6e22e">Now</span>())
</span></span><span style="display:flex;"><span>		<span style="color:#66d9ef">if</span> <span style="color:#a6e22e">err</span> <span style="color:#f92672">!=</span> <span style="color:#66d9ef">nil</span> {
</span></span><span style="display:flex;"><span>			<span style="color:#66d9ef">return</span> <span style="color:#ae81ff">0</span>, <span style="color:#a6e22e">err</span>
</span></span><span style="display:flex;"><span>		}
</span></span><span style="display:flex;"><span>		<span style="color:#75715e">// Reset read deadline
</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>		<span style="color:#66d9ef">defer</span> <span style="color:#66d9ef">func</span>() {
</span></span><span style="display:flex;"><span>			<span style="color:#66d9ef">if</span> <span style="color:#a6e22e">e</span> <span style="color:#f92672">:=</span> <span style="color:#a6e22e">conn</span>.<span style="color:#a6e22e">SetReadDeadline</span>(<span style="color:#a6e22e">time</span>.<span style="color:#a6e22e">Time</span>{}); <span style="color:#a6e22e">e</span> <span style="color:#f92672">!=</span> <span style="color:#66d9ef">nil</span> {
</span></span><span style="display:flex;"><span>				<span style="color:#a6e22e">err</span> = <span style="color:#a6e22e">errors</span>.<span style="color:#a6e22e">Join</span>(<span style="color:#a6e22e">err</span>, <span style="color:#a6e22e">e</span>)
</span></span><span style="display:flex;"><span>			}
</span></span><span style="display:flex;"><span>		}()
</span></span><span style="display:flex;"><span>		<span style="color:#75715e">// Make sure that the goroutine is done to avoid leaks
</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span>		<span style="color:#f92672">&lt;-</span><span style="color:#a6e22e">done</span>
</span></span><span style="display:flex;"><span>		<span style="color:#66d9ef">return</span> <span style="color:#ae81ff">0</span>, <span style="color:#a6e22e">errors</span>.<span style="color:#a6e22e">Join</span>(<span style="color:#a6e22e">err</span>, <span style="color:#a6e22e">ctx</span>.<span style="color:#a6e22e">Err</span>())
</span></span><span style="display:flex;"><span>	}
</span></span><span style="display:flex;"><span>}
</span></span></code></pre>
</body>
</html>
<p>I could blame the LLM on this, but this is the type of mistake that I could see
happening even in a Pull Request done by an actual human.</p>
<p>Ignoring errors is bad, especially because once the error is ignored it is
gone, forever. You may have strange issues in the code that are impossible to
debug because you can't know about something that doesn't exist. This is one of
the reasons I think exceptions are better, since this is the kind of error that
would be impossible to ignore if the language had exceptions, and the exception
would eventually propagate in your stack until it is captured or you get a
crash (with a stack trace to debug the issue).</p>
<p>Now, I don't think this is a fault of using error as values instead of
exceptions. Go has the tools to encapsulate the errors and propagate them
properly, like it is shown in the fixed code. But the fact that someone can do
this mistake without any warning or error from the compiler is bad.</p>
<p>If Go had a warning or error for unused return values, this would be different:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span><span style="color:#66d9ef">func</span> <span style="color:#a6e22e">foo</span>(<span style="color:#a6e22e">conn</span> <span style="color:#f92672">*</span><span style="color:#a6e22e">net</span>.<span style="color:#a6e22e">Conn</span>) {
</span></span><span style="display:flex;"><span> <span style="color:#75715e">// ...
</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> <span style="color:#a6e22e">_</span> = <span style="color:#a6e22e">conn</span>.<span style="color:#a6e22e">SetReadDeadline</span>(<span style="color:#a6e22e">time</span>.<span style="color:#a6e22e">Now</span>())
</span></span><span style="display:flex;"><span>}
</span></span></code></pre>
</body>
</html>
<p>Now in this case it is clear: I am ignoring the error explicitly, probably for
a good reason. I can ask during a Pull Request why the committer is ignoring if
it is lacking appropriate context, and maybe even ask for some comments to be
added why this would be safe. What can't happen is this being ignored by
mistake.</p>
<p>I am not completely sure if I want this only for errors or for any unused value
though. There are lots of cases where ignoring non-error values is ok, but I
also don't see the issue of having something like:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span><span style="color:#66d9ef">func</span> <span style="color:#a6e22e">foo</span>() {
</span></span><span style="display:flex;"><span> <span style="color:#75715e">// ...
</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> <span style="color:#a6e22e">_</span>, <span style="color:#a6e22e">_</span> = <span style="color:#a6e22e">FuncThatReturnIntAndError</span>()
</span></span><span style="display:flex;"><span>}
</span></span></code></pre>
</body>
</html>
<p>If anything, it is clear for whoever is reading this code later that you're
only calling this function only for its side-effects.</p>
<p>By the way, yes, <a href="https://github.com/kisielk/errcheck">errcheck</a> linter exists,
but if the language itself doesn't enforce this it means that there are lots of
libraries that are probably mistakenly ignoring errors. And if the library is
ignoring errors by mistake, you can't do much when you actually have them.</p>
</description>
<guid>https://github.com/thiagokokada/blog/blob/main/posts/2024-08-17/01-an-unordered-list-of-things-i-miss-in-go.md</guid>
<pubDate>Sat, 17 Aug 2024 00:00:00 +0000</pubDate>
</item>
<item>
<title>Quick bits: NixOS's boot.kernelPackages is a attrset</title>
<link>https://github.com/thiagokokada/blog/blob/main/posts/2024-08-15/01-quick-bits-nixoss-boot-kernelpackages-is-a-attrset.md</link>
<description><p>I don't know if this is just something that got me by surprise or not, but
<a href="https://github.com/NixOS/nixpkgs/blob/bb16119a4a7639ebbc91ad0f516b324b0f7c9b68/nixos/modules/system/boot/kernel.nix#L40-L71"><code>boot.kernelPackages</code></a>
option does not receive a derivation like most other packages, but instead
receives a function and returns an attribute set with all packages. Here is the
documentation:</p>
<blockquote>
<p>This option allows you to override the Linux kernel used by NixOS. Since
things like external kernel module packages are tied to the kernel you’re
using, it also overrides those. This option is a function that takes Nixpkgs
as an argument (as a convenience), and returns an attribute set containing at
the very least an attribute kernel. Additional attributes may be needed
depending on your configuration. For instance, if you use the NVIDIA X
driver, then it also needs to contain an attribute <code>nvidia_x11</code>.</p>
</blockquote>
<p>The kernel package itself is referenced by the
<a href="https://github.com/NixOS/nixpkgs/blob/bb16119a4a7639ebbc91ad0f516b324b0f7c9b68/nixos/modules/system/boot/kernel.nix#L331-L332"><code>kernel</code></a>
derivation inside this attribute set:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>nix-repl&gt; nixosConfigurations.sankyuu-nixos.config.boot.kernelPackages.kernel
</span></span><span style="display:flex;"><span>«derivation /nix/store/5zyjvf3qgfk52qmgxh36l4dkr9lf100x-linux-6.10.3.drv»
</span></span></code></pre>
</body>
</html>
<p>The reason for this is because it ensure that things like modules are built
with the same kernel version you are booting.</p>
<p>However one less obvious consequence about this is that if you want packages
that come from <code>linuxPackages</code>, say for example
<a href="https://github.com/NixOS/nixpkgs/blob/nixos-unstable/pkgs/os-specific/linux/cpupower/default.nix"><code>cpupower</code></a>
or
<a href="https://github.com/NixOS/nixpkgs/blob/nixos-unstable/pkgs/os-specific/linux/cpupower/default.nix"><code>turbostat</code></a>,
it is better to do:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>{ config<span style="color:#f92672">,</span> <span style="color:#f92672">...</span> }:
</span></span><span style="display:flex;"><span>{
</span></span><span style="display:flex;"><span> environment<span style="color:#f92672">.</span>systemPackages <span style="color:#f92672">=</span> [
</span></span><span style="display:flex;"><span> config<span style="color:#f92672">.</span>boot<span style="color:#f92672">.</span>kernelPackages<span style="color:#f92672">.</span>cpupower
</span></span><span style="display:flex;"><span> config<span style="color:#f92672">.</span>boot<span style="color:#f92672">.</span>kernelPackages<span style="color:#f92672">.</span>turbostat
</span></span><span style="display:flex;"><span> ];
</span></span><span style="display:flex;"><span>}
</span></span></code></pre>
</body>
</html>
<p>Instead of:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>{ pkgs<span style="color:#f92672">,</span> <span style="color:#f92672">...</span> }:
</span></span><span style="display:flex;"><span>{
</span></span><span style="display:flex;"><span> environment<span style="color:#f92672">.</span>systemPackages <span style="color:#f92672">=</span> <span style="color:#66d9ef">with</span> pkgs; [
</span></span><span style="display:flex;"><span> linuxPackages<span style="color:#f92672">.</span>cpupower
</span></span><span style="display:flex;"><span> linuxPackages<span style="color:#f92672">.</span>turbostat
</span></span><span style="display:flex;"><span> ];
</span></span><span style="display:flex;"><span>}
</span></span></code></pre>
</body>
</html>
<p>Now, I used the later for a long time and never had issues. But technically
those packages depends in a specific kernel version for a reason, so getting
them from <code>config.boot.kernelPackages</code> reduces the chance of you having some
compatibility issue in the future.</p>
</description>
<guid>https://github.com/thiagokokada/blog/blob/main/posts/2024-08-15/01-quick-bits-nixoss-boot-kernelpackages-is-a-attrset.md</guid>
<pubDate>Thu, 15 Aug 2024 00:00:00 +0000</pubDate>
</item>
<item>
<title>Abusing Makefiles for fun and profit</title>
<link>https://github.com/thiagokokada/blog/blob/main/posts/2024-08-13/01-abusing-makefiles-for-fun-and-profit.md</link>
<description><p>If you are following this blog for a while, it should be no surprise that most
of the workflow in this blog is <a href="https://github.com/thiagokokada/blog/blob/main/posts/2024-07-29/01-quick-bits-why-you-should-automate-everything.md">automated using
Go</a>. I
basically write Markdown files with some special rules inside the
<a href="https://github.com/thiagokokada/blog">repository</a>, commit and push it. In
seconds, the CI (currently <a href="https://github.com/thiagokokada/blog/blob/4e3f25485c6682f3e066b219df2290934bc0d256/.github/workflows/go.yml">GitHub
Actions</a>)
will take the latest commit, generate some files (since I use the <a href="https://github.com/thiagokokada/blog/blob/main/posts/2024-07-26/02-using-github-as-a-bad-blog-platform.md">repository
itself</a> as a
backup blog) and publish to the <a href="https://kokada.capivaras.dev/">capivaras.dev
website</a>.</p>
<p>Now, considering how much about <a href="https://nixos.org/">Nix</a> I talk in this blog,
it should be a surprise that the workflow above has <strong>zero</strong> Nix code inside
it. I am not saying this blog will never have it, but I am only going to add if
this is necessary, for example if I start using a tool to build this blog that
I generally don't expect it to be installed by the machine I am currently
using. Go is an exception of this rule since it is relatively straightfoward to
install (just download the <a href="https://go.dev/doc/install">binary</a>) and because
its <a href="https://go.dev/doc/go1compat">stability guarantee</a> means (hopefully) no
breakage. But most other things I consider moving targets, and I wouldn't be
comfortable to use unless I have Nix to ensure reproducibility.</p>
<p>This is why the other tool that this blog (ab)uses during its workflow is
<a href="https://en.wikipedia.org/wiki/Make_(software)"><code>Make</code></a>, one of the oldest
build automation tool that exist. It is basically available in any *nix (do not
confuse with <a href="https://nixos.org/">Nix</a>) system, from most Linux distros to
macOS, by default. So it is the tool I choose to automatise some tasks in this
blog, even if I consider writing a <code>Makefile</code> (the domain-specific language
that <code>Make</code> uses) kind of a lost, dark art.</p>
<p>To be clear, the idea of this post is not to be a <code>Makefile</code> tutorial. I will
explain some basic concepts, but if you want an actual tutorial a good one can
be found <a href="https://makefiletutorial.com/">here</a>. Also, while I am using <code>Make</code>
thanks to the reasons above, you can use many other tools for a similar
objective, like <a href="https://github.com/casey/just">Justfiles</a>,
<a href="https://taskfile.dev/">Taskfiles</a> (sadly it uses
<a href="https://github.com/thiagokokada/blog/blob/main/posts/2024-07-31/01-generating-yaml-files-with-nix.md">YAML</a>), or even a
small script written in any language you want. The reason that I am writing
this post is why you should do it, not how.</p>
<p>A quick recap on how this blog works: inside the
<a href="https://github.com/thiagokokada/blog">repository</a>, a post is basically a
Markdown post following the directory structure below
(<a href="https://github.com/thiagokokada/blog/tree/894a388c61ca3a38dfc9d4cbe88dc684fd964bb7">permalink</a>
for the current version of this blog):</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>.
</span></span><span style="display:flex;"><span>&lt;...&gt;
</span></span><span style="display:flex;"><span>├── 2024-08-07
</span></span><span style="display:flex;"><span>│ ├── 01-quick-bits-is-crostini-a-microvm.md
</span></span><span style="display:flex;"><span>│ └── 02-meta-are-quick-bits-really-quick.md
</span></span><span style="display:flex;"><span>├── 2024-08-11
</span></span><span style="display:flex;"><span>│ └── 01-building-static-binaries-in-nix.md
</span></span><span style="display:flex;"><span>├── 2024-08-12
</span></span><span style="display:flex;"><span>│ ├── 01-things-i-dont-like-in-my-chromebook-duet-3.md
</span></span><span style="display:flex;"><span>│ └── Screenshot_2024-08-12_20.50.42.png
</span></span><span style="display:flex;"><span>├── 2024-08-13
</span></span><span style="display:flex;"><span>│ ├── 01-abusing-makefiles-for-fun-and-profit.md &lt;-- this file
</span></span><span style="display:flex;"><span>├── .github
</span></span><span style="display:flex;"><span>│ └── workflows
</span></span><span style="display:flex;"><span>│ └── go.yml
</span></span><span style="display:flex;"><span>├── .gitignore
</span></span><span style="display:flex;"><span>├── go.mod
</span></span><span style="display:flex;"><span>├── go.sum
</span></span><span style="display:flex;"><span>├── LICENSE
</span></span><span style="display:flex;"><span>├── link_rewriter.go
</span></span><span style="display:flex;"><span>├── Makefile
</span></span><span style="display:flex;"><span>├── mataroa.go
</span></span><span style="display:flex;"><span>├── README.md
</span></span><span style="display:flex;"><span>├── rss.xml
</span></span><span style="display:flex;"><span>└── .scripts
</span></span><span style="display:flex;"><span> └── gen-post.sh
</span></span></code></pre>
</body>
</html>
<p>So I just create a new Markdown file following the
<code>YYYY-MM-DD/XX-title-slug.md</code> format. It <strong>must</strong> start with a <code>h1</code> header,
that will be automatically extract to be used as the post title, but otherwise
there is no other formatting rules. It is a highly optionated structure, but
the nice thing about being optionated is that we can extract lots of
information just from how the files are organised in the filesystem.</p>
<p>Most of the magic that converts those Markdown files to actual blog posts are
in the Go files that you can see above: <code>blog.go</code> is the main logic that walks
in the repository and extracts the necessary information, <code>mataroa.go</code> is
responsible for the <a href="https://capivaras.dev/">capivaras.dev</a> integration (that
uses <a href="https://mataroa.blog/">Mataroa</a> platform), while <code>link_rewriter.go</code> is
responsible to do some transformations in the Markdown files before posting.</p>
<p>While I could manage everything by just using <code>go</code> CLI and a few other *nix
commands, to make it easier to manager everything I have the following
<a href="https://github.com/thiagokokada/blog/blob/527466a2a7c8baae532281bff5db3f0695f018cb/Makefile"><code>Makefile</code></a>:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>MARKDOWN <span style="color:#f92672">:=</span> <span style="color:#66d9ef">$(</span>wildcard ./**/*.md<span style="color:#66d9ef">)</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#a6e22e">.PHONY</span><span style="color:#f92672">:</span> all
</span></span><span style="display:flex;"><span><span style="color:#a6e22e">all</span><span style="color:#f92672">:</span> README.md rss.xml
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#a6e22e">blog</span><span style="color:#f92672">:</span> *.go go.*
</span></span><span style="display:flex;"><span>	go build
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#a6e22e">README.md</span><span style="color:#f92672">:</span> blog <span style="color:#66d9ef">$(</span>MARKDOWN<span style="color:#66d9ef">)</span>
</span></span><span style="display:flex;"><span>	./blog &gt; README.md
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#a6e22e">rss.xml</span><span style="color:#f92672">:</span> blog <span style="color:#66d9ef">$(</span>MARKDOWN<span style="color:#66d9ef">)</span>
</span></span><span style="display:flex;"><span>	./blog -rss &gt; rss.xml
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#a6e22e">.PHONY</span><span style="color:#f92672">:</span> publish
</span></span><span style="display:flex;"><span><span style="color:#a6e22e">publish</span><span style="color:#f92672">:</span> blog
</span></span><span style="display:flex;"><span>	./blog -publish
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>DAY <span style="color:#f92672">:=</span> <span style="color:#66d9ef">$(</span>shell date<span style="color:#66d9ef">)</span>
</span></span><span style="display:flex;"><span>_PARSED_DAY <span style="color:#f92672">:=</span> <span style="color:#66d9ef">$(</span>shell date <span style="color:#e6db74">&#39;+%Y-%m-%d&#39;</span> -d <span style="color:#e6db74">&#39;$(DAY)&#39;</span><span style="color:#66d9ef">)</span>
</span></span><span style="display:flex;"><span><span style="color:#a6e22e">.PHONY</span><span style="color:#f92672">:</span> day
</span></span><span style="display:flex;"><span><span style="color:#a6e22e">day</span><span style="color:#f92672">:</span>
</span></span><span style="display:flex;"><span>	mkdir -p <span style="color:#e6db74">&#39;$(_PARSED_DAY)&#39;</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>TITLE <span style="color:#f92672">=</span> <span style="color:#66d9ef">$(</span>error TITLE is not defined<span style="color:#66d9ef">)</span>
</span></span><span style="display:flex;"><span><span style="color:#a6e22e">.PHONY</span><span style="color:#f92672">:</span> post
</span></span><span style="display:flex;"><span><span style="color:#a6e22e">post</span><span style="color:#f92672">:</span> blog day
</span></span><span style="display:flex;"><span>	./.scripts/gen-post.sh <span style="color:#e6db74">&#39;$(_PARSED_DAY)&#39;</span> <span style="color:#e6db74">&#39;$(TITLE)&#39;</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>FILE <span style="color:#f92672">=</span> <span style="color:#66d9ef">$(</span>error FILE is not defined<span style="color:#66d9ef">)</span>
</span></span><span style="display:flex;"><span><span style="color:#a6e22e">.PHONY</span><span style="color:#f92672">:</span> draft
</span></span><span style="display:flex;"><span><span style="color:#a6e22e">draft</span><span style="color:#f92672">:</span>
</span></span><span style="display:flex;"><span>	cd <span style="color:#e6db74">&#39;$(dir $(FILE))&#39;</span> <span style="color:#f92672">&amp;&amp;</span> mv <span style="color:#e6db74">&#39;$(notdir $(FILE))&#39;</span> <span style="color:#e6db74">&#39;.$(notdir $(FILE))&#39;</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#a6e22e">.PHONY</span><span style="color:#f92672">:</span> undraft
</span></span><span style="display:flex;"><span><span style="color:#a6e22e">undraft</span><span style="color:#f92672">:</span>
</span></span><span style="display:flex;"><span>	cd <span style="color:#e6db74">&#39;$(dir $(FILE))&#39;</span> <span style="color:#f92672">&amp;&amp;</span> mv <span style="color:#e6db74">&#39;$(notdir $(FILE))&#39;</span> <span style="color:#e6db74">&#39;$(patsubst .%,%,$(notdir $(FILE)))&#39;</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#a6e22e">.PHONY</span><span style="color:#f92672">:</span> clean
</span></span><span style="display:flex;"><span><span style="color:#a6e22e">clean</span><span style="color:#f92672">:</span>
</span></span><span style="display:flex;"><span>	rm -rf blog
</span></span></code></pre>
</body>
</html>
<p>For those unfamiliar with <code>Makefile</code>, a quick explanation on how it works from
<a href="https://en.wikipedia.org/wiki/Make_(software)#Makefile">Wikipedia</a>:</p>
<blockquote>
<p>Each rule begins with a <em>dependency line</em> which consists of the rule's target
name followed by a colon (:) and optionally a list of targets on which the
rule's target depends, its prerequisites.</p>
</blockquote>
<p>So if we look for example at the <code>blog</code> binary, the dependencies are all the
<code>.go</code> files and Go module files like <code>go.mod</code> and <code>go.sum</code>. We can make the
<code>blog</code> binary by running:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>$ make blog
</span></span><span style="display:flex;"><span>go build
</span></span></code></pre>
</body>
</html>
<p>One nice thing about <code>Makefile</code> is that they track if any of the source files
has a newer timestamp than the target file, and only trigger the build again if
there are changes, for example:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>$ make blog
</span></span><span style="display:flex;"><span>make: &#39;blog&#39; is up to date.
</span></span><span style="display:flex;"><span><span style="color:#960050;background-color:#1e0010">
</span></span></span><span style="display:flex;"><span><span style="color:#960050;background-color:#1e0010"></span>$ touch blog.go
</span></span><span style="display:flex;"><span><span style="color:#960050;background-color:#1e0010">
</span></span></span><span style="display:flex;"><span><span style="color:#960050;background-color:#1e0010"></span>$ make blog
</span></span><span style="display:flex;"><span>go build
</span></span></code></pre>
</body>
</html>
<p>But sometimes this property is undesirable. In those cases we can declare a
target as <code>.PHONY</code>, that basically instructs <code>Makefile</code> to always make the
target. One classic example is <code>clean</code> target, that removes build artifacts:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>$ make clean
</span></span><span style="display:flex;"><span>rm -rf blog
</span></span><span style="display:flex;"><span><span style="color:#960050;background-color:#1e0010">
</span></span></span><span style="display:flex;"><span><span style="color:#960050;background-color:#1e0010"></span>$ make clean
</span></span><span style="display:flex;"><span>rm -rf blog
</span></span></code></pre>
</body>
</html>
<p>By the way, it is better to declare a target as <code>.PHONY</code> than declaring
dependencies incorrectly, especially in languages that has fast build times
like e.g.: Go. The worst thing that can happen is something not being rebuild
when it needs to. So my recomendation if you are writing your first <code>Makefile</code>
is to just declare everything as <code>.PHONY</code>. You can always improve it later.</p>
<p>One last basic concept that I want to explain about <code>Makefile</code> is the default
target: it is the target that is run if you just run <code>make</code> without arguments
inside the directory that contains a <code>Makefile</code>. The default target is
generally the first target in the <code>Makefile</code>. It is common to have an <code>all</code>
target (that is also marked as <code>.PHONY</code>) that has as dependencies all the
targets that you want to build by default. In this particular case I declare
the <code>README.md</code> and <code>rss.xml</code> files to be build by default, and they themselves
depends in <code>blog</code> binary being build. So once I run <code>make</code> you get as result:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>$ make
</span></span><span style="display:flex;"><span>go build
</span></span><span style="display:flex;"><span>./blog &gt; README.md
</span></span><span style="display:flex;"><span>./blog -rss &gt; rss.xml
</span></span></code></pre>
</body>
</html>
<p>And this result above highlights the first reason I think you should have a
<code>Makefile</code> or something similar in your projects: you don't need to remember
the exactly steps that you need to get things working. If I see one project of
mine having a <code>Makefile</code>, I can be reasonably confident that I can get it
working by just running <code>make</code>.</p>
<p>But now let's focus in the other targets that I have in the <code>Makefile</code> that are
not related to the build process but are there to help me manage my blog posts.
Remember the rules I explained above? Maybe not, but it should be no problem,
because:</p>
<pre><code>$ make post TITLE=&quot;My new blog post&quot;
mkdir -p &quot;2024-08-13&quot;
./.scripts/gen-post.sh &quot;2024-08-13&quot; &quot;My new blog post&quot;
Creating file: 2024-08-13/02-my-new-blog-post.md

$ cat 2024-08-13/02-my-new-blog-post.md
# My new blog post
</code></pre>
<p>This command, <code>make post</code>, is responsible for:</p>
<ol>
<li>Create a new directory for today, if it doesn't exist</li>
<li>Run the
<a href="https://github.com/thiagokokada/blog/blob/6a3b06970729f7650e5bee5fb0e1f9f2541ffea8/.scripts/gen-post.sh"><code>gen-post.sh</code></a>
script, that:
<ol>
<li>Enumerates all posts from the day, so we can number the new post correctly
<ul>
<li>We already had this post planned for 2024-08-13, so the new post is 02</li>
</ul>
</li>
<li>Slugify the title, so we can create each Markdown file with the correct
filename</li>
<li>Creates a new Markdown file with the title as a <code>h1</code> header</li>
</ol>
</li>
</ol>
<p>The steps above may or may not seen trivial, and for a while I was doing them
manually. But not having to think what is the current date or if I already
posted that day or what is the slug is for the title make (pun intended) my
like much easier.</p>
<p>Yes, the code is ugly. The way variables works in <code>Make</code> is that you can
declare then inside the <code>Makefile</code>, but they can be overwritten in the terminal
if you pass them. I used this to allow <code>make post</code> to also work for future
posts:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>$ make post TITLE<span style="color:#f92672">=</span><span style="color:#e6db74">&#34;Another new blog post&#34;</span> DAY<span style="color:#f92672">=</span>2024-12-12
</span></span><span style="display:flex;"><span>mkdir -p &#34;2024-12-12&#34;
</span></span><span style="display:flex;"><span>./.scripts/gen-post.sh &#34;2024-12-12&#34; &#34;Another new blog post&#34;
</span></span><span style="display:flex;"><span>Creating file: 2024-12-12/01-another-new-blog-post.md
</span></span></code></pre>
</body>
</html>
<p>So in the above case, <code>DAY</code> is filled with the value passed in the terminal
instead of default (that would be the current day), and <code>_PARSED_DAY</code> is the
day we use to actually create the directory. We can actually pass any date
format recognised by
<a href="https://www.gnu.org/software/coreutils/manual/html_node/Examples-of-date.html"><code>date</code></a>,
not just <code>YYYY-MM-DD</code>.</p>
<p>I have 2 other phony targets that I want to talk, <code>draft</code> and <code>undraft</code>. They
expect a <code>FILE</code> to be passed, and I use them to either hide or unhide a file:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>$ make draft FILE<span style="color:#f92672">=</span>2024-12-12/01-another-new-blog-post.md
</span></span><span style="display:flex;"><span>mv &#34;2024-12-12/01-another-new-blog-post.md&#34; &#34;2024-12-12/.01-another-new-blog-post.md&#34;
</span></span><span style="display:flex;"><span><span style="color:#960050;background-color:#1e0010">
</span></span></span><span style="display:flex;"><span><span style="color:#960050;background-color:#1e0010"></span>$ make undraft FILE<span style="color:#f92672">=</span>2024-12-12/.01-another-new-blog-post.md
</span></span><span style="display:flex;"><span>mv &#34;2024-12-12/.01-another-new-blog-post.md&#34; &#34;2024-12-12/01-another-new-blog-post.md&#34;
</span></span></code></pre>
</body>
</html>
<p>Why? Because hidden files are <a href="https://github.com/thiagokokada/blog/blob/894a388c61ca3a38dfc9d4cbe88dc684fd964bb7/blog.go#L101-L104">explicit
ignored</a>
during my directory parser to mean they're a draft post and not ready to be
published. And the reason I created those targets is because I was tired of
trying to hide or unhide a file manually.</p>
<p>So that's it, for the same reason you <a href="https://github.com/thiagokokada/blog/blob/main/posts/2024-07-29/01-quick-bits-why-you-should-automate-everything.md">should probably automate
everything</a>,
you also need to have some way to automate your tasks. <code>Makefile</code> is one way to
do it, maybe not the best way to do it, but it works and it is available
anywhere.</p>
</description>
<guid>https://github.com/thiagokokada/blog/blob/main/posts/2024-08-13/01-abusing-makefiles-for-fun-and-profit.md</guid>
<pubDate>Tue, 13 Aug 2024 00:00:00 +0000</pubDate>
</item>
<item>
<title>Things I don't like in my Chromebook Duet 3</title>
<link>https://github.com/thiagokokada/blog/blob/main/posts/2024-08-12/01-things-i-dont-like-in-my-chromebook-duet-3.md</link>
<description><p>So this is kind of a continuation from my <a href="https://github.com/thiagokokada/blog/blob/main/posts/2024-08-05/01-my-favorite-device-is-a-chromebook.md">previous
post</a> talking why
my favorite device is a Chromebook. In this post I want to talk about what
makes me this device unhappy, and comment about things that if changed would
make it a much better device.</p>
<p>But before talking about the negative aspects, let me talk about a positive
aspect that I just briefly talked in the previous post: the screen. It is a
HiDPI screen (2000x1200 resolution in 10.95''), that is unexpected bright (400
nits according to the
<a href="https://www.lenovo.com/us/en/p/laptops/lenovo/lenovo-edu-chromebooks/ideapad-duet-3-chromebook-11-inch,-qlc/len101i0034">specs</a>).
It is difficult to find laptops at the same price with a screen that good. At
10.95'' in its default resolution I find it too small (like 1250x750), but I
find the font size acceptable at 115% scale (1087x652). Yes, it result in a
small workspace, but this is not a big issue for what I do in this device. It
is also only 60Hz, but I thought I would miss high refresh rate more than I
actually miss in this device.</p>
<p>Update: I forgot to say one thing about the screen: it scratches really easy. I
got my screen scratched after the first day of usage, and considering the price
I don't think the screen has a hardened glass. I bought a cheap glass screen
protector and this did the trick though, even hiding the previous scratch, and
I have zero issues with the screen now.</p>
<p>Now the first aspect that I don't like: the speakers. They sound tiny and even
at maximum volume it is not really loud. The speakers is the only reason why I
still keep my <a href="https://www.gsmarena.com/xiaomi_pad_5-11042.php">Xiaomi Pad 5</a>,
because I like to watch animes/videos before sleep and having good speakers is
a must.</p>
<p>The keyboard has that issue that I mentioned in the previous post: sometimes
the key get stuck, and I get duplicated characters. But it also has some minor
issues that I didn't talked about: the first one is the UK layout that has some
extra keys that I have no use for, but this also makes the keys that I use
smaller. Very much a &quot;me&quot; problem, since if I had got a US version I wouldn't
have those issues, but an issue nonetheless that gets worse considering how
small the keyboard is. I am actually suprised how fast I can type considering
how many issues this keyboard has, so maybe this is a testament that this
keyboard is not actually that bad.</p>
<p>The other keyboard issue is a problem that affects all Chromebooks: its custom
layout. Google replaced a few keys like Fn keys with shortcuts and replaced the
Caps Lock with a
<a href="https://chromeunboxed.com/chromebook-launcher-now-everything-button">&quot;Everything&quot;</a>
key (that is similar to the Windows Key), while removing Windows Key from its
place. I actually have less issue with this than I initially though: I don't
care too much about Fn keys (except when using IntelliJ, but that is something
that I only use at <code>$CURRENT_JOB</code>), and ChromeOS is surprisingly powerful in
its customisation, allowing you to swap key functionality. I remap Everything
key with Esc, and Esc for the Everything key, and I can get productive in my
<code>neovim</code> setup.</p>
<p>And finally, let me talk more about the performance: yes, it is bad, but
bearable once you get used to. The issue is both the CPU and IO. While the CPU,
a <a href="https://www.qualcomm.com/products/mobile/snapdragon/laptops-and-tablets/snapdragon-mobile-compute-platforms/snapdragon-7c-gen-2-compute-platform">Snapdragon 7c Gen
2</a>
is octa-core, it has only 2 high performance CPU cores vs 6 low performance
ones (2xARM Cortex A76 vs 6xARM Cortex A55). If it was something like 4x4, it
would be much better. The fact that the cores are old doesn't help either.</p>
<p>But the worst part is the IO. Not only it uses a eMMC module, it is slow:</p>
<p><a href="https://github.com/thiagokokada/blog/raw/main/posts/2024-08-12/Screenshot_2024-08-12_20.50.42.png"><img src="https://github.com/thiagokokada/blog/raw/main/posts/2024-08-12/Screenshot_2024-08-12_20.50.42.png" alt="CPDT Benchmark results from Chromebook Duet 3."></a></p>
<p>I don't know how much more expensive it would be to put a
<a href="https://en.wikipedia.org/wiki/Universal_Flash_Storage">UFS</a> instead of eMMC in
this device, but this is probably the choice that would most increase
performance in this device, especially considering how aggressive Chromebooks
use (z)swap.</p>
<p>Update 2: I forgot to talk about the fact that the exterior of the device is
covered in cloth. I thought I would hate this at first, but nowadays I kind
like it. And it is also nice that it will never get scratched, I don't care too
much about the exterior of this device and it is the only device that I have at
home that doesn't have additional protection (except the screen protector
mentioned above).</p>
</description>
<guid>https://github.com/thiagokokada/blog/blob/main/posts/2024-08-12/01-things-i-dont-like-in-my-chromebook-duet-3.md</guid>
<pubDate>Mon, 12 Aug 2024 00:00:00 +0000</pubDate>
</item>
<item>
<title>Building static binaries in Nix</title>
<link>https://github.com/thiagokokada/blog/blob/main/posts/2024-08-11/01-building-static-binaries-in-nix.md</link>
<description><p>I recently had laser eye surgery for my near-sightedness, and while evaluating
if I could have the surgery I discovered that I was suffering from dry eyes.
Thanks to this, my ophthalmologist recommended that every 20 minutes while
using screens, I look somewhere 20 feet away for 20 seconds, a technique known
as <a href="https://www.healthline.com/health/eye-health/20-20-20-rule">20-20-20 rule</a>.</p>
<p>I had issues following this rule because I never remembered to do the pauses. I
initially tried to setup an alarm, but this became annoying, so I decided to
try to find a program. I found
<a href="https://tonyh4156.github.io/20-20-20.github.io/">this</a> one for macOS that
worked fine, but it bothered me that it was compiled for x86_64 while I was
using a M1 MacBook Pro at the time, and also I needed something that worked in
Linux.</p>
<p>Since I never found a good multi-platform alternative, I decided to write my
own. This became
<a href="https://github.com/thiagokokada/twenty-twenty-twenty/">twenty-twenty-twenty</a>,
the first <a href="https://github.com/thiagokokada/blog/blob/main/posts/2024-07-29/02-go-a-reasonable-good-language.md">Go</a> program
that I ever wrote. I wrote it in Go because I wanted to learn the language, but
also because Go made it easy to build static binaries. And the first version I
could build static binaries without issues because I was using
<a href="https://github.com/gen2brain/beeep">beeep</a>, that uses pure Go code in all
supported platforms. However, it also meant that the notifications in macOS
looked ugly, since it used
<a href="https://github.com/gen2brain/beeep/blob/master/beep_darwin.go#L20">osascript</a>.</p>
<p>I wanted better integration with macOS, so this meant switching libraries.
After searching for a while, the
<a href="https://pkg.go.dev/gioui.org/x/notify">notify</a> library from
<a href="https://gioui.org/">GioUI</a> is the one that seemed to work better. It
implements notification in macOS using its native framework, so it works much
better, but sadly it meant losing static binaries because it depends in CGO.</p>
<p>Not a big loss initially, because I am only depending in Foundation inside
macOS (that should always be available), and in Linux I could still statically
compile. However I eventually added more features like sound
(via <a href="https://github.com/gopxl/beep">beep</a>) and tray icon (via
<a href="https://github.com/fyne-io/systray">systray</a>), that meant I needed CGO in both
macOS and Linux.</p>
<p>Losing static binaries in Linux is a much bigger deal, since Linux is a moving
target. The general recommendation for building CGO binaries statically is
using
<a href="https://eli.thegreenplace.net/2024/building-static-binaries-with-go-on-linux/">musl</a>,
but this also means building all dependencies that we need using musl (e.g.:
<a href="https://github.com/ebitengine/oto?tab=readme-ov-file#linux"><code>ALSA</code></a> for
[beep/oto]). This generally means pain, but Nix makes it easy.</p>
<p>Let's start by creating a <a href="https://github.com/thiagokokada/twenty-twenty-twenty/blob/main/twenty-twenty-twenty.nix">Nix
file</a>
that builds our Go module (simplified below for brevity):</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>{ lib
</span></span><span style="display:flex;"><span><span style="color:#f92672">,</span> stdenv
</span></span><span style="display:flex;"><span><span style="color:#f92672">,</span> alsa-lib
</span></span><span style="display:flex;"><span><span style="color:#f92672">,</span> buildGoModule
</span></span><span style="display:flex;"><span><span style="color:#f92672">,</span> pkg-config
</span></span><span style="display:flex;"><span><span style="color:#f92672">,</span> withStatic <span style="color:#f92672">?</span> <span style="color:#66d9ef">false</span>
</span></span><span style="display:flex;"><span>}:
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>buildGoModule {
</span></span><span style="display:flex;"><span> pname <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;twenty-twenty-twenty&#34;</span>;
</span></span><span style="display:flex;"><span> version <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;1.0.0&#34;</span>;
</span></span><span style="display:flex;"><span> src <span style="color:#f92672">=</span> lib<span style="color:#f92672">.</span>cleanSource <span style="color:#e6db74">./.</span>;
</span></span><span style="display:flex;"><span> vendorHash <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;sha256-NzDhpJRogIfL2IYoqAUHoPh/ZdNnvnhEQ+kn8A+ZyBw=&#34;</span>;
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> CGO_ENABLED <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;1&#34;</span>;
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> nativeBuildInputs <span style="color:#f92672">=</span> lib<span style="color:#f92672">.</span>optionals (stdenv<span style="color:#f92672">.</span>hostPlatform<span style="color:#f92672">.</span>isLinux) [
</span></span><span style="display:flex;"><span> pkg-config
</span></span><span style="display:flex;"><span> ];
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> buildInputs <span style="color:#f92672">=</span> lib<span style="color:#f92672">.</span>optionals (stdenv<span style="color:#f92672">.</span>hostPlatform<span style="color:#f92672">.</span>isLinux) [
</span></span><span style="display:flex;"><span> alsa-lib
</span></span><span style="display:flex;"><span> ];
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> ldflags <span style="color:#f92672">=</span> [ <span style="color:#e6db74">&#34;-X=main.version=</span><span style="color:#e6db74">${</span>version<span style="color:#e6db74">}</span><span style="color:#e6db74">&#34;</span> <span style="color:#e6db74">&#34;-s&#34;</span> <span style="color:#e6db74">&#34;-w&#34;</span> ]
</span></span><span style="display:flex;"><span> <span style="color:#f92672">++</span> lib<span style="color:#f92672">.</span>optionals withStatic [ <span style="color:#e6db74">&#34;-linkmode external&#34;</span> <span style="color:#e6db74">&#39;&#39;-extldflags &#34;-static&#34;&#39;&#39;</span> ];
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> meta <span style="color:#f92672">=</span> <span style="color:#66d9ef">with</span> lib; {
</span></span><span style="display:flex;"><span> description <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;Alerts every 20 minutes to look something at 20 feet away for 20 seconds&#34;</span>;
</span></span><span style="display:flex;"><span> homepage <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;https://github.com/thiagokokada/twenty-twenty-twenty&#34;</span>;
</span></span><span style="display:flex;"><span> license <span style="color:#f92672">=</span> licenses<span style="color:#f92672">.</span>mit;
</span></span><span style="display:flex;"><span> mainProgram <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;twenty-twenty-twenty&#34;</span>;
</span></span><span style="display:flex;"><span> };
</span></span><span style="display:flex;"><span>}
</span></span></code></pre>
</body>
</html>
<p>And we can build it with the following <code>flake.nix</code>:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>{
</span></span><span style="display:flex;"><span> description <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;twenty-twenty-twenty&#34;</span>;
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> inputs <span style="color:#f92672">=</span> {
</span></span><span style="display:flex;"><span> nixpkgs<span style="color:#f92672">.</span>url <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;nixpkgs/nixpkgs-unstable&#34;</span>;
</span></span><span style="display:flex;"><span> flake-compat<span style="color:#f92672">.</span>url <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;github:edolstra/flake-compat&#34;</span>;
</span></span><span style="display:flex;"><span> };
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> outputs <span style="color:#f92672">=</span> { self<span style="color:#f92672">,</span> nixpkgs<span style="color:#f92672">,</span> <span style="color:#f92672">...</span> }:
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">let</span>
</span></span><span style="display:flex;"><span> supportedSystems <span style="color:#f92672">=</span> [ <span style="color:#e6db74">&#34;x86_64-linux&#34;</span> <span style="color:#e6db74">&#34;aarch64-linux&#34;</span> ];
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#75715e"># Helper function to generate an attrset &#39;{ x86_64-linux = f &#34;x86_64-linux&#34;; ... }&#39;.</span>
</span></span><span style="display:flex;"><span> forAllSystems <span style="color:#f92672">=</span> nixpkgs<span style="color:#f92672">.</span>lib<span style="color:#f92672">.</span>genAttrs supportedSystems;
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#75715e"># Nixpkgs instantiated for supported system types.</span>
</span></span><span style="display:flex;"><span> nixpkgsFor <span style="color:#f92672">=</span> forAllSystems (system: <span style="color:#f92672">import</span> nixpkgs { <span style="color:#66d9ef">inherit</span> system; });
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">in</span>
</span></span><span style="display:flex;"><span> {
</span></span><span style="display:flex;"><span> packages <span style="color:#f92672">=</span> forAllSystems (system:
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">let</span> pkgs <span style="color:#f92672">=</span> nixpkgsFor<span style="color:#f92672">.</span><span style="color:#e6db74">${</span>system<span style="color:#e6db74">}</span>;
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">in</span> {
</span></span><span style="display:flex;"><span> default <span style="color:#f92672">=</span> self<span style="color:#f92672">.</span>packages<span style="color:#f92672">.</span><span style="color:#e6db74">${</span>system<span style="color:#e6db74">}</span><span style="color:#f92672">.</span>twenty-twenty-twenty;
</span></span><span style="display:flex;"><span> twenty-twenty-twenty <span style="color:#f92672">=</span> pkgs<span style="color:#f92672">.</span>callPackage <span style="color:#e6db74">./twenty-twenty-twenty.nix</span> { };
</span></span><span style="display:flex;"><span> twenty-twenty-twenty-static <span style="color:#f92672">=</span> pkgs<span style="color:#f92672">.</span>pkgsStatic<span style="color:#f92672">.</span>callPackage <span style="color:#e6db74">./twenty-twenty-twenty.nix</span> {
</span></span><span style="display:flex;"><span> withStatic <span style="color:#f92672">=</span> <span style="color:#66d9ef">true</span>;
</span></span><span style="display:flex;"><span> };
</span></span><span style="display:flex;"><span> });
</span></span><span style="display:flex;"><span> };
</span></span><span style="display:flex;"><span>}
</span></span></code></pre>
</body>
</html>
<p>I think this shows how powerful Nix is: the only difference between the normal
build and a static build the usage of <code>pkgs.pkgsStatic</code> instead of <code>pkgs</code>. This
automatically builds all packages statically with <code>musl</code>. Also we need pass
some <a href="https://honnef.co/articles/statically-compiled-go-programs-always-even-with-cgo-using-musl/">extra
flags</a>
to the Go compiler (i.e.: <code>-linkmode external -extldflags &quot;-static&quot;</code>), but this
is a requirement from Go.</p>
<p>So, does it work? Let's test:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>$ nix build .#twenty-twenty-twenty-static
</span></span><span style="display:flex;"><span><span style="color:#960050;background-color:#1e0010">
</span></span></span><span style="display:flex;"><span><span style="color:#960050;background-color:#1e0010"></span>$ file result/bin/twenty-twenty-twenty
</span></span><span style="display:flex;"><span>result/bin/twenty-twenty-twenty: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, stripped
</span></span><span style="display:flex;"><span><span style="color:#960050;background-color:#1e0010">
</span></span></span><span style="display:flex;"><span><span style="color:#960050;background-color:#1e0010"></span>$ ./result/bin/twenty-twenty-twenty
</span></span><span style="display:flex;"><span>ALSA lib pcm.c:2712:(snd_pcm_open_conf) Either /nix/store/dhn51w2km4fyf9ivi00rz03qs8q4mpng-pipewire-1.2.1/lib/alsa-lib/libasound_module_pcm_pipewire.so cannot be opened or _snd_pcm_pipewire_open was not defined inside
</span></span><span style="display:flex;"><span>ALSA lib pcm.c:2712:(snd_pcm_open_conf) Either /nix/store/ly9d7llymzjyf6gi1455qzayqipk2kab-pipewire-1.2.1/lib/alsa-lib/libasound_module_pcm_pipewire.so cannot be opened or _snd_pcm_pipewire_open was not defined inside
</span></span><span style="display:flex;"><span>ALSA lib pcm.c:2712:(snd_pcm_open_conf) Either /nix/store/dhn51w2km4fyf9ivi00rz03qs8q4mpng-pipewire-1.2.1/lib/alsa-lib/libasound_module_pcm_pipewire.so cannot be opened or _snd_pcm_pipewire_open was not defined inside
</span></span><span style="display:flex;"><span>ALSA lib pcm.c:2712:(snd_pcm_open_conf) Either /nix/store/ly9d7llymzjyf6gi1455qzayqipk2kab-pipewire-1.2.1/lib/alsa-lib/libasound_module_pcm_pipewire.so cannot be opened or _snd_pcm_pipewire_open was not defined inside
</span></span><span style="display:flex;"><span>ALSA lib pcm.c:2712:(snd_pcm_open_conf) Either /nix/store/dhn51w2km4fyf9ivi00rz03qs8q4mpng-pipewire-1.2.1/lib/alsa-lib/libasound_module_pcm_pipewire.so cannot be opened or _snd_pcm_pipewire_open was not defined inside
</span></span><span style="display:flex;"><span>ALSA lib pcm.c:2712:(snd_pcm_open_conf) Either /nix/store/ly9d7llymzjyf6gi1455qzayqipk2kab-pipewire-1.2.1/lib/alsa-lib/libasound_module_pcm_pipewire.so cannot be opened or _snd_pcm_pipewire_open was not defined inside
</span></span><span style="display:flex;"><span>2024-08-11T19:26:33+01:00 INFO Running twenty-twenty-twenty every 20.0 minute(s), with 20 second(s) duration and sound set to true
</span></span></code></pre>
</body>
</html>
<p>There are some warns and sadly the sound doesn't work. I think the issue is
related because of my usage of PipeWire and the binary may work in a pure ALSA
system, but I don't have access to one. Maybe adding <code>pipewire</code> to
<code>buildInputs</code> would fix this issue, but I can't get <code>pipewire</code> to be compiled
statically (because of its dependencies). I think this is a good show how easy
it is to statically compilation is in Nix, but also how complex static binaries
are to get correctly.</p>
<p>Bonus points for
<a href="https://nix.dev/tutorials/cross-compilation.html">cross-compilation</a>. We can
easily cross-compile by using <code>pkgsCross</code>:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>{
</span></span><span style="display:flex;"><span> <span style="color:#75715e"># ...</span>
</span></span><span style="display:flex;"><span> outputs <span style="color:#f92672">=</span> { self<span style="color:#f92672">,</span> nixpkgs<span style="color:#f92672">,</span> <span style="color:#f92672">...</span> }:
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">let</span>
</span></span><span style="display:flex;"><span> <span style="color:#75715e"># ...</span>
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">in</span>
</span></span><span style="display:flex;"><span> {
</span></span><span style="display:flex;"><span> packages <span style="color:#f92672">=</span> forAllSystems (system:
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">let</span> pkgs <span style="color:#f92672">=</span> nixpkgsFor<span style="color:#f92672">.</span><span style="color:#e6db74">${</span>system<span style="color:#e6db74">}</span>;
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">in</span> {
</span></span><span style="display:flex;"><span> twenty-twenty-twenty-static-aarch64 <span style="color:#f92672">=</span> pkgs<span style="color:#f92672">.</span>pkgsCross<span style="color:#f92672">.</span>aarch64-multiplatform<span style="color:#f92672">.</span>pkgsStatic<span style="color:#f92672">.</span>callPackage <span style="color:#e6db74">./twenty-twenty-twenty.nix</span> {
</span></span><span style="display:flex;"><span> withStatic <span style="color:#f92672">=</span> <span style="color:#66d9ef">true</span>;
</span></span><span style="display:flex;"><span> };
</span></span><span style="display:flex;"><span> });
</span></span><span style="display:flex;"><span> };
</span></span><span style="display:flex;"><span>}
</span></span></code></pre>
</body>
</html>
<p>The idea of <code>pkgsCross</code> is to select a target platform (e.g.:
<code>aarch64-multiplatform</code>) and use it as any other <code>pkgs</code>. We can even chain
<code>pkgsStatic</code> to statically cross compile binaries:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>$ nix build .#twenty-twenty-twenty-static-aarch64
</span></span><span style="display:flex;"><span><span style="color:#960050;background-color:#1e0010">
</span></span></span><span style="display:flex;"><span><span style="color:#960050;background-color:#1e0010"></span>$ file result/bin/twenty-twenty-twenty
</span></span><span style="display:flex;"><span>result/bin/twenty-twenty-twenty: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, stripped
</span></span></code></pre>
</body>
</html>
</description>
<guid>https://github.com/thiagokokada/blog/blob/main/posts/2024-08-11/01-building-static-binaries-in-nix.md</guid>
<pubDate>Sun, 11 Aug 2024 00:00:00 +0000</pubDate>
</item>
<item>
<title>Meta: are quick bits really quick?</title>
<link>https://github.com/thiagokokada/blog/blob/main/posts/2024-08-07/02-meta-are-quick-bits-really-quick.md</link>
<description><p>When I wrote my first <a href="https://github.com/thiagokokada/blog/blob/main/posts/2024-07-27/01-quick-bits-nix-shell-is-cursed.md">quick
bits</a> post in this
blog, I gave that title prefix without much thought: I knew it was supposed to
be a quicker post than my previous one, but I never thought too deeply about
it. But after:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>$ ls -lah **/*quick-bits*.md | wc -l
</span></span><span style="display:flex;"><span>4
</span></span></code></pre>
</body>
</html>
<p>Well, 4 blog posts starting with the same prefix, I was curious: are those
quick bits really quick, or at least quicker? Let's see:</p>
<pre><code>$ wc -w **/*.md
 1107 2024-07-26/01-writing-nixos-tests-for-fun-and-profit.md
 1220 2024-07-26/02-using-github-as-a-bad-blog-platform.md
 286 2024-07-27/01-quick-bits-nix-shell-is-cursed.md
 387 2024-07-29/01-quick-bits-why-you-should-automate-everything.md
 1060 2024-07-29/02-go-a-reasonable-good-language.md
 1380 2024-07-30/01-first-impressions-fpgbc.md
 1238 2024-07-31/01-generating-yaml-files-with-nix.md
 2308 2024-08-01/01-troubleshooting-zsh-lag-and-solutions-with-nix.md
 504 2024-08-01/02-quick-bits-realise-nix-symlinks.md
 834 2024-08-04/01-make-nixd-modules-completion-work-anywhere-with-flakes.md
 1147 2024-08-05/01-my-favorite-device-is-a-chromebook.md
 394 2024-08-07/01-quick-bits-is-crostini-a-microvm.md
 120 README.md
11985 total
</code></pre>
<p>While using <code>wc</code> is probably not the best way to measure word count (especially
in this blog, since I tend to write lots of code snippets), I think this at
least it gives me a good insight: yes, quick bits are quicker, and they're
basically posts with a soft limit around 500 words. So expect in future this
limit to be used.</p>
<p>By the way, at:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>$ wc -w 2024-08-07/02-meta-are-quick-bits-really-quick.md
</span></span><span style="display:flex;"><span>220 2024-08-07/02-meta-are-quick-bits-really-quick.md
</span></span></code></pre>
</body>
</html>
<p>This post is also technically a quick bits post, but &quot;quick bits meta&quot; would be
too much. And yes, that last block of code is also meta ;).</p>
</description>
<guid>https://github.com/thiagokokada/blog/blob/main/posts/2024-08-07/02-meta-are-quick-bits-really-quick.md</guid>
<pubDate>Wed, 07 Aug 2024 00:00:00 +0000</pubDate>
</item>
<item>
<title>Quick bits: is Crostini a micro VM?</title>
<link>https://github.com/thiagokokada/blog/blob/main/posts/2024-08-07/01-quick-bits-is-crostini-a-micro-vm.md</link>
<description><p><strong>Disclaimer</strong>: I am not a Virtual Machine specialist, this post is mainly the
conclusion I got after some searching.</p>
<p>Someone asked me in my <a href="https://github.com/thiagokokada/blog/blob/main/posts/2024-08-05/01-my-favorite-device-is-a-chromebook.md">previous
post</a> about my
Chromebook if Crostini could be considered a micro VM. This was a interesting
question, so I decided to do another post.</p>
<p>To start, it is really difficult to get a good definition of what a &quot;micro VM&quot;
is. <a href="https://firecracker-microvm.github.io/">Firecracker</a> defines itself as a
micro VM, and describes itself in its website as:</p>
<blockquote>
<p>Firecracker is a virtual machine monitor (VMM) that uses the Linux
Kernel-based Virtual Machine (KVM) to create and manage microVMs. Firecracker
has a minimalist design. It excludes unnecessary devices and guest
functionality to reduce the memory footprint and attack surface area of each
microVM. This improves security, decreases the startup time, and increases
hardware utilization.</p>
</blockquote>
<p>Now looking at Crostini, its heart is a VMM called
<a href="https://crosvm.dev/">crosvm</a>. It is described in its
<a href="https://chromium.googlesource.com/chromiumos/platform/crosvm/+/HEAD/README.md">README</a>
as:</p>
<blockquote>
<p>crosvm is a virtual machine monitor (VMM) based on Linux’s KVM hypervisor,
with a focus on simplicity, security, and speed. crosvm is intended to run
Linux guests, originally as a security boundary for running native
applications on the ChromeOS platform. Compared to QEMU, crosvm doesn’t
emulate architectures or real hardware, instead concentrating on
paravirtualized devices, such as the virtio standard.</p>
</blockquote>
<p>Similar descriptions right? Actually Firecracker website says it &quot;started from
Chromium OS's Virtual Machine Monitor, crosvm, an open source VMM written in
Rust&quot;. So I would say it is safe to say crosvm itself is a micro VM.</p>
<p>But
<a href="https://www.chromium.org/chromium-os/developer-library/guides/containers/containers-and-vms/">Crostini</a>
itself is a combination of virtualization AND containerization. Basically
inside the VM it runs a Linux kernel and <a href="https://linuxcontainers.org/">LXC</a>,
that can start arbitrary containers inside it. From the Crostini documentation
this choice seems to be to keep startup times down, and also to increase
security (e.g.: in case of a security issue inside the container).</p>
<p>This is definitely an interesting choice, since containers allow the overhead
of each distro that you run inside Crostini to be low, and the main VM itself
(called
<a href="https://chromium.googlesource.com/chromiumos/overlays/board-overlays/+/HEAD/project-termina/">Termina</a>)
should have low overhead too thanks to crosvm.</p>
<p>By the way, if you want to learn more how &quot;devices&quot; works inside a micro VM
like crosvm, I recommend <a href="https://prilik.com/blog/post/crosvm-paravirt/">this blog
post</a> talking about
paravirtualized devices in crosvm.</p>
</description>
<guid>https://github.com/thiagokokada/blog/blob/main/posts/2024-08-07/01-quick-bits-is-crostini-a-micro-vm.md</guid>
<pubDate>Wed, 07 Aug 2024 00:00:00 +0000</pubDate>
</item>
<item>
<title>My favorite device is a Chromebook</title>
<link>https://github.com/thiagokokada/blog/blob/main/posts/2024-08-05/01-my-favorite-device-is-a-chromebook.md</link>
<description><p>Most of the posts in this blog (including this one) and most of I would call
&quot;personal computing&quot; that I do nowadays is mostly done in one of the most
unremarkable devices that I own: a cheap <a href="https://www.lenovo.com/us/en/p/laptops/lenovo/lenovo-edu-chromebooks/ideapad-duet-3-chromebook-11-inch,-qlc/len101i0034">Chromebook Duet
3</a>,
that I bought for around EUR300. I was thinking why, because it is woefully
underpowered: a <a href="https://www.anandtech.com/show/16696/qualcomm-announces-snapdragon-7c-gen-2-entrylevel-pc-and-chromebook-refresh">Snapdragon 7c Gen
2</a>,
a CPU that was already considered slow 3 years ago, coupled with an eMMC for
storage, that is not much faster than a HDD. At least I have the 8GB RAM
version instead of the 4GB one.</p>
<p>It is a hybrid device, one that can be used as either a tablet or laptop, but
it is compromised experience in both cases: as a tablet, it lacks the better
touch optimised interface from iOS or Android; as a laptop, you have to depend
on the stand to adjust the screen, and the detachable keyboard is worse than
any laptop I have ever owned: getting keys stucked and characters being
duplicated as a result is a common occurence. It is not so bad that I can't get
things done though. About the trackpad: its biggest quality is that I never
feel the need to use the touchscreen in laptop mode, that is to say that it is
acceptable. Just crank up the pointer speed in ChromeOS settings, otherwise you
never get anywhere since the trackpad is so small. There is also an active
stylus, that helped me sometimes when I needed to sign something but otherwise
I can't comment too much.</p>
<p>But I really love this device. It is generally the only device that I bring in
trips nowadays, because while it is compromised it works well enough: I can use
to consume media in tablet mode (the fact that ChromeOS supports Android apps
is a plus in those cases), browse the web and even do Linux stuff (more about
this later). The fact that it is small (the size remembers me of a
<a href="https://en.wikipedia.org/wiki/Netbook">netbook</a>), lightweight (~1KG, including
the keyboard), has a good screen (that is bright and HiDPI) and good battery
life (I don't have numbers but I almost never worry about it) is what makes
this device the perfect companion to trips.</p>
<p>Also, it has 2 USB-C ports and supports DisplayPort alt-mode, so it means you
can charge it, connect to a external display and peripherals, all at the same
time. Sadly, the maximum output resolution I got was 1080p (2560x1080),
although some people at Reddit <a href="https://www.reddit.com/r/chromeos/comments/zh27tg/comment/izku724/?utm_source=share&amp;utm_medium=web3x&amp;utm_name=web3xcss&amp;utm_term=1&amp;utm_content=share_button">seems to have
success</a>
at 1440p, and the specs suggests it supports 4k. It may be my Dell S3423DWC
monitor being wonky, the fact that it is Ultrawide or the cable, who knows? I
even tried to change the monitor to &quot;High Resolution&quot; mode in settings, but to
no avail.</p>
<p><em>Update:</em> looking at the <a href="https://www.qualcomm.com/content/dam/qcomm-martech/dm-assets/documents/prod_brief_qcom_sd7c_gen2.pdf">product
brief</a>
for Snapdragon 7c Gen 2, it seems it supports up to QHD@60Hz (1440p) for the
external display, and not 4k. This explains why it doesn't work at maximum
resolution in my Dell S3423DWC, since while it is 1440p it is Ultrawide, so the
resolution is bigger than QHD (3440x1440 vs 2560x1440).</p>
<p>ChromeOS is also really interesting nowadays. To start, it is designed from the
ground up to be a <a href="https://support.google.com/chromebook/answer/3438631">secure computing
environment</a>, probably
the most secure OS for consumers right now. Being a Chrome-first OS makes it a
compromised experience, for example, it is the only device that I use Chrome as
my main browser (since I personally prefer Firefox). But having a OS that boots
fast is great: I never worry about OS updates because I know the device will be
ready in seconds after a reboot. And the whole desktop experience inside the
ChromeOS desktop is good, having shortcuts for many operations so you can get
things done fast, and support for virtual desktops (ChromeOS call it &quot;desks&quot;)
means you can organise your windows as much as you want.</p>
<p>And what I think makes ChromeOS really powerful is
<a href="https://chromeos.dev/en/linux">Crostini</a>, a full Linux VM that you can run
inside ChromeOS. It runs Debian (it seems you can <a href="https://www.reddit.com/r/Crostini/wiki/howto/run-other-distros/">run other
distros</a>
though) with a deep integration with ChromeOS, so you can run even graphical
programs without issues (including OpenGL!):</p>
<p><a href="https://github.com/thiagokokada/blog/raw/main/posts/2024-08-05/Screenshot_2024-08-05_21.22.29.png"><img src="https://github.com/thiagokokada/blog/raw/main/posts/2024-08-05/Screenshot_2024-08-05_21.22.29.png" alt="Fastfetch inside Crostini with gitk running side-by-side."></a></p>
<p><a href="https://github.com/thiagokokada/blog/raw/main/posts/2024-08-05/Screenshot_2024-08-05_21.39.58.png"><img src="https://github.com/thiagokokada/blog/raw/main/posts/2024-08-05/Screenshot_2024-08-05_21.39.58.png" alt="Running glxgears inside Crostini."></a></p>
<p>This is all thanks to
<a href="https://chromium.googlesource.com/chromiumos/platform2/+/HEAD/vm_tools/sommelier/README.md">Sommelier</a>,
a nested Wayland compositor that runs inside Crostini and allow both Wayland
and X11 applications to be forwarded to ChromeOS. The integration is so good
that I can run Firefox inside Crostini and works well enough, but sadly Firefox
is too slow in this device (I am not sure if the issue is ChromeOS or Firefox,
but I suspect the later since Google does some optimisation per device).</p>
<p>One interesting tidbit about the OpenGL situation in this device: this seems to
be the first Chromebook to ship with open source drivers, thanks to Freedreno.
There is <a href="https://www.youtube.com/watch?v=8mnjSmN03VM">this</a> very interesting
presentation done by Rob Clark in XDC 2021, that I recommended anyone
interested in free drivers to watch (the reference design of Duet 3 is
<a href="https://chromeunboxed.com/chromebook-tablet-snapdragon-7c-homestar-coachz-strongbad">Strongbad</a>).</p>
<p>The Crostini integration is probably the best VM integration with Linux I ever
saw in an OS: you can manage files inside the VM, share directories between the
OS and VM, copy and paste works between the two, GUI applications installed
inside the VM appear in the ChromeOS menu, memory allocation inside the VM is
transparent, etc. Even the themes for Linux GUI applications are customised to
match ChromeOS. It is unironically one of the best Linux desktop experiences I
ever had.</p>
<p>Of course I am using Nix, but since the Crostini integration depends on some
services being configured and installed, I decided to run Nix inside Debian
instead of NixOS and run <a href="https://nix-community.github.io/home-manager/index.xhtml#sec-install-standalone">Home-Manager
standalone</a>.
I recommend checking the official <a href="https://wiki.nixos.org/wiki/Installing_Nix_on_Crostini">NixOS Wiki article about
Crostini</a>, that details
how to register applications in ChromeOS (so desktop applications appear in
menu) and use <a href="https://github.com/nix-community/nixGL">nixGL</a> to make OpenGL
applications work.</p>
<p>Like I said at the start of the article, the device is woefully slow thanks to
its CPU and eMMC. It does mean that, for example, activating my Home-Manager
configuration takes a while (around 1 minute, vs a few seconds in my laptop).
But it is much faster than say,
<a href="https://github.com/nix-community/nix-on-droid-app">nix-on-droid</a>, that the
last time I tried in a much more powerful device (<a href="https://www.gsmarena.com/xiaomi_pad_5-11042.php">Xiaomi Pad
5</a>), took 30 minutes until I
just decided to cancel the operation. Having a proper VM instead of
<a href="https://wiki.termux.com/wiki/PRoot">proot</a> makes all the difference here.</p>
<p>I can even do some light programming here: using my
<a href="https://github.com/thiagokokada/blog/blob/main/posts/2024-08-01/01-troubleshoting-zsh-lag-and-solutions-with-nix.md">ZSH</a>
and neovim configuration (including LSP for coding) is reasonable fast. For
example, I did most of the code that <a href="https://github.com/thiagokokada/blog/blob/main/posts/2024-07-29/01-quick-bits-why-you-should-automate-everything.md">powers this
blog</a>
using this Chromebook. If I need more power, I can use the <a href="https://tailscale.com/kb/1267/install-chromebook">Tailscale app for
Android</a> to connect to any
other of my hosts via SSH. Yes, the Tailscale app works in Crostini, sadly
without MagicDNS, so you need to use the internal Tailscale IPs instead.</p>
<p>Until Google decides to give us a proper VM or user namespaces in Android or
release a hybrid Chromebook device with better specs, this small Chromebook
will probably stay as my travel companion, and is one of my favorite devices.</p>
</description>
<guid>https://github.com/thiagokokada/blog/blob/main/posts/2024-08-05/01-my-favorite-device-is-a-chromebook.md</guid>
<pubDate>Mon, 05 Aug 2024 00:00:00 +0000</pubDate>
</item>
<item>
<title>Make nixd module completion to work anywhere (with Flakes)</title>
<link>https://github.com/thiagokokada/blog/blob/main/posts/2024-08-04/01-make-nixd-module-completion-to-work-anywhere-with-flakes.md</link>
<description><p>If you want the TL;DR, go to the bottom of the post (search for &quot;final
result&quot;).</p>
<p>I recently switched from <a href="https://github.com/oxalica/nil">nil</a> to
<a href="https://github.com/nix-community/nixd">nixd</a> as my LSP of choice for Nix. I
was curious in <code>nixd</code> for a long time since the fact that it can eval Nix code
means it can offer much more powerful completion than the <code>nil</code>'s static
analysis, however it used to be difficult to setup. Nowadays it is much easier,
basically doing the right thing as long as you have <code>NIX_PATH</code> setup, and you
get both package and NixOS modules completion.</p>
<p>Getting Home-Manager modules though needs some setup. The recommended way to
setup accordingly to the <a href="https://github.com/nix-community/nixd/blob/fe202307eaf7e89c4366ed927af761482a6065c8/nixd/docs/configuration.md">official
documentation</a>
is to use the following for Flake based configurations (using neovim
configuration here, but it should be easy to adapt to other editors):</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>{
</span></span><span style="display:flex;"><span> nixpkgs <span style="color:#f92672">=</span> {
</span></span><span style="display:flex;"><span> expr <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;import &lt;nixpkgs&gt; { }&#34;</span>,
</span></span><span style="display:flex;"><span> },
</span></span><span style="display:flex;"><span> options <span style="color:#f92672">=</span> {
</span></span><span style="display:flex;"><span> nixos <span style="color:#f92672">=</span> {
</span></span><span style="display:flex;"><span> expr <span style="color:#f92672">=</span> <span style="color:#e6db74">&#39;(builtins.getFlake (&#34;git+file://&#34; + toString ./.)).nixosConfigurations.miku-nixos.options&#39;</span>,
</span></span><span style="display:flex;"><span> },
</span></span><span style="display:flex;"><span> home_manager <span style="color:#f92672">=</span> {
</span></span><span style="display:flex;"><span> expr <span style="color:#f92672">=</span> <span style="color:#e6db74">&#39;(builtins.getFlake (&#34;git+file://&#34; + toString ./.)).homeConfigurations.home-linux.options&#39;</span>,
</span></span><span style="display:flex;"><span> },
</span></span><span style="display:flex;"><span> },
</span></span><span style="display:flex;"><span> <span style="color:#75715e">-- ...</span>
</span></span><span style="display:flex;"><span>}
</span></span></code></pre>
</body>
</html>
<p>This works, but it should be pretty clear the limitations of using <code>./.</code>: this
will only work if you open your editor in your <a href="https://github.com/thiagokokada/nix-configs">Nix configuration
repository</a>. For any other
repository, <code>nixosConfigurations.miku-nixos</code> or <code>homeConfigurations.home-linux</code>
will not exist and the completion will not work.</p>
<p>It may look like this is easy to fix if you have
<a href="https://nixos.org/manual/nixos/unstable/options#opt-_module.args"><code>specialArgs</code></a>
(or <code>extraSpecialArgs</code> in Home-Manager) set to have your
Flakes inputs, but:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span><span style="color:#75715e"># By the way, ${self} does not exist in the Flake output by default, you need</span>
</span></span><span style="display:flex;"><span><span style="color:#75715e"># to explicit add `inherit self` to your outputs:</span>
</span></span><span style="display:flex;"><span><span style="color:#75715e"># https://discourse.nixos.org/t/who-is-self-in-flake-outputs/31859/4</span>
</span></span><span style="display:flex;"><span>nix-repl<span style="color:#f92672">&gt;</span> (builtins<span style="color:#f92672">.</span>getFlake <span style="color:#e6db74">&#34;git+file://</span><span style="color:#e6db74">${</span>self<span style="color:#e6db74">}</span><span style="color:#e6db74">&#34;</span>)<span style="color:#f92672">.</span>nixosConfigurations<span style="color:#f92672">.</span>miku-linux<span style="color:#f92672">.</span>options
</span></span><span style="display:flex;"><span>error:
</span></span><span style="display:flex;"><span> <span style="color:#960050;background-color:#1e0010">…</span> while calling the <span style="color:#960050;background-color:#1e0010">&#39;</span>getFlake&#39; builtin
</span></span><span style="display:flex;"><span> at <span style="color:#960050;background-color:#1e0010">«</span>string<span style="color:#960050;background-color:#1e0010">»</span>:<span style="color:#ae81ff">1</span>:<span style="color:#ae81ff">2</span>:
</span></span><span style="display:flex;"><span> <span style="color:#ae81ff">1</span><span style="color:#960050;background-color:#1e0010">|</span> (builtins<span style="color:#f92672">.</span>getFlake <span style="color:#e6db74">&#34;git+file://</span><span style="color:#e6db74">${</span>self<span style="color:#e6db74">}</span><span style="color:#e6db74">&#34;</span>)
</span></span><span style="display:flex;"><span> <span style="color:#960050;background-color:#1e0010">|</span> <span style="color:#960050;background-color:#1e0010">^</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#960050;background-color:#1e0010">…</span> while evaluating the argument passed to builtins<span style="color:#f92672">.</span>getFlake
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> error: the string <span style="color:#960050;background-color:#1e0010">&#39;</span><span style="color:#e6db74">git+file:///nix/store/avr1lcmznj8ghynh5vj1kakgfdf0zrxx-source&#39;</span> is not allowed to refer to a store path (such as <span style="color:#960050;background-color:#1e0010">&#39;</span>avr1lcmznj8ghynh5vj1kakgfdf0zrxx-source&#39;)
</span></span></code></pre>
</body>
</html>
<p>Well, it was worth a try. Another option would be to:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>(builtins<span style="color:#f92672">.</span>getFlake <span style="color:#e6db74">&#34;github:thiagokokada/nix-configs&#34;</span>)<span style="color:#f92672">.</span>nixosConfigurations<span style="color:#f92672">.</span>miku-linux<span style="color:#f92672">.</span>options
</span></span><span style="display:flex;"><span><span style="color:#75715e"># Or even something like this</span>
</span></span><span style="display:flex;"><span><span style="color:#75715e"># However, using ${rev} means this wouldn&#39;t work in dirty Flake repos, since</span>
</span></span><span style="display:flex;"><span><span style="color:#75715e"># ${rev} is not set in those cases</span>
</span></span><span style="display:flex;"><span>(builtins<span style="color:#f92672">.</span>getFlake <span style="color:#e6db74">&#34;github:thiagokokada/nix-configs/</span><span style="color:#e6db74">${</span>rev<span style="color:#e6db74">}</span><span style="color:#e6db74">&#34;</span>)<span style="color:#f92672">.</span>nixosConfigurations<span style="color:#f92672">.</span>miku-linux<span style="color:#f92672">.</span>options
</span></span></code></pre>
</body>
</html>
<p>But while it works, it is slow, because it needs network to evaluate (and it is
impure, since there is no <code>flake.lock</code>).</p>
<p>The default configuration for <code>nixd</code> makes NixOS completion work even outside
of my configuration repo, and it is fast. How? I decided to take a look at the
<code>nixd</code> source code and found
<a href="https://github.com/nix-community/nixd/blob/d938026c55c7c36a6e79afd9627459160b4924ed/nixd/lib/Controller/LifeTime.cpp#L33C11-L35C76">this</a>
(formatted here for legibility):</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>(
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">let</span>
</span></span><span style="display:flex;"><span> pkgs <span style="color:#f92672">=</span> <span style="color:#f92672">import</span> <span style="color:#e6db74">&lt;nixpkgs&gt;</span> { };
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">in</span>
</span></span><span style="display:flex;"><span> (pkgs<span style="color:#f92672">.</span>lib<span style="color:#f92672">.</span>evalModules {
</span></span><span style="display:flex;"><span> modules <span style="color:#f92672">=</span> (<span style="color:#f92672">import</span> <span style="color:#e6db74">&lt;nixpkgs/nixos/modules/module-list.nix&gt;</span>) <span style="color:#f92672">++</span> [
</span></span><span style="display:flex;"><span> ({ <span style="color:#f92672">...</span> }: { nixpkgs<span style="color:#f92672">.</span>hostPlatform <span style="color:#f92672">=</span> builtins<span style="color:#f92672">.</span>currentSystem; })
</span></span><span style="display:flex;"><span> ];
</span></span><span style="display:flex;"><span> })
</span></span><span style="display:flex;"><span>)<span style="color:#f92672">.</span>options
</span></span></code></pre>
</body>
</html>
<p>Interesting, so they're manually loading the modules using <code>evalModules</code>. As I
said above, it depends in <code>NIX_PATH</code> being correctly set. Can we fix this to
use our Flake inputs instead? After some tries in the Nix REPL, I got the
following:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>(
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">let</span>
</span></span><span style="display:flex;"><span> pkgs <span style="color:#f92672">=</span> <span style="color:#f92672">import</span> <span style="color:#e6db74">&#34;</span><span style="color:#e6db74">${</span>inputs<span style="color:#f92672">.</span>nixpkgs<span style="color:#e6db74">}</span><span style="color:#e6db74">&#34;</span> { };
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">in</span>
</span></span><span style="display:flex;"><span> (pkgs<span style="color:#f92672">.</span>lib<span style="color:#f92672">.</span>evalModules {
</span></span><span style="display:flex;"><span> modules <span style="color:#f92672">=</span> (<span style="color:#f92672">import</span> <span style="color:#e6db74">&#34;</span><span style="color:#e6db74">${</span>inputs<span style="color:#f92672">.</span>nixpkgs<span style="color:#e6db74">}</span><span style="color:#e6db74">/nixos/modules/module-list.nix&#34;</span>) <span style="color:#f92672">++</span> [
</span></span><span style="display:flex;"><span> ({ <span style="color:#f92672">...</span> }: { nixpkgs<span style="color:#f92672">.</span>hostPlatform <span style="color:#f92672">=</span> builtins<span style="color:#f92672">.</span>currentSystem; })
</span></span><span style="display:flex;"><span> ];
</span></span><span style="display:flex;"><span> })
</span></span><span style="display:flex;"><span>)<span style="color:#f92672">.</span>options
</span></span></code></pre>
</body>
</html>
<p>So we can adapt this to the neovim configuration:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>{
</span></span><span style="display:flex;"><span> options <span style="color:#f92672">=</span> {
</span></span><span style="display:flex;"><span> nixos <span style="color:#f92672">=</span> {
</span></span><span style="display:flex;"><span> expr <span style="color:#f92672">=</span> <span style="color:#e6db74">&#39;(let pkgs = import &#34;${inputs.nixpkgs}&#34; { }; in (pkgs.lib.evalModules { modules = (import &#34;${inputs.nixpkgs}/nixos/modules/module-list.nix&#34;) ++ [ ({...}: { nixpkgs.hostPlatform = builtins.currentSystem;} ) ] ; })).options&#39;</span>,
</span></span><span style="display:flex;"><span> },
</span></span><span style="display:flex;"><span> },
</span></span><span style="display:flex;"><span>}
</span></span></code></pre>
</body>
</html>
<p>This was easy. But the main issue is Home-Manager. How can we fix it? I needed
to take a look at the Home-Manager <a href="https://github.com/nix-community/home-manager/blob/master/docs/default.nix#L161-L169">source
code</a>
to find the answer:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>(
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">let</span>
</span></span><span style="display:flex;"><span> pkgs <span style="color:#f92672">=</span> <span style="color:#f92672">import</span> <span style="color:#e6db74">&#34;</span><span style="color:#e6db74">${</span>inputs<span style="color:#f92672">.</span>nixpkgs<span style="color:#e6db74">}</span><span style="color:#e6db74">&#34;</span> { };
</span></span><span style="display:flex;"><span> lib <span style="color:#f92672">=</span> <span style="color:#f92672">import</span> <span style="color:#e6db74">&#34;</span><span style="color:#e6db74">${</span>inputs<span style="color:#f92672">.</span>home-manager<span style="color:#e6db74">}</span><span style="color:#e6db74">/modules/lib/stdlib-extended.nix&#34;</span> pkgs<span style="color:#f92672">.</span>lib;
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">in</span>
</span></span><span style="display:flex;"><span> (lib<span style="color:#f92672">.</span>evalModules {
</span></span><span style="display:flex;"><span> modules <span style="color:#f92672">=</span> (<span style="color:#f92672">import</span> <span style="color:#e6db74">&#34;</span><span style="color:#e6db74">${</span>inputs<span style="color:#f92672">.</span>home-manager<span style="color:#e6db74">}</span><span style="color:#e6db74">/modules/modules.nix&#34;</span>) {
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">inherit</span> lib pkgs;
</span></span><span style="display:flex;"><span> check <span style="color:#f92672">=</span> <span style="color:#66d9ef">false</span>;
</span></span><span style="display:flex;"><span> };
</span></span><span style="display:flex;"><span> })
</span></span><span style="display:flex;"><span>)<span style="color:#f92672">.</span>options
</span></span></code></pre>
</body>
</html>
<p>The interesting part is: Home-Manager has its own extension of the module
system (including <code>evalModules</code>). This includes e.g.: extra types used in
Home-Manager only. Also, we need to disable <code>checks</code>, otherwise we will hit
some validations (e.g.: missing <code>stateVersion</code>). I am not sure if this causes
any issue for module completion yet, I may set it in the future.</p>
<p>And for the final result:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>{
</span></span><span style="display:flex;"><span> nixpkgs <span style="color:#f92672">=</span> {
</span></span><span style="display:flex;"><span> expr <span style="color:#f92672">=</span> <span style="color:#e6db74">&#39;import &#34;${flake.inputs.nixpkgs}&#34; { }&#39;</span>,
</span></span><span style="display:flex;"><span> },
</span></span><span style="display:flex;"><span> options <span style="color:#f92672">=</span> {
</span></span><span style="display:flex;"><span> nixos <span style="color:#f92672">=</span> {
</span></span><span style="display:flex;"><span> expr <span style="color:#f92672">=</span> <span style="color:#e6db74">&#39;(let pkgs = import &#34;${inputs.nixpkgs}&#34; { }; in (pkgs.lib.evalModules { modules = (import &#34;${inputs.nixpkgs}/nixos/modules/module-list.nix&#34;) ++ [ ({...}: { nixpkgs.hostPlatform = builtins.currentSystem;} ) ] ; })).options&#39;</span>,
</span></span><span style="display:flex;"><span> },
</span></span><span style="display:flex;"><span> home_manager <span style="color:#f92672">=</span> {
</span></span><span style="display:flex;"><span> expr <span style="color:#f92672">=</span> <span style="color:#e6db74">&#39;(let pkgs = import &#34;${inputs.nixpkgs}&#34; { }; lib = import &#34;${inputs.home-manager}/modules/lib/stdlib-extended.nix&#34; pkgs.lib; in (lib.evalModules { modules = (import &#34;${inputs.home-manager}/modules/modules.nix&#34;) { inherit lib pkgs; check = false; }; })).options&#39;</span>,
</span></span><span style="display:flex;"><span> },
</span></span><span style="display:flex;"><span> },
</span></span><span style="display:flex;"><span>}
</span></span></code></pre>
</body>
</html>
<p>Yes, it is quite a mouthful, but it makes module completion work in any
repository, as long as you're using Flakes. And it is fast, since it doesn't
need any network access. Since we are already here, let's define <code>nixpkgs</code> to
not depend in the <code>NIX_PATH</code> being set too.</p>
</description>
<guid>https://github.com/thiagokokada/blog/blob/main/posts/2024-08-04/01-make-nixd-module-completion-to-work-anywhere-with-flakes.md</guid>
<pubDate>Sun, 04 Aug 2024 00:00:00 +0000</pubDate>
</item>
<item>
<title>Quick bits: realise Nix symlinks</title>
<link>https://github.com/thiagokokada/blog/blob/main/posts/2024-08-01/02-quick-bits-realise-nix-symlinks.md</link>
<description><p>When you are using Nix, especially with
<a href="https://github.com/nix-community/home-manager/">Home-Manager</a>, there are times
when you want to test something or maybe debug some issue in your
configuration. Those times it would be really convenient if you could avoid a
rebuild of your Home-Manager configuration, since this takes time until
evaluation and activation.</p>
<p>For those times I have this small script in my Nix configuration called
<code>realise-symlinks</code>, that is defined as:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>{ pkgs<span style="color:#f92672">,</span> <span style="color:#f92672">...</span> }:
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">let</span>
</span></span><span style="display:flex;"><span> realise-symlink <span style="color:#f92672">=</span> pkgs<span style="color:#f92672">.</span>writeShellApplication {
</span></span><span style="display:flex;"><span> name <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;realise-symlink&#34;</span>;
</span></span><span style="display:flex;"><span> runtimeInputs <span style="color:#f92672">=</span> <span style="color:#66d9ef">with</span> pkgs; [ coreutils ];
</span></span><span style="display:flex;"><span> text <span style="color:#f92672">=</span> <span style="color:#e6db74">&#39;&#39;
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> for file in &#34;$@&#34;; do
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> if [[ -L &#34;$file&#34; ]]; then
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> if [[ -d &#34;$file&#34; ]]; then
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> tmpdir=&#34;</span><span style="color:#ae81ff">&#39;&#39;$</span><span style="color:#e6db74">{file}.tmp&#34;
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> mkdir -p &#34;$tmpdir&#34;
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> cp --verbose --recursive &#34;$file&#34;/* &#34;$tmpdir&#34;
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> unlink &#34;$file&#34;
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> mv &#34;$tmpdir&#34; &#34;$file&#34;
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> chmod --changes --recursive +w &#34;$file&#34;
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> else
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> cp --verbose --remove-destination &#34;$(readlink &#34;$file&#34;)&#34; &#34;$file&#34;
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> chmod --changes +w &#34;$file&#34;
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> fi
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> else
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> &gt;&amp;2 echo &#34;Not a symlink: $file&#34;
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> exit 1
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> fi
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> done
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> &#39;&#39;</span>;
</span></span><span style="display:flex;"><span> };
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">in</span>
</span></span><span style="display:flex;"><span>{
</span></span><span style="display:flex;"><span> home<span style="color:#f92672">.</span>packages <span style="color:#f92672">=</span> [ realise-symlink ];
</span></span><span style="display:flex;"><span>}
</span></span></code></pre>
</body>
</html>
<p>The idea of this script is that you can call it against a symlink against Nix
store and it will realise, e.g.: convert to an &quot;actual&quot; file, e.g.:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>$ ls -lah .zshrc
</span></span><span style="display:flex;"><span>lrwxrwxrwx 1 thiagoko users 69 Aug 1 00:10 .zshrc -&gt; /nix/store/glz018yyh0qfqc9lywx1yhr7c3l96lv7-home-manager-files/.zshrc
</span></span><span style="display:flex;"><span><span style="color:#960050;background-color:#1e0010">
</span></span></span><span style="display:flex;"><span><span style="color:#960050;background-color:#1e0010"></span>$ realise-symlink .zshrc
</span></span><span style="display:flex;"><span>removed &#39;.zshrc&#39;
</span></span><span style="display:flex;"><span>&#39;/nix/store/glz018yyh0qfqc9lywx1yhr7c3l96lv7-home-manager-files/.zshrc&#39; -&gt; &#39;.zshrc&#39;
</span></span><span style="display:flex;"><span>mode of &#39;.zshrc&#39; changed from 0444 (r--r--r--) to 0644 (rw-r--r--)
</span></span><span style="display:flex;"><span><span style="color:#960050;background-color:#1e0010">
</span></span></span><span style="display:flex;"><span><span style="color:#960050;background-color:#1e0010"></span>$ ls -lah .zshrc
</span></span><span style="display:flex;"><span>-rw-r--r-- 1 thiagoko users 5.8K Aug 1 00:16 .zshrc
</span></span></code></pre>
</body>
</html>
<p>It also add write permissions to the resulting file, to make it easier to edit.
By the way, it also works with directories:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>$ ls -lah zim-completion
</span></span><span style="display:flex;"><span>lrwxrwxrwx 1 thiagoko users 90 Aug 1 00:10 zim-completion -&gt; /nix/store/glz018yyh0qfqc9lywx1yhr7c3l96lv7-home-manager-files/.zsh/plugins/zim-completion
</span></span><span style="display:flex;"><span><span style="color:#960050;background-color:#1e0010">
</span></span></span><span style="display:flex;"><span><span style="color:#960050;background-color:#1e0010"></span>$ realise-symlink zim-completion
</span></span><span style="display:flex;"><span>&#39;zim-completion/init.zsh&#39; -&gt; &#39;zim-completion.tmp/init.zsh&#39;
</span></span><span style="display:flex;"><span>&#39;zim-completion/init.zsh.zwc&#39; -&gt; &#39;zim-completion.tmp/init.zsh.zwc&#39;
</span></span><span style="display:flex;"><span>&#39;zim-completion/LICENSE&#39; -&gt; &#39;zim-completion.tmp/LICENSE&#39;
</span></span><span style="display:flex;"><span>&#39;zim-completion/README.md&#39; -&gt; &#39;zim-completion.tmp/README.md&#39;
</span></span><span style="display:flex;"><span>mode of &#39;zim-completion/init.zsh&#39; changed from 0444 (r--r--r--) to 0644 (rw-r--r--)
</span></span><span style="display:flex;"><span>mode of &#39;zim-completion/init.zsh.zwc&#39; changed from 0444 (r--r--r--) to 0644 (rw-r--r--)
</span></span><span style="display:flex;"><span>mode of &#39;zim-completion/LICENSE&#39; changed from 0444 (r--r--r--) to 0644 (rw-r--r--)
</span></span><span style="display:flex;"><span>mode of &#39;zim-completion/README.md&#39; changed from 0444 (r--r--r--) to 0644 (rw-r--r--)
</span></span><span style="display:flex;"><span><span style="color:#960050;background-color:#1e0010">
</span></span></span><span style="display:flex;"><span><span style="color:#960050;background-color:#1e0010"></span>$ ls -lah zim-completion
</span></span><span style="display:flex;"><span>total 28K
</span></span><span style="display:flex;"><span>drwxr-xr-x 1 thiagoko users 72 Aug 1 00:18 .
</span></span><span style="display:flex;"><span>drwxr-xr-x 1 thiagoko users 130 Aug 1 00:18 ..
</span></span><span style="display:flex;"><span>-rw-r--r-- 1 thiagoko users 5.3K Aug 1 00:18 init.zsh
</span></span><span style="display:flex;"><span>-rw-r--r-- 1 thiagoko users 12K Aug 1 00:18 init.zsh.zwc
</span></span><span style="display:flex;"><span>-rw-r--r-- 1 thiagoko users 1.3K Aug 1 00:18 LICENSE
</span></span><span style="display:flex;"><span>-rw-r--r-- 1 thiagoko users 2.2K Aug 1 00:18 README.md
</span></span></code></pre>
</body>
</html>
<p>After you finish whatever you are testing, to return to your configuration you
can just delete those files and re-run your Home-Manager activation:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>$ rm -rf .zshrc
</span></span><span style="display:flex;"><span><span style="color:#960050;background-color:#1e0010">
</span></span></span><span style="display:flex;"><span><span style="color:#960050;background-color:#1e0010"></span>$ sudo systemctl restart home-manager-&lt;user&gt;.service <span style="color:#75715e"># or `home-manager switch`</span>
</span></span><span style="display:flex;"><span><span style="color:#960050;background-color:#1e0010">
</span></span></span><span style="display:flex;"><span><span style="color:#960050;background-color:#1e0010"></span>$ ls -lah .zshrc
</span></span><span style="display:flex;"><span>lrwxrwxrwx 1 thiagoko users 69 Aug 1 00:20 .zshrc -&gt; /nix/store/glz018yyh0qfqc9lywx1yhr7c3l96lv7-home-manager-files/.zshrc
</span></span></code></pre>
</body>
</html>
<p>It even works with system files (i.e.: NixOS):</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>$ sudo realise-symlink /etc/nix/nix.conf
</span></span><span style="display:flex;"><span>[sudo] password for thiagoko:
</span></span><span style="display:flex;"><span>removed &#39;nix.conf&#39;
</span></span><span style="display:flex;"><span>&#39;/etc/static/nix/nix.conf&#39; -&gt; &#39;nix.conf&#39;
</span></span><span style="display:flex;"><span>mode of &#39;nix.conf&#39; changed from 0444 (r--r--r--) to 0644 (rw-r--r--)
</span></span></code></pre>
</body>
</html>
<p>But I never needed for this case since it is more rare to me to experiment with
OS level configuration.</p>
</description>
<guid>https://github.com/thiagokokada/blog/blob/main/posts/2024-08-01/02-quick-bits-realise-nix-symlinks.md</guid>
<pubDate>Thu, 01 Aug 2024 00:00:00 +0000</pubDate>
</item>
<item>
<title>Troubleshooting: ZSH lag and solutions with Nix</title>
<link>https://github.com/thiagokokada/blog/blob/main/posts/2024-08-01/01-troubleshoting-zsh-lag-and-solutions-with-nix.md</link>
<description><p>Inspired by this <a href="https://lock.cmpxchg8b.com/slowterm.html">blog post from
Tavis</a>, I decided to document my own
recent journey of reducing terminal (ZSH) lag startup. This post is way less
interesting than the one from Tavis that uses a debugger to patch applications
on the fly, but should still be interesting for some. And it also shows how
powerful Nix can be for some things.</p>
<p>For context, I have basically 3 systems where I interact with terminal
frequently:</p>
<ul>
<li><a href="https://www.notebookcheck.net/Lenovo-ThinkPad-P14s-Gen-1-Laptop-Review-AMD-workstation-sans-dedicated-GPU.528746.0.html">Thinkpad P14s Gen 1</a>
running NixOS, with a reasonable fast CPU (<a href="https://www.notebookcheck.net/AMD-Ryzen-7-PRO-4750U-Processor-Benchmarks-and-Specs.464396.0.html">AMD Ryzen 7 PRO
4750U</a>)
and disk (NVMe)</li>
<li><a href="https://everymac.com/systems/apple/macbook_pro/specs/macbook-pro-m1-pro-10-core-cpu-16-core-gpu-16-2021-specs.html">MacBook
Pro</a>
with a really fast CPU (M1 Pro) and disk, but of course running macOS, being
phased-out since this is a job owned machine and I am changing jobs right
now, but should be replaced with another one soon™</li>
<li><a href="https://chromeunboxed.com/lenovo-chromebook-duet-3-review-perfect-sequel">Chromebook Duet
3</a>
running ChromeOS, with slow CPU (<a href="https://www.notebookcheck.net/Qualcomm-Snapdragon-7c-Gen-2-Processor-Benchmarks-and-Specs.554426.0.html">Snapdragon 7c Gen
2</a>)
and disk (eMMC, really?)</li>
</ul>
<p>My experience is similar to Tavis, at around 300ms of startup time I don't care
too much, but around 500ms+ is where I start to notice. I never had any issues
with startup time in NixOS itself (I had issues with macOS before, but it was
not actually the fault of macOS), but in the Chromebook it was awful: 600ms+
with <a href="https://www.instabug.com/blog/understanding-cold-hot-and-warm-app-launch-time">hot
start</a>,
while cold start it could take multiple seconds.</p>
<p>We can check how long ZSH takes to start by using:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>$ time zsh -ci exit
</span></span><span style="display:flex;"><span>zsh -ic exit 0.04s user 0.10s system 100% cpu 0.143 total
</span></span></code></pre>
</body>
</html>
<p>The <code>-i</code> flag here is important, because we are interested in the interactive
use of ZSH. Without this flag ZSH will ignore your <code>~/.zshrc</code> file, and the
results will be meaningless.</p>
<p>To do a more interesting benchmark, we can use
<a href="https://github.com/sharkdp/hyperfine"><code>hyperfine</code></a>:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>$ hyperfine <span style="color:#e6db74">&#34;zsh -ic exit&#34;</span>
</span></span><span style="display:flex;"><span>Benchmark 1: zsh -ic exit
</span></span><span style="display:flex;"><span> Time (mean ± σ): 145.4 ms ± 4.2 ms [User: 49.8 ms, System: 97.3 ms]
</span></span><span style="display:flex;"><span> Range (min … max): 138.6 ms … 155.3 ms 19 runs
</span></span></code></pre>
</body>
</html>
<p>Hyperfine will run the command multiple times and take care of things like
shell startup time. A really great tool to have in your toolbox by the way, but
I digress.</p>
<p>So let's do a little time travelling. Going back to commit
<a href="https://github.com/thiagokokada/nix-configs/tree/b12757f90889653e359a1ab0a8cfd2f90cfabf31"><code>b12757f</code></a>
from <a href="https://github.com/thiagokokada/nix-configs/">nix-configs</a>. Running
<code>hyperfine</code> like above from my NixOS laptop, we have:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>$ hyperfine <span style="color:#e6db74">&#34;zsh -ic exit&#34;</span>
</span></span><span style="display:flex;"><span>Benchmark 1: zsh -ic exit
</span></span><span style="display:flex;"><span> Time (mean ± σ): 218.6 ms ± 5.1 ms [User: 70.6 ms, System: 151.5 ms]
</span></span><span style="display:flex;"><span> Range (min … max): 210.3 ms … 227.0 ms 13 runs
</span></span></code></pre>
</body>
</html>
<p>This doesn't look that bad, but let's see the same commit in my Chromebook:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>$ hyperfine <span style="color:#e6db74">&#34;zsh -ic exit&#34;</span>
</span></span><span style="display:flex;"><span>Benchmark 1: zsh -ic exit
</span></span><span style="display:flex;"><span> Time (mean ± σ): 679.7 ms ± 40.2 ms [User: 230.8 ms, System: 448.5 ms]
</span></span><span style="display:flex;"><span> Range (min … max): 607.3 ms … 737.0 ms 10 runs
</span></span></code></pre>
</body>
</html>
<p>Yikes, this is much worse. And those are the results after I retried the
benchmark (so it is a hot start). The cold start times were above 3s. So let's
investigate what is happening here. We can profile what is taking time during
the startup of ZSH using <a href="https://www.bigbinary.com/blog/zsh-profiling">zprof</a>.
You can add the following in your <code>~/.zshrc</code>:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span><span style="color:#75715e"># At the top of your ~/.zshrc file</span>
</span></span><span style="display:flex;"><span>zmodload zsh/zprof
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#75715e"># ...</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#75715e"># At the end of your ~/.zshrc file</span>
</span></span><span style="display:flex;"><span>zprof
</span></span></code></pre>
</body>
</html>
<p>Or if using Home-Manager, use the
<a href="https://nix-community.github.io/home-manager/options.xhtml#opt-programs.zsh.zprof.enable"><code>programs.zsh.zprof.enable</code></a>
option. Once we restart ZSH, we will have something like:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>num calls time self name
</span></span><span style="display:flex;"><span>-----------------------------------------------------------------------------------
</span></span><span style="display:flex;"><span> 1) 1 36.91 36.91 34.29% 30.47 30.47 28.31% (anon) [/home/thiagoko/.zsh/plugins/zim-completion/init.zsh:13]
</span></span><span style="display:flex;"><span> 2) 1 25.43 25.43 23.63% 25.43 25.43 23.63% (anon) [/home/thiagoko/.zsh/plugins/zim-ssh/init.zsh:6]
</span></span><span style="display:flex;"><span> 3) 1 22.00 22.00 20.45% 21.92 21.92 20.36% _zsh_highlight_load_highlighters
</span></span><span style="display:flex;"><span> 4) 1 12.32 12.32 11.45% 12.32 12.32 11.45% autopair-init
</span></span><span style="display:flex;"><span> 5) 1 6.44 6.44 5.98% 6.44 6.44 5.98% compinit
</span></span><span style="display:flex;"><span> 6) 1 3.56 3.56 3.31% 3.48 3.48 3.23% prompt_pure_state_setup
</span></span><span style="display:flex;"><span> 7) 2 3.79 1.89 3.52% 2.85 1.43 2.65% async
</span></span><span style="display:flex;"><span> 8) 1 0.93 0.93 0.87% 0.93 0.93 0.87% async_init
</span></span><span style="display:flex;"><span> 9) 6 0.93 0.15 0.86% 0.93 0.15 0.86% is-at-least
</span></span><span style="display:flex;"><span>10) 6 0.67 0.11 0.63% 0.67 0.11 0.63% add-zle-hook-widget
</span></span><span style="display:flex;"><span>11) 1 8.25 8.25 7.66% 0.61 0.61 0.57% prompt_pure_setup
</span></span><span style="display:flex;"><span>12) 1 0.40 0.40 0.37% 0.40 0.40 0.37% (anon) [/nix/store/p1zqypy7600fvfyl1v571bljx2l8zhay-zsh-autosuggestions-0.7.0/share/zsh-autosuggestions/zsh-autosuggestions.zsh:458]
</span></span><span style="display:flex;"><span>13) 5 0.31 0.06 0.29% 0.31 0.06 0.29% add-zsh-hook
</span></span><span style="display:flex;"><span>14) 1 0.60 0.60 0.56% 0.29 0.29 0.27% (anon) [/home/thiagoko/.zsh/plugins/zim-input/init.zsh:5]
</span></span><span style="display:flex;"><span>15) 1 0.21 0.21 0.20% 0.21 0.21 0.20% compdef
</span></span><span style="display:flex;"><span>16) 1 0.10 0.10 0.09% 0.10 0.10 0.09% _zsh_highlight__function_is_autoload_stub_p
</span></span><span style="display:flex;"><span>17) 1 0.26 0.26 0.24% 0.08 0.08 0.08% _zsh_highlight__function_callable_p
</span></span><span style="display:flex;"><span>18) 1 0.08 0.08 0.08% 0.08 0.08 0.08% prompt_pure_is_inside_container
</span></span><span style="display:flex;"><span>19) 1 0.07 0.07 0.07% 0.07 0.07 0.07% _zsh_highlight__is_function_p
</span></span><span style="display:flex;"><span>20) 1 0.01 0.01 0.01% 0.01 0.01 0.01% __wezterm_install_bash_prexec
</span></span><span style="display:flex;"><span>21) 1 0.00 0.00 0.00% 0.00 0.00 0.00% _zsh_highlight_bind_widgets
</span></span><span style="display:flex;"><span># ...
</span></span></code></pre>
</body>
</html>
<p>I ommited some output for brevit. The first 2 things that shows are from the
<a href="https://github.com/zimfw/zimfw">zimfw</a>, the framework that I use to configure
my ZSH (similar to Oh-My-Zsh). I actually don't use <code>zimfw</code> directly, instead I
just load some modules that I find useful, like the <code>zim-completion</code> and
<code>zim-ssh</code> that we can see above. By the way, Zim is generally really well
optimised for startup time, but those 2 modules are kind slow.</p>
<p>For <a href="https://github.com/zimfw/completion"><code>zim-completion</code></a>, after taking a
look at it, there isn't much I could do. It seems that the reason
<code>zim-completion</code> takes so long during startup is because it is trying to decide
if it needs to recompile the completions (and replacing it with just a naive
<code>autoload -U compinit &amp;&amp; compinit</code> is even worse for startup performance). I
may eventually replace it for something else, but I really like what Zim brings
here, so I decided to not touch it for now.</p>
<p>However <a href="https://github.com/zimfw/ssh"><code>zim-ssh</code></a> is another history. The only
reason I used it is to start a <code>ssh-agent</code> and keep it between multiple ZSH
sessions. It shouldn't have that much influence in startup time. So I took a
look the code (since it is small, I am reproducing it here):</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span><span style="color:#75715e">#</span>
</span></span><span style="display:flex;"><span><span style="color:#75715e"># Set up ssh-agent</span>
</span></span><span style="display:flex;"><span><span style="color:#75715e">#</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#75715e"># Don&#39;t do anything unless we can actually use ssh-agent</span>
</span></span><span style="display:flex;"><span><span style="color:#f92672">((</span> <span style="color:#e6db74">${</span>+commands[ssh-agent]<span style="color:#e6db74">}</span> <span style="color:#f92672">))</span> <span style="color:#f92672">&amp;&amp;</span> <span style="color:#f92672">()</span> <span style="color:#f92672">{</span>
</span></span><span style="display:flex;"><span> ssh-add -l &amp;&gt;/dev/null
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">if</span> <span style="color:#f92672">((</span> ? <span style="color:#f92672">==</span> <span style="color:#ae81ff">2</span> <span style="color:#f92672">))</span>; <span style="color:#66d9ef">then</span>
</span></span><span style="display:flex;"><span> <span style="color:#75715e"># Unable to contact the authentication agent</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#75715e"># Load stored agent connection info</span>
</span></span><span style="display:flex;"><span> local -r ssh_env<span style="color:#f92672">=</span><span style="color:#e6db74">${</span>HOME<span style="color:#e6db74">}</span>/.ssh-agent
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">if</span> <span style="color:#f92672">[[</span> -r <span style="color:#e6db74">${</span>ssh_env<span style="color:#e6db74">}</span> <span style="color:#f92672">]]</span> source <span style="color:#e6db74">${</span>ssh_env<span style="color:#e6db74">}</span> &gt;/dev/null
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> ssh-add -l &amp;&gt;/dev/null
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">if</span> <span style="color:#f92672">((</span> ? <span style="color:#f92672">==</span> <span style="color:#ae81ff">2</span> <span style="color:#f92672">))</span>; <span style="color:#66d9ef">then</span>
</span></span><span style="display:flex;"><span> <span style="color:#75715e"># Start agent and store agent connection info</span>
</span></span><span style="display:flex;"><span> <span style="color:#f92672">(</span>umask 066; ssh-agent &gt;! <span style="color:#e6db74">${</span>ssh_env<span style="color:#e6db74">}</span><span style="color:#f92672">)</span>
</span></span><span style="display:flex;"><span> source <span style="color:#e6db74">${</span>ssh_env<span style="color:#e6db74">}</span> &gt;/dev/null
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">fi</span>
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">fi</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#75715e"># Load identities</span>
</span></span><span style="display:flex;"><span> ssh-add -l &amp;&gt;/dev/null
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">if</span> <span style="color:#f92672">((</span> ? <span style="color:#f92672">==</span> <span style="color:#ae81ff">1</span> <span style="color:#f92672">))</span>; <span style="color:#66d9ef">then</span>
</span></span><span style="display:flex;"><span> local -a zssh_ids
</span></span><span style="display:flex;"><span> zstyle -a <span style="color:#e6db74">&#39;:zim:ssh&#39;</span> ids <span style="color:#e6db74">&#39;zssh_ids&#39;</span>
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">if</span> <span style="color:#f92672">((</span> <span style="color:#e6db74">${#</span>zssh_ids<span style="color:#e6db74">}</span> <span style="color:#f92672">))</span>; <span style="color:#66d9ef">then</span>
</span></span><span style="display:flex;"><span> ssh-add <span style="color:#e6db74">${</span>HOME<span style="color:#e6db74">}</span>/.ssh/<span style="color:#e6db74">${</span>^zssh_ids<span style="color:#e6db74">}</span> 2&gt;/dev/null
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">else</span>
</span></span><span style="display:flex;"><span> ssh-add 2&gt;/dev/null
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">fi</span>
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">fi</span>
</span></span><span style="display:flex;"><span><span style="color:#f92672">}</span>
</span></span></code></pre>
</body>
</html>
<p>Well, this is bad. Let's assume the common path, where the <code>ssh-agent</code> is
already running but you open a new shell instance (that doesn't have the
connection info yet so it will need to load). This will run <code>ssh-add</code> at 4
times. How long does <code>ssh-add</code> takes to run?</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>$ hyperfine -Ni <span style="color:#e6db74">&#34;ssh-add -l&#34;</span>
</span></span><span style="display:flex;"><span>Benchmark 1: ssh-add -l
</span></span><span style="display:flex;"><span> Time (mean ± σ): 4.6 ms ± 1.1 ms [User: 2.0 ms, System: 2.0 ms]
</span></span><span style="display:flex;"><span> Range (min … max): 3.4 ms … 8.7 ms 619 runs
</span></span><span style="display:flex;"><span><span style="color:#960050;background-color:#1e0010">
</span></span></span><span style="display:flex;"><span><span style="color:#960050;background-color:#1e0010"></span> Warning: Ignoring non-zero exit code.
</span></span></code></pre>
</body>
</html>
<p>For those curious, <code>-N</code> disables the Shell usage, that works better when the
command being tested is too fast.</p>
<p>In average we have 4x4ms=16ms of startup time. But keep in mind the worst case
can be much worse. The question is, how can we improve the situation here?</p>
<p>After taking a look, I decided to write my own code, based in some ideas stolen
from <a href="https://github.com/ohmyzsh/ohmyzsh/blob/67581c53c6458566e174620361e84b364b9034d2/plugins/ssh-agent/ssh-agent.plugin.zsh">Oh-My-Zsh ssh-agent
plugin</a>.
Here is final version of my
<a href="https://github.com/thiagokokada/nix-configs/blob/e45a888f2bf3ce5644c3966f0b6371414d0291e2/home-manager/cli/ssh/ssh-agent.zsh">code</a>:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>zmodload zsh/net/socket
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>_check_agent<span style="color:#f92672">(){</span>
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">if</span> <span style="color:#f92672">[[</span> -S <span style="color:#e6db74">&#34;</span>$SSH_AUTH_SOCK<span style="color:#e6db74">&#34;</span> <span style="color:#f92672">]]</span> <span style="color:#f92672">&amp;&amp;</span> zsocket <span style="color:#e6db74">&#34;</span>$SSH_AUTH_SOCK<span style="color:#e6db74">&#34;</span> 2&gt;/dev/null; <span style="color:#66d9ef">then</span>
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">return</span> <span style="color:#ae81ff">0</span>
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">fi</span>
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">return</span> <span style="color:#ae81ff">1</span>
</span></span><span style="display:flex;"><span><span style="color:#f92672">}</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>_start_agent<span style="color:#f92672">()</span> <span style="color:#f92672">{</span>
</span></span><span style="display:flex;"><span> <span style="color:#75715e"># Test if $SSH_AUTH_SOCK is visible, in case we start e.g.: ssh-agent via</span>
</span></span><span style="display:flex;"><span> <span style="color:#75715e"># systemd service</span>
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">if</span> _check_agent; <span style="color:#66d9ef">then</span>
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">return</span> <span style="color:#ae81ff">0</span>
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">fi</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#75715e"># Get the filename to store/lookup the environment from</span>
</span></span><span style="display:flex;"><span> local -r ssh_env_cache<span style="color:#f92672">=</span><span style="color:#e6db74">&#34;</span>$HOME<span style="color:#e6db74">/.ssh-agent&#34;</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#75715e"># Check if ssh-agent is already running</span>
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">if</span> <span style="color:#f92672">[[</span> -f <span style="color:#e6db74">&#34;</span>$ssh_env_cache<span style="color:#e6db74">&#34;</span> <span style="color:#f92672">]]</span>; <span style="color:#66d9ef">then</span>
</span></span><span style="display:flex;"><span> source <span style="color:#e6db74">&#34;</span>$ssh_env_cache<span style="color:#e6db74">&#34;</span> &gt; /dev/null
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#75715e"># Test if $SSH_AUTH_SOCK is visible, e.g.: the ssh-agent is still alive</span>
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">if</span> _check_agent; <span style="color:#66d9ef">then</span>
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">return</span> <span style="color:#ae81ff">0</span>
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">fi</span>
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">fi</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#75715e"># start ssh-agent and setup environment</span>
</span></span><span style="display:flex;"><span> <span style="color:#f92672">(</span>
</span></span><span style="display:flex;"><span> umask <span style="color:#ae81ff">066</span>
</span></span><span style="display:flex;"><span> ssh-agent -s &gt;! <span style="color:#e6db74">&#34;</span>$ssh_env_cache<span style="color:#e6db74">&#34;</span>
</span></span><span style="display:flex;"><span> <span style="color:#f92672">)</span>
</span></span><span style="display:flex;"><span> source <span style="color:#e6db74">&#34;</span>$ssh_env_cache<span style="color:#e6db74">&#34;</span> &gt; /dev/null
</span></span><span style="display:flex;"><span><span style="color:#f92672">}</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>_start_agent
</span></span><span style="display:flex;"><span>unfunction _check_agent _start_agent
</span></span></code></pre>
</body>
</html>
<p>The idea here is simple: using
<a href="https://zsh.sourceforge.io/Doc/Release/Zsh-Modules.html#The-zsh_002fnet_002fsocket-Module"><code>zsocket</code></a>
module from ZSH itself to check if the <code>ssh-agent</code> is working instead of
executing <code>ssh-add -l</code>. The only case we run any program now is to start the
agent itself if needed. Let's run <code>hyperfine</code> again:</p>
<pre><code>$ hyperfine &quot;zsh -ic exit&quot;
Benchmark 1: zsh -ic exit
 Time (mean ± σ): 188.3 ms ± 8.2 ms [User: 61.1 ms, System: 130.0 ms]
 Range (min … max): 170.9 ms … 198.4 ms 16 runs
</code></pre>
<p>Got a good improvement here already. Let's see <code>zprof</code> again:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>num calls time self name
</span></span><span style="display:flex;"><span>-----------------------------------------------------------------------------------
</span></span><span style="display:flex;"><span> 1) 1 41.23 41.23 48.66% 33.52 33.52 39.56% (anon) [/home/thiagoko/.zsh/plugins/zim-completion/init.zsh:13]
</span></span><span style="display:flex;"><span> 2) 1 22.23 22.23 26.24% 22.12 22.12 26.10% _zsh_highlight_load_highlighters
</span></span><span style="display:flex;"><span> 3) 1 8.90 8.90 10.51% 8.90 8.90 10.51% Gautopair-init
</span></span><span style="display:flex;"><span> 4) 1 7.71 7.71 9.10% 7.71 7.71 9.10% compinit
</span></span><span style="display:flex;"><span> 5) 1 5.74 5.74 6.77% 5.60 5.60 6.60% prompt_pure_state_setup
</span></span><span style="display:flex;"><span> 6) 6 1.19 0.20 1.41% 1.19 0.20 1.41% add-zle-hook-widget
</span></span><span style="display:flex;"><span> 7) 2 1.97 0.99 2.33% 1.14 0.57 1.34% async
</span></span><span style="display:flex;"><span> 8) 6 0.87 0.15 1.03% 0.87 0.15 1.03% is-at-least
</span></span><span style="display:flex;"><span> 9) 1 0.84 0.84 0.99% 0.84 0.84 0.99% async_init
</span></span><span style="display:flex;"><span>10) 1 9.30 9.30 10.97% 0.72 0.72 0.84% prompt_pure_setup
</span></span><span style="display:flex;"><span>11) 5 0.63 0.13 0.75% 0.63 0.13 0.75% add-zsh-hook
</span></span><span style="display:flex;"><span>12) 1 0.41 0.41 0.48% 0.41 0.41 0.48% _start_agent
</span></span><span style="display:flex;"><span>13) 1 0.31 0.31 0.37% 0.31 0.31 0.37% (anon) [/nix/store/p1zqypy7600fvfyl1v571bljx2l8zhay-zsh-autosuggestions-0.7.0/share/zsh-autosuggestions/zsh-autosuggestions.zsh:458]
</span></span><span style="display:flex;"><span>14) 1 0.55 0.55 0.64% 0.24 0.24 0.28% (anon) [/home/thiagoko/.zsh/plugins/zim-input/init.zsh:5]
</span></span><span style="display:flex;"><span>15) 1 0.14 0.14 0.16% 0.14 0.14 0.16% prompt_pure_is_inside_container
</span></span><span style="display:flex;"><span>16) 1 0.14 0.14 0.16% 0.14 0.14 0.16% compdef
</span></span><span style="display:flex;"><span>17) 1 0.09 0.09 0.11% 0.09 0.09 0.11% _zsh_highlight__function_is_autoload_stub_p
</span></span><span style="display:flex;"><span>18) 1 0.25 0.25 0.29% 0.08 0.08 0.09% _zsh_highlight__function_callable_p
</span></span><span style="display:flex;"><span>19) 1 0.07 0.07 0.09% 0.07 0.07 0.09% _zsh_highlight__is_function_p
</span></span><span style="display:flex;"><span>20) 1 0.01 0.01 0.01% 0.01 0.01 0.01% __wezterm_install_bash_prexec
</span></span><span style="display:flex;"><span>21) 1 0.01 0.01 0.01% 0.01 0.01 0.01% _zsh_highlight_bind_widgets
</span></span><span style="display:flex;"><span># ...
</span></span></code></pre>
</body>
</html>
<p>Well, there is nothing interesting here anymore. I mean, <code>zim-completion</code> is
still the main culprit, but nothing to do for now. Instead of looking at
<code>zproof</code>, let's take a look at my <code>~/.zshrc</code> instead:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span><span style="color:#75715e"># ...</span>
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">if</span> <span style="color:#f92672">[[</span> $options<span style="color:#f92672">[</span>zle<span style="color:#f92672">]</span> <span style="color:#f92672">=</span> on <span style="color:#f92672">]]</span>; <span style="color:#66d9ef">then</span>
</span></span><span style="display:flex;"><span> eval <span style="color:#e6db74">&#34;</span><span style="color:#66d9ef">$(</span>/nix/store/sk6wsgp4h477baxypksz9rl8ldwwh9yg-fzf-0.54.0/bin/fzf --zsh<span style="color:#66d9ef">)</span><span style="color:#e6db74">&#34;</span>
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">fi</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#75715e"># ...</span>
</span></span><span style="display:flex;"><span>/nix/store/x3yblr73r5x76dmaanjk3333mvzxc49r-any-nix-shell-1.2.1/bin/any-nix-shell zsh | source /dev/stdin
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#75715e"># ...</span>
</span></span><span style="display:flex;"><span>eval <span style="color:#e6db74">&#34;</span><span style="color:#66d9ef">$(</span>/nix/store/330d6k81flfs6w46b44afmncxk57qggv-zoxide-0.9.4/bin/zoxide init zsh <span style="color:#66d9ef">)</span><span style="color:#e6db74">&#34;</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#75715e"># ...</span>
</span></span><span style="display:flex;"><span>eval <span style="color:#e6db74">&#34;</span><span style="color:#66d9ef">$(</span>/nix/store/8l9j9kdv9m0z0s30lp4yvrc9s5bcbgmx-direnv-2.34.0/bin/direnv hook zsh<span style="color:#66d9ef">)</span><span style="color:#e6db74">&#34;</span>
</span></span></code></pre>
</body>
</html>
<p>So you see, starting all those programs during ZSH startup can hurt the shell
startup considerable. Not necessary for commands fast like <code>fzf</code> (that is
written in Go), but let's see
<a href="https://github.com/haslersn/any-nix-shell"><code>any-nix-shell</code></a>, that is written
in shell script:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>$ hyperfine <span style="color:#e6db74">&#34;any-nix-shell zsh&#34;</span>
</span></span><span style="display:flex;"><span>Benchmark 1: any-nix-shell zsh
</span></span><span style="display:flex;"><span> Time (mean ± σ): 16.0 ms ± 1.8 ms [User: 5.6 ms, System: 10.5 ms]
</span></span><span style="display:flex;"><span> Range (min … max): 11.3 ms … 20.3 ms 143 runs
</span></span></code></pre>
</body>
</html>
<p>This is bad, consistently bad actually. Even for commands that are fast, keep
in mind that there is a difference between the cold and hot start again. For
example, <code>fzf</code>:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>$ hyperfine -N <span style="color:#e6db74">&#34;fzf --zsh&#34;</span>
</span></span><span style="display:flex;"><span>Benchmark 1: fzf --zsh
</span></span><span style="display:flex;"><span> Time (mean ± σ): 2.9 ms ± 0.9 ms [User: 0.6 ms, System: 2.3 ms]
</span></span><span style="display:flex;"><span> Range (min … max): 1.7 ms … 6.8 ms 1113 runs
</span></span></code></pre>
</body>
</html>
<p>See the range? While 1.7ms is something that is probably difficult to notice,
6.8ms can be noticiable, especially if this accumulates with other slow
starting apps.</p>
<p>And the thing is, all those commands are just generating in the end a fixed
output, at least for the current version of the program. Can we pre-generate
them instead? If using Nix, of course we can:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span><span style="color:#75715e"># You need to disable the default integration</span>
</span></span><span style="display:flex;"><span>programs<span style="color:#f92672">.</span>direnv<span style="color:#f92672">.</span>enableZshIntegration <span style="color:#960050;background-color:#1e0010">=</span> <span style="color:#66d9ef">false</span>;
</span></span><span style="display:flex;"><span>programs<span style="color:#f92672">.</span>fzf<span style="color:#f92672">.</span>enableZshIntegration <span style="color:#960050;background-color:#1e0010">=</span> <span style="color:#66d9ef">false</span>;
</span></span><span style="display:flex;"><span>programs<span style="color:#f92672">.</span>zoxide<span style="color:#f92672">.</span>enableZshIntegration <span style="color:#960050;background-color:#1e0010">=</span> <span style="color:#66d9ef">false</span>;
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>programs<span style="color:#f92672">.</span>zsh<span style="color:#f92672">.</span>initExtra <span style="color:#960050;background-color:#1e0010">=</span>
</span></span><span style="display:flex;"><span> <span style="color:#75715e"># bash</span>
</span></span><span style="display:flex;"><span> <span style="color:#e6db74">&#39;&#39;
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> # any-nix-shell
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> source </span><span style="color:#e6db74">${</span>
</span></span><span style="display:flex;"><span> pkgs<span style="color:#f92672">.</span>runCommand <span style="color:#e6db74">&#34;any-nix-shell-zsh&#34;</span> { } <span style="color:#e6db74">&#39;&#39;
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> </span><span style="color:#e6db74">${</span>lib<span style="color:#f92672">.</span>getExe pkgs<span style="color:#f92672">.</span>any-nix-shell<span style="color:#e6db74">}</span><span style="color:#e6db74"> zsh &gt; $out
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> &#39;&#39;</span>
</span></span><span style="display:flex;"><span> <span style="color:#e6db74">}</span><span style="color:#e6db74">
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74">
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> # fzf
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> source </span><span style="color:#e6db74">${</span>config<span style="color:#f92672">.</span>programs<span style="color:#f92672">.</span>fzf<span style="color:#f92672">.</span>package<span style="color:#e6db74">}</span><span style="color:#e6db74">/share/fzf/completion.zsh
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> source </span><span style="color:#e6db74">${</span>config<span style="color:#f92672">.</span>programs<span style="color:#f92672">.</span>fzf<span style="color:#f92672">.</span>package<span style="color:#e6db74">}</span><span style="color:#e6db74">/share/fzf/key-bindings.zsh
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74">
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> # zoxide
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> source </span><span style="color:#e6db74">${</span>
</span></span><span style="display:flex;"><span> pkgs<span style="color:#f92672">.</span>runCommand <span style="color:#e6db74">&#34;zoxide-init-zsh&#34;</span> { } <span style="color:#e6db74">&#39;&#39;
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> </span><span style="color:#e6db74">${</span>lib<span style="color:#f92672">.</span>getExe config<span style="color:#f92672">.</span>programs<span style="color:#f92672">.</span>zoxide<span style="color:#f92672">.</span>package<span style="color:#e6db74">}</span><span style="color:#e6db74"> init zsh &gt; $out
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> &#39;&#39;</span>
</span></span><span style="display:flex;"><span> <span style="color:#e6db74">}</span><span style="color:#e6db74">
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74">
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> # direnv
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> source </span><span style="color:#e6db74">${</span>
</span></span><span style="display:flex;"><span> pkgs<span style="color:#f92672">.</span>runCommand <span style="color:#e6db74">&#34;direnv-hook-zsh&#34;</span> { } <span style="color:#e6db74">&#39;&#39;
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> </span><span style="color:#e6db74">${</span>lib<span style="color:#f92672">.</span>getExe config<span style="color:#f92672">.</span>programs<span style="color:#f92672">.</span>direnv<span style="color:#f92672">.</span>package<span style="color:#e6db74">}</span><span style="color:#e6db74"> hook zsh &gt; $out
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> &#39;&#39;</span>
</span></span><span style="display:flex;"><span> <span style="color:#e6db74">}</span><span style="color:#e6db74">
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> &#39;&#39;</span>;
</span></span></code></pre>
</body>
</html>
<p>So we can use <code>pkgs.runCommand</code> to run those commands during build time and
<code>source</code> the result. <code>fzf</code> actually doesn't need this since we have the files
already generated in the package. I think this is one of those things that
really shows the power of Nix: I wouldn't do something similar if I didn't use
Nix because the risk of breaking something later is big (e.g.: forgetting to
update the generated files), but Nix makes those things trivial.</p>
<p>Let's run <code>hyperfine</code> again:</p>
<pre><code>$ hyperfine &quot;zsh -ic exit&quot;
Benchmark 1: zsh -ic exit
 Time (mean ± σ): 162.3 ms ± 4.9 ms [User: 52.7 ms, System: 111.1 ms]
 Range (min … max): 153.0 ms … 173.4 ms 19 runs
</code></pre>
<p>Another good improvement. The last change I did is switching between
<a href="https://github.com/zsh-users/zsh-syntax-highlighting"><code>zsh-syntax-highlighting</code></a>
to
<a href="https://github.com/zdharma-continuum/fast-syntax-highlighting"><code>zsh-fast-syntax-highlighting</code></a>,
that is supposed to be faster and have better highlighting too. I got that from
<code>_zsh_highlight_load_highlighters</code> using 26% of the time from my <code>zprof</code> above.
And for the final <code>hyperfine</code> in my laptop:</p>
<pre><code>$ hyperfine &quot;zsh -ic exit&quot;
Benchmark 1: zsh -ic exit
 Time (mean ± σ): 138.3 ms ± 7.1 ms [User: 47.5 ms, System: 91.9 ms]
 Range (min … max): 123.8 ms … 157.9 ms 21 runs
</code></pre>
<p>A ~36% improvement, not bad. Let's see how it fares in my Chromebook:</p>
<pre><code>$ hyperfine &quot;zsh -ic exit&quot;
Benchmark 1: zsh -ic exit
 Time (mean ± σ): 278.2 ms ± 46.9 ms [User: 88.0 ms, System: 184.8 ms]
 Range (min … max): 204.7 ms … 368.5 ms 11 runs
</code></pre>
<p>An even more impressive ~59% improvement. And yes, the shell startup now feels
much better.</p>
</description>
<guid>https://github.com/thiagokokada/blog/blob/main/posts/2024-08-01/01-troubleshoting-zsh-lag-and-solutions-with-nix.md</guid>
<pubDate>Thu, 01 Aug 2024 00:00:00 +0000</pubDate>
</item>
<item>
<title>Generating YAML files with Nix</title>
<link>https://github.com/thiagokokada/blog/blob/main/posts/2024-07-31/01-generating-yaml-files-with-nix.md</link>
<description><p>I hate YAML. Instead of writing an essay on why I hate YAML, I can just link to
<a href="https://noyaml.com/">noyaml.com</a>. In my personal projects I will never use it,
preferring either JSON, <a href="https://toml.io/en/">TOML</a> or even plain old
<a href="https://en.wikipedia.org/wiki/INI_file">INI</a> files depending on the use case.
However the ship has sailed already, there are tons of projects everywhere that
uses YAML: from most CI systems (<a href="https://docs.github.com/en/actions/writing-workflows/workflow-syntax-for-github-actions">GitHub
Actions</a>,
<a href="https://circleci.com/docs/introduction-to-yaml-configurations/">CircleCI</a>,
<a href="https://docs.travis-ci.com/user/build-config-yaml">Travis</a>, <em>et tu</em>
<a href="https://man.sr.ht/builds.sr.ht/">builds.sr.ht</a>), to
<a href="https://kubernetes.io/docs/concepts/overview/working-with-objects/">Kubernetes</a>,
or in almost every
<a href="https://guides.rubyonrails.org/configuring.html#configuring-a-database">Rails</a>
application.</p>
<p>One way to avoid at least some issues with the language is to write YAML in
another language. I will show my solution in one of my <a href="https://github.com/thiagokokada/nix-configs/">personal
repositories</a>, writing Nix to
generate GitHub Actions configuration files. Bonus points for validating the
result against the schema of GitHub Actions, so the famous &quot;this is supposed to
be string instead of a list of strings&quot; is gone.</p>
<p>Let's start with the basics: YAML is supposed to be a <a href="https://stackoverflow.com/a/1729545">superset of
JSON</a>. What that means is that a JSON file
<a href="https://yaml.org/spec/1.2-old/spec.html#id2759572">can be parsed</a> by a YAML
parser. And Nix itself generates JSON natively, after all, Nix can be imagined
as <a href="https://nix.dev/tutorials/nix-language.html">&quot;JSON with functions&quot;</a>.</p>
<p>To make things easier, I will assume that you have the <code>nix-commands</code> and
<code>flakes</code> enabled as <code>experimental-features</code> in your Nix configuration. If not,
go <a href="https://wiki.nixos.org/wiki/Flakes">here</a>.</p>
<p>Using the <code>nix eval</code> command, we can generate a JSON expression from Nix by:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>$ nix eval --expr <span style="color:#e6db74">&#39;{ foo = &#34;bar&#34;; }&#39;</span> --json
</span></span><span style="display:flex;"><span>{&#34;foo&#34;:&#34;bar&#34;}
</span></span></code></pre>
</body>
</html>
<p>However, typing long excerpts of Nix code inside the console would be
impractical. We can write the following code inside a <code>foo.nix</code> file instead:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>{
</span></span><span style="display:flex;"><span> foo <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;bar&#34;</span>;
</span></span><span style="display:flex;"><span>}
</span></span></code></pre>
</body>
</html>
<p>And:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>$ nix eval --file foo.nix --json
</span></span><span style="display:flex;"><span>{&#34;foo&#34;:&#34;bar&#34;}
</span></span></code></pre>
</body>
</html>
<p>While you can use a JSON output as an input for YAML parsers, it is probably
not the <a href="https://metacpan.org/pod/JSON::XS#JSON-and-YAML">best idea</a>. Sadly (or
maybe not), Nix has no native functionality to export data to YAML. However,
since we are using Nix, it is trivial to use <code>nixpkgs</code> to use some program to
convert from JSON to YAML.</p>
<p>To start, let's create a new directory, move our <code>foo.nix</code> file to it, create a
new <code>flake.nix</code> file and put the following contents:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>{
</span></span><span style="display:flex;"><span> description <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;Generate YAML files with Nix&#34;</span>;
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> inputs <span style="color:#f92672">=</span> {
</span></span><span style="display:flex;"><span> nixpkgs<span style="color:#f92672">.</span>url <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;github:NixOS/nixpkgs/nixos-unstable&#34;</span>;
</span></span><span style="display:flex;"><span> };
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> outputs <span style="color:#f92672">=</span> { nixpkgs<span style="color:#f92672">,</span> <span style="color:#f92672">...</span> }:
</span></span><span style="display:flex;"><span> {
</span></span><span style="display:flex;"><span> packages<span style="color:#f92672">.</span>x86_64-linux <span style="color:#f92672">=</span>
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">let</span>
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">inherit</span> (nixpkgs) lib;
</span></span><span style="display:flex;"><span> pkgs <span style="color:#f92672">=</span> <span style="color:#f92672">import</span> nixpkgs { system <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;x86_64-linux&#34;</span>; };
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">in</span>
</span></span><span style="display:flex;"><span> {
</span></span><span style="display:flex;"><span> toYAML <span style="color:#f92672">=</span> pkgs<span style="color:#f92672">.</span>runCommand <span style="color:#e6db74">&#34;toYAML&#34;</span> {
</span></span><span style="display:flex;"><span> buildInputs <span style="color:#f92672">=</span> <span style="color:#66d9ef">with</span> pkgs; [ yj ];
</span></span><span style="display:flex;"><span> json <span style="color:#f92672">=</span> builtins<span style="color:#f92672">.</span>toJSON (<span style="color:#f92672">import</span> <span style="color:#e6db74">./foo.nix</span>);
</span></span><span style="display:flex;"><span> passAsFile <span style="color:#f92672">=</span> [ <span style="color:#e6db74">&#34;json&#34;</span> ]; <span style="color:#75715e"># will be available as `$jsonPath`</span>
</span></span><span style="display:flex;"><span> } <span style="color:#e6db74">&#39;&#39;
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> mkdir -p $out
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> yj -jy &lt; &#34;$jsonPath&#34; &gt; $out/go.yaml
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> &#39;&#39;</span>;
</span></span><span style="display:flex;"><span> };
</span></span><span style="display:flex;"><span> };
</span></span><span style="display:flex;"><span>}
</span></span></code></pre>
</body>
</html>
<p>We are loading the <code>./foo.nix</code> as a Nix file, converting it to JSON with
<code>builtins.toJSON</code> function, and finally, using <code>pkgs.runCommand</code> and its
<code>passAsFile</code> option to load the contents of the JSON file into
<a href="https://github.com/sclevine/yj">yj</a>, that converts between serialisation
formats (<code>-jy</code> flag means &quot;JSON to YAML&quot;). The reason I choose <code>yj</code> is mostly
because it is a single binary Go program, but you can use whatever you prefer.</p>
<p>By the way, there is a
<a href="https://github.com/NixOS/nixpkgs/blob/9f918d616c5321ad374ae6cb5ea89c9e04bf3e58/lib/generators.nix#L805"><code>lib.generators.toYAML</code></a>
inside <code>nixpkgs.lib</code>, but as of the day of this post it only calls
<code>lib.strings.toJSON</code> (that in turn, calls <code>builtins.toJSON</code>). So it doesn't
really help here. Another option would be <code>pkgs.formats.yaml.generate</code>, that
converts between formats, but it calls
<a href="https://github.com/NixOS/nixpkgs/blob/008ceae1a2b47a84d7aa01e55f8468272c70b9ee/pkgs/pkgs-lib/formats.nix#L77-L84"><code>remarshal</code></a>
(in Python), so not my favorite choice.</p>
<p>If we run the following commands, we can see the result:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>$ nix build .#packages.x86_64-linux.toYAML
</span></span><span style="display:flex;"><span>$ cat result/foo.yaml
</span></span><span style="display:flex;"><span>foo: bar
</span></span></code></pre>
</body>
</html>
<p>That is the basic idea. To have a more realistic example, let's convert the
<a href="https://github.com/thiagokokada/blog/blob/4e3f25485c6682f3e066b219df2290934bc0d256/.github/workflows/go.yml"><code>go.yml</code></a>,
that builds this blog, to Nix:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>{
</span></span><span style="display:flex;"><span> name <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;Go&#34;</span>;
</span></span><span style="display:flex;"><span> on<span style="color:#f92672">.</span>push<span style="color:#f92672">.</span>branches <span style="color:#f92672">=</span> [ <span style="color:#e6db74">&#34;main&#34;</span> ];
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> jobs <span style="color:#f92672">=</span> {
</span></span><span style="display:flex;"><span> build <span style="color:#f92672">=</span> {
</span></span><span style="display:flex;"><span> runs-on <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;ubuntu-latest&#34;</span>;
</span></span><span style="display:flex;"><span> permissions<span style="color:#f92672">.</span>contents <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;write&#34;</span>;
</span></span><span style="display:flex;"><span> steps <span style="color:#f92672">=</span> [
</span></span><span style="display:flex;"><span> { uses <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;actions/checkout@v4&#34;</span>; }
</span></span><span style="display:flex;"><span> {
</span></span><span style="display:flex;"><span> name <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;Set up Go&#34;</span>;
</span></span><span style="display:flex;"><span> uses <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;actions/checkout@v4&#34;</span>;
</span></span><span style="display:flex;"><span> <span style="color:#e6db74">&#34;with&#34;</span><span style="color:#f92672">.</span>go-version <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;1.21&#34;</span>;
</span></span><span style="display:flex;"><span> }
</span></span><span style="display:flex;"><span> {
</span></span><span style="display:flex;"><span> name <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;Update&#34;</span>;
</span></span><span style="display:flex;"><span> run <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;make&#34;</span>;
</span></span><span style="display:flex;"><span> }
</span></span><span style="display:flex;"><span> {
</span></span><span style="display:flex;"><span> name <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;Publish&#34;</span>;
</span></span><span style="display:flex;"><span> run <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;make publish&#34;</span>;
</span></span><span style="display:flex;"><span> env<span style="color:#f92672">.</span>MATAROA_TOKEN <span style="color:#f92672">=</span> <span style="color:#e6db74">&#39;&#39;</span><span style="color:#ae81ff">&#39;&#39;$</span><span style="color:#e6db74">{{ secrets.MATAROA_TOKEN }}&#39;&#39;</span>;
</span></span><span style="display:flex;"><span> }
</span></span><span style="display:flex;"><span> {
</span></span><span style="display:flex;"><span> name <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;Commit&#34;</span>;
</span></span><span style="display:flex;"><span> uses <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;stefanzweifel/git-auto-commit-action@v5&#34;</span>;
</span></span><span style="display:flex;"><span> <span style="color:#e6db74">&#34;with&#34;</span><span style="color:#f92672">.</span>commit_message <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;README/rss:update&#34;</span>;
</span></span><span style="display:flex;"><span> }
</span></span><span style="display:flex;"><span> ];
</span></span><span style="display:flex;"><span> };
</span></span><span style="display:flex;"><span> };
</span></span><span style="display:flex;"><span>}
</span></span></code></pre>
</body>
</html>
<p>Some interesting things to highlight: <code>with</code> is a reserved word in Nix, so we
need to quote it. Not a problem, but something to be aware. And the template
string in GitHub Actions uses the same <code>${}</code> that Nix uses, so we need to
escape.</p>
<p>And after running the following commands:</p>
<pre><code>$ nix build .#packages.x86_64-linux.toYAML
$ cat result/go.yaml
jobs:
 build:
 permissions:
 contents: write
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v4
 - name: Set up Go
 uses: actions/checkout@v4
 with:
 go-version: &quot;1.21&quot;
 - name: Update
 run: make
 - env:
 MATAROA_TOKEN: ${{ secrets.MATAROA_TOKEN }}
 name: Publish
 run: make publish
 - name: Commit
 uses: stefanzweifel/git-auto-commit-action@v5
 with:
 commit_message: README/rss:update
name: Go
&quot;on&quot;:
 push:
 branches:
 - main
</code></pre>
<p>Yes, the keys are not in the same order as we defined, since Nix, like most
programming languages (with the exception of
<a href="https://mail.python.org/pipermail/python-dev/2017-December/151283.html">Python</a>),
do not guarantee the insertion order in maps/dicts/attrsets/whatever. But I
really hope whatever is consuming your YAML is not relying in the order the
keys are defined (this would be more cursed than YAML already is).</p>
<p>So that is basically it. For the bonus points that I talked at the start of the
post, we can modify <code>pkgs.runCommand</code> to run some kind of validator. I use
<a href="https://github.com/mpalmer/action-validator"><code>action-validator</code></a>, one that I
particularly packaged in
<a href="https://github.com/NixOS/nixpkgs/pull/260217">nixpkgs</a> to use in those cases.
But you could use e.g.: a validator of Kubernetes YAML. Or a generic YAML lint
like this <a href="https://github.com/adrienverge/yamllint">one</a>. The possibilities are
endless.</p>
<p>Let's modify our <code>flake.nix</code> to add the validation:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>{
</span></span><span style="display:flex;"><span> <span style="color:#75715e"># ...</span>
</span></span><span style="display:flex;"><span> outputs <span style="color:#f92672">=</span> { nixpkgs<span style="color:#f92672">,</span> <span style="color:#f92672">...</span> }:
</span></span><span style="display:flex;"><span> {
</span></span><span style="display:flex;"><span> packages<span style="color:#f92672">.</span>x86_64-linux <span style="color:#f92672">=</span>
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">let</span>
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">inherit</span> (nixpkgs) lib;
</span></span><span style="display:flex;"><span> pkgs <span style="color:#f92672">=</span> <span style="color:#f92672">import</span> nixpkgs { system <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;x86_64-linux&#34;</span>; };
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">in</span>
</span></span><span style="display:flex;"><span> {
</span></span><span style="display:flex;"><span> toYAML <span style="color:#f92672">=</span> pkgs<span style="color:#f92672">.</span>runCommand <span style="color:#e6db74">&#34;toYAML&#34;</span> {
</span></span><span style="display:flex;"><span> buildInputs <span style="color:#f92672">=</span> <span style="color:#66d9ef">with</span> pkgs; [ action-validator yj ];
</span></span><span style="display:flex;"><span> json <span style="color:#f92672">=</span> builtins<span style="color:#f92672">.</span>toJSON (<span style="color:#f92672">import</span> <span style="color:#e6db74">./go.nix</span>);
</span></span><span style="display:flex;"><span> passAsFile <span style="color:#f92672">=</span> [ <span style="color:#e6db74">&#34;json&#34;</span> ];
</span></span><span style="display:flex;"><span> } <span style="color:#e6db74">&#39;&#39;
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> mkdir -p $out
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> yj -jy &lt; &#34;$jsonPath&#34; &gt; $out/go.yaml
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> action-validator -v $out/go.yaml
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> &#39;&#39;</span>;
</span></span><span style="display:flex;"><span> };
</span></span><span style="display:flex;"><span> };
</span></span><span style="display:flex;"><span>}
</span></span></code></pre>
</body>
</html>
<p>And let's add an error in our <code>go.nix</code> file:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>diff --git a/go.nix b/go.nix
</span></span><span style="display:flex;"><span>index 25e0596..8c00033 100644
</span></span><span style="display:flex;"><span><span style="color:#f92672">--- a/go.nix
</span></span></span><span style="display:flex;"><span><span style="color:#f92672"></span><span style="color:#a6e22e">+++ b/go.nix
</span></span></span><span style="display:flex;"><span><span style="color:#a6e22e"></span><span style="color:#75715e">@@ -5,7 +5,7 @@
</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> jobs = {
</span></span><span style="display:flex;"><span> build = {
</span></span><span style="display:flex;"><span> runs-on = &#34;ubuntu-latest&#34;;
</span></span><span style="display:flex;"><span><span style="color:#f92672">- permissions.contents = &#34;write&#34;;
</span></span></span><span style="display:flex;"><span><span style="color:#f92672"></span><span style="color:#a6e22e">+ permissions.contents = [ &#34;write&#34; ];
</span></span></span><span style="display:flex;"><span><span style="color:#a6e22e"></span> steps = [
</span></span><span style="display:flex;"><span> { uses = &#34;actions/checkout@v4&#34;; }
</span></span><span style="display:flex;"><span> {
</span></span></code></pre>
</body>
</html>
<p>Finally, let's try to build our YAML file again:</p>
<pre><code>$ nix build .#packages.x86_64-linux.toYAML
error: builder for '/nix/store/j8wr6j1pvyf986sf74hqw8k31lvlzac5-toYAML.drv' failed with exit code 1;
 last 25 log lines:
 &gt; &quot;Additional property 'runs-on' is not allowed&quot;,
 &gt; ),
 &gt; path: &quot;/jobs/build&quot;,
 &gt; title: &quot;Property conditions are not met&quot;,
 &gt; },
 &gt; Properties {
 &gt; code: &quot;properties&quot;,
 &gt; detail: Some(
 &gt; &quot;Additional property 'steps' is not allowed&quot;,
 &gt; ),
 &gt; path: &quot;/jobs/build&quot;,
 &gt; title: &quot;Property conditions are not met&quot;,
 &gt; },
 &gt; Required {
 &gt; code: &quot;required&quot;,
 &gt; detail: None,
 &gt; path: &quot;/jobs/build/uses&quot;,
 &gt; title: &quot;This property is required&quot;,
 &gt; },
 &gt; ],
 &gt; },
 &gt; ],
 &gt; },
 &gt; ],
 &gt; }
 For full logs, run 'nix log /nix/store/j8wr6j1pvyf986sf74hqw8k31lvlzac5-toYAML.drv'.
</code></pre>
<p>Yes, the output of <code>action-validator</code> is awfully verbose, but it is still
better than making <a href="https://x.com/eric_sink/status/1430954572848287744">&quot;8 commits/push in one
hour&quot;</a>.</p>
<p>If you are interested in how a more advantage usage of this technique is,
including usage of functions and constants to share common steps between
different actions, please take a look at the
<a href="https://github.com/thiagokokada/nix-configs/tree/master/actions">actions</a>
(<a href="https://github.com/thiagokokada/nix-configs/tree/fbbc8f935cc7f774f691196e1612bbf7dd2054b5/actions">permalink</a>)
in my <code>nix-config</code> repository.</p>
</description>
<guid>https://github.com/thiagokokada/blog/blob/main/posts/2024-07-31/01-generating-yaml-files-with-nix.md</guid>
<pubDate>Wed, 31 Jul 2024 00:00:00 +0000</pubDate>
</item>
<item>
<title>First impressions: FPGBC</title>
<link>https://github.com/thiagokokada/blog/blob/main/posts/2024-07-30/01-first-impressions-fpgbc.md</link>
<description><p>Here is something for nostalgia: I just put together a <a href="https://en.wikipedia.org/wiki/Game_Boy_Color">Game Boy
Color</a> made of completely new
parts for a friend: here is the
<a href="https://funnyplaying.com/products/fpgbc-kit">FPGBC</a>.</p>
<p>The <em>FP</em> part of the name comes from
<a href="https://en.wikipedia.org/wiki/Field-programmable_gate_array">FPGA</a>, because
instead of software emulation this device use FPGA to reproduce the device.
While I am not convinced that FPGA is necessary more accurate than a good
software emulator, one advantage of FPGA is the (possible) lower input latency
thanks to the avoidance of complexity to handle the user input (e.g.: the
Operational System). A quick playthrough against <a href="https://en.wikipedia.org/wiki/Motocross_Maniacs">Motocross
Maniacs</a> seems to be fine, but
I can't see much difference from my <a href="https://retrogamecorps.com/2022/05/15/miyoo-mini-v2-guide/">Miyoo
Mini+</a> (I will do
more comparisons between the two devices later), that is a software emulation
device.</p>
<p>But I think focusing in accuracy is wrong, the main reason of getting a device
like this one is for nostalgia, and this definitely hit the mark. The quality
of the case is as good as I remember the original, and most of the details are
replicate perfectly, including reproduction stickers in the back of the device.
The only differences that I can find is the usage of USB-C port for charging in
place of the barrel jack power adapter (thanks!), and the fact that the screen
bezels are smaller compared to the original (because the screen is bigger) and
doesn't include the Game Boy Color logo (that is fine in my opinion, since it
would look weird in the fine bezels). It even has a supposedly working <a href="https://en.wikipedia.org/wiki/Game_Link_Cable">Link
Cable</a> (I don't have another
Game Boy to test). Sadly it is missing the infrared sensor, but the usage of
that was pretty limited anyway.</p>
<p><a href="https://github.com/thiagokokada/blog/raw/main/posts/2024-07-30/PXL_20240729_175245569.jpg"><img src="https://github.com/thiagokokada/blog/raw/main/posts/2024-07-30/PXL_20240729_175245569.jpg" alt="FPGBC running Tetris."></a></p>
<p><a href="https://github.com/thiagokokada/blog/raw/main/posts/2024-07-30/PXL_20240729_175131157.jpg"><img src="https://github.com/thiagokokada/blog/raw/main/posts/2024-07-30/PXL_20240729_175131157.jpg" alt="Back of FPGBC. It includes even reproduction stickers of the original."></a></p>
<p>So how well does it work? I can't say for sure. I don't have any original games
with me, so I am relying in backups and a
<a href="https://en.wikipedia.org/wiki/Flashcard">flashcard</a> for now. Many games that I
tested works fine, a few of them have graphical issues that can be fixed in the
menu (more about it later), and some of them doesn't boot. But I don't know if
the issue with the games not booting are because of the roms, the flashcard or
the device itself.</p>
<p>By the way, the flashcard I am using is a cheap knockoff of an <a href="https://gbatemp.net/review/everdrive-gb.141/">Everdrive
GB</a>. This FPGBC came with
firmware v1.09, while there is an update available for v1.10 in the
<a href="https://funnyplaying.com/products/fpgbc-kit">website</a>. I had an weird issue in
the new firmware where no games would boot with this knockoff Everdrive so I
had to go back to v1.09, but again, I am not sure if the issue was fact that I
am using a knockoff device or this would happen with an original Everdrive GB.
If you are going to buy a proper Everdrive, you probably wouldn't get a
Everdrive GB anyway since it is discontinued, and it seems the <a href="https://www.reddit.com/r/Gameboy/comments/1atwjh3/fpgbc_everdrive_compatibility/">newer
versions</a>
have better compatibility with FPGBC.</p>
<p>Sadly that the update didn't work, since there is this
<a href="https://github.com/makhowastaken/GWGBC_FW">repository</a> that patches the
firmware to boot the original logo instead of the ugly FPGBC one. And yes, for
some reason the v1.09 firmware from this repository still doesn't work with my
knockoff Everdrive.</p>
<p>By the way, it seems the device is not easy to brick: I borked the firmware
update process once while trying to downgrade back to v1.09, resulting in a
black screen when I turned on the console. But just connecting the device to
the computer and powering on, I could flash the firmware again and the device
came back to life.</p>
<p>About the features of the device: if you press the volume button (yes, you can
press it now), it opens the following menu:</p>
<p><a href="https://github.com/thiagokokada/blog/raw/main/posts/2024-07-30/PXL_20240729_210604830.jpg"><img src="https://github.com/thiagokokada/blog/raw/main/posts/2024-07-30/PXL_20240729_210604830.jpg" alt="FPGBC menu."></a></p>
<p>The first 2 options are the LCD backlight (<code>BKLT</code>) and volume (<code>VOL</code>). I didn't talk about
those, but the LCD screen seems to be IPS, and the quality is really good, and
also looks bright enough to play even under bad lightining conditions. And the
speaker has good quality, the sound is better than I remember, but sadly the
maximum volume is kind low. Still should be enough for playing in a quiet room.</p>
<p><code>DISPMOD</code> is probably the most controversial option: it allow you to set which
scale you want. Anything with <code>EMU</code> at the end means emulating the original
colors, and as far I remember it gets really close. You can also chose betwen
<code>X4</code>, <code>X4P</code> and <code>FUL</code>, the last one is the one shown in the photos where the
image fills the whole screen at the cost of non-integer scaling. <code>X4</code> is
integer scaling, however the image doesn't fill the whole screen. The <code>X4P</code>
also includes a pixel effect that makes the image closer than the original
screen. It actually looks good, but the fact that I chose a white border for
this FPGBC makes the border really distracting. Maybe the black one is a better
choice if you want integer scale.</p>
<p><code>CORE</code> is simple: you can choose between <code>GB</code> (Game Boy) or <code>GBC</code> (Game Boy
Color). For those who don't know, you can run Game Boy games in Game Boy Color
and they will be automatically colorised. Some people don't like this and
prefer the colors of <code>GB</code>, so you have this option. The <code>GB_PALETTE</code> allows you
to chose the color in GB mode, for example, the green-ish colors from the
original Game Boy or the blue-ish colors from <a href="https://nintendo.fandom.com/wiki/Game_Boy_Light">Game Boy
Light</a>. And yes, you can
choose the color palette for Game Boy games running in <code>GBC</code> mode by pressing a
<a href="https://gbstudiocentral.com/tips/game-boy-color-modes/">button combination</a> at
the boot screen, but it seems not working in my unit and again, not sure if the
fault is my knockoff Everdrive.</p>
<p><code>FRAME_MIX</code> basically is an option that makes some effects, like transparency
in <a href="https://en.wikipedia.org/wiki/Wave_Race">Wave Race</a>, to work at the cost of
introducing blurriness. The reason for this is that those effects depends in
the fact that the Game Boy screen was slow refresh, so you could rely on it by
rapidly changing pixels to create some interesting effects, but sadly those
effects doesn't work well in modern displays.</p>
<p><code>GB_CLRFIX</code> is the option I mentioned before, where some Game Boy games just
get completely wrong colors for some reason, e.g.: <a href="https://en.wikipedia.org/wiki/The_Addams_Family_(video_game)">The Addams
Family</a>. Turning
on fixes those games, but I am not sure if this option breaks other games.</p>
<p>Finally, <code>SPD</code> allows you to increase or decrease the CPU clock, slowing or
speeding up the games (including the sound). The result can be hilarious, so I
think this is a nice addition to the features. Sadly you can't know what the
default speed is, so you need to rely on sound to adjust back to the default.</p>
<p>So in the end, can I recommend a FPGBC? I am not sure. If you want a device to
play games, I still think something like a Miyoo Mini+ is a better choice. Not
only you will have access to more games from different platforms, you also
don't need to rely on flashcards or cartridges. Also it has way more features
than FPGBC, like wireless multiplayer,
<a href="https://retroachievements.org/">RetroArchivements</a> and save states.</p>
<p>But the actual reason to get a FPGBC is nostalgia, and for that I think the
FPGBC is difficult to beat. The price of the <a href="https://funnyplaying.com/products/fpgbc-kit">kit to
assemble</a> ($69.90) is cheaper than
most Game Boy's in good condition you can find in eBay, and you get for that
price a rechargable battery, an amazing quality screen, the PCB and the
speaker. You need to buy separately the case and the buttons, but in total you
will still end up paying less, and allows you to fully customise your build.
And the result device is not only in mint condition, it is really convenient
too: recharging batteries (via USB-C even) is much more convenient than buying
AA batteries, and the screen not only is better but it even has backlight. You
can also buy a fully built console for
<a href="https://funnyplaying.com/products/fpgbc-console">$99.00</a>, but you have less
options of customisation.</p>
<p>This is the classic case of do what I say, don't do what I do. This FPGBC is a
gift, and I will buy another one soon. Can't wait to play <a href="https://en.wikipedia.org/wiki/Pok%C3%A9mon_Gold_and_Silver">Pokémon
Gold</a> in (almost)
original hardware again.</p>
<p><a href="https://github.com/thiagokokada/blog/raw/main/posts/2024-07-30/PXL_20240729_123847458.jpg"><img src="https://github.com/thiagokokada/blog/raw/main/posts/2024-07-30/PXL_20240729_123847458.jpg" alt="The kit before assemble."></a></p>
</description>
<guid>https://github.com/thiagokokada/blog/blob/main/posts/2024-07-30/01-first-impressions-fpgbc.md</guid>
<pubDate>Tue, 30 Jul 2024 00:00:00 +0000</pubDate>
</item>
<item>
<title>Go, a reasonable good language</title>
<link>https://github.com/thiagokokada/blog/blob/main/posts/2024-07-29/02-go-a-reasonable-good-language.md</link>
<description><p>Go was one of the languages that I always was interested to learn, but never
got the hang of it. I first got interested in the language when I was in my
first job, between 2016-2018. At the time the language was a completely
different beast: no modules, no generics, no easy way to error wrap yet, etc.</p>
<p>Go forward 2023 (no pun indented), I wrote my <a href="https://github.com/thiagokokada/twenty-twenty-twenty/">first project in
Go</a>, wrote some scripts
at <code>$CURRENT_JOB</code> in the language, and now wrote <a href="https://github.com/thiagokokada/hyprland-go/">my first
library</a> (with an <a href="https://github.com/thiagokokada/blog/blob/main/posts/2024-07-26/01-writing-nixos-tests-for-fun-and-profit.md">interesting
way to run
CI</a>). I am also
writing more
<a href="https://github.com/thiagokokada/nix-configs/blob/8c559527ed12e1d4f57a3fc5c72630b956f4c290/home-manager/desktop/wayland/hyprland/hyprtabs/hyprtabs.go">scripts</a>
in the language, where I would prefer to use Bash or Python before. Heck, even
this blog is automatically published with a <a href="https://github.com/thiagokokada/blog/blob/main/posts/2024-07-29/01-quick-bits-why-you-should-automate-everything.md">Go
script</a>,
that used to be a <a href="https://github.com/thiagokokada/blog/blob/main/posts/2024-07-26/02-using-github-as-a-bad-blog-platform.md">Python
one</a> before. I can
say that nowadays it is another language in my toolbox, and while it is still a
love and hate relationship, recently it is more about love and less about hate.</p>
<p>The points that I love about Go is probably obvious for some, but still
interesting to talk about anyway. The fact that the language generates static
binaries by default and have fast compilation times is something that I
apreciate since I first heard about the language, and now that I am using the
language frequently, are points thatI appreciate even more. Something about
getting almost instant feedback after changing a line of code and running <code>go run</code> (even with its quirks) are great for the developer experience. This is the
main reason why I am using the language more frequently for scripts.</p>
<p>Then we have the fast startup times. I am kind of sensitive to latency,
especially of command line utilities that need to answer fast when I expect
them to be fast (e.g.: <code>foo --help</code>). This is one part where I could have
issues in Python, especially for more complex programs, but in Go it is rarely
an issue.</p>
<p>Modules are also fantastic. It is not without its weirdness (like everything in
Go ecossystem), but the fact that it is so easy to add and manage dependencies
in a project using only the <code>go</code> CLI is great. I also like that it generates a
hash of every dependency, making it reproducible (well, probably not at Nix
level, but still reproducible).</p>
<p>Since I started to talk about <code>go</code> CLI, what a great tool! The fact that you
can manage dependencies, generate documentation, format code, lint, run
tests/<a href="https://blog.logrocket.com/benchmarking-golang-improve-function-performance/">benchmarks</a>/<a href="https://go.dev/doc/security/fuzz/">fuzzing</a>,
check code for <a href="https://go.dev/doc/articles/race_detector">races</a> etc., all
with just the &quot;compiler&quot; for the language is excelent. Still probably one of
the best developer experiences I know in any programming language.</p>
<p>I will not even talk about the things that everyone talks about Go, like
<a href="https://go.dev/doc/effective_go#goroutines">goroutines</a>, because I just don't
think I can add anything interesting to the topic.</p>
<p>Now for the parts that I like less, the test part still quirks me that it is
not based in assertions, but thankfully it is easy to write assertions with
generics nowadays:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span><span style="color:#66d9ef">func</span> <span style="color:#a6e22e">Equal</span>[<span style="color:#a6e22e">T</span> <span style="color:#a6e22e">comparable</span>](<span style="color:#a6e22e">t</span> <span style="color:#f92672">*</span><span style="color:#a6e22e">testing</span>.<span style="color:#a6e22e">T</span>, <span style="color:#a6e22e">got</span>, <span style="color:#a6e22e">want</span> <span style="color:#a6e22e">T</span>) {
</span></span><span style="display:flex;"><span>	<span style="color:#a6e22e">t</span>.<span style="color:#a6e22e">Helper</span>()
</span></span><span style="display:flex;"><span>	<span style="color:#66d9ef">if</span> <span style="color:#a6e22e">got</span> <span style="color:#f92672">!=</span> <span style="color:#a6e22e">want</span> {
</span></span><span style="display:flex;"><span>		<span style="color:#a6e22e">t</span>.<span style="color:#a6e22e">Errorf</span>(<span style="color:#e6db74">&#34;got: %#v, want: %#v&#34;</span>, <span style="color:#a6e22e">got</span>, <span style="color:#a6e22e">want</span>)
</span></span><span style="display:flex;"><span>	}
</span></span><span style="display:flex;"><span>}
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">func</span> <span style="color:#a6e22e">GreaterOrEqual</span>[<span style="color:#a6e22e">T</span> <span style="color:#a6e22e">cmp</span>.<span style="color:#a6e22e">Ordered</span>](<span style="color:#a6e22e">t</span> <span style="color:#f92672">*</span><span style="color:#a6e22e">testing</span>.<span style="color:#a6e22e">T</span>, <span style="color:#a6e22e">actual</span>, <span style="color:#a6e22e">expected</span> <span style="color:#a6e22e">T</span>) {
</span></span><span style="display:flex;"><span>	<span style="color:#a6e22e">t</span>.<span style="color:#a6e22e">Helper</span>()
</span></span><span style="display:flex;"><span>	<span style="color:#66d9ef">if</span> <span style="color:#a6e22e">actual</span> &lt; <span style="color:#a6e22e">expected</span> {
</span></span><span style="display:flex;"><span>		<span style="color:#a6e22e">t</span>.<span style="color:#a6e22e">Errorf</span>(<span style="color:#e6db74">&#34;got: %v; want: &gt;=%v&#34;</span>, <span style="color:#a6e22e">actual</span>, <span style="color:#a6e22e">expected</span>)
</span></span><span style="display:flex;"><span>	}
</span></span><span style="display:flex;"><span>}
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#75715e">// etc...
</span></span></span></code></pre>
</body>
</html>
<p>Just one of those things that I end up re-writing in every project. Yes, I know
about <a href="https://github.com/stretchr/testify">testify</a> and other assertion
libraries, but quoting <a href="https://www.youtube.com/watch?v=PAAkCSZUG1c&amp;t=568s">Rob
Pike</a> here, &quot;a little
copying is better than a little dependency&quot;. As long the code you write is
trivial, it is better to duplicate the code than try to import a dependency.</p>
<p>About another piece of code that generics allows me to write and I always end
up re-writing in every project is the <code>must*</code> family of functions:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span><span style="color:#66d9ef">func</span> <span style="color:#a6e22e">must</span>(<span style="color:#a6e22e">err</span> <span style="color:#66d9ef">error</span>) {
</span></span><span style="display:flex;"><span>	<span style="color:#66d9ef">if</span> <span style="color:#a6e22e">err</span> <span style="color:#f92672">!=</span> <span style="color:#66d9ef">nil</span> {
</span></span><span style="display:flex;"><span>		panic(<span style="color:#a6e22e">err</span>)
</span></span><span style="display:flex;"><span>	}
</span></span><span style="display:flex;"><span>}
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">func</span> <span style="color:#a6e22e">must1</span>[<span style="color:#a6e22e">T</span> <span style="color:#a6e22e">any</span>](<span style="color:#a6e22e">v</span> <span style="color:#a6e22e">T</span>, <span style="color:#a6e22e">err</span> <span style="color:#66d9ef">error</span>) <span style="color:#a6e22e">T</span> {
</span></span><span style="display:flex;"><span>	<span style="color:#a6e22e">must</span>(<span style="color:#a6e22e">err</span>)
</span></span><span style="display:flex;"><span>	<span style="color:#66d9ef">return</span> <span style="color:#a6e22e">v</span>
</span></span><span style="display:flex;"><span>}
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">func</span> <span style="color:#a6e22e">must2</span>[<span style="color:#a6e22e">T1</span> <span style="color:#a6e22e">any</span>, <span style="color:#a6e22e">T2</span> <span style="color:#a6e22e">any</span>](<span style="color:#a6e22e">v1</span> <span style="color:#a6e22e">T1</span>, <span style="color:#a6e22e">v2</span> <span style="color:#a6e22e">T2</span>, <span style="color:#a6e22e">err</span> <span style="color:#66d9ef">error</span>) <span style="color:#a6e22e">T</span> {
</span></span><span style="display:flex;"><span>	<span style="color:#a6e22e">must</span>(<span style="color:#a6e22e">err</span>)
</span></span><span style="display:flex;"><span>	<span style="color:#66d9ef">return</span> <span style="color:#a6e22e">v1</span>, <span style="color:#a6e22e">v2</span>
</span></span><span style="display:flex;"><span>}
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#75715e">// must3, must4, etc...
</span></span></span></code></pre>
</body>
</html>
<p>Those functions are so useful, especially for scripts where I generally don't
want to handle each error: if I have an error, I want the program to halt and
print a stack trace (exactly as I would have with a language with exceptions).
It basically allow me to convert code from:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span><span style="color:#a6e22e">contents</span>, <span style="color:#a6e22e">err</span> <span style="color:#f92672">:=</span> <span style="color:#a6e22e">os</span>.<span style="color:#a6e22e">ReadFile</span>(<span style="color:#e6db74">&#34;file&#34;</span>)
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">if</span> <span style="color:#a6e22e">err</span> <span style="color:#f92672">!=</span> <span style="color:#66d9ef">nil</span> {
</span></span><span style="display:flex;"><span> panic(<span style="color:#a6e22e">err</span>)
</span></span><span style="display:flex;"><span>}
</span></span></code></pre>
</body>
</html>
<p>To:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span><span style="color:#a6e22e">contents</span> <span style="color:#f92672">:=</span> <span style="color:#a6e22e">must1</span>(<span style="color:#a6e22e">os</span>.<span style="color:#a6e22e">ReadFile</span>(<span style="color:#e6db74">&#34;file&#34;</span>))
</span></span></code></pre>
</body>
</html>
<p>This brings Go closer to Python to me, and I think for scripts this is
something great.</p>
<p>Finally, for the things that I hate, well the biggest one currently is the lack
of nullability (or in Go terms,
<a href="https://github.com/golang/go/issues/49202">nillability</a>). After using
languages that has it, like Kotlin, or even something like
<a href="https://www.mypy-lang.org/">mypy</a>, this is one of those things that completely
changes the developer experience. I also still don't like the error handling
(but <code>must*</code> goes far by improving the situation, when it is possible to use
it), especially because it is easy to lose context on it:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span><span style="color:#75715e">// bad
</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span><span style="color:#66d9ef">func</span> <span style="color:#a6e22e">readFileContents</span>(<span style="color:#a6e22e">file</span>) ([]<span style="color:#66d9ef">byte</span>, <span style="color:#66d9ef">error</span>) {
</span></span><span style="display:flex;"><span> <span style="color:#a6e22e">contents</span>, <span style="color:#a6e22e">err</span> <span style="color:#f92672">:=</span> <span style="color:#a6e22e">os</span>.<span style="color:#a6e22e">ReadFile</span>(<span style="color:#a6e22e">file</span>)
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">if</span> <span style="color:#a6e22e">err</span> <span style="color:#f92672">!=</span> <span style="color:#66d9ef">nil</span> {
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">return</span> <span style="color:#66d9ef">nil</span>, <span style="color:#a6e22e">err</span>
</span></span><span style="display:flex;"><span> }
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">return</span> <span style="color:#a6e22e">contents</span>, <span style="color:#66d9ef">nil</span>
</span></span><span style="display:flex;"><span>}
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#75715e">// good
</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span><span style="color:#66d9ef">func</span> <span style="color:#a6e22e">readFileContents</span>(<span style="color:#a6e22e">file</span>) ([]<span style="color:#66d9ef">byte</span>, <span style="color:#66d9ef">error</span>) {
</span></span><span style="display:flex;"><span> <span style="color:#a6e22e">contents</span>, <span style="color:#a6e22e">err</span> <span style="color:#f92672">:=</span> <span style="color:#a6e22e">os</span>.<span style="color:#a6e22e">ReadFile</span>(<span style="color:#a6e22e">file</span>)
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">if</span> <span style="color:#a6e22e">err</span> <span style="color:#f92672">!=</span> <span style="color:#66d9ef">nil</span> {
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">return</span> <span style="color:#66d9ef">nil</span>, <span style="color:#a6e22e">fmt</span>.<span style="color:#a6e22e">Errorf</span>(<span style="color:#e6db74">&#34;readFileContents: error while reading a file: %w&#34;</span>, <span style="color:#a6e22e">err</span>)
</span></span><span style="display:flex;"><span> }
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">return</span> <span style="color:#a6e22e">contents</span>, <span style="color:#66d9ef">nil</span>
</span></span><span style="display:flex;"><span>}
</span></span></code></pre>
</body>
</html>
<p>I also have some grips about the mutate everything approach of the language. I
prefer immutability by default, but I find that in general as long as you split
your functions at a reasonable size it is generally fine.</p>
<p>And for a language that is supposed to be straightforward, it is strange on how
much magic the language relies on, in the form of things like <a href="https://go.dev/doc/modules/layout"><code>internal</code> and
<code>main</code></a> packages, name capitalisation to
indicate visibility (<code>private</code> vs <code>Public</code>), conditional compiling by filenames
(e.g.: <code>foo_amd64.go</code>, <code>bar_linux.go</code>), magic comments (e.g.: <code>//go:build</code>),
etc.</p>
<p>I expect to write more Go code going forward. Not because it is the perfect
language or whatever, but just because it a is language that has some really
good qualities that makes the language attractive even with the issues that I
have. That makes it a reasonable good language, and at least for me this is
good enough.</p>
</description>
<guid>https://github.com/thiagokokada/blog/blob/main/posts/2024-07-29/02-go-a-reasonable-good-language.md</guid>
<pubDate>Mon, 29 Jul 2024 00:00:00 +0000</pubDate>
</item>
<item>
<title>Quick bits: why you should automate everything</title>
<link>https://github.com/thiagokokada/blog/blob/main/posts/2024-07-29/01-quick-bits-why-you-should-automate-everything.md</link>
<description><p>If everything works as expected this blog post should appear in <a href="https://kokada.capivaras.dev/">in my
blog</a> without I ever touching the
<a href="https://capivaras.dev">capivaras.dev</a> website. I rewrote my <a href="https://github.com/thiagokokada/blog/blob/main/posts/2024-07-26/02-using-github-as-a-bad-blog-platform.md">previous Python
script</a> to Go
(<a href="https://github.com/thiagokokada/blog/blob/3c39e0f7cd58b1af885f69871490b05bf6fc7d99/blog.go">permalink</a>)
since my attempt to generate proper description to the RSS feed resulted in
slow startup times (not because of Python, but because of my usage of
<code>nix-shell</code> since I didn't want to deal with
<a href="https://docs.python.org/3/library/venv.html">venv</a> or anything to manage my
Python dependencies).</p>
<p>My previous workflow of this blog already involved me writing the texts in
<a href="https://neovim.io/">neovim</a>, copying and pasting the result in the
<a href="https://capivaras.dev">capivaras.dev</a> website and publishing. This was not
that bad, except that it seems I have a heavy tendency of editing my posts
multiple times. Copying and pasting data between neovim and the website became
tedious, so I decided to give up and automate the whole process.</p>
<p><a href="https://mataroa.blog/">Mataroa</a> (the blog platform
<a href="https://capivaras.dev">capivaras.dev</a> run) has a reasonable good
<a href="https://mataroa.blog/api/docs/">API</a>, and it only took a few hours to get a
version of publishing working (it would take less if
<a href="https://www.djangoproject.com/">Django</a>, the framework Mataroa is written, did
not have a weird behavior with URLs missing a trailing <code>/</code>). An additional few
lines of
<a href="https://github.com/thiagokokada/blog/blob/51b20612335c7f4312a51a0f436235b4b701ce8b/.github/workflows/go.yml">YAML</a>
to make GitHub Actions trigger a pipeline and now I should never have to
manually update my blog again.</p>
<p>I could have not done this. I mean, I probably wasted more time writing an
automation than I actually wasted publishing manually. But the manual process
is easy to miss, and I already did multiple mistakes publishing in the manual
method. For example, when writing the Markdown files, each post is written in a
particular format, where the first header is considered the title, so I need to
remove it from the contents during publication. But of course, this is easy to
miss, and I had to fix this multiple times already.</p>
<p>So yes, I think this is a good lesson on why you should automate everything. It
is more than just about <a href="https://xkcd.com/1205/">time savings</a>, it is about
reducing mistakes and even documenting (even if poorly) a process. I mean, the
code I wrote is not that great, but I can definitely rely on it in the future
to remember what I need to do. It will be much faster than trying to learn from
scratch again.</p>
</description>
<guid>https://github.com/thiagokokada/blog/blob/main/posts/2024-07-29/01-quick-bits-why-you-should-automate-everything.md</guid>
<pubDate>Mon, 29 Jul 2024 00:00:00 +0000</pubDate>
</item>
<item>
<title>Quick bits: nix-shell is cursed</title>
<link>https://github.com/thiagokokada/blog/blob/main/posts/2024-07-27/01-quick-bits-nix-shell-is-cursed.md</link>
<description><p>The other day I had to run a PHP project in my machine. I have no idea how PHP
ecosystem work, I just wanted to get it to run.</p>
<p>The easiest way to get a script to run if you use Nix is to use <code>nix-shell</code>. As
many of you probably know, you can add <code>nix-shell</code> as a shebang in your scripts
to run them as <code>./script</code>. This was a PHP script so I wanted to do the same.
Easy right?</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span><span style="color:#75715e">#!/use/bin/env nix-shell
</span></span></span><span style="display:flex;"><span><span style="color:#75715e">#!nix-shell -i php -p php83
</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span><span style="color:#f92672">&lt;?</span><span style="color:#a6e22e">php</span>
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">declare</span>(<span style="color:#a6e22e">strict_types</span><span style="color:#f92672">=</span><span style="color:#ae81ff">1</span>);
</span></span></code></pre>
</body>
</html>
<p>And:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>$ ./index.php
</span></span><span style="display:flex;"><span>Fatal error: strict_types declaration must be the very first statement in the script in index.php on line 4
</span></span></code></pre>
</body>
</html>
<p>So it seems that <code>declare(strict_types=1)</code> needs to be the first line in a PHP
script if used. I removed <code>declare(strict_types=1)</code> and while the script works,
I don't have enough expertise in PHP to know if this would be safe or not.</p>
<p>I decided to try something that initially looked really dumb:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span><span style="color:#75715e">#!/use/bin/env nix-shell
</span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span><span style="color:#f92672">&lt;?</span><span style="color:#a6e22e">php</span>
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">declare</span>(<span style="color:#a6e22e">strict_types</span><span style="color:#f92672">=</span><span style="color:#ae81ff">1</span>);
</span></span><span style="display:flex;"><span><span style="color:#75715e">#!nix-shell -i php -p php83
</span></span></span></code></pre>
</body>
</html>
<p>And:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>$ ./index.php
</span></span><span style="display:flex;"><span>Works
</span></span></code></pre>
</body>
</html>
<p>Wat? I mean, it is not dumb if it works, but this at least looks cursed.</p>
<p>Eventually I found this
<a href="https://github.com/NixOS/nix/issues/2570#issuecomment-446220517">comment</a> in a
Nix issue talking about cases where <code>nix-shell</code> shebang doesn't work. It looks
like the classic case of a <a href="https://github.com/NixOS/nix/issues/2570#issuecomment-446222206">bug that becomes a
feature</a>.</p>
<p><em>Update:</em> after posting this in
<a href="https://lobste.rs/s/gkcgza/quick_bits_nix_shell_is_cursed">Lobte.rs</a>, it seems
someone decided to open a <a href="https://github.com/NixOS/nix/pull/11202">Pull
Request</a> to document this behavior.
Also the equivalent for the new <a href="https://nix.dev/manual/nix/2.23/command-ref/new-cli/nix#shebang-interpreter">nix
CLI</a>
explicitly documents this behavior:</p>
<blockquote>
<p>Note that the <code>#! nix</code> lines don't need to follow after the first line, to
accomodate other interpreters.</p>
</blockquote>
</description>
<guid>https://github.com/thiagokokada/blog/blob/main/posts/2024-07-27/01-quick-bits-nix-shell-is-cursed.md</guid>
<pubDate>Sat, 27 Jul 2024 00:00:00 +0000</pubDate>
</item>
<item>
<title>Using GitHub as a (bad) blog platform</title>
<link>https://github.com/thiagokokada/blog/blob/main/posts/2024-07-26/02-using-github-as-a-bad-blog-platform.md</link>
<description><p>I finally started a new blog, thanks to the offer of
<a href="https://gluer.org/">@ratsclub</a> to give me free access to
<a href="https://capivaras.dev/">capivaras.dev</a>. But considering how small this blog
platform is supposed to be, I want to have at least somewhere to have a backup
of my posts. I know <a href="https://mataroa.blog/">Mataroa</a>, the blog platform that
<a href="https://capivaras.dev/">capivaras.dev</a> runs, has automatic e-mail backups, but
I want something more reliable.</p>
<p>I am writing all my posts in Markdown (the format that <a href="https://1.mataroa.blog/guides/markdown/">Mataroa
supports</a>) files inside
<a href="https://neovim.io/">neovim</a> anyway, so why not store all my Markdown files in
Git? So this is what I did, I now have an unofficial mirror in
<a href="https://github.com/thiagokokada/blog">GitHub</a>.</p>
<p>While I am here, why not overcomplicate? Can I make an usable blog platform
from GitHub? And by that I don't mean GitHub pages, the repository itself. I
mean, it already renders Markdown files by default, so no need to do anything
in that space. To reach feature parity with
<a href="https://capivaras.dev/">capivaras.dev</a>, I only need to have an index and RSS
(since comments are not supported anyway). No need for newsletter since GitHub
has a <a href="https://docs.github.com/en/account-and-profile/managing-subscriptions-and-notifications-on-github/managing-subscriptions-for-activity-on-github/viewing-your-subscriptions">watch
feature</a>
already.</p>
<p>After a couple of hours hacking a Python script, you can see the result of this
monstrosity <a href="https://github.com/thiagokokada/blog">here</a>. The script, called
<code>gen_blog.py</code>, is available at the same repository (here is a
<a href="https://github.com/thiagokokada/blog/blob/c8986d1ab1b94c0986fd814629bb8eb4034fb6e7/gen_blog.py">permalink</a>).
It automatically generates an index at
<a href="https://github.com/thiagokokada/blog/blob/main/README.md"><code>README.md</code></a> with
each blog post and a
<a href="https://raw.githubusercontent.com/thiagokokada/blog/main/rss.xml"><code>rss.xml</code></a>
file at the root of the repository.</p>
<p>Instead of trying to explain the code, I am going to explain the general idea,
because I think that if you want to replicate this idea it is better to rewrite
it in a way that you understand. It shouldn't take more than 2 hours in any
decent programming language. But if you really want, the script itself is
licensed in <a href="https://en.wikipedia.org/wiki/WTFPL">WTFPL</a> license. The code only
uses Python 3's standard library and should work in any relatively recent
version (anything newer than 3.9 should work).</p>
<p>So the idea is basically to organise the repository and the Markdown files in a
easy way that makes it trivial to parse in a deterministic way. For example, my
repository is organised in the following way:</p>
<pre><code>.
├── 2024-07-26
│ ├── 01-writing-nixos-tests-for-fun-and-profit.md
│ └── 02-using-github-as-a-bad-blog-platform.md &lt;- this file
├── gen_blog.py
├── README.md
└── rss.xml
</code></pre>
<p>Each day that you write a new blog post will be on its own directory. This is
nice because Markdown files may include extra files than the posts themselves,
e.g.: images, and this organisation make it trivial to organise everything.</p>
<p>Each post has its own Markdown file. I put a two digit number before each post,
to ensure that when publishing multiple posts at the same day I keep them in
the same order of publishing. But if you don't care about it, you can just name
the files whatever you want.</p>
<p>Also, I am assuming that each Markdown file has a header starting with <code># </code>,
and that is the title of the blog post.</p>
<p>Using the above organisation, I have this function that scrapes the repository
and collect the necessary information to generate the index and RSS files:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span><span style="color:#66d9ef">def</span> <span style="color:#a6e22e">grab_posts</span>(pwd: Path):
</span></span><span style="display:flex;"><span> posts <span style="color:#f92672">=</span> defaultdict(list)
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">for</span> dir <span style="color:#f92672">in</span> sorted(pwd<span style="color:#f92672">.</span>iterdir(), reverse<span style="color:#f92672">=</span><span style="color:#66d9ef">True</span>):
</span></span><span style="display:flex;"><span> <span style="color:#75715e"># Ignore non-directories or hidden files</span>
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">if</span> <span style="color:#f92672">not</span> dir<span style="color:#f92672">.</span>is_dir() <span style="color:#f92672">or</span> dir<span style="color:#f92672">.</span>name[<span style="color:#ae81ff">0</span>] <span style="color:#f92672">==</span> <span style="color:#e6db74">&#34;.&#34;</span>:
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">continue</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#75715e"># Try to parse date from directory name</span>
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">try</span>:
</span></span><span style="display:flex;"><span> date <span style="color:#f92672">=</span> datetime<span style="color:#f92672">.</span>strptime(dir<span style="color:#f92672">.</span>name, <span style="color:#e6db74">&#34;%Y-%m-</span><span style="color:#e6db74">%d</span><span style="color:#e6db74">&#34;</span>)
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">except</span> <span style="color:#a6e22e">ValueError</span>:
</span></span><span style="display:flex;"><span> print(<span style="color:#e6db74">f</span><span style="color:#e6db74">&#34;WARN: ignoring non-date directory: </span><span style="color:#e6db74">{</span>dir<span style="color:#e6db74">}</span><span style="color:#e6db74">&#34;</span>, file<span style="color:#f92672">=</span>sys<span style="color:#f92672">.</span>stderr)
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">continue</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#75715e"># Iterate between the files in the date directory</span>
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">for</span> post <span style="color:#f92672">in</span> sorted(dir<span style="color:#f92672">.</span>iterdir(), reverse<span style="color:#f92672">=</span><span style="color:#66d9ef">True</span>):
</span></span><span style="display:flex;"><span> <span style="color:#75715e"># Ignore non-markdown files or hidden files (draft)</span>
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">if</span> <span style="color:#f92672">not</span> post<span style="color:#f92672">.</span>suffix <span style="color:#f92672">==</span> <span style="color:#e6db74">&#34;.md&#34;</span> <span style="color:#f92672">or</span> post<span style="color:#f92672">.</span>name[<span style="color:#ae81ff">0</span>] <span style="color:#f92672">==</span> <span style="color:#e6db74">&#34;.&#34;</span>:
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">continue</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#75715e"># Grab the first H1 section to parse as title</span>
</span></span><span style="display:flex;"><span> text <span style="color:#f92672">=</span> post<span style="color:#f92672">.</span>read_text()
</span></span><span style="display:flex;"><span> mTitle <span style="color:#f92672">=</span> re<span style="color:#f92672">.</span>match(<span style="color:#e6db74">r</span><span style="color:#e6db74">&#34;# (?P&lt;title&gt;.*)\r?\n&#34;</span>, text)
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">if</span> mTitle <span style="color:#f92672">and</span> (title <span style="color:#f92672">:=</span> mTitle<span style="color:#f92672">.</span>groupdict()<span style="color:#f92672">.</span>get(<span style="color:#e6db74">&#34;title&#34;</span>)):
</span></span><span style="display:flex;"><span> posts[date]<span style="color:#f92672">.</span>append({<span style="color:#e6db74">&#34;title&#34;</span>: title, <span style="color:#e6db74">&#34;file&#34;</span>: str(post)})
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">else</span>:
</span></span><span style="display:flex;"><span> print(<span style="color:#e6db74">f</span><span style="color:#e6db74">&#34;WARN: did not find title for file: </span><span style="color:#e6db74">{</span>post<span style="color:#e6db74">}</span><span style="color:#e6db74">&#34;</span>, file<span style="color:#f92672">=</span>sys<span style="color:#f92672">.</span>stderr)
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">return</span> posts
</span></span></code></pre>
</body>
</html>
<p>Some interesting tidbits: if a Markdown file has a <code>.</code> at the start I assume it
is a draft post, and ignore it from my scraper. I added a bunch of <code>WARN</code>
prints to make sure that the me in the future doesn't do anything dumb. Also,
sorting in reverse since reverse chronological order is the one most people
expect in blogs (i.e.: more recent blog posts at top).</p>
<p>After running the function above, I have a resulting dictionary that I can use
to generate either a <code>README.md</code> file or Markdown:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span><span style="color:#66d9ef">def</span> <span style="color:#a6e22e">gen_readme</span>(posts):
</span></span><span style="display:flex;"><span> titles <span style="color:#f92672">=</span> []
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">for</span> date, dayPosts <span style="color:#f92672">in</span> posts<span style="color:#f92672">.</span>items():
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">for</span> post <span style="color:#f92672">in</span> dayPosts:
</span></span><span style="display:flex;"><span> <span style="color:#75715e"># This creates a relative link to the Markdown file, .e.g.:</span>
</span></span><span style="display:flex;"><span> <span style="color:#75715e"># ./02-using-github-as-a-bad-blog-platform.md</span>
</span></span><span style="display:flex;"><span> link <span style="color:#f92672">=</span> os<span style="color:#f92672">.</span>path<span style="color:#f92672">.</span>join(<span style="color:#e6db74">&#34;.&#34;</span>, post[<span style="color:#e6db74">&#34;file&#34;</span>])
</span></span><span style="display:flex;"><span> <span style="color:#75715e"># This formats the title, e.g.:</span>
</span></span><span style="display:flex;"><span> <span style="color:#75715e"># - [Using GitHub as a (bad) blog platform](./2024-07-26/02-using-github-as-a-bad-blog-platform.md) - 2024-07-26</span>
</span></span><span style="display:flex;"><span> title <span style="color:#f92672">=</span> date<span style="color:#f92672">.</span>strftime(<span style="color:#e6db74">f</span><span style="color:#e6db74">&#34;- [</span><span style="color:#e6db74">{</span>post[<span style="color:#e6db74">&#39;title&#39;</span>]<span style="color:#e6db74">}</span><span style="color:#e6db74">](</span><span style="color:#e6db74">{</span>link<span style="color:#e6db74">}</span><span style="color:#e6db74">) - %Y-%m-%d&#34;</span>)
</span></span><span style="display:flex;"><span> <span style="color:#75715e"># This appends to the list to generate the content later</span>
</span></span><span style="display:flex;"><span> titles<span style="color:#f92672">.</span>append(title)
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#75715e"># README_TEMPLATE is a string with the static part of the README</span>
</span></span><span style="display:flex;"><span> print(README_TEMPLATE<span style="color:#f92672">.</span>format(posts<span style="color:#f92672">=</span><span style="color:#e6db74">&#34;</span><span style="color:#ae81ff">\n</span><span style="color:#e6db74">&#34;</span><span style="color:#f92672">.</span>join(titles)))
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">def</span> <span style="color:#a6e22e">gen_rss</span>(posts):
</span></span><span style="display:flex;"><span> <span style="color:#75715e"># Got most of the specification from here:</span>
</span></span><span style="display:flex;"><span> <span style="color:#75715e"># https://www.w3schools.com/XML/xml_rss.asp</span>
</span></span><span style="display:flex;"><span> rss <span style="color:#f92672">=</span> ET<span style="color:#f92672">.</span>Element(<span style="color:#e6db74">&#34;rss&#34;</span>, version<span style="color:#f92672">=</span><span style="color:#e6db74">&#34;2.0&#34;</span>)
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#75715e"># Here are the RSS metadata for the blog itself</span>
</span></span><span style="display:flex;"><span> channel <span style="color:#f92672">=</span> ET<span style="color:#f92672">.</span>SubElement(rss, <span style="color:#e6db74">&#34;channel&#34;</span>)
</span></span><span style="display:flex;"><span> ET<span style="color:#f92672">.</span>SubElement(channel, <span style="color:#e6db74">&#34;title&#34;</span>)<span style="color:#f92672">.</span>text <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;kokada&#39;s blog&#34;</span>
</span></span><span style="display:flex;"><span> ET<span style="color:#f92672">.</span>SubElement(channel, <span style="color:#e6db74">&#34;link&#34;</span>)<span style="color:#f92672">.</span>text <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;https://github.com/thiagokokada/blog&#34;</span>
</span></span><span style="display:flex;"><span> ET<span style="color:#f92672">.</span>SubElement(channel, <span style="color:#e6db74">&#34;description&#34;</span>)<span style="color:#f92672">.</span>text <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;dd if=/dev/urandom of=/dev/brain0&#34;</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#75715e"># You create one item for each blog post</span>
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">for</span> date, dayPost <span style="color:#f92672">in</span> posts<span style="color:#f92672">.</span>items():
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">for</span> post <span style="color:#f92672">in</span> dayPost:
</span></span><span style="display:flex;"><span> item <span style="color:#f92672">=</span> ET<span style="color:#f92672">.</span>SubElement(channel, <span style="color:#e6db74">&#34;item&#34;</span>)
</span></span><span style="display:flex;"><span> link <span style="color:#f92672">=</span> urljoin(RSS_POST_LINK_PREFIX, post[<span style="color:#e6db74">&#34;file&#34;</span>])
</span></span><span style="display:flex;"><span> ET<span style="color:#f92672">.</span>SubElement(item, <span style="color:#e6db74">&#34;title&#34;</span>)<span style="color:#f92672">.</span>text <span style="color:#f92672">=</span> post[<span style="color:#e6db74">&#34;title&#34;</span>]
</span></span><span style="display:flex;"><span> ET<span style="color:#f92672">.</span>SubElement(item, <span style="color:#e6db74">&#34;guid&#34;</span>)<span style="color:#f92672">.</span>text <span style="color:#f92672">=</span> link
</span></span><span style="display:flex;"><span> ET<span style="color:#f92672">.</span>SubElement(item, <span style="color:#e6db74">&#34;link&#34;</span>)<span style="color:#f92672">.</span>text <span style="color:#f92672">=</span> link
</span></span><span style="display:flex;"><span> ET<span style="color:#f92672">.</span>SubElement(item, <span style="color:#e6db74">&#34;pubDate&#34;</span>)<span style="color:#f92672">.</span>text <span style="color:#f92672">=</span> date<span style="color:#f92672">.</span>strftime(<span style="color:#e6db74">&#39;</span><span style="color:#e6db74">%a</span><span style="color:#e6db74">, </span><span style="color:#e6db74">%d</span><span style="color:#e6db74"> %b %Y %H:%M:%S GMT&#39;</span>)
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#75715e"># Generate the XML and indent</span>
</span></span><span style="display:flex;"><span> tree <span style="color:#f92672">=</span> ET<span style="color:#f92672">.</span>ElementTree(rss)
</span></span><span style="display:flex;"><span> ET<span style="color:#f92672">.</span>indent(tree, space<span style="color:#f92672">=</span><span style="color:#e6db74">&#34;</span><span style="color:#ae81ff">\t</span><span style="color:#e6db74">&#34;</span>, level<span style="color:#f92672">=</span><span style="color:#ae81ff">0</span>)
</span></span><span style="display:flex;"><span> tree<span style="color:#f92672">.</span>write(<span style="color:#e6db74">&#34;rss.xml&#34;</span>, xml_declaration<span style="color:#f92672">=</span><span style="color:#66d9ef">True</span>, encoding<span style="color:#f92672">=</span><span style="color:#e6db74">&#34;UTF-8&#34;</span>)
</span></span></code></pre>
</body>
</html>
<p>To publish a new Post, a basically write a Markdown file, run `./gen_readme.py</p>
<blockquote>
<p>README.md` at the root of the repository, and see the magic happen.</p>
</blockquote>
<p>It works much better than I initially anticipated. The <code>README.md</code> is properly
populated with the titles and links. The RSS is kind of empty since it has no
description, but it seems to work fine (at least in
<a href="https://www.inoreader.com/">Inoreader</a>, my RSS reader of choice). I can
probably fill the post description with more information if I really want, but
it is enough for now (update: it is working now, you just need to render the
Markdown as HTML and escape the tags;
<a href="https://github.com/thiagokokada/blog/blob/9506051cc3d49f203304174b335ff62de9d17a05/gen_blog.py">permalink</a>
for the updated script). Not sure who is that interested in my writing that
will want to use this RSS feed instead the one available in
<a href="https://kokada.capivaras.dev/rss/">capivaras.dev</a> anyway.</p>
<p>Also, while I am using GitHub here, the same idea would work in GitLab, Gitea,
sr.ht or whatever. As long as your source hub supports Markdown files it should
work.</p>
<p>So that is it. I am not saying this is a good idea for your primary blog
platform or whatever, and I still prefer to publish to a platform that doesn't
track users or have tons of JavaScript or whatever. But if you want a backup of
your posts and you are already writing Markdown anyway, well, there are worse
ways to do it I think.</p>
<p><em>Update:</em> I rewrote the script again using Go
(<a href="https://github.com/thiagokokada/blog/blob/3ece3853e7dd32977e7ec5238d2955e530ca4beb/blog.go">permalink</a>).
The reason for it is because when I started rendering Markdown (for
descriptions) the Python version got quite slow (not the fault of Python
itself, mostly because of the usage of <code>nix-shell</code> to manage dependencies;
something that Go doesn't need). Took about half an hour, showing how easy it
is to do the same.</p>
</description>
<guid>https://github.com/thiagokokada/blog/blob/main/posts/2024-07-26/02-using-github-as-a-bad-blog-platform.md</guid>
<pubDate>Fri, 26 Jul 2024 00:00:00 +0000</pubDate>
</item>
<item>
<title>Writing NixOS tests for fun and profit</title>
<link>https://github.com/thiagokokada/blog/blob/main/posts/2024-07-26/01-writing-nixos-tests-for-fun-and-profit.md</link>
<description><p>I recently started a <a href="https://github.com/thiagokokada/hyprland-go">new side
project</a> writing an IPC library in
Go for <a href="https://hyprland.org/">Hyprland</a>, a Window Manager for Wayland.</p>
<p>Once I got past the Work-in-Progress phase, I realise I had an issue: I wrote
some tests, but I was running then inside my system running Hyprland. And the
tests themselves were annoying: since they send commands to the current running
Hyprland instance, I was having programs being opened and settings being
changed, because this was the only way to have a reasonable good confidence
that what I was doing was correct. So I need to do like any good developer and
implement a CI, but how?</p>
<p>One approach would be to create something like a mock client and test against
my mock. Since this mock wouldn't need a running Hyprland instance the tests
could run everywhere (even in non-Linux systems!), but they wouldn't be much
useful. Mocks are great for testing business logic, but not really for making
sure everything is working correctly.</p>
<p>I need something more akin to an integration test, but this is tricky. It is
not like I am doing integration with e.g.: PostgreSQL that has thousands of
libraries available to make integration tests easier, I am doing integration
with a Window Manager that is a moving target with multiple breaking changes in
each release. And this is where NixOS tests enter, a way to run tests inside
Virtual Machines configured in Nix.</p>
<p>I am a long time NixOS user and commiter, but I never wrote a NixOS test
outside of <a href="https://github.com/NixOS/nixpkgs">nixpkgs</a> itself. However I knew
it was possible, and after doing a quick reading of the <a href="https://wiki.nixos.org/wiki/NixOS_VM_tests">Wiki
entry</a> about it, I was ready to
start.</p>
<p>The first part is to call <code>pkgs.testers.runNixOSTest</code> and configure the machine
as any other NixOS system, e.g.:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>{ pkgs<span style="color:#f92672">,</span> <span style="color:#f92672">...</span> }:
</span></span><span style="display:flex;"><span>pkgs<span style="color:#f92672">.</span>testers<span style="color:#f92672">.</span>runNixOSTest {
</span></span><span style="display:flex;"><span> name <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;hyprland-go&#34;</span>;
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> nodes<span style="color:#f92672">.</span>machine <span style="color:#f92672">=</span>
</span></span><span style="display:flex;"><span> { config<span style="color:#f92672">,</span> pkgs<span style="color:#f92672">,</span> lib<span style="color:#f92672">,</span> <span style="color:#f92672">...</span> }:
</span></span><span style="display:flex;"><span> {
</span></span><span style="display:flex;"><span> <span style="color:#75715e"># bootloader related configuration</span>
</span></span><span style="display:flex;"><span> boot<span style="color:#f92672">.</span>loader<span style="color:#f92672">.</span>systemd-boot<span style="color:#f92672">.</span>enable <span style="color:#f92672">=</span> <span style="color:#66d9ef">true</span>;
</span></span><span style="display:flex;"><span> boot<span style="color:#f92672">.</span>loader<span style="color:#f92672">.</span>efi<span style="color:#f92672">.</span>canTouchEfiVariables <span style="color:#f92672">=</span> <span style="color:#66d9ef">true</span>;
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#75715e"># enable hyprland</span>
</span></span><span style="display:flex;"><span> programs<span style="color:#f92672">.</span>hyprland<span style="color:#f92672">.</span>enable <span style="color:#f92672">=</span> <span style="color:#66d9ef">true</span>;
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#75715e"># create a user called alice</span>
</span></span><span style="display:flex;"><span> users<span style="color:#f92672">.</span>users<span style="color:#f92672">.</span>alice <span style="color:#f92672">=</span> {
</span></span><span style="display:flex;"><span> isNormalUser <span style="color:#f92672">=</span> <span style="color:#66d9ef">true</span>;
</span></span><span style="display:flex;"><span> };
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#75715e"># add some extra packages that we need during tests</span>
</span></span><span style="display:flex;"><span> environment<span style="color:#f92672">.</span>systemPackages <span style="color:#f92672">=</span> <span style="color:#66d9ef">with</span> pkgs; [ go kitty ];
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#75715e"># auto login as alice</span>
</span></span><span style="display:flex;"><span> services<span style="color:#f92672">.</span>getty<span style="color:#f92672">.</span>autologinUser <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;alice&#34;</span>;
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#75715e"># configure VM, increase memory and CPU and enable OpenGL via LLVMPipe</span>
</span></span><span style="display:flex;"><span> virtualisation<span style="color:#f92672">.</span>qemu <span style="color:#f92672">=</span> {
</span></span><span style="display:flex;"><span> options <span style="color:#f92672">=</span> [
</span></span><span style="display:flex;"><span> <span style="color:#e6db74">&#34;-smp 2&#34;</span>
</span></span><span style="display:flex;"><span> <span style="color:#e6db74">&#34;-m 4G&#34;</span>
</span></span><span style="display:flex;"><span> <span style="color:#e6db74">&#34;-vga none&#34;</span>
</span></span><span style="display:flex;"><span> <span style="color:#e6db74">&#34;-device virtio-gpu-pci&#34;</span>
</span></span><span style="display:flex;"><span> ];
</span></span><span style="display:flex;"><span> };
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#75715e"># Start hyprland at login</span>
</span></span><span style="display:flex;"><span> programs<span style="color:#f92672">.</span>bash<span style="color:#f92672">.</span>loginShellInit <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;Hyprland&#34;</span>;
</span></span><span style="display:flex;"><span> };
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> testScript <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;start_all()&#34;</span>;
</span></span><span style="display:flex;"><span>}
</span></span></code></pre>
</body>
</html>
<p>A few details that I want to bring to attention. The first one is how easy it
is to setup things like a normal user account, add some extra packages we need
for testing, add Hyprland itself and configure auto-login. I have no idea how
painful it would be to automatise all those steps in e.g.: Ansible, but here we
are in a few lines of Nix code. This is, of course, thanks to all the
contributors to nixpkgs that implement something that help their own use case,
but once combined make it greater than the sum of the parts.</p>
<p>Second is something that I took a while to figure out: how to enable GPU
acceleration inside the VM. You see, Hyprland, different from other Window
Managers, requires OpenGL support. This is basically why the flag <code>-device virtio-gpu-pci</code> is in <code>virtualisation.qemu.options</code>, this enables OpenGL
rendering via LLVMPipe, that while being slow since it is rendered in CPU, is
sufficient for this case.</p>
<p>Putting the above code inside a
<a href="https://wiki.nixos.org/wiki/Flakes"><code>flake.nix</code></a> for reproducibility, I had
something similar to:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>{
</span></span><span style="display:flex;"><span> description <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;Hyprland&#39;s IPC bindings for Go&#34;</span>;
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> inputs <span style="color:#f92672">=</span> {
</span></span><span style="display:flex;"><span> nixpkgs<span style="color:#f92672">.</span>url <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;github:NixOS/nixpkgs/nixos-unstable&#34;</span>;
</span></span><span style="display:flex;"><span> };
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> outputs <span style="color:#f92672">=</span> { nixpkgs<span style="color:#f92672">,</span> <span style="color:#f92672">...</span> }:
</span></span><span style="display:flex;"><span> {
</span></span><span style="display:flex;"><span> checks<span style="color:#f92672">.</span>x86_64-linux <span style="color:#f92672">=</span>
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">let</span>
</span></span><span style="display:flex;"><span> pkgs <span style="color:#f92672">=</span> <span style="color:#f92672">import</span> nixpkgs { system <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;x86_64-linux&#34;</span>; };
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">in</span>
</span></span><span style="display:flex;"><span> {
</span></span><span style="display:flex;"><span> testVm <span style="color:#f92672">=</span> pkgs<span style="color:#f92672">.</span>testers<span style="color:#f92672">.</span>runNixOSTest {
</span></span><span style="display:flex;"><span> <span style="color:#75715e"># the code above</span>
</span></span><span style="display:flex;"><span> };
</span></span><span style="display:flex;"><span> }
</span></span><span style="display:flex;"><span> };
</span></span><span style="display:flex;"><span>}
</span></span></code></pre>
</body>
</html>
<p>I can now run <code>nix build .#checks.x86_64-linux.testVm -L</code> to build and run the
VM. However it is not really useful right now, since we didn't add any useful
code in <code>testScript</code>, the core of the NixOS test framework. We can also run
<code>nix build .#checks.x86_64-linux.testVm.driverInteractive</code> and
<code>./result/bin/nixos-test-driver</code>: this will start a Python console where we can
manually play with the VM (try typing <code>start_all()</code> for example).</p>
<p>The <code>testScript</code> is a sequence of Python statements that perform various
actions, such as starting VMs, executing commands in the VMs, and so on. More
about it in the official
<a href="https://nixos.org/manual/nixos/stable/index.html#sec-nixos-tests">documentation</a>.
For our case we can start with something like this:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>{
</span></span><span style="display:flex;"><span> testScript <span style="color:#f92672">=</span> <span style="color:#75715e">/* python */</span> <span style="color:#e6db74">&#39;&#39;
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> start_all()
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74">
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> machine.wait_for_unit(&#34;multi-user.target&#34;)
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> machine.wait_for_file(&#34;/home/alice/test-finished&#34;)
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> &#39;&#39;</span>;
</span></span><span style="display:flex;"><span>}
</span></span></code></pre>
</body>
</html>
<p>The first statement, <code>start_all()</code>, starts all VMs, in this case we have only
one, called <code>machine</code>. We send two further commands to <code>machine</code>:
<code>wait_for_unit(&quot;multi-user.target&quot;)</code> and
<code>wait_for_file(&quot;/home/alice/test-finished&quot;)</code>.</p>
<p>The first command waits until systemd's <code>multi-user.target</code> is ready, a good
way to ensure that the system is ready for further commands. The second one we
wait for a file called <code>test-finished</code> to appear in Alice's <code>$HOME</code> (basically,
a canary), but how can we generate this file?</p>
<p>Remember that we added <code>programs.bash.loginShellInit = &quot;Hyprland&quot;</code>, that
automatically starts Hyprland when Alice logs in. We need to modify that
command to run the Go tests from our library. The good thing is that Hyprland
configuration file supports a
<a href="https://wiki.hyprland.org/Configuring/Keywords/#executing"><code>exec-once</code></a>
command that runs a command during Hyprland launch. We can abuse this to launch
a terminal emulator and run our tests:</p>
<html>
<body style="color:#f8f8f2;background-color:#272822">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;"><code><span style="display:flex;"><span>{
</span></span><span style="display:flex;"><span> programs<span style="color:#f92672">.</span>bash<span style="color:#f92672">.</span>loginShellInit <span style="color:#f92672">=</span>
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">let</span>
</span></span><span style="display:flex;"><span> testScript <span style="color:#f92672">=</span> pkgs<span style="color:#f92672">.</span>writeShellScript <span style="color:#e6db74">&#34;hyprland-go-test&#34;</span> <span style="color:#e6db74">&#39;&#39;
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> set -euo pipefail
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74">
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> trap &#39;echo $? &gt; $HOME/test-finished&#39; EXIT # creates the canary when the script finishes
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74">
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> cd </span><span style="color:#e6db74">${</span><span style="color:#e6db74">./.</span><span style="color:#e6db74">}</span><span style="color:#e6db74"> # go to the library directory
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> go test -v ./... &gt; $HOME/test.log 2&gt;&amp;1 # run Go tests
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> &#39;&#39;</span>;
</span></span><span style="display:flex;"><span> hyprlandConf <span style="color:#f92672">=</span> pkgs<span style="color:#f92672">.</span>writeText <span style="color:#e6db74">&#34;hyprland.conf&#34;</span> <span style="color:#e6db74">&#39;&#39;
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> exec-once = kitty sh -c </span><span style="color:#e6db74">${</span>testScript<span style="color:#e6db74">}</span><span style="color:#e6db74">
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> &#39;&#39;</span>;
</span></span><span style="display:flex;"><span> <span style="color:#66d9ef">in</span> <span style="color:#e6db74">&#39;&#39;
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> Hyprland --config </span><span style="color:#e6db74">${</span>hyprlandConf<span style="color:#e6db74">}</span><span style="color:#e6db74">
</span></span></span><span style="display:flex;"><span><span style="color:#e6db74"> &#39;&#39;</span>;
</span></span><span style="display:flex;"><span>}
</span></span></code></pre>
</body>
</html>
<p>So we are basically creating a custom Hyprland config that starts a
<a href="https://sw.kovidgoyal.net/kitty/">Kitty</a> terminal emulator, that then launches
a shell script that runs the test. Since we have no way to get the results of
the test, we pipe the output to a file that we can collect later (e.g.:
<code>machine.succeded(&quot;cat /home/alice/test.log&quot;)</code>). And once the script exit, we
create the canary file <code>$HOME/test-finished</code>, that allows the <code>testScript</code>
knows that the test finished and it can destroy the VM safely.</p>
<p>If you want to take a look at the final result, it is
<a href="https://github.com/thiagokokada/hyprland-go/blob/v0.0.1/flake.nix">here</a>. This
tests run in any Linux machine that supports KVM, and also works in <a href="https://github.com/thiagokokada/hyprland-go/actions/workflows/nix.yaml">GitHub
Actions</a>
thanks to the the
<a href="https://github.com/DeterminateSystems/nix-installer-action">nix-installer-action</a>.</p>
<p>And now I have a proper CI pipeline in a way that I never imagined would be
possible, especially considering how simple it was.</p>
</description>
<guid>https://github.com/thiagokokada/blog/blob/main/posts/2024-07-26/01-writing-nixos-tests-for-fun-and-profit.md</guid>
<pubDate>Fri, 26 Jul 2024 00:00:00 +0000</pubDate>
</item>
</channel>
</rss>