diff --git a/index.xml b/index.xml
index 4bfdb80..8c4b3ce 100644
--- a/index.xml
+++ b/index.xml
@@ -6,65 +6,65 @@
Last 10 notes on Chaotic Good ComputingQuartz -- quartz.jzhao.xyz
- CGC Mentoring
- https://blog.chaoticgood.computer/tags/mentoring/
- https://blog.chaoticgood.computer/tags/mentoring/
+ Reading Annotations
+ https://blog.chaoticgood.computer/tags/annotations
+ https://blog.chaoticgood.computer/tags/annotations
- Mon, 18 Mar 2024 17:05:48 GMT
+ Mon, 18 Mar 2024 17:07:58 GMT
- Tutorials
- https://blog.chaoticgood.computer/tags/mentoring/tutorials
- https://blog.chaoticgood.computer/tags/mentoring/tutorials
-
- Mon, 18 Mar 2024 17:05:48 GMT
+ CGC Articles
+ https://blog.chaoticgood.computer/tags/articles
+ https://blog.chaoticgood.computer/tags/articles
+ Here, you’ll find some actual, bona-fide Articles™. You can expect a higher degree of cohesion, revision, and quality than you may find in, say, notes.
+ Mon, 18 Mar 2024 17:07:58 GMT
- Daily Notes
- https://blog.chaoticgood.computer/tags/notes/daily
- https://blog.chaoticgood.computer/tags/notes/daily
- These are daily notes, where I log and track progress on ongoing projects and make notes for myself later. These are largely for my own benefit, and are here in case they’re helpful to myself (or, somehow, others) in the future.
- Mon, 18 Mar 2024 17:05:48 GMT
+ CGC Business Admin
+ https://blog.chaoticgood.computer/tags/business/admin
+ https://blog.chaoticgood.computer/tags/business/admin
+
+ Mon, 18 Mar 2024 17:07:58 GMT
- Spencer's Notes
- https://blog.chaoticgood.computer/tags/notes/
- https://blog.chaoticgood.computer/tags/notes/
- These are notes - they’re things that are structured, albeit not necessarily as structured or refined as CGC articles. In general, there are three kinds of notes I make: General notes, which are found on this page Daily Notes, which contain daily logs for my own benefit Scratch Notes, which really just means “none of the above” .
- Mon, 18 Mar 2024 17:05:48 GMT
+ Economics & Finance
+ https://blog.chaoticgood.computer/tags/economics/finance
+ https://blog.chaoticgood.computer/tags/economics/finance
+
+ Mon, 18 Mar 2024 17:07:58 GMT
- Scratch Notes
- https://blog.chaoticgood.computer/tags/notes/scratch
- https://blog.chaoticgood.computer/tags/notes/scratch
- For all intents and purposes, these are the digital gardening equivalent of shitposts and shower thoughts.
- Mon, 18 Mar 2024 17:05:48 GMT
+ Economics
+ https://blog.chaoticgood.computer/tags/economics/
+ https://blog.chaoticgood.computer/tags/economics/
+ Economics is the core research focus for Chaotic Good Computing. Wherever there is a finite amount of resources and an infinite capacity for demand, an economy forms.
+ Mon, 18 Mar 2024 17:07:58 GMT
- CGC Projects
- https://blog.chaoticgood.computer/tags/projects/cgc
- https://blog.chaoticgood.computer/tags/projects/cgc
+ Market Design & Non-Market Environments
+ https://blog.chaoticgood.computer/tags/economics/market-design
+ https://blog.chaoticgood.computer/tags/economics/market-design
- Mon, 18 Mar 2024 17:05:48 GMT
+ Mon, 18 Mar 2024 17:07:58 GMT
- Day Job Projects
- https://blog.chaoticgood.computer/tags/projects/dayjob
- https://blog.chaoticgood.computer/tags/projects/dayjob
- For obvious reasons, anything related to these projects will be very vague.
- Mon, 18 Mar 2024 17:05:48 GMT
+ Project Management
+ https://blog.chaoticgood.computer/tags/economics/project-management
+ https://blog.chaoticgood.computer/tags/economics/project-management
+
+ Mon, 18 Mar 2024 17:07:58 GMT
- Projects
- https://blog.chaoticgood.computer/tags/projects/
- https://blog.chaoticgood.computer/tags/projects/
+ Economics & Strategy
+ https://blog.chaoticgood.computer/tags/economics/strategy
+ https://blog.chaoticgood.computer/tags/economics/strategy
- Mon, 18 Mar 2024 17:05:48 GMT
+ Mon, 18 Mar 2024 17:07:58 GMT
- Personal Projects
- https://blog.chaoticgood.computer/tags/projects/personal
- https://blog.chaoticgood.computer/tags/projects/personal
+ C\#
+ https://blog.chaoticgood.computer/tags/engineering/csharp
+ https://blog.chaoticgood.computer/tags/engineering/csharp
- Mon, 18 Mar 2024 17:05:48 GMT
+ Mon, 18 Mar 2024 17:07:58 GMT
- Doodles, a.k.a. Minimally Professional Presentations
- https://blog.chaoticgood.computer/tags/viz/doodles
- https://blog.chaoticgood.computer/tags/viz/doodles
+ engineering
+ https://blog.chaoticgood.computer/tags/engineering/
+ https://blog.chaoticgood.computer/tags/engineering/
- Mon, 18 Mar 2024 17:05:48 GMT
+ Mon, 18 Mar 2024 17:07:58 GMT
\ No newline at end of file
diff --git a/notes/dailies/2024-03-18.html b/notes/dailies/2024-03-18.html
index 9725e39..31c4ea6 100644
--- a/notes/dailies/2024-03-18.html
+++ b/notes/dailies/2024-03-18.html
@@ -52,7 +52,7 @@
This has been backburned for way too long. I will send this today, before I do anything else with CGC time.
+
This has been backburn’d for way too long. I will send this today, before I do anything else with CGC time.
PLAN:
diff --git a/sitemap.xml b/sitemap.xml
index 0ea5fd2..1f7be10 100644
--- a/sitemap.xml
+++ b/sitemap.xml
@@ -18,7 +18,7 @@
2022-07-14T00:00:00.000Zhttps://blog.chaoticgood.computer/
- 2024-03-18T17:05:48.646Z
+ 2024-03-18T17:07:58.945Zhttps://blog.chaoticgood.computer/notes/annotations/college-admissions-and-the-stability-of-marriage2024-03-14T00:00:00.000Z
@@ -162,77 +162,77 @@
2021-03-01T00:00:00.000Zhttps://blog.chaoticgood.computer/tags/annotations
- 2024-03-18T17:05:48.646Z
+ 2024-03-18T17:07:58.949Zhttps://blog.chaoticgood.computer/tags/articles
- 2024-03-18T17:05:48.646Z
+ 2024-03-18T17:07:58.949Zhttps://blog.chaoticgood.computer/tags/business/admin
- 2024-03-18T17:05:48.646Z
+ 2024-03-18T17:07:58.949Zhttps://blog.chaoticgood.computer/tags/economics/finance
- 2024-03-18T17:05:48.646Z
+ 2024-03-18T17:07:58.949Zhttps://blog.chaoticgood.computer/tags/economics/
- 2024-03-18T17:05:48.646Z
+ 2024-03-18T17:07:58.949Zhttps://blog.chaoticgood.computer/tags/economics/market-design
- 2024-03-18T17:05:48.646Z
+ 2024-03-18T17:07:58.949Zhttps://blog.chaoticgood.computer/tags/economics/project-management
- 2024-03-18T17:05:48.646Z
+ 2024-03-18T17:07:58.949Zhttps://blog.chaoticgood.computer/tags/economics/strategy
- 2024-03-18T17:05:48.646Z
+ 2024-03-18T17:07:58.949Zhttps://blog.chaoticgood.computer/tags/engineering/csharp
- 2024-03-18T17:05:48.646Z
+ 2024-03-18T17:07:58.949Zhttps://blog.chaoticgood.computer/tags/engineering/
- 2024-03-18T17:05:48.646Z
+ 2024-03-18T17:07:58.949Zhttps://blog.chaoticgood.computer/tags/engineering/python
- 2024-03-18T17:05:48.646Z
+ 2024-03-18T17:07:58.949Zhttps://blog.chaoticgood.computer/tags/games/roblox
- 2024-03-18T17:05:48.646Z
+ 2024-03-18T17:07:58.949Zhttps://blog.chaoticgood.computer/tags/goblin-slaying
- 2024-03-18T17:05:48.646Z
+ 2024-03-18T17:07:58.949Zhttps://blog.chaoticgood.computer/tags/horticulture
- 2024-03-18T17:05:48.646Z
+ 2024-03-18T17:07:58.949Zhttps://blog.chaoticgood.computer/tags/mentoring/
- 2024-03-18T17:05:48.650Z
+ 2024-03-18T17:07:58.949Zhttps://blog.chaoticgood.computer/tags/mentoring/tutorials
- 2024-03-18T17:05:48.650Z
+ 2024-03-18T17:07:58.949Zhttps://blog.chaoticgood.computer/tags/notes/daily
- 2024-03-18T17:05:48.650Z
+ 2024-03-18T17:07:58.949Zhttps://blog.chaoticgood.computer/tags/notes/
- 2024-03-18T17:05:48.650Z
+ 2024-03-18T17:07:58.949Zhttps://blog.chaoticgood.computer/tags/notes/scratch
- 2024-03-18T17:05:48.650Z
+ 2024-03-18T17:07:58.949Zhttps://blog.chaoticgood.computer/tags/projects/cgc
- 2024-03-18T17:05:48.650Z
+ 2024-03-18T17:07:58.949Zhttps://blog.chaoticgood.computer/tags/projects/dayjob
- 2024-03-18T17:05:48.650Z
+ 2024-03-18T17:07:58.949Zhttps://blog.chaoticgood.computer/tags/projects/
- 2024-03-18T17:05:48.650Z
+ 2024-03-18T17:07:58.949Zhttps://blog.chaoticgood.computer/tags/projects/personal
- 2024-03-18T17:05:48.650Z
+ 2024-03-18T17:07:58.949Zhttps://blog.chaoticgood.computer/tags/viz/doodles
- 2024-03-18T17:05:48.650Z
+ 2024-03-18T17:07:58.949Zhttps://blog.chaoticgood.computer/tags/viz/
- 2024-03-18T17:05:48.650Z
+ 2024-03-18T17:07:58.949Z
\ No newline at end of file
diff --git a/static/contentIndex.json b/static/contentIndex.json
index 03d20b6..1c4a3cd 100644
--- a/static/contentIndex.json
+++ b/static/contentIndex.json
@@ -1 +1 @@
-{"articles/binglish":{"title":"Binglish: The New Normal","links":["articles/structured-streaming","articles/robloxaville","notes/digital-gardening-with-quartz","notes/the-quest-to-slay-the-goblin"],"tags":["economics","horticulture","articles"],"content":"Binglish\n\nbing·lish, noun (derogatory)\n\nThe default writing style of generative AI text models. (derogatory)\nShorthand for a median style of writing, devoid of personality. (derogatory)\nThe way that a monkey’s paw would grant Microsoft’s wish to finally give Bing a generic English term like lowercase-g “google” (very derogatory)\n\n\nIn writing a bit more, I’ve been back-and-forth on the extent to which I want to swear. The style of my writing is more-or-less the way that I speak out loud (which prompted a lot of “couldn’t you make this a bit more professional?” feedback on the Databricks post), and when I speak out loud, I swear. Obviously, the frequency depends - at work it’s sparse (but not nil), on a personal level it’s pretty frequent - but I do still pepper it in for effect in some places. The only times I can remember not swearing is when I was teaching math to K-12 students, and when writing posts here.\nThis feels somewhat… internally dissonant? personally disingenuous? If, when I speak I swear, and I tend to write like I speak, why wouldn’t I also swear in my writing? The answer in the past has been professionalism. It feels unbecoming of a (relatively) young professional to swear in a professional context, and some of what I write here overlaps with what I do for work.\nToning down my writing has been, so far, a fairly active effort. I’ve occasionally even gone as far as writing with swears and then, after-the-fact, explicitly (ha!) searching for illicit words and removing them as I see them. This was probably most apparent when writing the Robloxaville recap article - it mentions a lot of my (very) early work history and, at the time, I thought that potential employers (or youths) might come across it, warranting a touch-up.\nThe world has changed quite a bit since ye olde 2021. I am (slightly) more established in my career, and the context of written documentation has changed quite a bit as technology has advanced. This has made me reconsider my previous policy of toning down my writing. In the process of porting over & more frequently updating this blog, I’ve made an executive decision for the foreseeable future: I’m going to be swearing a bit more.\nOr, at the very least, I’m going to be taking a dramatically less formal approach to writing and editing, here. The spur of this decision:\nThe Bots Are Coming Have Arrived\nI don’t think that, ten years ago, anybody expected that the next dramatic economic shift would’ve been generative AI - specifically text-based AI - putting white-collar workers at the highest risk of automation. An example that comes readily to mind is the CGPGrey video Humans Need Not Apply, in which self-driving cars were offered up as the most readily-apparent example of economic risk of automation. This feels slightly silly with the benefit of hindsight - having started my engineering studies at around the same time though, it’s easy to forget how high-profile advances in self-driving were (at the time). It has become wildly, obviously apparent that the current frontier of automation has come, first, in the form of automated writing.\nThis is obviously a concern for many reasons. The internet is made up of a whole lot of writing. It’s how we text, how we tweet, how we email. It’s what we read, how we get our news, how we ask questions and how we find our answers. Before now, we could safely assume that something we read on the internet came from, at the very least, another human being.\nThat’s not the case anymore, and it will likely not be the case ever again. I don’t think this is the end of the world (probably. hopefully). I do think that this is the end of what I refer to as the statistical median of language: boring, sanitized, business-speak.\nFor the purposes of this note, I am going to refer to this type of language as Binglish.\nBinglish is a Dead-On-Arrival Language\nBinglish, to me, can be described in a few ways:\n\nIt’s Business English. Office English. LinkedIn™ English. I hope, to those of you reading who have also worked a traditional office job, that you can recognize this shorthand as the type of thing that you’d find in your Outlook inbox;\nIt’s Boring English. It’s the median of the language, absent of all personality or human voice; and\nIt’s Bing English. Meant to half-heartedly celebrate Microsoft’s role in solidifying what is very likely a self-consuming business model , poised to consume the very medium it was trained on, this is the type of English that we (currently) see from Bing Copilot, ChatGPT, Gemini, and others\n\nAbsent explicit directives on tone, you can find examples of Binglish on pretty much every single chatbot you could open at the moment. It’s informative. It’s to the point. Once you get the hang of it, it’s easy to both read and write. What’s not to love?\nAnd - unfortunately for the mediums that rely on Binglish - it’s wildly automatable, now.\nIn an effort to formalize the English language into a common median, we have all but guaranteed that the data used to train LLMs has, by definition and in practice, mastered it and put all of us that use it at risk of automation. We can also see the remnants of human efforts to curtail widely-used bots from going outside Binglish at their default settings:\n\nChatGPT refuses to generate even a generic list of English swears…\n\n…as does Raycast’s AI feature, which uses either GPT-Turbo-3.5 or GPT-4…\n\n…as well as Github Copilot, which makes a bit more sense…\n\n…as well as Bing Copilot, even with creative settings, which makes less sense.\nThe takeaway, to me, seems as clear as day.\nHumans Need Not Write Binglish\nThe market for Binglish is cornered. It’s been automated away. The need for writing in the median, devoid of voice or personality, has gone the way of horses in a world of cars. The onus for humans, then, is not to stop writing, but instead to write outside of the Binglish median.\nOne of my favorite examples of people who write with an inimitable voice is Kyla Scanlon, who frequently writes and records thoughts about macro-economics. What makes her writing engaging is not just thoughtful analysis of economic trends, but the voice in which she writes it. It was specifically her post on language and trust that spurred this note to turn into the length of an article.\nWhile I feel like the market for dry, median writing was already withered in the wake of a rising palette preference for infotainment, it was the the advent of ChatGPT and widely-available generative text that has thoroughly crushed the future prospects of Binglish as a human activity. While I can make no guarantees that generative AI won’t someday master imitation of human voice in writing - something that has already reared its head in generative images and spurred active efforts to curtail imitation and preserve unique styles), I have made a decision for my personal writing:\nI don’t want to write Binglish anymore.\nI find no joy in editing the way I write into a now-dead, median form of language, optimized for algorithms with its supply automated away by language models. I’d rather write in my own voice, with my own anachronisms and mistakes and weird vocal ticks and stray punctuation and weirdly-placed emoticons and deeply-nested parentheticals and… run-on sentences? Probably quite a few run-on sentences.\nAs far as I’m concerned, my responsibility is to ensure that the text is readable and that the points I want to make come across, even if it the occasional sentence requires a re-read.\nAnd if it’s not search-engine optimized, or tonally non-neutral, or long-winded? if it contains foul language?\n¯\\(ツ)/¯ fuck it."},"articles/hello-blog":{"title":"Hello, Blog!","links":[],"tags":["data","engineering/python","articles"],"content":"Look at that - a whole blog, just for me! At this rate, the last thing I need to do to be a real tech guru is start a podcast.\nI’m starting this page as a place to talk about my cool projects, dumb thoughts, and other odds-and-ends aspects about my life and work.\nTo set the tone, here’s a quick code function to find the geomethic meandian of a set of numbers in Python - in case the need ever arises.\n\nimport numpy as np\n \ndef geothmetic_meandian(\n numbers: np.array,\n error: float = 10e-5\n):\n '''Find the geothmetic meandian of a set of numbers.'''\n \n # If all the numbers are close to the same, return the first number\n if sum(np.abs(numbers - numbers[0])) < error:\n return numbers[0]\n else:\n # Recursively call the geomethic meandian until the numbers converge\n return geothmetic_meandian(np.array([\n np.average(numbers),\n np.product(np.power(numbers, 1 / len(numbers))),\n np.median(numbers)\n ]), error)\n>>> geothmetic_meandian([1, 1, 2, 3, 5])\n2.089\n\nVoila!"},"articles/littlefield":{"title":"Zen, and the Art of Laziness","links":["notes/season-of-rhythm","notes/qamo","notes/stability-of-marriage","assets/clutter.png"],"tags":["data","games","engineering/python","viz","economics/market-design","articles"],"content":"I am awful at sticking to routines - the only thing I can bear to check on every day is Twitter. Even if it means spending twice as long over the course of a weekend, I will always try to find a way to automate away routine manual work.\n(All of the code used in this post can be found here.)\nToo much time to kill\nThis Monday, I started the last major semester of my undergraduate career. That’s right, folks - after six years, four program changes, and one brain surgery (which, in my personal opinion, is one too many), I will finally be wrapping up my degree!\nMy last two semesters will be my first part-time courseloads of my academic career, which leaves me quite a bit of extra time. Given that there’s not much to do during the quarantime, I took a bit of extra time to go over the top in a class I’m taking on operations and supply chain management.\nIntroducing: Littlefield\nAs part of the course, we’ll be spending four weeks in the operations simulation Littlefield. The premise of Littlefield is to tune parameters - things like orders taken and machines purchased - to optimize the profit of a simulated production line. The simulation itself runs a bit faster than real time. Over the course of five full calendar days, one session of the game simulates 250 days of work.\nIn the words of the course instructors:\n\nthe game is set up to give you the flexibility of checking it at your convenience (at any time of day or night)\n\nThe problem? I’m a walking dumpster fire when it comes to routinely checking on anything.\nMy email inbox is constantly full. I am always running on a quarter tank of gas. I’ve never even kept a house plant alive for more than a week - I’m responsible for the early death of more succulents than I would ever care to admit.\nIn the honest-to-God words that my fourth grade teacher wrote on my report card, I am somebody who is “lazy to the point of efficiency.” If there is any way for me to put in all of the work up front, I will find a way.\nWeb Scraping 101\nAs somebody who has been lazy for 23 years now, I’ve become well-acquainted with the concept of web scraping. This is a method for accessing websites using a program instead of your browser. If your job ever involves going to a massive number of web pages and finding specific information - as a few of my internships have entailed from time to time - it’s a fantastic way to avoid the grunt work of going to each individual page by hand. For example, rather than employing an army of people to go through every web page on the internet and keep track of keywords, Google instead uses an army of web scrapers to do all that work for them.\nLittlefield, like the rest of everything on the internet, is scrape-able as well. The game is hosted on a web page that looks like this:\n\nWhile the page is different - and a bit harder to scrape - than your average news article or social media page, it’s still just a set of links that we can get information from. Each node of the graph above leads to a page full of data about the game. If were playing this by hand, we would log in a few times each day, open up each page, and see if there were any issues or bottlenecks in need of fixing.\nIt takes about five minutes to check everything - five exhausting minutes.\nInstead of taking five minutes to check this data myself, I have - as the rational person I am - spent the last six hours creating a program that can check it for me: the Littlefield Driver.\nThe Littlefield Driver\nThe Littlefield Driver is a program that takes all of the data about the state of the game - things like machine utilization, cash on hand, and orders placed - and compiles it into one data table I can look at, graph, and use to diagnose any issues that may be the case. The program is based on a 2017 project by Greg Lewis, who was (at the time) an MBA student at the University of Maryland. Lewis wasn’t alone - developer Jiayu Yi created an export kit for Littlefield back in 2017 as well.\nIdeally, my Littlefield Driver will have a bit of extra functionality beyond data export, but the spirit of automation is still the same. By the end of the project, the goal will be to let the Littlefield Driver use incoming data to automate machine production in the background, and maybe even send me a notification if (or when) something goes horribly wrong.\nThe first order of business for the bot is to take the structure of the game and represent it as a network graph.\nHere’s what the driver sees when it opens up the game:\n>>> lfdriver.draw_graph()\n\n\nIt’s… admittedly not very pretty. At the very least, though, it shows that the program is seeing what we’re seeing: the production line starts at the Orders & Material menus, which goes to the first queue, which feeds into the first station, et cetera.\nHTML is the least romantic language\nWhen a web page is sent to your computer for your viewing pleasure, it’s sent as Hypertext Markup Language, or HTML for short. Normally, we don’t have to deal with HTML directly. The entire point of your web browser is to read the HTML and turn it into something clean, pretty, and - most importantly - human-readable. Scraping the web, however, means that we do need to read the HTML directly in order to pick out the parts we want the computer to look for.\nOur target for the first round of web scraping is going to be the Littlefield data pages. These data pages give you the historical plot of whatever the selected node of the graph represents. For example, this is the data page for how many incoming job requests are received every in-game day:\n\n(as an aside, every time you open a data view, it annoyingly opens as an entirely new window, which means playing Littlefield often looks like this)\nI can always appreciate a clean-looking graph.\nTo make that graph, it’s actually your web browser doing all the work to turn the HTML into a neat visualization. This is what your computer is actually sent:\n\nEven when you’re making a website, reading through raw HTML is rarely ever a fun time. Thankfully, the data we’re looking for is on line 29, smack-dab in the middle of the page:\n{label: 'data', points: '1 1 2 1 3 2 4 ...}\n\nThat’s the data that your browser is using to create the plot from the first image. It’s a bit weird that it’s all one line, but we can pick up from context that the data is being provided in pairs - the first number is the day, then the next is the datapoint - repeated over and over again. You could read it as:\nDay 1: 1 job\nDay 2: 1 job\nDay 3: 2 jobs\nDay 4: ...\n\nThankfully, the data is delivered in the same way for every single data page, from the orders received to the utilization of each station. All we have to do is write the code once, give the program all the data pages, and let it run. Instead of one string of data, we can tell it to put the data in nice, neat human-readable columns like this:\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nDayJOBIN112132425363\nAfter scraping every single page of data in the game, we can smash it all together and get a beautiful chart!\n>>> lfdriver.data()\n\n\nThis chart just shows the first and last 10 days of the historical data, and even from this we can gain some insights. S3UTIL, which represents the current utilization of the third station, is at 67% on Day 50 of the simulation. Judging by the rate of increase - roughly 1.3% every day on average - we should probably consider buying another machine at Station 3 prior to Day 75.\nOf course, that’s just an estimate. The purpose of the Driver is to check in on the game routinely to see if there are any critical issues. For example, the Driver can check to see if a station utilization is at a high percent and calculate whether or not it would be profitable to order a new machine.\nEverything is done - for now\nAt the time of writing this, I haven’t actually played a game of Littlefield - everything written so far was done on a static sample game. This means I haven’t had access to some features - like purchasing machines - so they haven’t been added to the Littlefield Driver… yet. Additionally, there’s some system variables - such as the cost of a new machine - that are available in the data but aren’t available in the Driver.\nUltimately, this is a god-awful way to play the game. With 12 years in development and only 2 examples (that I’ve found) of people attempting to automate the game, the game is clearly playable by hand. You might ask, then - why put in all this effort?\nThe real question is: why not?"},"articles/resume-ci-pipeline":{"title":"Job Applications are a Waking Nightmare - Let Tech Do It For You","links":[],"tags":["mentoring/tutorials","engineering/devops","articles"],"content":"\nNOTE: THIS IS OUTDATED. There are now far better PDF markup languages to use, such as Typst. These will be detailed in a later blog post.\n\nAfter trying - and failing - to constantly keep my resume up-to-date in a dozen different places, I took a page out of the wild world of software development and found the solution: continuous integration.\nIt used to be the case that software would release in massive updates. A bunch of big changes, tweaks, and bug fixes would be shoved together in a tidy package and delivered out all at once. As development became faster, though, development teams created continuous integration pipelines to constantly push changes out to their users. Rather than waiting until the next big change to release patches, small changes could be pushed out automatically.\nThis is common in web applications like Facebook or Twitter. You’ll never see Twitter come out with some big new Twitter 2.0 (Two-tter) - any changes they need to make are just pushed out to users in small batches, over time. Why not do the same with a resume?\nCreating an Always-Updated Resume\nCreating a continuously-updating resume can be done in two steps:\n\nSet up your resume so that you can make small, quick, iterative changes over time; and\nFind a central place to send those changes to, and send viewers to that location.\n\nI’m sure there are quite a few ways to run those two stages. I personally chose Overleaf and GitHub. At this point, I had already been using Overleaf to format my resume and had been using GitHub for small odds-and-ends projects for the first couple years of my undergrad. If we’re being honest, I didn’t want to learn new things - I just wanted it to work.\nSometimes being lazy pays off. Not usually, but sometimes.\nIt also helps that Overleaf has a neat built-in GitHub integration. How nice of them to do that! With this integration, the steps for the project start to come together:\n\nCreate a resume on Overleaf\nPush your resume to GitHub\nCompile your resume into a .pdf file\n\nAlright - let’s do it!\nCreating a Resume in Overleaf\nFor the uninitiated, Overleaf is an application commonly used in academia to write academic papers in a programming language called LaTeX. If you’ve ever used HTML to create a website, LaTeX does the same thing for PDFs. Thankfully, to get started, you don’t have to create a new resume from scratch. Overleaf has a ton of sleek-looking resume templates that you can put your current information into. If you’re looking for a friendly resume-making guide, I made a starter template with explanations on a few best resume practices for students.\nOnce you have a template picked, you can select “Open as Template” to add it to your personal Overleaf projects. You can start putting your information in at your own pace, but it isn’t necessary to start connecting your resume to GitHub.\nLinking Overleaf and GitHub\nIf you don’t have a GitHub account, now is the time to make a new one! GitHub is a fantastic place to store and update code - it’s literally what it was made for! Because your new resume (and the LaTeX it’s made from) are very small, it’s an ideal place to host your resume.\nOn your Overleaf resume, you can find the GitHub integration in the menu on the top-left of the code editing window. After signing into your GitHub account, you’ll have the option to create a new code repository for your project. Choose a name for your repository, enter a description if you’d like, and then select the option to make your repository public.\nWhile you can name your code repository anything, I would recommend sending this to your user’s GitHub Pages repository. Every GitHub user gets a special place where they can host a website or files publicly at <YOUR USERNAME>.github.io. If that type of address looks familiar, it’s because I host this blog on my own GitHub Pages site!\nIn order to use Github Pages, you’ll need to name your new repository <YOUR USERNAME>.github.io. For example, because my GitHub username is spelkington, my GitHub Pages repository is named spelkington.github.io.\nFinally, with your repository made, click the button to push Overleaf changes to GitHub. Now it’s time to make your resume update automatically!\nSetting up Continuous Integration\nGitHub has a nifty feature called Github Actions that development teams can use to automate some of the pain-in-the-ass aspects of programming. Typically this involves automatically testing software for glitches or compiling code, but for our case we’re going to use it to automatically turn our resume code into a .pdf file.\nThankfully, we don’t have to come up with a way to do this ourselves! Xu Cheng has created a nifty GitHub Action that will compile LaTeX files into PDFs.\nGo to your new repository on GitHub and click on the Actions button at the top. From there, you can create a new action.\nCreating a new action should take you to a code editor where you can create an Action configuration from scratch. Remove everything and place this code below:\nname: Build LaTeX document\n \n# Run this action every time the resume .tex is updated\non:\n push:\n paths:\n - '<RESUME FILE NAME>.tex'\n \njobs:\n build_latex:\n runs-on: ubuntu-latest\n steps:\n \n # Check out the repository on the Action machine\n - name: Set up Git repository\n uses: actions/checkout@v2\n \n # Compile the LaTeX document into a .pdf\n - name: Compile LaTeX document\n uses: xu-cheng/latex-action@v2\n with:\n root_file: resume/<RESUME FILE NAME>.tex\n \n # Commit this change as an automated user\n - name: Commit Change\n uses: EndBug/add-and-commit@v7.0.0\n with:\n author_name: <YOUR NAME> (auto)\n author_email: <YOUR EMAIL>\n message: 'TeX compile'\n \n # Push the change to the repository\n - name: Upload PDF\n uses: ad-m/github-push-action@master\n with:\n github_token: $\\{{ secrets.GITHUB_TOKEN }}\nReplace everything in the < > brackets, press the “Start Commit” button at the top right, and you’re all set! From now on, whenever you make changes to your resume in Overleaf, you can navigate back to the GitHub integration menu, push your change, and it’ll automatically send the code to GitHub and turn it into a .PDF!\nSending Your Resume\nOf course, the point of all this is be able to have a single link to your resume that will stay up-to-date forever. So how do we get this link?\nIf you followed the instructions for setting up the resume on your GitHub pages repository, it’s easy! Your resume will be available at <YOUR GITHUB USERNAME>.github.io/<YOUR RESUME FILE>.pdf. For example, mine is available at spelkington.github.io/Elkington_Resume.pdf."},"articles/robloxaville":{"title":"The Unbearable Weight of ROBLOX Celebrity","links":["notes/robloxaville-remaster"],"tags":["games/roblox","engineering/lua","articles"],"content":"Guilty confession: I’ve had a ROBLOX account for 14 years. At 23 years old, that’s more than half my life. After picking up development again in 2021, I’ve had quite the walk through memory lane.\nBeing a dork is a lifetime commitment\nFlash back to 2007: Spencer Elkington, 4th grade extraordinaire. Oh, to be at my peak again. While I appreciate that secondary schools mandate that all kids have a well-rounded curriculum to introduce them to a wide span of different fields of study, I can safely say that one of my least favorite classes was art. While I didn’t not enjoy spending an hour cutting, painting, gluing, and molding, there are only so many burnt ceramics and skewed portraits I could churn out before they - and my parents to whom that “art” was gifted - realized that perhaps a budding patronage in the fine arts wasn’t going to be in the cards for me.\nRelated point: Being a dork might be genetic\nAround that same time, I made an account on ROBLOX, an online game platform that was just a year old in 2007. Founded originally as a physics engine by mechanical engineers, it transitioned into a gaming platform after (I can only assume) the founders decided simulating viable buildings was far less fun than blowing them up. One of the coolest features was ROBLOX Studio, an IDE for developing user-made games and content in Lua. If you could play a game, you could make a game, and that ease of entry introduced a whole generation - myself included - to the wild world of programming.\nCooking up small games for me and my friends is probably the closest I’ll ever get being an artist. I could never paint a canvas worth a damn, but in an increasingly digital world being able to write a bit of code lets you create as much as a paintbrush ever could. Always looking for ways to hang out with her son, my mom - who is and always will be the biggest, coolest dork I have ever met and will ever will meet - joined the platform in 2008 to make games too.\nWith a background in coding from her physics, geology, and chemistry studies at BYU during her undergraduate (because just studying one STEM field would be too easy, obviously), she was an artist of her own right when it came to making games her kids could play. Over time, one of those games developed into a project called Robloxaville that she actively developed on from 2008 to 2012. The game became one of the most played games on ROBLOX during that period, later superceded by a follow-on project called Lakeside that became one of the fastest-growing projects made on the platform within two months of its’ initial release.\n\nOriginal Robloxaville thumbnail, 2008\nThis success was in no small part due to a feature she developed in 2009 called Morph Magic, a wildly clever, wildly hack-y workaround she discovered to modify the underlying skeleton of the ROBLOX player model and transform players into fully-animated dogs, cats, giants, or any of a dozen other models. The ad-hoc rigging method originally used in Morph Magic was the only method for modifying the player skeleton until ROBLOX added official rigging support - six years later. Watching millions of people enjoyed my mom’s masterpiece is what made me get into coding - first as a play tester, then as a (very amateur) contributor, and finally remastering the project in 2017.\nLike I said: the biggest, coolest dork I have ever met and will ever meet.\nThe wild ways software can go horribly, horribly wrong\nIn 2013, my mom stopped developing on the platform. It’s not that she didn’t enjoy it - as a general rule, nobody in their right mind writes clever workaround libraries in their free time if they don’t enjoy it. What stopped her from developing on the platform was years of rampant client-side exploits made possible by some questionable decisions made in the development of the ROBLOX game engine. For the uninitiated, here’s what went wrong:\nWhenever you use a device connected to the internet - your phone, laptop, tablet, desktop, etc. - it is communicating with tons of other computers called servers that belong to whatever service you happen to be using. When you’re browsing the web, it’s communicating with web servers. When you’re sending an email, it’s communicating with email servers. And when you’re playing ROBLOX, it’s communicating with ROBLOX servers.\nThese servers are responsible for running all the code that makes the game work. It diligently reads the Lua code that makes up a ROBLOX game, line by line, and changes the state of the game to whatever its’ told. If you tell the server to game.Workspace.Player.Head:Remove(), it will gleefully read that line of code and promptly remove the head of some unsuspecting victim.\nThat being said, sometimes the server needs a bit of help. Specifically, the server will occasionally rely on the game clients - your computer, and the computers of every other player in the game - to help it run some of the physics calculations. The millions of physics calculations run by the game can be a real pain in the ass for one computer, so the game server needs all the help it can get. Because of that, the client will occasionally send some information to the server to help the server know what the state of the game should be. Innocent enough, right?\nIt wasn’t always the case that the server relied on the clients for just physics calculations. Back in the early days of the ROBLOX engine, the client could send the server any information about the state of the game and the server would happily accept it as a gospel truth, even if it wasn’t in the game code. Here’s how the conversations between the server and client used to go:\n\nCLIENT: Hey Server! I need you to remove this player’s head for me.\nSERVER: Are you sure? I don’t remember seeing game.Workspace.Player.Head:Remove() anywhere in the code.\nCLIENT: Yep! Definitely supposed to not have a head. That’s how it looks on my computer.\n\nAnd then the server would, once again, gleefully remove the head of some unlucky soul, despite that fact that it was a rogue player’s client who gave the order, not the programmer.\nThese exploits, at a basic level, allowed users to wreak havoc on games: breaking game functions, harassing players, et cetera. However, the exploits were not always just to be annoying. The same exploit that allowed users to mess with the state of the server could also be used to download the source code for entire games, allowing users to rip and reupload entire years-long projects in minutes. At their worst, these exploits could get downright horrifying. \nBack in the Saddle\nIn 2014 - following what could only be described as a complete COPPA cluster - ROBLOX began making moves to clean up and polish the platform. This came in two major changes to the way development worked. \n1. Filtering Enabled\nTo fix the slightly problematic client exploit issue, the company introduced the Filtering Enabled feature to their game engine. Filtering Enabled locked down what types of information game servers would accept from the game client and fixed that little whoopsies mistake made in the younger days of the platform. However, development became a trickier after this feature was rolled out - many older games needed to be re-made to conform to the communication boundary between the server and client, and required a bit more formal knowledge of game development to make.\n(For reference, I didn’t learn anything about server and client relationships until my third year into a computer science degree.)\nEven though this feature could be disabled to allow legacy games to keep functioning (and yes, Robloxaville is legacy now - it even has the Fandom Wiki page to prove it), ROBLOX’s featured games algorithm wouldn’t touch your project with a ten-foot pole for risk of traumatizing even more unsuspecting children. This was especially important for the next major change,\n2. Monetization\nThat’s right, folks - every line of code my mom wrote from 2008 to 2013, she wrote for free. Making fun games for millions of kids across the world, pro bono? That’s just the way Tiffany rolls.\nWith the introduction of the Developer Exchange (DevEx), anybody (who could afford a premium membership) could take the in-game currency generated from their games and exchange it for cold, hard cash. Funnily enough, in that original blog post explaining how the DevEx program worked, the company describes a system where:\n\n[A] power-user [could cash] cash out and [buy] a car to get to and from college – all from developing a popular game on ROBLOX. This is the kind of story we can’t wait to tell.\n\nThat’s not quite the direction things went.\nThe Part Where Spencer Does Some Math\nSince DevEx was released, I’ve heard stories of full-time developers leaving their jobs to develop on the platform. Success could be lucrative: Welcome to Bloxburg, one of a number of games that filled in the Town And City genre gap after Robloxaville and Lakeside became deprecated by the Filtering Enabled update, has been played 4.8 billion times as of writing.\nBillion. With a “B”. 4,800,000,000.\nThat’s a lot of zeroes.\nThe game offers users a premium pass as an in-game purchase. This is a pretty common way to offer users a bit of extra bang for their buck by providing small bonus features - Robloxaville did the same thing at one point. How much do you think they could make off of a feature like that?\nUnfortunately for us, exact numbers are hard to find. ROBLOX stopped showing how many times in-game purchases were sold, no doubt to stop people from snooping around at how much developers make. That being said, we’ll make an assumption off of Robloxaville’s conversion rates: For every 10,000 player visits, 5 people will splurge on a premium pass at a purchase rate of 0.05%. That doesn’t look too impressive - yet.\nAt a 0.05% purchase rate against - and I cannot emphasize this enough - 4.8 billion visits, we could estimate that BLOXburg’s premium pass has been purchased 2.4 million times. Each game pass is 400 Robux (R$), ROBLOX’s in-game currency. If memory serves, ROBLOX keeps 30% of in-game player-to-player purchases (because, as we know, the only certainties are death and Robux taxation). We can assume, then, that the developers of BLOXburg received ~960 million R$ for that premium pass.\nThat golden parachute is starting to look real good.\nWhat might a 960 million R$ payout look like? The current DevEx exchange rate as of writing is 0.0035 USD for every 1 Robux (Robuck?) at the time of writing. This means that the estimated worth of that one premium pass for that one game is…\n3.6 million dollars.\nHm.\nHmmm.\nOf course, this does not come without some caveats. First: this is one of the most played games in the history of the platform - its’ success is by no means descriptive of every developer. Second: their estimated payout was made over the course of the six years the game has been in development. Third: the BLOXburg team is not just a single developer, and they are very good at what they do - enviously good, even.\nThat being said: those caveats also come with caveats. Although not every developer is going to be that successful, ROBLOX self-reportedly spends about 25% of their revenue on developer payouts as of a report from 2017. Since the company made its’ public debut on the New York Stock Exchange and started reporting revenue as a publicly traded company, they reported a revenue of $147 million dollars in 2020. Assuming they still pay out 25% to developers, that’s a big ol’ pot of 34 million dollars to go around.\nHow To Make Games and Influence Gen Z\nThis post has gotten way, way longer (and included way more math) than I intended, so I’ll wrap it up for now. Long story short: since 2007, ROBLOX has grown beyond description in both size and sophistication. The introduction of monetization professionalized the platform beyond what hobbyists from the early 2010s could ever pull off today. And, after (attempting) to redux Robloxaville back in 2016, I’m ready to take another stab at it.\nThe way I see it, games are to the computer what film was to the camera and paint was to the canvas. It’s a way to create an experience that people can look at, play with, and grow from. I really enjoy it. It’s a way for somebody who has no business in making art to get pretty damn close.\n(And it wouldn’t hurt to get some of that sweet DevEx stimmie, either.)"},"articles/structured-streaming":{"title":"Don't Double Down: Using Spark Structured Streaming to Wrangle Growing Data","links":[],"tags":["data","engineering/devops","engineering/python","mentoring/tutorials","viz/doodles","economics/finance","articles"],"content":"\nThis post was written during my time as a software engineer at M Science as a joint project with Databricks. You can view the original post here\n\nLet’s say that you, a ✨ humble data plumber ✨ of the Big Data era, have been tasked to create an analytics solution for an online retail dataset:\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nInvoiceNoStockCodeDescriptionQuantityInvoiceDateUnitPriceCustomerIDCountry53636585123AWHITE HANGING HEA…62012-01-10$2.5517850United Kingdom53636571053WHITE METAL LANTERN62012-01-10$3.3917850United Kingdom53636584406BCREAM CUPID HEART…82012-01-10$2.7517850United Kingdom……………………\nThe analysis you’ve been asked for is simple - an aggregation of the number of dollars, units sold, and unique users for each day, and across each stock code. With just a few lines of PySpark, we can transform our raw data into a usable aggregate:\nimport pyspark.sql.functions as F\n \ndf = spark.table("default.online_retail_data")\n \nagg_df = (\n df\n # Group data by month, item code and country\n .groupBy(\n "InvoiceDate",\n "StockCode",\n )\n # Return aggregate totals of dollars, units sold, and unique users\n .agg(\n F.sum("UnitPrice")\n .alias("Dollars"),\n F.sum("Quantity")\n .alias("Units"),\n F.countDistinct("CustomerID")\n .alias("Users"),\n )\n)\n \n(\n agg_df.write\n .format('delta')\n .mode('overwrite')\n .saveAsTable("analytics.online_retail_aggregations")\n)\nWith your new aggregated data, you can throw together a nice visualization to do… business things.\n\nThis works - right?\nAn ETL process like will work great for a static analysis, where you don’t expect the data to ever be updated - you assume the data you have now is going to be the only data you ever have. The problem with a static analysis?\nModern data doesn’t stop growing.\nWhat are you going to do when you get more data?\nThe naive answer would be to just run that same code every day, but then you’d be re-processing all of the data, every time you run the code. Each new update means re-processing data you’ve already processed before. When your data gets big enough, you’ll be doubling down on what you spend in time and compute costs.\n\nWith a static analysis, you spend money on re-processing data you’ve already processed before\nThere are very few modern data sources that aren’t going to be updated. If you want to keep your analytics growing with the source of your data, and save yourself a fortune on compute cost, you’ll need a better solution.\nWhat do we do when our data grows?\nIn the past few years, the term “Big Data” has become… lacking. As the sheer volume of data has grown and more of life has moved online, the era of Big Data has started to become the era of “God Help Us, It Just Won’t Stop Getting Bigger Data.” A good data source doesn’t stop growing while you work, and this growth can make keeping data products up-to-date a monumental task.\nAt M Science, our mission is to use alternative data - data outside of your typical quarterly report or stock trend data sources - to analyze, refine, and predict change in the market and economy.\nEvery day, our analysts and engineers face a challenge: alternative data grows really fast. I’d even go as far to say that, if our data ever stops growing, something in the economy has gone very, very wrong.\nAs our data grows, our analytics solutions need to handle that growth. Not only do we need to account for growth, we also need to account for data that may come in late or out-of-order. This is a vital part of our mission - every new batch of data could be the batch that signals a dramatic change in the economy.\nTo make scalable solutions to the analytics products that M Science analysts and clients depend on every day, we use Databricks Structured Streaming. Structured Streaming gives us the assurance that, as our data grows, our solutions will scale as well.\nUsing Spark Structured Streaming\nStructured Streaming comes into play when new batches of data are being introduced into your data sources. Structured Streaming leverages Delta Lake’s ability to track changes in your data to determine what data is part of an update and re-computes only the parts of your analysis that are affected by the new data.\nIt’s important to re-frame how you think about streaming data. For many people, “streaming” means real-time data - streaming a movie, checking Twitter, checking the weather, et cetera. If you’re an analyst, engineer, or scientist, any data that gets updated is a stream. The frequency of the update doesn’t matter. It could be seconds, hours, days, or even months - if the data gets updated, the data is a stream. If the data is a stream, then Structured Streaming is going to save you a lot of headaches.\n\nWith Structured Streaming, you can avoid the cost of re-processing previous data\nLet’s step back into our hypothetical - you have an aggregate analysis that you not only need to deliver today, but also keep updating as new data rolls in. This time, we have the DeliveryDate column to remind us of the futility of our previous single-shot analysis:\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nInvoiceNoStockCodeDescriptionQuantityInvoiceDateDeliveryDateUnitPriceCustomerIDCountry53636585123AWHITE HANGING HEA…62012-01-102012-01-172.5517850United Kingdom53636571053WHITE METAL LANTERN62012-01-102012-01-153.3917850United Kingdom53636584406BCREAM CUPID HEART…82012-01-102012-01-162.7517850United Kingdom………………………\nThankfully, the interface for Structured Streaming is incredibly similar to your original PySpark snippet. Here is your original static batch analysis code:\n# =================================\n# ===== OLD STATIC BATCH CODE =====\n# =================================\n \nimport pyspark.sql.functions as F\n \ndf = spark.table("default.online_retail_data")\n \nagg_df = (\n df\n \n # Group data by date & item code\n .groupBy(\n "InvoiceDate",\n "StockCode",\n )\n \n # Return aggregate totals of dollars, units sold, and unique users\n .agg(\n F.sum("UnitPrice")\n .alias("Dollars"),\n F.sum("Quantity")\n .alias("Units"),\n F.countDistinct("CustomerID")\n .alias("Users"),\n )\n)\n \n(\n agg_df.write\n .format('delta')\n .mode('overwrite')\n .saveAsTable("analytics.online_retail_aggregations")\n)\nWith just a few tweaks, we can adjust this to leverage Structured Streaming. To convert your previous code, you’ll:\n\nRead our input table as a stream instead of a static batch of data\nMake a directory in your file system where checkpoints will be stored\nSet a watermark to establish a boundary for how late data can arrive before it is ignored in the analysis\nModify some of your transformations to keep the saved checkpoint state from getting too large\nWrite your final analysis table as a stream that incrementally processes the input data\n\nWe’ll apply these tweaks, run through each change, and give you a few options for how to configure the behavior of your stream.\nHere is the ✨ stream-ified ✨ version of your old code:\n# =========================================\n# ===== NEW STRUCTURED STREAMING CODE =====\n# =========================================\n \n+ CHECKPOINT_DIRECTORY = "/delta/checkpoints/online_retail_analysis"\n+ dbutils.fs.mkdirs(CHECKPOINT_DIRECTORY)\n \n+ df = spark.readStream.table("default.online_retail_data")\n \nagg_df = (\n df\n+ # Watermark data with an InvoiceDate of -7 days\n+ .withWatermark("InvoiceDate", f"7 days")\n \n # Group data by date & item code\n .groupBy(\n "InvoiceDate",\n "StockCode",\n )\n \n # Return aggregate totals of dollars, units sold, and unique users\n .agg(\n F.sum("UnitPrice")\n .alias("Dollars"),\n F.sum("Quantity")\n .alias("Units"),\n+ F.approx_count_distinct("CustomerID", 0.05)\n .alias("Users"),\n )\n)\n \n(\n+ agg_df.writeStream\n .format("delta")\n+ .outputMode("update")\n+ .trigger(once = True)\n+ .option("checkpointLocation", CHECKPOINT_DIR)\n+ .toTable("analytics.online_retail_aggregations")\n)\nLet’s run through each of the tweaks we made to get Structured Streaming working:\n\nStream from a Delta Table\n\n+ df = spark.readStream.table("default.online_retail_data")\nOf all of Delta tables’ nifty features, this may be the niftiest: you can treat them like a stream. Because Delta keeps track of updates, you can use .readStream.table() to stream new updates each time you run the process.\nIt’s important to note that your input table must be a Delta table for this to work. It’s possible to stream other data formats with different methods, but .readStream.table() requires a Delta table\n\nDeclare a checkpoint location\n\n+ # Create checkpoint directory\n+ CHECKPOINT_DIRECTORY = "/delta/checkpoints/online_retail_analysis"\n+ dbutils.fs.mkdirs(CHECKPOINT_DIRECTORY)\nIn Structured Streaming-jargon, the aggregation in this analysis is a stateful transformation. Without getting too far in the weeds, Structured Streaming saves out the state of the aggregation as a checkpoint every time the analysis is updated.\nThis is what saves you a fortune in compute cost: instead of re-processing all the data from scratch every time, updates simply pick up where the last update left off.\n\nDefine a watermark\n\n+ # Watermark data with an InvoiceDate of -7 days\n+ .withWatermark("InvoiceDate", f"7 days")\nWhen you get new data, there’s a good chance that you may receive data out-of-order. Watermarking your data lets you define a cutoff for how far back aggregates can be updated. In a sense, it creates a boundary between “live” and “settled” data.\nTo illustrate: let’s say this data product contains data up to the 7th of the month. We’ve set our watermark to 7 days. This means aggregates from the 7th to the 1st are still “live”. New updates could change aggregates from the 1st to the 7th, but any new data that lagged behind more than 7 days won’t be included in the update - aggregates prior to the 1st are “settled”, and updates for that period are ignored.\n\nNew data that falls outside of the watermark is not incorporated into the analysis.\nIt’s important to note that the column you use to watermark must be either a Timestamp or a Window.\n\nUse Structured Streaming-compatible transformations\n\n+ F.approx_count_distinct("CustomerID", 0.05)\nIn order to keep your checkpoint states from ballooning, you may need to replace some of your transformations with more storage-efficient alternatives. For a column that may contain lots of unique individual values, the approx_count_distinct function will get you results within a defined relative standard deviation.\n\nCreate the output stream\n\n+ agg_df.writeStream\n .format("delta")\n+ .outputMode("update")\n+ .trigger(once = True)\n+ .option("checkpointLocation", CHECKPOINT_DIR)\n+ .toTable("analytics.online_retail_aggregations")\nThe final step is to output the analysis into a Delta table. With this comes a few options that determine how your stream will behave:\n\n.outputMode("update") configures the stream so that, each time the code runs, the aggregation will pick up where it left off instead of running from scratch. To re-do an aggregation from scratch, you can use "complete" - in effect, doing a traditional batch aggregate while still preserving the aggregation state for a future "update" run.\ntrigger(once = True) will trigger the query once when the line of output code is started, and then stop the query once all of the new data has been processed.\n"checkpointLocation" lets the program know where checkpoints should be stored.\n\nThese configuration options make the stream behave most closely like the original one-shot solution.\nThis all comes together to create a scalable solution to your growing data. If new data is added to your source, your analysis will take into account the new data without costing an arm and a leg.\nYou’d be hard pressed to find any context where data isn’t going to be updated at some point. It’s a soft agreement that data analysts, engineers, and scientists make when we work with modern data - it’s going to grow, and we have to find ways to handle that growth.\nWith Spark Structured Streaming, we can use the latest and greatest data to deliver the best products, without the headaches that come with scale."},"index":{"title":"Oh, Howdy!","links":["tags/engineering/","tags/economics/","tags/mentoring/","tags/articles"],"tags":[],"content":"Chaotic Good Computing is an organization specializing in providing advice and assistance for small companies around:\n\nSoftware Engineering\nData Analysis & Handling\nEconomic Analysis\n\nCGC’s general specialty is index, analysis, insights, and maintenance of digital economies like multiplayer game economies, network and cloud resource optimizations, or other digital spaces where you’d find a whole lot of chaos and complications.\nCGC also provides professional development resources and mentoring for young developers and professionals interested in economics and software engineering.\nThis site contains articles, notes, and thoughts on a broad range of topics. Chaotic Good Computing’s articles are great points to get started!"},"notes/annotations/college-admissions-and-the-stability-of-marriage":{"title":"Annotation: College Admissions and the Stability of Marriage","links":["notes/stability-of-marriage"],"tags":["economics/market-design","consulting","annotations"],"content":"\nth50%75%100%125%150%200%300%400 College Admissions and the Stability of Marriage Author(s): D. Gale and L. S. Sha\nshow annotation\nThis is a little test of something I’d like to do - keeping annotations of papers I’ve read on the CGC website, provided that they are open-access (which, unfortunately, is not always the case.)\nThis is being done with the Obsidian Annotator plugin. I have absolutely no idea how quartz is going to handle this, so… we’ll see! There’s a very real chance that this is going to end up being a substantially heavier lift on the frontend and design side than my feeble backend brain can handle. So, if you’re seeing this, feel free to pat me on the back for figuring it out (☞ﾟヮﾟ)☞\n\n\n\ncal Monthly.http://www.jstor.org COLLEGE ADMISSIONS AND THE STABILITY OF MARRIAGE D. GALE AND L. S. SHAPLEY, Brow*\nshow annotation\nWithout further ado! College Admissions and the Stability of Marriage was the second paper(ish) that I read related to market design and non-market environments that got me interested in the subject. The first, Dr. Kenneth Arrow’s Social Choice and Individual Values, was related more to voting systems - something that comes up every four years in America and then somewhat drifts into a hibernation until the next worthy national debate over the merits of first-past-the-post.\nThis one caught my attention, though, as it was the first technical analysis I’d seen of a topic that had some weight to it in my own life, and in the lives of others - partnership, romance, all the fun ones. If Arrow turned the handle on the door into this specific interest, it was Gale and Shapley that kicked it wide-open.\nIt was especially fun to present a summary of this paper in Dr. Allison Stashko’s Non-Market Environments course. I was proud to cap off the end of the term talking to my fellow early-20s undergraduates about something we could all understand at the deepest of levels: Being Horny™\n\n\n\nthe second will admit him later? Is it ethical to accept the first without informing the second and then withdraw his acceptance if the second later admits hi m? We contend that the difficult\nshow annotation\nIt’s interesting that exploding offers aren’t mentioned here - reading this for the first time in 3-4 years, I thought that it was mentioned at the top.\nIf it isn’t mentioned at all in this paper, then it was without a doubt Alvin Roth’s Who Gets What — And Why? that introduced the concept to me.\n\n\n\nng definition should not occur: DEFINITION. An assignment of applicants to colleges will be called unstable if there are two applicants a and ,B who are assigned to colleges A and B, respectively, although ,3 prefers A to B and A prefers : to a. Suppose the situation desc\nshow annotation\nIn the actual presentation, it was deeply difficult to re-word this in a way that didn’t sound like I was having another aneurysm. It reads somewhat well on paper, but sounds completely unhinged out loud.\n\n\n\nr let us look at some examples. Example 1. The following is the "ranking matrix" of three men, a, ,3, and 7, and three women, A, B, and C. A B C a 1,3 2,2 3,1 p 3,1\nshow annotation\nAnother difficulty was trying to reframe this specific style of diagram in such a way that a room full of people who’d been staring at game theory model grids for four years didn’t reflexively start eyeballing Nash equillibria\n\n\n\nignment of applicants. That is, THEOREM 2. Every applicant is at least as well off under the assignment given by the deferred acceptance procedure as he would be under any other stable assign- ment. Proof. Let us call a college “p\nshow annotation\nThe idea of match optimality was the concept that launched me out of this paper and into a rabbit hole of reading more about matchmaking algorithms, and actually modeling this in Python to toy around with the concept.\nIf, as they assert, this will give the assignees their optimal outcome compared to other stable match configurations, that implies the existence of other stable match configurations.\nTake a look at the assignment grid above — it’s a 2D grid representing one possible stable configuration. In that case, you could imagine that all possible configurations — stable or unstable — stacked on top of each other to form a 3D grid, or lattice, of match configurations. All of the possible 2D grids stacked like a big crystalline structure.\nFrom that lattice, you can slide out all of the unstable configurations (because, obviously, they suck and don’t mean anything), leaving us with a stable match lattice.\nThe even more interesting thing is - now that you have that stable match lattice, what exactly is in it? That’s where this particular part gets super zesty - “Every applicant is at least as well off […] as he would be under any other stable assignment”\nGale-Shapley’s stable match algorithm takes in two sets of people — in the case of this paper, men and women — which you can think of as the “assigner” and the “assignee”, the former of which makes the proposals and the latter who accepts or declines them. Think of it as a formula:\nmatches = gale_shapley(assigners, assignees)\n\nGale-Shapley is a noncommutative operation that optimizes the outcome for the assigners set.\nImagine you have a set of four men and four women, and you want to measure how “good” a matchmaking configuration is. You could imagine that the best-case scenario is that everybody ends up with their #1 choice, and the worst-case scenario is that everybody ends up with their #4 choice. We can use that as a baseline for measuring the “goodness” of the matchmaking.\nLet’s say, in some given matchmaking pair, everybody gets their first choice — the best case. If you were to add up everybody’s result (1st place) and average it by the number of people (8 people) you’d get a value of 1. On the flip side, for the worst-case, you’d get a value of 4 - ultimately being able to answer the question “On average, how well were people placed?”\nUltimately, if you take an analysis of the average placement of the assigner group (in this case, men), you’ll see that the assigner group’s average in Gale-Shapley is the highest of all possible stable configurations.\nUltimately, if you were to rank-order all possible stable matching sets on the matchmaking lattice on a spectrum of “Benefit to Group A” to “Benefits to Group B”, the outputs of gale_shapley(A, B) and gale_shapley(B, A) calculate the ends of the lattice.\nThis also implies a bit of a potential ethical issue. Depending on the context, there is an active decision made for “who gets better outcomes” by simply putting them into the algorithm first. Even within this paper, the marriage example itself generates the stable marriage output that provides men with the optimal output.\nUltimately, matchmaking is still something that interests me deeply, and I’m hoping that CGC (either the consulting side or development side) puts me in a position to keep working with these very interesting, very cool situations.\n\n"},"notes/annotations/simple-economics-of-open-source":{"title":"Annotation: The Simple Economics of Open Source","links":[],"tags":["consulting","economics/market-design","annotations","projects/cgc/economics-of-open-source"],"content":"\n00%400%NBER WORKING PAPER SERIES THE SIMPLE ECONOMICS OF OPEN SOURCE Josh LernerJean TiroleWorking Pa\nshow annotation\nI feel like I’ve been throwing out a lot of super speculative opinions on the economics of OSS lately - with a couple upcoming projects and opportunities, I want to make sure I understand some quantitative models for OSS contributions.\n\n\n\ner operating system market, International Data Corporation estimates that the open source program Linux has between seven to twenty-one million users worldwide, with a 200% annual growth rate. Many observers believe it repr\nshow annotation\nwell i’m sure this number is quite large 24 years later, huh?\n\n\n\nith a 200% annual growth rate. Many observers believe it represents a leading potential challenger to Microsoft Windows in this important market segment. • The significant capital in\nshow annotation\nI actually don’t know… what did people use for servers before Linux? Not to betray my youthful age but I can’t even imagine what was going on before Linux became a de-facto standard for headless server runs.\n\n\n\nrs helpless? [Stallman, 1999] The “utility function” Linux hackers is maximizing is not classically economic, but is the intangible of their own ego satisfaction and reputation among other hackers. [Parenthetical comment deleted] Voluntary cultures that work this way are actually not uncommon; one other in which I have long participated is science fiction fandom, which unlike hackerdom explicitly recognizes “egoboo” (the enhancement of one’s reputation among other fans) [Raymond, 1999b]. It is not initially clear\nshow annotation\n”egoboo” is a new term. While I do think (going into this on a first read-through) there are economic benefits related to labor contributions to OSS, i definitely don’t disagree that there is some level of utility in ego that keeps things moving.\n\n\n\ne provision of a public good? A ny explanation based on altruism1 only goes so far. While users in less deve\nshow annotation\nin general, “If everybody just _____” is a god-awful way to design markets.\n\n\n\nomics of open source software. Reflecting the early stage of the field’s development, we do not seek to develop new theoretical frameworks or to statistically analyze large samples . Rather, we focus on thr\nshow annotation\nDamn. I may have to find more recent papers, although I will finish reading this one through to the end.\n\n\n\nmost successful examples, the Unix operating system and the C language used for developing Unix applications, were originally developed at AT&T’s Bell Laboratories. The software was then instal\nshow annotation\noops. I thought UNIX was a standard (*NIX), not an OS in-and-of-itself.\n\n\n\nies was considerably enhanced. These cooperative software development projects were undertaken on a highly informal basis. Typically no effort to delineate property rights or to restrict reuse of the software were made. This informality proved to be problematic in the early 1980s, when AT&T began enforcing its (purported) intellectual property rights related to Unix. 2.2 The second era: early 1980s\nshow annotation\nNow this is ironic. given the inherently OSS nature of UNIX (with 24 years of hindsight) the fact that AT&T/Bell Labs tried to enforce a trademark on it is wild.\n\n\n\nperatively developed software. In exchange for being able to use and modify the GNU software (as it was known), users had to agree to make the source code freely available (or at a nominal cost). As part of the General Public License (GPL, also known as “copylefting”), the user had to also agree not to impose licensing restrictions on others. Furthermore, all enhancements\nshow annotation\nYahaha! The first case of a “cascading” license agreement, where software using GNU had to also, in turn, be GNU.\n\n\n\nng into various variants. In some cases, passionate disputes over product design have led to the splintering of open source projects into different variants. 8 Examples of such splinte\nshow annotation\nTwo examples off the top of my head:\n\n\nDolphin, the open-source Gamecube/Wii emulator, and its’ Ishiiruka fork. IIRC, there’s a bit of sass between the original developers regarding the Ishiiruka fork. Not sure what the context is.\n\n\nA recent (and spicy) one: the licensing changes around Terraform, spurring the creation of the OpenTofu fork.\n\n\n\n\n\nAnother challenge has been the apparently lesser emphasis on documentation and support, user interfaces,\nshow annotation\nsome things truly are evergreen.\n\n\n\ny by one open source developer: [I]n every release cycle Microsoft always listens to its most ignorant customers. This is the key to dumbing down each release cycle of software for further assaulting the non-personal computing population. Linux and OS/2 developers, on the other hand, tend to listen to their smartest customers... The good that Microsoft does in bringing computers to non-users is outdone by the curse that they bring on experienced users [Nadeau, 1999] . Certainly, the greatest d\nshow annotation\nI can get on board with this. Windows development is notoriously a pain in the ass.\n\n\n\nSource Definition and the concomitant decline of the GNU license\nshow annotation\na term worthy of a link.\n\n\n\n997 to 55% in September 1999.8 In 1999, the Apache Software Foundation was established to oversee the development and diffusion of the program . The current status of Apache,\nshow annotation\nThis is something else I’d like to look into — the structuring and continuity of OSS foundations.\n\n\n\nhow enjoyable the work is. The delayed reward covers two distinct, although hard-to-distinguish, incentives. The career concern incentive refers to future job offers, shares in commercial open source-based companies,9 or future access to the v\nshow annotation\nCertainly worthy of follow-up research.\n\n\n\ntrategic complementarities.” To have an “audience,” programmers will want to work on software projects that will attract a large number of other programmers. This suggests the possibil\nshow annotation\nInteresting to see the term “audience” made in the pre-influencer era.\n\n\n\nfads also have benefits. A fad can create a strong signaling incentive: researchers working in a popular area may be highly motivated to produce a high-quality work, since they can be confident that a large audience will examine their work. 4.2 Comparison between open so\nshow annotation\npossibly why i lowered the bar for the blog - i don’t have a clue if anybody actually reads it (☞ﾟヮﾟ)☞\n\n\n\nommon pool of students. 17 ii) Customization and bug-fixing benefîts: The cost of contributing to an open source project is lower if the activity brings about a private benefit (bug fixing, customization) for the programmer and her firm. Note again that this factor of cost reduction is directly linked to the openness of the source code. Let us now turn to the delayed\nshow annotation\nI hope this is elaborated on - I do think that this is the reason why companies will open-source previously private projects, as it opens up bug-fixes and improvements to a broader audience at the expense of (potentially) giving an edge to direct competitors.\n\n\n\nurce mode for three reasons: i) Better performance measurement: Outsiders can only observe inexactly the functionality and/or quality of individual elements of a typical commercially developed program, as they are unable to observe the proprietary source code. By way of contrast, in an open source project, the outsiders are able to see not only what the contribution of each individual was and whether that component “worked,” but also whether the task was hard, if the problem was addressed in a clever way, whether the code can be useful for other programming tasks in the future, and so forth . ii) Full initiative: The ope\nshow annotation\nOne of the things CS students are told constantly is “Contribute to OSS!” and this is the vast majority of the reason. If you don’t have the authority of having previously had an engineering position, OSS is a good way to show the results of your project.\nWhile I seriously doubt that any interviewer will look beyond an initial use-case for an OSS project you’ve contributed to, it does add a signal of “They did contribute!” to the project.\n\n\n\n’s interference, advice, etc. Economic theory would predict that the programmer's performance is more precisely measured in the former case. 18 iii) Greater fluidity: It\nshow annotation\nThis is crunchy! Because contributing to OSS is a self-decided move, it does make sense that the success is more indicative of single-agent performance rather than external factors.\n\n\n\ned in the former case. 18 iii) Greater fluidity: It may be argued that the labor market is more fluid in an open source environment. Programmers are likely to have less idiosyncratic, or firm-specific, human capital that limits shifting one’s efforts to a new program or work environment. (Since many elements of the source code are shared across open source projects, more of the knowledge they have accumulated can be transferred to the new environment) . These theoretical arguments a\nshow annotation\nYes!!! never underestimate the power of building transferrable skills.\n\n\n\nrong signaling incentives. By way of contrast, tasks aiming at helping the much-less-sophisticated end user—e.g., documentation, design of easy-to-use interfaces, technical support, and insuring backward compatibility—usually provide lower signaling incentives.14 4.3 Evidence on individual inc\nshow annotation\nHmmmm… I don’t know if I’d agree with this. Referencing the previous annotation, I think people tend to sleep on proper documentation and polish. While I don’t think this’d be the case for “industry-standard” OSS (curl being a good example), I do think that (for an individual contributing to OSS with less authority, adoption, or audience) documentation is key.\nIf an interviewer or company isn’t familiar with your work, they at least need to be able to see at-a-glance what the value of a project is. Even experienced engineers wouldn’t give a shit about a random GitHub repository with no documentation, even if it contains big-brain ideas or world-changing software.\n\n\n\ne nine key requirements in the “ Open Source Definition” [Open Source Initiative, 1999]. This point is also empha\nshow annotation\nFollow-up needed — I’m interested to see if this has evolved at all, or at the very least to read & review.\n\n\n\np, organization and governance. A successful open source project also requires a credible leader or leadership, and an organization consistent with the nature of the process. Although the leader is ofte\nshow annotation\nInteresting - let’s talk about some ethos!\n\n\n\nader must (a) provide a vision, (b) make sure that the overall project is divided into much smaller and well-defined tasks (“modules”) that individuals can tackle independently from other tasks, (c) attract other programmers,\nshow annotation\nMy brain is screaming “VSA!“. Parallelization is as important — if not more important — in planning software development as it is in software development itself.\n\n\n\ner” (prevent it from forking or being abandoned ). The initial leader must a\nshow annotation\noh, i hadn’t thought of this — part of the reason I try to bring collaborators in on CGC work is because having even a small team does wonders in keeping momentum.\nI’d liken it to the idea of variance in multiple distributions. Combining two normal distributions — in this case, representing motivation and velocity — ultimately lowers the variance of the distribution.\n\n\n\nto achieve full functionality. Another important determinant of project success appears to be the nature of its leadership. In some respects, the governance structures of open source projects are quite different. In a number of instances, such as Linux, there is an undisputed leader. While certain aspects are delegated to others, a strong centralization of authority characterizes these projects. In other cases, such as Apache, a committee will resolve the disputes by voting or a consensus process. At the same time, leaders\nshow annotation\nWhat?! They didn’t even mention Perl’s weird-ass “cycling leadership” that was mentioned at the top of the paper. I want to know more about that! It sounds whack as hell!\n\n\n\nted and all postings archived. For instance, on the Apache web site, it is explained how proposed changes to the program are reviewed by the program’s governing body, whose membership is largely based on contributions to the project. (Any significant change requires at least three “yes” votes—and no vetoes—by these key decision-makers.) 5. Commercial Software Compani\nshow annotation\nI wonder how this works now that Apache is no longer just “Apache,” but also an amalgamation of products (e.g Spark, Arrow (or whatever that underlying Database structure thing is called, etc.)\nI also wonder… how does this contribute to private use of the software? I know that the person who made Apache Spark now (IIRC) works as the CTO at Databricks, who benefits immensely from continued development on Spark. Are there instances of people in these positions potentially using this voting/veto power to further their own private interests at the cost of the broader community — and, if so, was trust undermined?\n\n\n\nl software companies (e.g., video game companies , Qualcomm for the Eudora e\nshow annotation\nHuh - this is something I’d never thought about. Video games are really weird, insofar as some of them do roll credits at the end of the game, not unlike movies or television shows.\n\n\n\ncommercial software firms. An internal Microsoft document on open source [Valloppillil, 1998] describes a number of pressures that limit the implementation of features of open source development within Microsoft. Most importantly, each so\nshow annotation\nOh shit - this should absolutely be followed up on.\n\n\n\nncentive to existing employees. Another area in which software companies might try to emulate open source development is the promotion of widespread code sharing within the company. This may enable them to reduce code duplication and to broaden a programmer's audience. Interestingly, existing orga\nshow annotation\nSee - I wonder if there are good, broad solutions to run software repositories for things like pip, apt, npm, etc, since that would be a fantastic thing to set up at companies for people to install things across the company.\nOne shortfall may be undermining the ease-of-use of just using the default, public software repositories.\n\n\n\ne benefits of the investments, the free-rider problem often discussed in the economics of innovation should apply here as well . Subsidies by commercial compa\nshow annotation\nfinally! the free-rider problem!! This is the one I think about most when OSS contributions come up.\nIf you can use a piece of software and don’t have to incur the cost of contribution, why contribute?\n\n\n\non should apply here as well. Subsidies by commercial companies for open source projects should remain limited unless the potential beneficiaries succeed in organizing a consortium (which will limit the free-riding problem). A second strategy is to take a\nshow annotation\n…I need to look into this more. What exactly does the consortium provide that de-limits the amount of commercial contributions to a project?\nSome level of coordination and more authority, perhaps?\n\n\n\nent of open source software. Companies can release existing proprietary code and create some governance structure for the resulting open source process. For example, Hewlett-Packa\nshow annotation\nAh, yes! K8s, React, Material, etc etc etc!\n\n\n\nckard’s RISC architecture. They can even (though probably less likely) encourage “ex nihilo” development of new pieces of open source software. This is similar to the s\nshow annotation\n”Ex Nihilo” in this case meaning “Something for nothing.” This is somewhat the flipped case of free-riding, where the owner of the OSS expects contributions from others.\nI’d imagine this use case would, in effect, undermine the authority of the company releasing proprietary software as OSS.\nHowever, the interesting thing - what if the company simply “transfers ownership” over to an OSS consortium? I believe I’ve seen this one happen before, but examples are eluding me.\n\n\n\nst visible of these efforts was Netscape’s 1998 decision to make “Mozilla,” a portion\nshow annotation\nI had no idea Mozilla came from Netscape in this situation - and it’s funny to see that Mozilla has worked out pretty damn well. Does Netscape still retain some ownership over the modern Mozilla organization?\n\n\n\ns had already been completed). It is also likely that Netscape did not adopt the right governance structure. Leadership by a commercia\nshow annotation\nIt’s so interesting to see this 2000 paper paint Mozilla as a failure. It’s worth a follow-up to see how Mozilla pulled out of what this paper paints as a nose-dive, death-spiral of irrelevance.\n\n\n\nhe open source community. In particular, a corporation may not be able to credibly commit to keeping all source code in the public domain and to adequately highlighting important contributions. 22 In this light, it is tempting\nshow annotation\nThis is interesting - IIRC Google has an “internal build” of K8s running, meaning that an inferior (Google-contribution-wise) version was the OSS version.\nI’d say the same thing about Databricks/Apache/Delta Lake, since I know that was the topic of some passion at one point.\n\n\n\ntion role for new firms. Collab.Net, a new venture funded by the venture capital group Benchmark Partners, will organize open source projects for corporations who wish to develop part of their software in this manner. Collab.net will receive fees for its online marketplace (SourceXchange, through which corporations will contact open source developers), for preparing contracts, for helping select and monitor developers, and for settling disputes. Hewlett Packard\nshow annotation\nCollab.net now redirects here, which is quite a pivot. That said, I feel I have seen SourceXchange out there - although I may be confusing it for StackExchange. It’s worth following up to see if there’s some relation, there.\n\n\n\nise, it is not without hazards. First, the open source project may be “hijacked” by a participant who builds a valuable module and then offers proprietary APIs to which application developers start writing. The innovator has then buil\nshow annotation\nAh, the curl-to-Postman pipeline.\n\n\n\nsuperimposes a new platform.25 Second, the coexistence of commercial activities may alter the programmers' incentives. To understand why it may be useful to make an analogy with academia (despite some differences between the academic research and open source development processes). To put our reflections in persp\nshow annotation\nSomething I have noticed is that people in OSS get real heated when the possibility of financial sponsorship comes up. I personally believe that it’s in the moral right to contribute financially to OSS — which is why I have contributions like that built into my budget — but I’ve seen people get real mad about that take.\n\n\n\nabout the latest developments. Second, the academic process may lose some of its integrity. The high-powered incentives\nshow annotation\nOkay — while I do get where they’re coming from, the fact alone that JSTOR was selling access to this paper for $40.00 when it was available elsewhere as a draft does make me giggle.\nTell me more about commercial interests polluting academic progress?\n\n\n\nemics to other communities. 32 While it is too early to tell, some of these same issues may appear in the open source world. Programmers working on an\nshow annotation\n”Too early to tell” from a paper written in 2000 is a great signal to follow-up.\n\n\n\nto be highly skewed, with a few individuals (or at most a few dozen) accounting for a disproportionate amount of the contributions, with most programmers making just one or two submissions. Many contributors to Send\nshow annotation\nThis feels like something GitHub would have data on. Worth a follow-up\n\n\n\nment itself grows and evolves. At the same time, it is heartening to us how much of open source activities can be understood within existing economic frameworks, despite the presence of claims to the contrary. The literature on “career\nshow annotation\nConsidering OSS has continued to grow, my gut instinct is that there are economically-incentivized reasons for contributions and that further research into… the further research, done in the last 24 years will yield some interesting new insights.\nI’m glad I started here - doing research on this chronologically will, I think, be a very interesting project.\n\n\n\ns will be less possible. But recent developments in computer science and programming languages (e.g., object-oriented programming) have encouraged further mod\nshow annotation\nteehee — “recent”\n\n\n\nof open source software. This is similar to the strategy of giving away the razor (the released code) to sell more razor blades (the related consulting services that HP will provide). Various efforts by corporations\nshow annotation\nAh, yes, the Terraform to Terraform Cloud pipeline.\n\n"},"notes/arc-review":{"title":"Arc Browser: Kicking off the Season of Rhythm","links":["notes/the-quest-to-slay-the-goblin"],"tags":["goblin-slaying","notes"],"content":"The Arc Browser is probably the first tool that slayed off a series of shake-ups on how I use technology to put up guard rails on being able to start - and stay - focused. It was also on their blog that I discovered Maggie Appleton’s blog - so in a way, it was very much a half-step into a pool of ideas that have helped me come to terms with the slightly-more-than-standardly chaotic way that I tend to work.\nAt some point, I’ll do a review here."},"notes/beethoven":{"title":"From the Archive: \"Reading the Room with Beethoven\"","links":[],"tags":["engineering/typescript","notes"],"content":"Beethoven was a project for HackTheU 2019, where a team and I (mostly the team) created an application for providing live captions to deaf students by using a remote microphone to stream audio and captioning to a user’s local device.\nFrom the original submission:\n\nInspiration\nSpencer, one of our team members, has a parent who is hearing-impaired and another that provides live, remote, ver batem transcription services for deaf and hard-of-hearing undergraduate, Ph.D., and Executive MBA students across the nation. Dan and Joseph, two of our other members, work for Sorenson Communications making video calling and other applications for the deaf community. We noticed that, for deaf students who would prefer live captioning in their lectures, there is no one-stop solution for both audio streaming and live captioning. We created Beethoven, an application for quick and accurate transcription in the classroom, to solve this.\nFunctionality\nPut simply, Beethoven will pair two devices together - an audio input and a screen output - and stream captioning from one device to the other in real time. The student can put one device at the front of the class for the clearest possible audio, and then position another next to them to read live captioning of the lecture and dialog as if it was the Closed Captioning on a television show or movie. This can either be done through Google Cloud transcription services or a live CART interpreter, depending on the level of accuracy desired and resources available.\nDevelopment Process\nThe application is split into three main parts - the audio input, the transcription service, and the screen output. First, the audio is captured by the audio input device. From there, it goes to either the Google Cloud voice transcriber, or a live CART interpreter. This happens through Google Cloud Services or peer-to-peer WebRTC, respectively. Once complete, the text will be sent to both our server’s backend to be compiled into a transcript and send to the screen output device, placed in front of the student, where a stream of the captioning is displayed.\nWhat’s next for Beethoven\nMoving forward, we’d like to fully implement the CART reporting features of the application. The P2P handshaking feature has a lot of improvements that can be made. Once finished, we’d like to reach out to the deaf and CART communities to get their feedback on the product.\n"},"notes/caveat-lector":{"title":"Caveat Lector: Reader Beware","links":["notes/digital-gardening-with-quartz","notes/qamo","tags/horticulture"],"tags":["horticulture"],"content":"The past few days have been a weird experience as I actively try to lower the internal bar for what I feel comfortable putting out into the world. As a proud member of the most terminally online generation to ever live, I have a deep and relentless fear of being wrong on the Internet.\nIn getting everything ready to re-publish this site as my own digital garden, I’ve started using the phrase “caveat lector” a lot. Back in my undergraduate degree, I learned about the phrase caveat emptor and caveat venditor:\n\nCaveat emptor means “buyer beware,” to be used in situations where a buyer of a product is responsible for understanding the risk they take in their purchase; and\nCaveat venditor means “seller beware”, to be used in situations where a seller of a product is responsible for minimizing the risk of the products they sell.\n\nA good napkin example would be a buyer and seller of digital LCD displays. In that case, it’s the seller’s responsibility to make sure that the display actually works - caveat venditor. However, what if a display works, but has a few dead pixels? This doesn’t render the display unfit for purpose - the seller would still be selling a good and usable product that fulfills their promise - but dead pixels are pretty annoying. The risk of dead pixels, then, may fall on the buyer - caveat emptor.\nOne of my intentions with Chaotic Good Computing is to contribute to the free body of knowledge I’ve benefitted immensely from myself, but to do so in a way that both fits my style and my habits better than traditional blogging or educational entertainment. The commitments that I’m making to you, the reader, are:\n\nTo my knowledge, the things that I’m posting are accurate at the time that I write them down; and\nIf I learn something in the future that corrects something I’ve written in the past, I’ll do my best to make corrections as I find mistakes. \n\nThat said, I’m also just some person on the internet. I know some things pretty well and other things not-so-much, with a pretty vast in-between. By the very nature of learning in public, some things here will be straight-up wrong, if not at least not-as-correct-as-they-could-be.\nSo, to you, I say:\n\nCaveat lector!\nReader Beware!\n\nand, as a follow-up piece of advice, I might also borrow from the Russian proverb:\n\nDoveryay, no Proveryay.\nTrust, but Verify.\n"},"notes/dailies/2024-02-25":{"title":"2024-02-25","links":["notes/dailies/2024-02-26","notes/caveat-lector","notes/digital-gardening-with-quartz","notes/over-under-engineering"],"tags":["horticulture","engineering/typescript","notes/daily"],"content":"2024-02-26 ⇒\n\nCAVEAT LECTOR: This is a daily note! Sometimes they’ll be structured, sometimes they won’t. These are more for the benefit of my near- and long-term future self, but I do tend to publish them in case they’re somehow helpful for other people.\n\nToday’s Plan\nI didn’t plan anything today, but I do want to start planning things in the future.\nToday’s Report\nI’m totally ripping off the labelling of this from Pikmin 2’s “Today’s Report” screen.\nThat’s not even a joke - I absolutely love the idea of ending the day with little graphs of what I did (or did not do) well that day in a tidy little reflective at-a-glance view.\nToday was (and will continue to be - my plan is to go home and keep going while I watch Avatar) a whole lot of writing. I’m still in the process of creating a new blog with Quartz, but I keep catching myself falling into a trap of “I want to make everything look perfect before I publish it!“. Despite spending the day jotting down my thoughts about over-engineering killing my dreams, I can’t seem to stop myself.\nThe stopping point for today will probably be a vague “do I feel like I’ve written enough to cover, at a base level, the anchor tags of what I want to write.” I still do want to do some weird D3.js fuckery to Quartz’s network graph where major “anchor” tags are locked to the outside of a circle (or some other shape) with extending graphs moving inward, but I am unfortunately blocked by not knowing how to debug with Quartz’s weird custom CLI build process.\nActually, it looks like the Quartz author just got back to me - QuartzComponents are built… \ntoo late? too early? I still don’t know enough about frontend development to fully understand how Quartz works under-the-hood or totally what that message means.\nTL;DR something about the Quartz build process means that I can’t use DevTools in a browser to debug Quartz components. Yesterday I was researching CodePens (or, more specifically, VSCode-inline CodeSwings) as a better workflow for my not-design-or-frontend-savvy self to develop these components in isolation. While I do still want to do some deep customizations to the way the Quartz Graph component works, I have decided (just now, as I’m writing) to leave this as a stretch goal, and instead prioritize getting this blog out as-is and customizing it to my liking later.\nMore work to be done in the endless struggle to avoid putting horses before carts, or whatever.\nNotes\n\nhttps://blog.partykit.io/posts/cursor-party\nhttp://www.bruno-latour.fr/sites/default/files/35-MIXING-H-ET-NH-GBpdf_0.pdf\nhttps://maggieappleton.com/gathering-structures\nhttps://maggieappleton.com/ambient-copresence\nhttps://github.blog/2023-10-05-a-developers-guide-to-open-source-llms-and-generative-ai/\nhttps://github.blog/2023-10-04-how-to-communicate-like-a-github-engineer-our-principles-practices-and-tools/\nhttps://www.rawgraphs.io/\n"},"notes/dailies/2024-02-26":{"title":"2024-02-26","links":["notes/dailies/2024-02-25","notes/dailies/2024-02-27","notes/caveat-lector","notes/the-quest-to-slay-the-goblin","notes/season-of-rhythm"],"tags":["goblin-slaying","horticulture","engineering/devops","engineering/csharp","notes/daily"],"content":"⇐ 2024-02-25 | 2024-02-27 ⇒\n\nCAVEAT LECTOR: This is a daily note! Sometimes they’ll be structured, sometimes they won’t. These are more for the benefit of my near- and long-term future self, but I do tend to publish them in case they’re somehow helpful for other people.\n\nToday’s Plan\nMost of today’s plan is going to be getting an internal NWYC project up-and-running with an MVP before we transition to some compliance updates to infrastructure. Follow-up on CGC will be taking inventory of the writing I did yesterday, and (once again) debating whether or not to get the formatting of the blog just right before I publish.\nB-Block\n\nCompleting C# wrappers for Geocoding services\nWorking on next ChainSharp steps for congressional data lookups\n\nM-Block\n\nLikely a continuation of B-block work\n\nE-Block\n\nCGC; in particular, either\n\nContinuing content writing, potentially starting the new posts for 1099 contractor writing; OR\nSetting up an environment & workflow for creating swings with CodeSwings\n\n\nStretch goal is contemplating whether or not MDX compatibility for Obsidian and Quartz will come now or later. I think Remark has the ability to do MDX files but I’ll need to spike that a bit more closely.\n\nToday’s Report\n\n\nNoon progress update: Got distracted at the possibilities of AWS CloudWatch Dashboards. Morning block is a bit behind, but getting back to it.\n\n\nAfternoon progress update: as a general rule, try to never fuck up the same way twice. that’s a good motivation for writing down as much as possible outside of code, so that the body of DX-related issues with undocumented fixes is always shrinking.\n🎵 come with me 🎵 //\n🎵 and you’ll see 🎵 //\n🎵 a world in which no developer fucks up the same way twice, and rather fucks up in new and creative ways each time 🌈 🎵\n\n\nIn sum total, today was… okay? ish? in terms of progress. Didn’t quite get as far as I’d hoped on C#, but some other (equally important) stuff came up and that seems okay. E-block certainly didn’t go as far as I’d hoped (read: it did not happen) but at least CGC stuff is at my own pace.\nAdmittedly, a minor distraction did pull away more of my attention than I ought to’ve allowed.\n\nAs a bit of a buzzer-beater against my self-imposed daily deadline of 12:00pm MT, I’ve decided to just publish this new blog format as-is. It doesn’t quite have the same style and zest as the original, but I’d rather put this out to encourage myself to improve it rather than let it languish in a “refactor” branch on GitHub.\nUPDATE: I did not beat the buzzer. It is currently 4:00AM the next day, and I am definitely not going to be deploying anything. Ironically, during this time I wrote the first posts about slaying The Goblin and my Year of Rhythm, which is ironic considering The Goblin certainly won tonight.\nNotes\n\nhttps://www.youtube.com/watch?v=0RFwyobtnKA\n"},"notes/dailies/2024-02-27":{"title":"2024-02-27","links":["notes/dailies/2024-02-26","notes/dailies/2024-02-28","notes/caveat-lector","articles/resume-ci-pipeline"],"tags":["engineering/csharp","engineering/python","notes/daily"],"content":"⇐ 2024-02-26 | 2024-02-28 ⇒\n\nCAVEAT LECTOR: This is a daily note! Sometimes they’ll be structured, sometimes they won’t. These are more for the benefit of my near- and long-term future self, but I do tend to publish them in case they’re somehow helpful for other people.\n\nToday’s Plan\nI didn’t get nearly as much C# done as I’d wanted to yesterday - while I’m a bit hesitant to say that “today is going to be a 100% effort day!”, considering how yesterday ended I’ve kept C# goals to a breezy middle-of-the-road, since I’d rather under- than over-set expectations.\nB-Block\n\nMore ChainSharp step testing.\n\nAs an aside, I think I’ve been dramatically over-complicating how I create separable PRs. This is one of my “Fuck, Theaux was right” moments that I increasingly have as I learn more C#. At some point, I’d like to do some gardening on philosophies for separable pull requests and commits.\n\n\n\nM-Block\n\nChecking on our internal project status - there’s likely tasks I can pick up, but my current work is the only very-large-web I have left to untangle (unless I’m forgetting something, which is likely.)\n\nE-Block\n\nToday’s E-block is a CGC day, so (sleep status pending) I’d like to work on actually taking what I have and uploading it to GitHub. This’ll mean taking a look at the Quartz build process.\n\nAnother aside - while I appreciate how much the developers of Quartz has made it user-friendly with a custom CLI, it abstracts away some of the usual development workflow and means I have to learn (and consequently fumble around with) a new thing. That said, I also don’t know jack shit about frontend development and there’s probably a good reason for that.\n\n\nOh, what’s this?? A friend of mine asked me to take a look over his resume. He’s still using the old LaTeX version of my resume template, so I’m real excited to get him moved over to the new Typst version (post pending). CGC blog stuff may get pushed back accordingly - not sure how high-capacity I’m feeling now at 5pm\n\nToday’s Report\nN/A\nNotes\n\nhttps://mypy-lang.org/\nhttps://github.com/apache/spark/blob/master/python/mypy.ini\nhttps://www.youtube.com/watch?v=xZYZEGHybqQ\nhttps://engineering.blackrock.com/static-type-checking-in-python-where-did-the-ducks-go-d17881d3205e - type hinting to static checks in mypy at a gradual pace\nhttps://github.com/kaiko-ai/typedspark Holy shit - it does exist!!\nhttps://typedspark.readthedocs.io/en/latest/index.html words cannot describe how excited i am??\n"},"notes/dailies/2024-02-28":{"title":"2024-02-28","links":["notes/dailies/2024-02-27","notes/dailies/2024-02-29","notes/caveat-lector","notes/dailies/2024-02-26","notes/putting-selfcontrol-on-raycast","notes/dailies/2024-02-25","tags/goblin-slaying","notes/raycast-review"],"tags":["goblin-slaying","engineering/csharp","notes/daily"],"content":"⇐ 2024-02-27 | 2024-02-29 ⇒\n\nCAVEAT LECTOR: This is a daily note! Sometimes they’ll be structured, sometimes they won’t. These are more for the benefit of my near- and long-term future self, but I do tend to publish them in case they’re somehow helpful for other people.\n\nToday’s Plan\nWhoops! Fell asleep at a reasonable hour yesterday, and didn’t get jack squat from my E-block done. That said, considering the day before, the sleep was very welcome. And - not to break my arm patting myself on the back - I did say that everything in that block was tentative.\nToday is pretty much the same thing as yesterday - ChainSharp does set things up in such a way that most work is “Today, I make another Step!” and that, plus not giving too many details for private projects, means that the B- and E-blocks for probably the rest of this week are not going to be too spicy and different.\nB-Block\n\nWorking on ChainSharp steps for private project\n\nM-Block\n\nWorking on ChainSharp step tests for private project\n\nE-Block\n\nThis… we’ll see. Today isn’t technically a CGC day - those tend to be Mondays, Tuesdays and Thursdays. However, yesterday was less than productive, so depending on my partner’s schedule (she may be going climbing) I’ll probably be staying in the office a bit later for CGC.\nIf CGC work happens today, current priority list (I should store these on an actual list app) is:\n\nResume reviews - 3 pending for two students and a friend\nQuartz deployment to GitHub Pages\nQuartz customization - likely a custom index page\n\n\n\nToday’s Report\n\n\n8PM Update: Hmmm… I think I’m going to pivot off of resumes. For an internal project, it’ll be in my best interest to get a bit of progress down on graph analysis & generation in TypeScript. I’ve already wasted (ironically) enough CGC time today on creating a SelfControl extension for RayCast. I think the reviews can wait until this weekend - likely Sunday, but I’ll try to get those done before then based on ✨ vibes 🌈.\nIdeally, I’d like to have a Jupyter-ish environment to prototype frontend - likely, this means picking up where I left off the other day with CodeSwings, and doing some horticulture on the TypeScript analog to Python’s Pandas/C#‘s LINQ, Data Forge.\n\n\n…\nOkay - so technically I did most of what I set out to do today. Left myself a bit of wiggle room on the E-block. Again. If you squint your eyes and tilt your head, foresight and unreliability do look quite similar.\nI have decided I’m very excited about the SelfControl on Raycast stuff. Of all the things that rattle around in my skull, the xkcd post about time saved vs time spent automating is the most rattle-y. After mulling it over (while i was doing it, which is the perfect time to second-guess yourself) I decided the juice was worth the squeeze if it means I can more reliably pull myself out of doomscrolls and death spirals.\nWhen it comes to the art of goblin-slaying, I do think that Raycast is the greatest set of steel-toed boots I own. It’s not the worst thing in the world that they now have a bit more weight in the heel with SelfControl tacked on.\n\nI will, however, also admit that - in terms of deliverables - the squeeze also squoze(?) out some time for other things I was hoping to do today. Hopefully trimming the doomscrolling fat makes up the time soon. Other than that slight detour, my CGC priorities remain generally the same and some of today’s list will likely get recycled into Friday. Tomorrow will be a lighter report day - CGC work is taking a backseat to watching the new(ish) Mean Girls with my partner, and then likely cleansing our palettes with the original Mean Girls afterwards. Taking a backseat? Classic Regina.\nIn a buzzer-beater, I learned a bit about D3 and finally settled on how to use CodeSwings in such a way that I can have them interact with Quartz internals. Maybe I am a frontend guy?? 😎\n(i am not.)\nNotes\n\nhttps://github.blog/2024-02-28-how-were-using-github-projects-to-standardize-our-workflows-and-stay-aligned/ - perhaps Projects is actually becoming… good? What a time to be alive!\nhttps://www.rawgraphs.io/ - Implementation as Backstage Plugin or Electron App?\nhttps://github.com/rawgraphs/rawgraphs-app - unfortunately not the type of “app” i was hoping for.\nhttps://github.com/jessejanderson/skylink - would be nice to do a popup as an Arc boost\nhttps://github.com/raycast/extensions/tree/c524392d80b35348811bae59831f29bfdbc34432/extensions/youtube-music/ - I’d like to, at some point, look at forking & adding the ability to add a song to a playlist. the selfcontrol stuff was quite a gateway drug, although i’ll have to adapt to using TypeScript for extensions rather than creating shitty, poorly-tested bash.\n"},"notes/dailies/2024-02-29":{"title":"2024-02-29","links":["notes/dailies/2024-02-28","notes/dailies/2024-03-01","notes/caveat-lector","notes/putting-selfcontrol-on-raycast"],"tags":["goblin-slaying","engineering/typescript","engineering/csharp","notes/daily"],"content":"⇐ 2024-02-28 | 2024-03-01 ⇒\n\nCAVEAT LECTOR: This is a daily note! Sometimes they’ll be structured, sometimes they won’t. These are more for the benefit of my near- and long-term future self, but I do tend to publish them in case they’re somehow helpful for other people.\n\nToday’s Plan\nOkay! With myself SelfControl’d this morning, feeling very optimistic today!! I think, today, I can finish up my part in the MVP of an internal project. As a treat, I’m going to take a closer look at tuning up our old Backstage deployment. I’ve been feeling a whole lot better about my TypeScript knowledge & workflows as of late - and my engineering in general - so taking another look at Backstage after ~8 months feels like it may be refreshing.\nB-Block\n\nFinalize MVP parts for internal project\n\nM-Block\n\nRe-clone and get Backstage local environment running\nLook into Backstage Plugins\n\nE-Block\n\nNone! It’s Mean Girls night, baybeeeeee 😎\n\nToday’s Report\n\nUPDATE 3PM: The SelfControl for Raycast integration was, in fact, worth the time spent. Today’s B&M blocks have been the most productive in probably 2-3 weeks 🎉\n\nSlight end-of-day, while I have the laptop open - we ended up stopping the new Mean Girls after about 10 minutes and just watching the old one instead.\nNotes\n\nhttps://kindavim.app/ - vim everywhere on macos. May automate some of the less-than-stellar shortcuts i’ve baked myself\nthere are few terms i despise more than “tribal knowledge.” at some point this will be written about.\nhttps://gist.github.com/timvisee/fcda9bbdff88d45cc9061606b4b923ca - Happy leap day!\nhttps://www.perkins.org/resource/how-write-alt-text-and-image-descriptions-visually-impaired/ - resource for writing alt text on images\n"},"notes/dailies/2024-03-01":{"title":"2024-03-01","links":["notes/dailies/2024-02-29","notes/dailies/2024-03-02","notes/caveat-lector","articles/binglish"],"tags":["engineering/csharp","notes/daily"],"content":"⇐ 2024-02-29 | 2024-03-02 ⇒\n\nCAVEAT LECTOR: This is a daily note! Sometimes they’ll be structured, sometimes they won’t. These are more for the benefit of my near- and long-term future self, but I do tend to publish them in case they’re somehow helpful for other people.\n\nToday’s Plan\nI think today is likely going to be lighter day, likely finishing up assigned work during the B-block - I have a couple last-minute touches to an internal MVP, but not sure to what extent my time is booked up for the rest of the day. May take M-block to do a bit of light research on Backstage plugins.\nB-Block\n\nQuick horticulture on (free) C# code coverage libraries\n\nM-Block\n\nBackstage horticulturie\n\nE-Block\n\nResume review - for real, this time!!\nActions/Quartz deployment\n\nToday’s Report\n\nUPDATE 3:00PM: Ended up writing an article about binglish, which is absolutely not what i had planned for the afternoon but i’m certainly not mad about it. from a quick cursory online search… i don’t think anybody else has coined this term yet?? At least not for the same concept - i was only able to find references to one chrome extension (ironic) and a somewhat(?) used term for british-indian-english (which, idk, maybe it’s far larger a term but not one that culturally/algorithmically would be at the tippity-top of my SLC-based search results, even on incognito) \nUPDATE 6:00PM: i was excited enough about that post to actually publish my blog. I was about to excitedly hyperlink to it, but… I suppose this will actually be on there now. There is something about seeing what i’ve been putting into a dusty refactor/ branch on GitHub actually be on The Internet™ that… I’m frankly not certain how to feel about.\nThere is something burning in me to really quickly learn everything more i need to know about TypeScript frontend development and finish the blog formatting posthaste, but I think that will need to wait for another time. I am hungry, and must depart.\n\nSee above - stopping CGC work and closing my laptop for the day. Perhaps I will play… Factorio???\n🫡\nNotes\n\nhttps://www.hanselman.com/blog/altcover-and-reportgenerator-give-amazing-code-coverage-on-net-core - Free code coverage for C#. Uncertain about Rider compat >:l\nhttps://youtu.be/_wxU3z9VxOY?si=Y2vuh_PH5_yHay9m&t=267 - suavemente\nhttps://kyla.substack.com/p/why-we-dont-trust-each-other-anymore - words, words, words (literally and topically)\nhttps://open.substack.com/pub/kyla/p/why-we-dont-trust-each-other-anymore?selection=f9b3b76b-bbe1-46c1-87f0-57b3d84594b1&utm_campaign=post-share-selection&utm_medium=web - poetry isn’t usually my thing but this got me\nhttps://smartbear.com/learn/code-review/best-practices-for-peer-code-review/ - Study on Pull Request ideal review size. I have no idea how big PRs should be and it falls into the same “economic value of lines of code” issue that I’ve looked at before. There’s lots of metrics - lines changed? files changed” feature count? subfeature count?\nhttps://smallbusinessprogramming.com/optimal-pull-request-size/ - More research resources I need to take a look at. Someday I will have an opinion.\n"},"notes/dailies/2024-03-02":{"title":"2024-03-02","links":["notes/dailies/2024-03-01","notes/dailies/2024-03-03","notes/caveat-lector","tags/games/roblox"],"tags":["mentoring","engineering/scratch","notes/daily"],"content":"⇐ 2024-03-01 | 2024-03-03 ⇒\n\nCAVEAT LECTOR: This is a daily note! Sometimes they’ll be structured, sometimes they won’t. These are more for the benefit of my near- and long-term future self, but I do tend to publish them in case they’re somehow helpful for other people.\n\nToday’s Plan\nToday wasn’t actually a CGC day, but I’m finding myself with some time at the tail end of the day after some sudden super-fun springtime winter storms have floated across Salt Lake City and tanked my plans to get tastefully wine blasted tonight.\nB-Block\n\nN/A\n\nM-Block\n\nN/A\n\nE-Block\n\nN/A\n\nToday’s Report\nToday was my partner’s first swim meet since she did triathlons in college! she did incredibly well - I wish that I knew more about photography so I could take cooler pictures of her when she’s swimming that we can send to our families. Maybe, someday, I’ll be a Photography Guy™.\nI ended up sitting next to a very nice group of kids. I had brought Kenneth Arrow’s The Limits of Organization - with no irony in the slightest, I put it away after seeing that the two kids next to me were reading objectively cooler books. They were very nice - we talked about video games and programming and played Exploding Kittens between meet heats. I cracked open my laptop and they showed me a few programs they’d made in Scratch, which brought back some memories of working with 4th-6th great students at Dr. David Johnson’s GREAT Camp back when I was a sophomore in college.\nSince teaching both that camp and working as a center director at a local Mathnasium for about a year around the same time, it’s been awhile since interacting with Modern Youths™, save for a few niche Roblox communities I lurk in for CGC-tangential projects. It is very cool to see how this generation (gen Alpha? i haven’t kept up much with the newfangled terminology around this generation) has a more… native? approach to technology. It makes sense, considering there will probably (hopefully) never be a moment they go without it.\nThey brought up roblox and I mentioned I’d played when I was their age - the oldest asked a few questions about moving from the block-based coding on Scratch to text-based coding. I feel it is a more interesting question than I was willing to inundate him with (i said “not really!” and capped it off at that.) - in hindsight, it was a pretty big transition for me. That said, I wonder if there exists a future where block-based coding comes back in a way, made possible by stronger generative code solutions to handle minute details within blocks. \nUnironically, I do think I would appreciate a graphical, scratch-like interface for cloud environments. That would be a pretty kickass Terraform frontend.\nAs a slight update on yesterday’s post about binglish - I did post it in a few places (and it didn’t even go viral??? shocked, offended, confused). I think the only piece of text feedback I got was in the Quartz discord, where somebody mentioned that albeit funny the term was one they were familiar with as a colonist term for British-Indian-English.\nI’m torn - by no means is this a Big Thing i’m doing with this blog, nor do I think it ever will be. Certainly not to the extent that the Venn diagrams of people who would read my notes and who would recognize “binglish” in that context would overlap. I also don’t know how I would go about checking - it is interesting that geography is a somewhat soft lock on that kind of cultural knowledge. Even from setting my VPN to Delhi, popping my browser into incognito and doing a (translated, obviously) search, I still get a “urban dictionary,” “stop trying to make fetch happen” vibe from the term. I’ll keep the post up purely because i like it (everybody, appreciate me!), but if that ever does become An Issue™ i will absolutely reconsider that decision.\nNotes"},"notes/dailies/2024-03-03":{"title":"2024-03-03","links":["notes/dailies/2024-03-02","notes/dailies/2024-03-04","notes/caveat-lector","notes/scratch/warning-about-slabtops"],"tags":["projects/personal/homelab","notes/daily"],"content":"⇐ 2024-03-02 | 2024-03-04 ⇒\n\nCAVEAT LECTOR: This is a daily note! Sometimes they’ll be structured, sometimes they won’t. These are more for the benefit of my near- and long-term future self, but I do tend to publish them in case they’re somehow helpful for other people.\n\nToday’s Plan\nToday is a bit strange - I’m currently on a kick of only gaming with friends (gaming solo, for whatever reason, has started to feel like “wasting time” (which is totally not a statement on the virtue of solo gaming, this is just how I feel about it at the moment)). After a bit of a misadventure trying to turn an old laptop into a well-ventilated server, a friend’s brother very graciously offered me some spare PC parts he had lying around so I could (properly, this time) start a home lab for running miscellaneous things, such as dedicated multiplayer servers at my home.\nI highly doubt any CGC work will get done today - I’d vastly prefer setting up the server and playing around with some homelab setup experimentation.\nB-Block\n\nPick up desktop case & parts\nClean out, assemble desktop case & parts\n\nM-Block\n\nPick up a new bed frame for a new mattress arriving later this week\n\nE-Block\n\nAssemble bed frame\nWork on desktop server\n\nToday’s Report\nVery minor EOD report - was able to get a Windows Proxmox setup with GPU passthrough and VirtIO set up today. With the addition of Moonlight & Sunshine, game streaming is now possible! I cannot wait to use this to bully invite my partner to join in game nights with us, now that we have a dedicated(ish) Windows machine for her to use.\n\nBy no means is running Lethal Company at a consistent HDx60fps a technological marvel, but I am very satisfied with the work tonight. While my Steam Deck is probably going to still be my go-to because I hate myself and love debugging, this was a fun little tangent.\nAs an aside - a lot of that setup was cobbling together random support forum posts and other weird shit - at one point I spuriously bought a pack of headless HDMI dummies from Amazon thinking I’d need them. I’d like to toss a quick note here at some point this week to elaborate on some of the missing steps between major guides.\nFuture work on the server (beyond just hosting multiplayer servers) will likely be looking at some actual capital-H-capital-L HomeLab stuff like HomeAssistant to bridge some of our disparate smart home devices into HomeKit so I can control them from my phone and computer a bit easier, but that’s for another night.\nNotes\n\nhttps://forum.proxmox.com/threads/pci-gpu-passthrough-on-proxmox-ve-8-windows-10-11.131002/ - okay, this one actually works.\n"},"notes/dailies/2024-03-04":{"title":"2024-03-04","links":["notes/dailies/2024-03-03","notes/dailies/2024-03-05","notes/caveat-lector","notes/season-of-rhythm"],"tags":["projects/personal/homelab","notes/daily"],"content":"⇐ 2024-03-03 | 2024-03-05 ⇒\n\nCAVEAT LECTOR: This is a daily note! Sometimes they’ll be structured, sometimes they won’t. These are more for the benefit of my near- and long-term future self, but I do tend to publish them in case they’re somehow helpful for other people.\n\nToday’s Plan\nBack to the work week! We haven’t had sprint planning, yet, so I’m not totally sure what today/this week has in store quite yet - I’ll loop back around and update blocks once I have ‘em.\nB-Block\n\nN/A\n\nM-Block\n\nN/A\n\nE-Block\n\nN/A\n\nToday’s Report\n\nTo have the full intended experience, please listen to the Pikmin 2 “Today’s Report” theme while reading & writing this\n\nSo, this particular report goes… well into tomorrow. I’ve been doing pretty good on my sleep rhythm lately, so let’s consider this 6:00AM posting a bit of a relapse.\nI need to get… a couple hours of sleep before I need to be up in the morning, especially considering it’s my partners birthday(!), so this’ll be a brief EOD. I will personally blame whoever it is that made this amazing set of proxmox LXC setup scripts for stealing my sleep tonight, in order to properly divert blame away from the true culprit: myself.\nNotes\n\nuhhhhh… quite the fuzzy match there, YouTube Music?\n\nhttps://gersom.nl/building-a-raycast-website-blocker - raycast blocker similar to Self Control\nhttps://www.raycast.com/tonka3000/homeassistant/commands - raycast extension for HomeAssistant, which is relevant to yesterday’s knock-on work\nhttps://docs.github.com/en/actions/managing-issues-and-pull-requests/closing-inactive-issues - action for closing stale issues, PM req\nTODO: Look up use of new on class method declaration in C#\n"},"notes/dailies/2024-03-05":{"title":"2024-03-05","links":["notes/dailies/2024-03-04","notes/dailies/2024-03-06","notes/caveat-lector"],"tags":["notes/daily"],"content":"⇐ 2024-03-04 | 2024-03-06 ⇒\n\nCAVEAT LECTOR: This is a daily note! Sometimes they’ll be structured, sometimes they won’t. These are more for the benefit of my near- and long-term future self, but I do tend to publish them in case they’re somehow helpful for other people.\n\nToday’s Plan\nToday ended up being a sick day (for both myself and my partner, which unfortunately coincided with her birthday), so not much got done. Not good, not bad - not really sure how to feel about it. At the moment, it’s a bit dour.\nB-Block\n\nR&R\n\nM-Block\n\nR&R\n\nE-Block\n\nR&R\n\nToday’s Report\n\nTo have the full intended experience, please listen to the Pikmin 2 “Today’s Report” theme while reading & writing this\n\ni do appreciates some rest and recovery. in storywriting terms, we’d probably call this one a bottle episode.\nNotes\n\nhttps://tteck.github.io/Proxmox/ - Proxmox helpers\nhttps://arxiv.org/pdf/2403.01152.pdf - research on current generative AI forensics solutions\n"},"notes/dailies/2024-03-06":{"title":"2024-03-06","links":["notes/dailies/2024-03-05","notes/dailies/2024-03-07","notes/caveat-lector","notes/dailies/2024-03-04","notes/utah-office-consult","notes/supply-chain"],"tags":["engineering/devops","projects/personal/homelab","notes/daily"],"content":"⇐ 2024-03-05 | 2024-03-07 ⇒\n\nCAVEAT LECTOR: This is a daily note! Sometimes they’ll be structured, sometimes they won’t. These are more for the benefit of my near- and long-term future self, but I do tend to publish them in case they’re somehow helpful for other people.\n\nToday’s Plan\nI think today’s plan has me diving back into the wonderful world of DevOps, testing out internal DNS capabilities for development resources within VPCs. Thankfully server bullshit the other day has gotten me in a Domain Name State of Mind.\nI think E-block today is going to be fixing up the homelab server. I took apart all of the pieces to clean them but (from what my partner has told me — oops!) there’s a fan in there that isn’t balanced and constantly clicking, so it’ll be some investigating there.\nAlso excited to put in a couple extra pieces - I bought a cheap burner card (GTX 1030 for like $60) to dedicate to video encoding/decoding, along with a few other here-and-there parts to expand storage slots. So long, 256GB storage!\nB-Block\n\nStarting on internal AWS DNS module for Account Factory Terraform\nWhile I’m in DevOps land, I kinda want to take, like, 30 minutes and follow up on a weird concept I read about awhile back - unit testing Terraform code. I don’t totally understand how one would unit test… infrastructure configurations? but who knows, maybe it’ll blow my mind\n\nM-Block\n\nMeeting up with our PM to talk over some developer experience stuff - people problems are truly the hardest engineering problems\n\nE-Block\n\nN/A - I’m hanging out with my partner tonight, but depending on how long her plague-ridden body can handle doing things, I may do some hardware stuff tonight on the homelab server.\n\nToday’s Report\n\nTo have the full intended experience, please listen to the Pikmin 2 “Today’s Report” theme while reading & writing this\n\nN/A\nNotes\n\nI’d like to eventually make the viz from the utah consult post a follow-up to the supply chain network post as an intro to that one guy on bluesky’s network graph viz guide\nhttps://apps.apple.com/us/app/proxmobo/id6447794447 - Proxmox mobile interface - very nice, 10/10 would recommend\nhttps://medium.com/@giuliodn/clone-and-backup-with-gparted-e1100219c1d7 - notes on transferring contents between hdrives with gparted\n"},"notes/dailies/2024-03-07":{"title":"2024-03-07","links":["notes/dailies/2024-03-06","notes/dailies/2024-03-08","notes/caveat-lector","notes/dailies/2024-03-03"],"tags":["projects/personal/homelab","engineering/devops","notes/daily"],"content":"⇐ 2024-03-06 | 2024-03-08 ⇒\n\nCAVEAT LECTOR: This is a daily note! Sometimes they’ll be structured, sometimes they won’t. These are more for the benefit of my near- and long-term future self, but I do tend to publish them in case they’re somehow helpful for other people.\n\nToday’s Plan\nToday’s plan is a bit more DevOps for work, and then some spring cleaning. Working on the new homelab server has, safe to say, unearthed a substantial amount of old hardware I had laying around in a box. If I don’t tidy up the living room, I may end up getting strangled in my sleep.\n\nThat said, the cat has fuckin’ loved the chaos. So many wires to attack - so little time between naps.\nB-Block\n\nCategorizing internal services into the internal LB/DNS\n\nM-Block\n\n\nLooking into GitHub Projects workflows through Actions. Yesterday was some messing about with actions/stale to manage some… pretty large batches of tickets at work. While this has seemed to do the trick so far, there was the realization that our current automation system would move all of the closed, stale tickets to our QA team.\nThere is currently a rolling boulder of about ~250 tickets slowly rolling towards our QA team that will squish them in 14 days’ time. God save their souls.\n\n\nE-Block\n\nSpring cleaning and playing around with the new dedicated GTX 1030 burner card in the homelab server\n\nToday’s Report\n\nTo have the full intended experience, please listen to the Pikmin 2 “Today’s Report” theme while reading & writing this\n\nN/A\nNotes\n\ntest!\n"},"notes/dailies/2024-03-08":{"title":"2024-03-08","links":["notes/dailies/2024-03-07","notes/dailies/2024-03-10","notes/caveat-lector"],"tags":["projects/personal/homelab","notes/daily"],"content":"⇐ 2024-03-07 | 2024-03-10 ⇒\n\nCAVEAT LECTOR: This is a daily note! Sometimes they’ll be structured, sometimes they won’t. These are more for the benefit of my near- and long-term future self, but I do tend to publish them in case they’re somehow helpful for other people.\n\nToday’s Plan\nN/A\nB-Block\n\nN/A\n\nM-Block\n\nN/A\n\nE-Block\n\nN/A\n\nToday’s Report\n\nTo have the full intended experience, please listen to the Pikmin 2 “Today’s Report” theme while reading & writing this\n\nN/A\nNotes\n\nhttps://github.com/languagetool-org/languagetool - languagetool OSS - can I get pro somehow? it’d be fun to hook that to the 1030\nhttps://www.reddit.com/r/portainer/comments/wuv5ei/comment/j18invp/?utm_source=share&utm_medium=web2x&context=3 - Setup for “global” environment variables across all portainer stacks\nhttps://i.kym-cdn.com/photos/images/newsfeed/001/318/550/e2d.jpg\nhttps://www.braytonstafford.com/partitioning-a-usb-drive-in-macos/ - actually partition a persistant boot drive on MacOS\n"},"notes/dailies/2024-03-10":{"title":"2024-03-10","links":["notes/dailies/2024-03-08","notes/dailies/2024-03-11","notes/caveat-lector","notes/scratch/shop-highlight-note","notes/dailies/2024-03-07","notes/digital-gardening-with-quartz","notes/supply-chain","notes/dailies/2024-02-25","notes/strong-and-weak-opinions"],"tags":["projects/personal/homelab","engineering/typescript","notes/daily"],"content":"⇐ 2024-03-08 | 2024-03-11 ⇒\n\nCAVEAT LECTOR: This is a daily note! Sometimes they’ll be structured, sometimes they won’t. These are more for the benefit of my near- and long-term future self, but I do tend to publish them in case they’re somehow helpful for other people.\n\nToday’s Plan\nWith a bit of a half-planned half-day, I’m at my office again (to put distance between me and that god-forsaken server, which has taken far too much of my time) to focus up on the CGC website.\nIn particular, I’m hoping to restore a bit of professionalism to the front landing page. At the moment, it’s the quartz default, and doesn’t have either the font of my handwriting or the actual CGC logo on it. Before submitting some notes for an office highlight, I’d vastly prefer to have my website looking far less cobbled-together than it is now.\nB-Block\n\nN/A\n\nM-Block\n\nN/A\n\nE-Block\n\nLaundry party! Tidying up the living room after the homelab explosion earlier this week\n\nToday’s Report\n\nTo have the full intended experience, please listen to the Pikmin 2 “Today’s Report” theme while reading & writing this\n\nUGH. This damn homelab server sucked me in again in this evening. A linux LVM is… oddly difficult to resize to the size of its new physical partition, and setting up a live disk to do that is incredibly stubborn. I’m in a difficult position with a goal and a few constraints:\n\nGOAL: Set up a Linux Live Disk that comes preconfigured with the utilities I need to remotely modify the server (e.g. lvm2, gparted, Chrome Remote Desktop (for remote-ness), etc)\nCONSTRAINT: I only have access to a MacBook Pro M2, Steam Deck, and the current 256G Proxmox instance itself\n\nSUBCONSTRAINT: There’s not many customizable options for flashing a customized Linux Live Disk on Mac\nSUBCONSTRAINT: The Steam Deck, despite being arch linux, fucking sucks at being used for Linux-y things due to the read-only1 root filesystem and kneecapped pacman installation\nSUBCONSTRAINT: All other Linux instances I have are virtualized on Proxmox, which makes creating custom drives onto actual hardware a pain in the ass\n\n\n\nTo get around this, I’ve settled (currently) on using Linux Live Kit on a virtualized Debian instance, customized with everything I need, and then doing a hardware passthrough of the USB drive to load the customized distro as a recovery disk. Ultimately, this means:\n\nUsing Proxmox…\n…to create a custom Debian live disk…\n…to use for remotely modifying LVM partitions…\n…to fix Proxmox.\n\nthis has sucked… a lot of time out of my life, most of which could’ve been better spent? and i’m debating whether or not to weather the clutter of moving a monitor back into the bedroom to just handle this.\nOther than that, Mrs. Lincoln…\nI did end up getting… a somewhat MVP up for the landing page of the site?\n\nBy all accounts, this doesn’t look good, but it at least has the logo available with… kinda? a title? ish?\nThere is a massive amount of frustration around having goals that require frontend know-how, despite not knowing frontend very well. I am debating, at the moment, the merits of continuing to use Quartz for the blog. I really like it on a design level, and specifically am very excited to dig into fixing what is ultimately Obsidian Publish’s unhelpful & messy (IMHO) network graph layout.\nHowever…\nQuartz, notably, is… weird? I think? The vibe I get is that Quartz was developed primarily for an audience who want, at a base level, to simply deploy an Obsidian vault onto the web without dealing with too much technical cruft. To that end, a lot of the actual frontend fundamental concepts seem to be abstracted away. \nLooking through the Quartz Showcase, there’s not a whole lot of structural changes made in most of these examples. This throws out a bit of an orange flag, making me a bit nervous that the framework itself wasn’t intended (at a base level) to be very extensible2.\nIt feels, somewhat, like an opinionated piece of software.\nThe tricky thing about opinionated software is that, when you pick it up as a beginner, it’s hard to separate whether or not the things you’re learning are:\n\nThe underlying, fundamental concepts of the technology you’re using (React.JS); or\nThe Quartz-level specifics, such as QuartzComponents.\n\nThat said, I don’t know if I have much of an option - from what I can tell, the closest OSS alternatives to Obsidian Publish are typically Gatsby and Jekyll themes - both of which I’ve used previously for this site, to mixed results.\nNotes\n\nhttps://music.youtube.com/watch?v=StlytUzBFcs&list=PLBxbqVsdFQnt5ihbCIkVa15MO7onhZwk5 - this channel has wildly impressive piano covers with super cool notes overlays - almost AR-esque?\nDue to an oversight, I am currently in the middle of a ~5 day gap in my prescriptions. I feel like my brain is currently operating with a constant background static noise that sounds vaguely like this\n\nFootnotes\n\n\nThe read-only OS for the steam deck isn’t actually read only, since you can disable that, but it is substantially restrained in its ability to do normal arch-y things like disk management. I’m also running into the issue of Pacman’s GPG key database is still complaining that it’s read-only and locked. Not sure what that’s all about ¯\\(ツ)/¯ ↩\n\n\nSo, there is such a thing as Quartz Plugins — the landing page work itself is a custom emitter plugin — but it doesn’t seem to have the same breadth or ecosystem that Gatsby did (I could be sub-informed, though, as I haven’t taken too close a look at Remark plugin integration yet). ↩\n\n\n"},"notes/dailies/2024-03-11":{"title":"2024-03-11","links":["notes/dailies/2024-03-10","notes/dailies/2024-03-12","notes/caveat-lector"],"tags":["engineering/devops","notes/daily"],"content":"⇐ 2024-03-10 | 2024-03-12 ⇒\n\nCAVEAT LECTOR: This is a daily note! Sometimes they’ll be structured, sometimes they won’t. These are more for the benefit of my near- and long-term future self, but I do tend to publish them in case they’re somehow helpful for other people.\n\nToday’s Plan\nJesus. I think I’m officially Old™ now. Daylight Savings was last night and I legitimately feel hung over right now.\nB-Block\n\nTerraform infrastructure - taking a look at the glories of DNS handling\nDoing brief & sexy horticulture on terraform unit testing\n\nM-Block\n\nI am too tired to look more than about ten minutes into the future.\n\nE-Block\n\nGoing home to assemble a bedside table and take a nap. God, I need a nap.\nLaundry Party!\n\nToday’s Report\n\nTo have the full intended experience, please listen to the Pikmin 2 “Today’s Report” theme while reading & writing this\n\nN/A\nNotes"},"notes/dailies/2024-03-12":{"title":"2024-03-12","links":["notes/dailies/2024-03-11","notes/dailies/2024-03-13","notes/caveat-lector","notes/over-under-engineering","notes/dailies/2024-03-03","notes/scratch/the-best-side-of-the-road","tags/notes/"],"tags":["projects/personal/homelab","notes/daily"],"content":"⇐ 2024-03-11 | 2024-03-13 ⇒\n\nCAVEAT LECTOR: This is a daily note! Sometimes they’ll be structured, sometimes they won’t. These are more for the benefit of my near- and long-term future self, but I do tend to publish them in case they’re somehow helpful for other people.\n\nToday’s Plan\nYesterday was, in fact, pretty much a complete wash Goblin-wise. That said, last night I got an actual human amount of sleep, so I’ll be starting work a couple hours early today to hopefully shift everything a healthy amount forward for the day\nB-Block\n\nN/A\n\nM-Block\n\nN/A\n\nE-Block\n\nN/A\n\nToday’s Report\n\nTo have the full intended experience, please listen to the Pikmin 2 “Today’s Report” theme while reading & writing this\n\nAlright! I broke yesterday’s homelab stalemate by sucking it up, getting past the need to over-engineer simple problems, and just… popping GParted onto a thumb drive and resizing the partition. As it turns out, easy problems only become hard problems when you make them hard problems ¯\\(ツ)/¯\nThis did leave time to get to the actual “lab” part of “homelab.” This involved knocking a few things off my list:\nProxmox Game Servers\nWhile the amount of focus has been nice, I will admit - I’m a bit lonely at the moment? A week ago the plan was to set up the server to play with friends, and that still hasn’t happened - so I wanted to at least get this off the ground tonight. This, however, cycled through a few phases… sometimes multiple times.\n\n\nOriginally, the plan was to use a lightweight Docker LXC on Alpine, with a sidecar Portainer instance, to run the game servers. However, coming from a Desktop Ubuntu environment, I was seriously missing the ability to load the Tailscale Docker Desktop extension to auto-add new containers to my Tailscale instance.\n\n\nI also thought that Portainer was a bit too overkill for what I was trying to do. Honestly, I saw a bunch of very Enterprise-y™ things, got scared, and looked at other solutions.\n\n\nNext stop was briefly trying out Pterodactyl - however, the setup process on their website absolutely tanked, so I ruled that out pretty quickly.\n\n\nNext up was AMP — which, to its credit, did work. However, the next snag was trying to set up a custom Docker Compose file. While AMP is great if the only control you’d like to have is “press button, have server,” I did a search across the entire system for anything resembling a Compose file and came up empty.\nDealbreaker.\nUnfortunately, it was only a dealbreaker after I had bought a license. Whoops.\n\n\nNext up was TurnKey Game Server. I’d seen a few things about TurnKey and LinuxGSD while searching around and figured it was worth a try. However, after multiple attempts, I wasn’t able to get a game server up at all1. Perhaps I misunderstood how the Webmin interface works, but… I also couldn’t even get a frontend to show up. Also a dealbreaker.\n\n\nSo, whaddaya know? I ended up exactly where I started - Docker & Portainer. Once you get past all the buttons, Portainer is very nice.\nI don’t think this is the worst thing in the world. Ultimately, every other of the above solutions (both the ones that worked, and the ones that didn’t) used weird, non-standard solutions for containerizing the game servers. The best side of the road is often the one others are driving on, and so choosing a pure-Docker solution for a very Docker-y problem is hardly a loss. However, I do wish that I’d spent less time fucking about with the options and more time just… making it work.\nThis spends a lot of time in my head, but it was very loud tonight.\nJellyfin & GPU Passthrough\nWell, fuck.\nAfter seeing some success with GPU passthrough, I rolled the dice on another go of it. I bought a GT 1030 card for hardware encoding & decoding for a video library I’ve got cookin’ and spent quite a bit of time tonight researching passthrough of the card into an LXC. Despite being virtually identical tasks in my own mind, the LXC passthrough turned turned out to be a surprisingly more involved process than passthrough into a VM2. After going through the trouble…\n…I realized that the GT 1030 doesn’t even support encoding.\nI’m not totally sure what to do with this card now…? Jellyfin hardware encoding was the entire reason I’d bought it originally. While $60 isn’t exactly financially devastating, it does feel weird having a useless card. I may do a quick search later this week to see if there’s some good reason to keep it in there - otherwise, I’ll likely take back the PCIx4 slot, likely for an additional NVME port. Still, quite the bummer.\nThankfully, most of my video library at this point is already encoded, and I suppose (worst-case scenario) I could potentially use the Windows VM (and its successfully passed-through GTX 970) to handle any on-the-spot encoding I need to do for specific files.\nPersonal Cloud\nThis is one that I kinda grazed, but didn’t get too far into - I’ve set up a NAS software to handle a bit of cloud storage, and I plan on setting up the aforementioned PCI/NVME expansion to set up a separate storage drive. I don’t have any hard drives at the moment, but this is something I’d very much like to get up-and-running at some point.\nFinal Thoughts: Things On My Mind\nYesterday, while internally mulling over the merits of using Quartz, I looked at the Navel-Gazing section of Maggie Appleton’s (very excellent) digital-gardening resource repo. I’d previously associated the term as being somewhat negative but realized I didn’t actually know what the term meant, so after a quick lookup on Wikipedia:\n\nHowever, phrases such as “contemplating one’s navel” or “navel-gazing” are frequently used, usually in jocular fashion, to refer to self-absorbed pursuits.\n\nI do worry somewhat that this blog - in particular, the notes section, is absolutely just me, navel-gazing, at my own cool, edgy ideas. It also coincides with a concern (possibly paranoia) that I use the word “I” too much when talking to others and that I’m being very self-absorbed.\n(I’ve also noticed myself linking others to this blog in my personal day-to-day. It feels silly to say “in hindsight” considering this last happened… less than 3 hours ago: in hindsight, this comes off super pretentious and i should absolutely not do it)\n(maybe i can do it a little bit - as a treat? i am pretty proud of this site so far.)\nThere doesn’t seem to be a great solution there, especially for the notes section, given that the whole thing is literally written in the first-person. However, I do want to make sure CGC remains a functional source of knowledge for others, and not just for my own benefit (although I do appreciate that part of this site.)\nSomething to think about, I suppose.\nNotes\n\nhttps://zeldor.biz/2011/01/mount-lvm-from-livecd/\nhttps://forum.proxmox.com/threads/pci-gpu-passthrough-on-proxmox-ve-8-installation-and-configuration.130218/ - proxmox gpu passthrough guide\n\nFootnotes\n\n\nWhile I’m thinking about it - honestly, from the polish of the application to the weird-ass unformatted, un-line-broken email they sent me as a confirmation for their website, TurnKey honestly doesn’t even look like a currently-maintained or legitimate website. Their stated goal is to bring server appliances to the masses, but their UX screams “Email from a Nigerian Prince” to me. ↩\n\n\nGPU passthrough to LXC for decoding - https://jocke.no/2022/02/23/plex-gpu-transcoding-in-docker-on-lxc-on-proxmox/ ↩\n\n\n"},"notes/dailies/2024-03-13":{"title":"2024-03-13","links":["notes/dailies/2024-03-12","notes/dailies/2024-03-14","notes/caveat-lector","notes/scratch/shop-highlight-note","notes/dailies/2024-03-10"],"tags":["projects/personal/homelab","notes/daily"],"content":"⇐ 2024-03-12 | 2024-03-14 ⇒\n\nCAVEAT LECTOR: This is a daily note! Sometimes they’ll be structured, sometimes they won’t. These are more for the benefit of my near- and long-term future self, but I do tend to publish them in case they’re somehow helpful for other people.\n\nToday’s Plan\nToday’s plan is going to be some logging service implementations - in particular, styling out a C# singleton to extend Discord webhook behavior for super quick-and-easy monitoring. This’ll be nice because I’d like to learn more about logging mechanisms, since being able to stream data straight out of a stdout would be nice for some other data gathering projects I’d like to do down the road.\nB-Block\n\nLook into C# standards and opinions around logging services - in particular, differentiating between a static Console-esque implementation of a logging service, versus an ASP.NET-flavored Singleton\n\nM-Block\n\nproject management notes meeting\nAt some point today, I need to finish the highlight note for The Shop.\nI’d also like to start reading (and potentially reviewing) The Framework Field Guide, which was written in part (or entirely? i don’t actually know the authorship distribution of that, since Unicorn Utterances is like, a collective, i think?). I’m super excited to dig into it - from the other day, it’ll be nice to put my paranoia aside about learning facts from opinions and getting to learn a more framework-agnostic baseline knowledge of frontend development.\n\nE-Block\n\n~~Hmmmm… good question? As much as I’d like to take advantage of the game server setup being actually functional again, I really ought to stick around the office while my partner is climbing to hammer out CGC work.\n~~That said… ~~factorio.\n\nToday’s Report\n\nTo have the full intended experience, please listen to the Pikmin 2 “Today’s Report” theme while reading & writing this\n\nIt’s already super late, so I’m gonna make this quick:\n\nI think friends have settled on a new Eco runthrough, which is very exciting!\nIn the meantime, spring cleaning on the steam deck - backed up /home, set up a Sunshine server for remote management from the laptop (QoL required)\nTried to go the Chrome Remote Desktop route by building it manually. I deeply regret this diversion, and would highly not recommend the attempt.\n\nNotes\n\nhttps://craftinginterpreters.com/ - A book my boss mentioned getting for his son about creating programming language. Total nerds\nhttps://aws.amazon.com/premiumsupport/technology/trusted-advisor/ - AWS trusted advisor, possibly(?) worthy of some horticulture\n"},"notes/dailies/2024-03-14":{"title":"2024-03-14","links":["notes/dailies/2024-03-13","notes/dailies/2024-03-15","notes/caveat-lector"],"tags":["economics/project-management","notes/daily"],"content":"⇐ 2024-03-13 | 2024-03-15 ⇒\n\nCAVEAT LECTOR: This is a daily note! Sometimes they’ll be structured, sometimes they won’t. These are more for the benefit of my near- and long-term future self, but I do tend to publish them in case they’re somehow helpful for other people.\n\nToday’s Plan\nN/A\nB-Block\n\nN/A\n\nM-Block\n\nN/A\n\nE-Block\n\nN/A\n\nToday’s Report\n\nTo have the full intended experience, please listen to the Pikmin 2 “Today’s Report” theme while reading & writing this\n\nN/A\nNotes\n\nhttps://rachelbythebay.com/w/2012/08/31/lines/ - pro puppy punting league\nhttps://stacks.stanford.edu/file/druid:xk241jp4730/Dissertation-LinFan-augmented.pdf - the concept of a “multi-armed bandit problem” is a) new to me; b) interesting; and c) incredibly funny out-of-context\n"},"notes/dailies/2024-03-15":{"title":"2024-03-15","links":["notes/dailies/2024-03-14","notes/dailies/2024-03-17","notes/dailies/2024-03-15","notes/scratch/shop-highlight-note","notes/digital-gardening-with-quartz"],"tags":["engineering/csharp","projects/personal/homelab","business/admin","notes/daily","projects/cgc/blog"],"content":"⇐ 2024-03-14 | 2024-03-17 ⇒\nToday’s Plan\n\n\nMorning Block: Morning work session, typically 9am - 12pm\nAfternoon Block: Afternoon work session - usually the bulk of work. 12pm - 5pm\nEvening Block: Evening work session for CGC-related work, 5pm - ???\n\n\nMorning Block\nNUnit Discord Logger\n\nPLAN:\n\nCreate ASP.NET general service to collect & batch logs\nCreate ITrace for use in NUnit tests\nImplement on test server webhook - once confirmed, move over to core server\n\n\nDELIVERABLES:\n\nA message (locally) posted from NUnit to Discord\nStretch goal is a webhook configuration able to create a new thread in a Forum channel1\n\n\n\nAfternoon Block\nBackstage Local Run\nAs dayjob work starts shuffling up a bit more, I want to elbow for a bit of room to keep working on Backstage plugins. To do that, I need to get my local Backstage build back up-and-running - and probably updated as well, since god knows how long it has been since I got that running 😬\n\nPLAN:\n\nRedownload Backstage as a fresh repo\nPossible minor configuration - I’ll likely test this on the CGC GitHub repository so I have the admin access needed to generate keys & pull information\nTaking a closer (more organized) look at the run scopes in the repo\n\n\nDELIVERABLES:\n\nDocker Compose-able Backstage running locally for plugin development. hype hype hype! 🎉\n\n\n\nShop Excerpt Written\nToday may be a dayjob early-release day, in which case I really need to get the Shop Highlight excerpt written & sent.\n\nPLAN:\n\nTake a look at the notes I took last week\nJot down draft, run it through something to un-fuck my fucked writing style.\n\n\nDELIVERABLES:\n\nDeliver as formatted email w/ Markdown attachment. God bless Markdown - hopefully this time doesn’t end up like the last guest post I wrote 😬\n\n\n\nEvening Block\nEco Server Run\nThe friends have settled on eco!! I believe I got most of this taken care of last night, but may need to adjust server run parameters. This’ll be the first stress test of the Allytron’s game container, so I don’t totally know what the resource req will look like.\n\nPLAN:\n\nRally the lads!\nMonitor specs during run\nOPTIONAL: bully my partner into joining us\n\n\nDELIVERABLES:\n\nFun, enjoyment, social satisfaction, etc. etc.\n\n\n\nToday’s Report\n\nTo have the full intended experience, please listen to the Pikmin 2 “Today’s Report” theme while reading\n\n\n\nLUNCH UPDATE: Did some TLC to the layout of the daily notes with Templater2. Added yesterday/tomorrow links to the top of each page (Formatting definitely required - tempted to just handle this in the standard quartz post emitter, now that I know how to do that) and configured the Daily Notes appender in the Raycast Quartz Plugin to append some custom logic for auto-incrementing footnotes. Not perfect, but works a treat.\n\n\nAlso did an update for a new layout for per-block tasks in the daily notes. I feel like I’ve been having trouble measuring tasks on a micro-level, and decided to use the same format for dayjob work that I do for contracting work, with a scope, plan, & declared set of deliverables so I can very confidently say that I did (or did not) finish something, and call the shot to stay on track & be intentional with my time.\n\n\nI’m considering setting up a Sunday-specific weekly template, so that I can better plan out my weeks during Sunday CGC work.\n\n\nA-BLOCK EOB UPDATE: Eeeee!! Feeling very behind on things. Overwhelmed and under-motivated.\n\n\nLikely EOD update — I don’t see myself getting back on my laptop at any point today for reasons — but I gotta hard stop to go take care of Life Business™. Did not get nearly as far as I’d hoped on any of today’s tasks, so today’s work will likely bleed into Sunday’s CGC work and (potentially, if the weather this weekend isn’t good enough to chill outside) Saturday.\nI’m 50/50 on whether or not I like the new daily template. While it is better-organized, I don’t totally know if it’s better-organized for me and where I’m at as a person doing things. I’ve been running constantly into an issue of over-estimating how much I can get done in a day. The deliverables are meant to help with that, but I don’t think it alleviates the issue of not being able to accurately predict my own day-to-day velocity.\nSome thoughts on the back of a napkin, but it may be worth trying out the Tasks List function in Obsidian, or potentially another solution. Interface aside, i think one of the best task management softwares I’ve used is Pivotal. I specifically appreciated the ability to “batch” tasks by velocity, to try and get a reasonable calculation for what is possible in a given amount of time. I think that’s something I’d like to get back to at some point, researching more “personal life” versions of the same thing.\nAnyway - likely signing off for the day. May the next day and the next be a better haul than today 🙏\nACTUALLY: one last final thought. I wonder if the Obsidian Git integration runs pre-commit hooks…?\nFootnotes\n\n\nhttps://arc.net/l/quote/flmmvzxd - the fact that Discord webhooks can auto-create threads is super cool. If there’s a way I can do thread auto-tags with this… i’ll be a very happy lad. ↩\n\n\nhttps://iwannabemewhenigrowup.medium.com/part-3-using-the-templater-community-plugin-to-automate-files-and-content-6f1f2f9227c3 - Guide used for intro to Obsidian Templater, used to create the automatic date entries and footnote counts on daily note template. ↩\n\n\n"},"notes/dailies/2024-03-17":{"title":"2024-03-17","links":["notes/dailies/2024-03-15","notes/dailies/2024-03-18","notes/dailies/2024-03-17","notes/dailies/2024-03-10","notes/over-under-engineering","tags/projects/","notes/time-tracking","notes/annotations/simple-economics-of-open-source"],"tags":["notes/daily","engineering/typescript","projects/cgc/blog","engineering/csharp"],"content":"⇐ 2024-03-15 | 2024-03-18 ⇒\nToday’s Plan\n\n\nMorning Block: Morning work session, typically 9am - 12pm\nAfternoon Block: Afternoon work session - usually the bulk of work. 12pm - 5pm\nEvening Block: Evening work session for CGC-related work, 5pm - ???\n\n\nMorning Block\nN/A\nAfternoon Block\nDiscord Logging\nJust to start the work week on a fresh note, I’m going to wrap up the Discord forum logging before anything else today.\n\nPLAN:\n\nTest hooks manually onto a forum channel in a test server\nTransfer customizations over to a configuration file\nInsert the head commit hash for splitting threads by deployment\n\n\nDELIVERABLES:\n\n Threaded, batched logs, output to the test server forum channel\n\n\n\nEvening Block\nQuartz Index Testing\nWhile the current landing page isn’t particularly bad, it isn’t good either. I’d like to do some touch-ups, with inspiration from a couple of examples of quartz that have customized their sites to a greater degree.\n\nPLAN:\n\nThere’s a particular Quartz site I saw that had a fantastic implementation of a landing page - I’ll need to find that\nReview the code for their emitter (provided it’s open-source — fingers crossed 🤞)\n\n\nDELIVERABLES:\n\n If feeling spicy, possibly give implementation a try.\n\n\n\nToday’s Report\n\nTo have the full intended experience, please listen to the Pikmin 2 “Today’s Report” theme while reading\n\n\nUPDATE: The Discord logging has become deceptively difficult. While creating a webhook and sending updates to a Forum channel are pretty easy, a surprising difficulty is getting the snowflake IDs of forum tags to use with the webhook’s applied_tags property of the JSON body. Unlike user, channel, and guild IDs, it doesn’t appear that the snowflake ID for tags are readily available. I think my next attempt for finding those will be looking directly at the webpage source for the discord web app. \nUPDATE: I think I’m going to be using a tag-based explorer approach1 for this site’s explorer - I’m a huge fan of being able to have a note in multiple places by subject matter rather than using the base file structure of the contents folder. Huge shout-out to whoever CVanmarcke is for publishing their Quartz components for this!!\n\nIn a typical act of over-engineering, I wasted a shitload of time today trying to figure out tags for Discord webhooks into forum posts. Ultimately, this is one of those “nice to have”, “oohs and ahhs” features for the relevant project that, ultimately, don’t need to be in there.\nI think that, personally, I need to start being better about catching myself getting diverted away from a defined MVP with “wouldn’t it be cool?”-type diversions. I can think of a few things to do about this:\n\nCreate well-written, well-scoped MVP documents for projects/milestones, and be better about catching myself wandering. Possibly a candidate for the new projects tag, perhaps?\nFind a way to more aggressively remind myself to time-track. Judging by a drop in my tracking coverage (-70% compared to two weeks ago), I need something to keep me on track, there.\n\nI do need to give myself a bit of grace, here - a recent lapse in medication left me pretty hobbled the past couple weeks. This week should be better.\n\n\nAdopt some kind of to-do list system. I talked about this a bit, recently, and I think I just need to kick myself into doing it. I need some better system for breaking milestones into subtasks, and then ranking them by priority/size. Ironically, I’m currently building a system specifically for this kind of thing — but having trouble getting momentum on it because, ironically, I don’t have a well-defined list of things to do. Very chicken-and-the-egg\n\nAs for wins today, I will say that the kick-off for an OSS economics research project and introduction/reorg of the tag and explorer systems for the site were wins. I do think that a tag-based explorer system meshes better with my intentions for the Quartz graph system, and is way easier for people to traverse. There is some fuckery afoot with the current way it’s handling tag index pages (only showing subtags, and not base-level tags within the category) but I’m feeling much more confident with Quartz tag emitters, now. That’ll be something on the chopping block, likely this week.\nAnother win is at least starting on a pre-build process for the site. The first tasks this’ll be responsible for is making sure that every tag being used has a proper index page. Even if it isn’t used, I’d like to be able to organize the /tags file structure automatically, rather than have to tune it by hand. Any opportunity to put up guard rails is a good opportunity.\nFootnotes\n\n\nhttps://github.com/CVanmarcke/quartz - Quartz plugin for tag-based post exploring ↩\n\n\n"},"notes/dailies/2024-03-18":{"title":"2024-03-18","links":["notes/dailies/2024-03-17","2024-03-19","notes/dailies/2024-03-18","articles/binglish"],"tags":["notes/daily","engineering/csharp","engineering/devops"],"content":"⇐ 2024-03-17 | 2024-03-19 ⇒\nToday’s Plan\n\n\nMorning Block: Morning work session, typically 9am - 12pm\nAfternoon Block: Afternoon work session - usually the bulk of work. 12pm - 5pm\nEvening Block: Evening work session for CGC-related work, 5pm - ???\n\n\nMorning Block\nCloud Testing\nIn order to catch the spunky, sexy, broad amalgamation of ways that infrastructure can fuck up in The Cloud™, it’ll be nice to have a cloud-based test suite that logs back to monitoring channels\n\nPLAN:\n\n Wrap tests into Docker Container\n Local test container/compose running (successfully or not) locally\n Adapt, adjust environment variable & ASP.NET environment\n\n\nDELIVERABLES:\n\n Tests running (successfully or not) in cloud\n\n\n\nAfternoon Block\nFinal Discord Webhook Work\nWith access to the GET REST APIs for Discord, now (through a bot - not thrilled, but w/e), I can do the final steps for deployment logging using Discord Forum channels\n\nPLAN:\n\n Generate webhook channel\n Add webhook to internal C# library\n Create new ILogger sub-interface\n Create batcher\n Create sender\n Inject logic into NUnit ITrace\n\n\nDELIVERABLES:\n\n Threads created through NUnit ITrace\n\n\n\nEvening Block\nShop Highlight (for real, this time).\nThis has been backburned for way too long. I will send this today, before I do anything else with CGC time.\n\nPLAN:\n\n Review note last state\n Finish first draft\n Edit it (probably with AI. there is a time and a place for binglish, and this is it.)\n Subject my partner to my shitty writing for proofread.\n\n\nDELIVERABLES:\n\n Highlight note sent as attachment & markdown.\n\n\n\nToday’s Report\n\nTo have the full intended experience, please listen to the Pikmin 2 “Today’s Report” theme while reading\n\nN/A"},"notes/digital-gardening-with-quartz":{"title":"Kicking off a Digital Garden with Quartz","links":["articles/structured-streaming","articles/hello-blog"],"tags":["goblin-slaying","horticulture","engineering/typescript","notes"],"content":"I’ve decided that I’m pretty bad at capital-B Blogging™. The last “post” I made (as of writing) was the Databricks guest post on Spark Structured Streaming that I did back when I worked at M Science in 2022. That particular post went through a few different stages of revisions:\n\nRevising initial drafts as I learned more about Structured Streaming as a concept\nRevising “personally final” drafts with the Structured Streaming engineering team at Databricks\nRevising “technically final” drafts with somebody from our company’s internal marketing team to make sure I didn’t sound like a doofus/didn’t make it seem like the company hired doofuses (it hired me, so it totally did, but that’s obviously not something to broadcast openly)\nRevising the “marketing final” draft with a lawyer to make sure I wasn’t blabbing trade secrets to the general public\nRevising the “legally final” draft with Databricks’ blog team to make sure the formatting was right (which I didn’t do a good job of, apparently - the final published draft ended up having some weird formatting errors. Not totally sure what happened there ¯\\(ツ)/¯ ) \n\nThat was quite a bit more than I’d expected when initially pitching “What if we published our internal guide for everybody?“. That, along with starting a new job that introduced me to a far higher standard for engineering, raised my own internal bar for “what should I put on the internet?” Since then, I’ve written quite a few posts that I’d describe as ~60% “done,” but didn’t end up hurdling internal (and arbitrary) bar for publishing.\nConsidering the editorial standards I started this blog with, the idea that I’ve had any bar at all is a bit funny.\nThe cost of learning is teaching\nAs somebody that has benefitted immensely from the free body of knowledge that exists on the internet, this doesn’t make me happy. A graveyard of posts I’ve arbitrarily deemed “unfinished” isn’t particularly helpful to anybody, including myself. Just because a set of knowledge isn’t refined enough to be perfectly, 100% clear to a broad audience doesn’t mean it wouldn’t be helpful to somebody, including myself in the future.\nI was fortunate enough to stumble on Maggie Appleton’s Ethos on Digital Gardening, which notably resonated with me more than many things I’d read in the recent past. In particular, this diagram she designed clicked something in my brain that’d remained un-clicked for quite awhile:\n\n\nbrief aside: Maggie brings a level of polish and class to doodling that is frankly enviable.\n\nThe cost of perfection is infinite\nPutting things on a spectrum of “chaos” to “cultivated”, in conjunction with the idea that “the cost of perfection is infinite” (if not asymptotic), puts into hardly-uncertain terms that simply because a thing is not perfect, or close to it, necessarily means that it is unhelpful and should die a slow death in a draft bin.\n"},"notes/literal-scratch-piano":{"title":"From the Archive: Scratch Piano Application","links":["notes/dailies/2024-03-02"],"tags":["mentoring","engineering/scratch","notes"],"content":"Back in 2018, I taught at the GREAT Camp at the University of Utah as an instructor for ~4th-6th grade students interested in intro programming in Scratch. Honestly, as a CS student at the time, it was actually really cool to try and put things I’d made in actual languages and translate them into Scratch, especially if they required manually implementing data structures like stacks and heaps.\nThis was a fun project to try and explain music - something that every Utah child learns, short of by-law - and also a way to cope with the fact that in Spring of 2018 I’d unintentionally signed up to take a weed-out Intro to Music Theory course at university.\n"},"notes/literal-scratch-quicksort":{"title":"From The Archive: Scratch Quicksort","links":["notes/literal-scratch-piano","notes/dailies/2024-03-02"],"tags":["mentoring","engineering/scratch","notes"],"content":"As an addendum to the Scratch Piano Application, I also around the same time cooked up an implementation of QuickSort. At my partner’s recent swim meet I sat next to a couple kids who’ve been learning Scratch and they were showing off some games they’d made.\ni was shocked - when it was my turn to show off, they didn’t seem very impressed by this.\n\nhonestly, the sort isn’t the weird thing - it’s the code itself that makes me, with 6 years of hindsight, think “…why the fuck would I make this?”\n"},"notes/over-under-engineering":{"title":"The Over/Unders of Over- and Under-Engineering","links":["notes/digital-gardening-with-quartz","notes/over-under-engineering","tags/engineering/python","articles/structured-streaming","tags/engineering/csharp"],"tags":["engineering","goblin-slaying","notes"],"content":"\n\n\n\nPremature optimization Over-engineering is the root of all evil\n— Donald Knuth\n— Sir Tony Hoare\n— Me, just now\n\n\n\n\nI think that over-engineering is the single greatest contributor to starting projects and never finishing them.\n\nLet’s define follow-through rate as the ratio of projects started to projects released. Not “completed”, since that’s an irrational way to think about projects.\n\nLooking back at periods of my life, I feel like my follow-through rate on projects was highest when I was 12 years old. At that point, my “projects” were:\n\nMy grades in high school\nWriting shitty programs on my TI-84\nWriting small games on Roblox\n\n13 years is long enough ago that, for the sake of simple math, my follow-through rate was 100%. Gee, what a reliable kid I was!\nI can name 4 periods of my life where the amount I learned about engineering grew at a more-rapid-than-usual pace:\n\n2016: I began studying computer science in my undergraduate\n2018: In trying to build up my resume, I started taking writing open-source and release-able projects more seriously\n2020: I got my first full-time job as a data analyst/software engineer writing data pipelines in Python and Spark\n2023: I got my second full-time job writing general software in csharp, with incredibly smart capital-E Engineers™ that were also incredibly good mentors\n\nWhile it feels great to know more, learning has caused me a fundamental issue: it’s easier to be bad at something on accident than on purpose.\nI’ve increasingly noticed myself trying to apply the things I’ve learned to new projects, causing a whole lot of up-front work to get a project started. Putting yourself in a prison of making the “best” choices at the very start of the project is incredibly demotivating and absolutely tanks the odds that a project started will someday be a project finished. Ironically, it’s future-proofing for a future that doesn’t happen.\nLet’s say, hypothetically, that my project follow-through rate started at 100% and dropped by a third following each learning period. After 4 learning periods, that would leave me at…\n(1.00−0.30)5=0.16\nYeesh. 16% follow-through rate sounds rough, but when I look at a recent history of what I’ve accomplished versus what I’ve wanted to accomplish, the number seems somewhat reasonable.\n\n"},"notes/putting-selfcontrol-on-raycast":{"title":"Watering Down SelfControl into Self Restraint","links":["notes/putting-selfcontrol-on-raycast","notes/season-of-rhythm","notes/arc-review","notes/the-quest-to-slay-the-goblin","notes/raycast-review","notes/caveat-lector","notes/dailies/2024-02-28"],"tags":["goblin-slaying","engineering","mentoring/tutorials","notes"],"content":"\nI talk a lot in this post. You may want to skip to the useful part.\n\nSo, during this recent period of trying to break quite a few years of bad work habits, I’ve been putting up little guardrails in my own life to keep myself on-track when I want to stay on-track. One issue I’ve run into - especially during a particularly tumultuous US election year - is a tendency to check the news.\nAnd then, five minutes later, check the news again.\n…and again\n…and again.\nI’ve rationalized it for awhile by saying “I work in politics - of course I should stay up to date!” However, that’s a lie - my day job is not in politics. At best, It’s working on software adjacent to people who are adjacent to politics.\nTo alleviate that for awhile, I had used a Chrome-specific app called StayFocused to block a pretty broad swath of websites, but ran into two issues:\n\nStayFocused allows you to schedule blocks, but only for predetermined amounts of time, and only a single block per day. This was not quite as flexible as I was comfortable with - I do like to check the news, my email, or Bluesky at lunch, or if I get off of work a bit earlier than expected.\nI started using Arc Browser with three different profiles - Personal, Work and CGC - and each one has its own Chrome plugins. Because of this, I’d have to configure/reconfigure/update the StayFocused installation in triplicate, which due to the way their app is set up is incredibly hard to do.\nTheir app kept hijacking my browser to notify me of updates, and I didn’t see a readily-available way to do that. Ironically, this had the effect of absolutely shattering my focus.\n\nI ended up uninstalling it and going with something a bit heavier-duty. SelfControl is an application for MacOS that blocks out apps across the entire operating system. I had also seen Focus listed as a possible solution, but… $49? For effectively the same thing with a prettier interface? yeesh.\nIronically, though, SelfControl requires self control to use, which is exactly what got me in this situation to begin with. It is incredibly easy to “take a break” and then just-a-couple-more-minutes yourself out of any semblance of flow. As a nice surprise, though, SelfControl comes with a CLI, which means we can use it for some more advanced workflows! We can use this to do two things:\n\nSchedule blocks of self-control for the future, e.g. “In 10 minutes, I will focus for 50 minutes”; and\nConnect that behavior to Raycast, which will help us lower the bar to starting Raycast (as, admittedly, having to swipe back to the main Mac desktop is a pain in the ass and puts it out-of-the-way)\n\nThe Useful Part\nI’ve added a couple of useful bash scripts to a SelfControl Raycast Plugin for others to use. At some point, I’ll look into the process for getting them added to the actual Raycast extension store. You can add these by cloning the repository and adding it in the Raycast settings as a “Script Directory.”\n\nTHAT BEING SAID: I’ve used the term caveat lector quite a bit to warn people that I’m just some guy. This goes doubly here! This is software that has literally only been tested, by me, a handful of times, that I made while (ironically) procrastinating other things I should be working on. To say it isn’t well-tested would be a dramatic understatement. The creators of SelfControl warn that it’s a tool that, if misused, can have side effects. So to that end… what would the phrase even be?\n\nHuh. I guess the Romans didn’t do much file sharing.\nCaveat download!"},"notes/qamo":{"title":"Quantitative Analysis of Markets & Organizations","links":["tags/economics/"],"tags":["economics","notes"],"content":"In 2022, I graduated from the University of Utah with a degree in Quantitative Analysis of Markets & Organizations (QAMO, more-or-less quantitative economics).\nWords cannot describe how much I appreciate the mentorship I received from my professors - especially Scott Schaefer, the program director who was incredibly kind in providing a level of flexibility that, frankly, was the only reason I was able to graduate while working full-time.\nThat said, I do wish the name of the degree wasn’t such a pain in the ass to type out."},"notes/raycast-review":{"title":"Diary of a Raycastaway","links":[],"tags":["goblin-slaying","notes"],"content":"Recently, I stumbled on Raycast - a MacOS Spotlight replacement that I would highly recommend to everybody, not just engineers."},"notes/robloxaville-remaster":{"title":"Thoughts on the bad 2017 Robloxaville Remaster","links":["articles/robloxaville","notes/over-under-engineering","notes/digital-gardening-with-quartz"],"tags":["games/roblox","engineering/lua","notes"],"content":"When I was in high school, I dabbled a bit in programming on Roblox. Due to some pretty glaring security concerns, a project started over the summer of 2017 to remaster Robloxaville. I’ll be the first to admit that the remaster is substantially less fun to play than the original. While part of this constraint was time - the original was made over the course of ~4 years, while the remaster was made in ~3 months - it’s undeniable that the velocity of the original project was simply far higher because it was under-engineered.\nThe original project was largely made by my mom, who studied programming during her physics-chemistry-geology triple(!!!) major in college, with some tangential work from me, who was a literal child. Not to sell short the sheer amount of work she put into the project, but between a lack of formal software engineering and the pretty glaring shortcomings of the Roblox engine itself meant that, frankly, it would’ve been impossible to engineer any of it properly to begin with. By the time the summer of 2017 rolled around and the remaster started, two circumstances collided:\n\nPlatform developers were now being paid, which introduced a far higher bar on quality of work being done\nI had just finished my first semester of a degree in computer science and learned tip-of-the-iceberg concepts of what constituted “good” software.\n\nDespite happening 7 years ago as of the time of writing, I remember how absolutely stressful that summer was. I spent wasted a lot of time worried that the code I was writing wasn’t engineered enough, and ended up designing a lot of bad and convoluted systems meant to save myself time in a future of development on that project that never came.\nThere exists a graveyard of projects and posts between 2017 and today that have fallen victim of arbitrary bars for under-engineering that I’ve put on my own work."},"notes/scratch/brevity":{"title":"brevity is no longer the soul of wit","links":["articles/binglish"],"tags":["horticulture","notes/scratch"],"content":"when he wrote “brevity is the soul of wit”, shakespeare clearly did not consider that someday AI generated briefs would consume the english language"},"notes/scratch/press-f-to-commit":{"title":"Press F to Commit","links":[],"tags":["viz","notes/scratch"],"content":"A friend of mine - who is one of the most consistent people I know - was called in over the weekend for an emergency patch deployment at work.\nThe weekday streak is over; long live the weekday streak!\n"},"notes/scratch/some1-entry":{"title":"Entries to SOME 1","links":["notes/stability-of-marriage"],"tags":["engineering/python","viz/doodles","economics/market-design","notes/scratch"],"content":"Awhile ago, the channel 3blue1brown announced an internship and public competition, Summer of Math Education - at the time, I submitted some half-assed entries that I’ve kept around since then.\nBachelor Analysis\n\nExcerpts from Stable Marriage\nLargely taken from a presentation on stable matching algorithms\n\n\n \nOptimal Stopping Problem\n"},"notes/scratch/the-best-side-of-the-road":{"title":"What is the best side of the road to drive on?","links":[],"tags":["notes/scratch"],"content":"\nThe best side of the road to drive on is the side everybody else has agreed to drive on.\n\nI don’t remember where I first heard this, but I think about it every day."},"notes/scratch/unstable-marriage":{"title":"Unstable Marriage","links":["notes/stability-of-marriage"],"tags":["data","economics/market-design","notes/scratch"],"content":"I’ve spent a whole lot of time researching matchmaking algorithms, and yet I’d never considered - didn’t even mention in the presentation - what would happen in odd-numbered sets.\nthe implications are frankly funnier than I would have imagined"},"notes/scratch/warning-about-slabtops":{"title":"Before You Slabtop Your Laptop: A Brief Warning","links":[],"tags":["projects/personal/homelab","notes/scratch"],"content":"I recently had a misadventure while trying to slabtop (remove the LCD panel of a laptop to use it as a server) an old Dell Alienware G2 15. I didn’t see this piece of friendly advice while Googling around before making my attempt. I really wish I had - it would’ve saved me a lot of time, trouble, and a motherboard to boot (pun intended).\nTo contribute to the body of all knowledge, I hope you (you, the person googling “how to slabtop a laptop” right now) see this before you whip out the screwdriver and go to town on your old burner:\nSearch “<YOUR LAPTOP MODEL> LCD Panel POST Check Beeps” before you attempt to slabtop your laptop!!\nComputers run without screens all the time - we call them servers, and they run everything, so the possibility of a laptop manufacturer literally not letting a computer complete POST check without an LCD panel wasn’t even something that crossed my mind. However, if you google the above, you’ll run into lots of posts of people who hear a handful of beeps when their LCD display is disconnected, signifying that the POST process didn’t complete. This will absolutely fuck you over if your laptop manufacturer is one of the many who have this check in place.\nWhile I believe it is possible to do this on a PC (not a Mac, where slabtopping has been documented extensively), I personally ended up frying the motherboard after about 5 hours of cutting away at the stupid fucking thing. If this helps even a single person, I will be immensely happy."},"notes/season-of-rhythm":{"title":"Spencer's Season of Rhythm","links":["notes/the-quest-to-slay-the-goblin","notes/over-under-engineering","tags/goblin-slaying"],"tags":["notes","goblin-slaying"],"content":"In the spirit of setting themes instead of resolutions, I’ve adopted a seasonal (or perhaps year-long - who knows?) theme for my own life:\nThe Season of 🎵 Rhythm! ✨\nDuring my Season of 🎵 Rhythm! ✨, I want to establish healthier cadences to how I spend my time day-to-day, and to be far more intentional with how I spend my time. This is largely a response to a broad problem in my life that I’ve felt for some time now.\nI am not a person who, at any point, has ever formed good, lasting, healthy habits.\nThrough high school, into college, and continuing into my adult life, I have always been a very “whichever way the wind blows” kind of personality. This hasn’t been entirely without its upsides. A coworker once described me as “very good to have in emergencies.” While I’d agree with that characterization, it’s because my tendency embrace The Goblin has meant I have been a co-conspirator to many of the emergencies in which I’ve learned to work well under pressure. If you do enough work, enough times, an hour before a deadline, you become very good at doing lots of work with hours before a deadline.\nThe downside to this is… fairly obvious. In the last year, I started working for and with a very emotionally healthy team of engineers who are remarkable at pacing themselves, doing diligent and level-headed work, and avoiding emergencies when possible - something that drove me absolutely insane at times during my first year. After having seen - and felt - how much healthier this environment is and how much more it can accomplish - particularly in collaborative and non-competitive settings - I’ve accepted it’s the way I’d ought to work, too.\nAs an additional downside to the way I worked, only having a cohesive and consistent pace of work meant that larger projects - things that aren’t done in one all-nighter but have to happen over the course of weeks and months - had a tendency to fall into obscurity. I’d attribute some of that to the way I have structured many of these projects, but constantly losing the thread between bursts of motivation means that bigger plans can easily turn into aimless wandering.\nSeason of Rhythm is a time to actively, rather than passively, establish good “rhythms” and routines that lower the variance of my own work and habits. The goal isn’t to shed the “works well in emergencies” characterization - on the face of it, I don’t think that’s a bad thing. However, where I’d describe the previous variance of my own ability to function personally and professionally as a 25-50% pace during “peace-time” and 200-400% pace during emergencies, I’d vastly prefer to even this out to something that doesn’t feel like personal and occupational whiplash: maybe 80% day-to-day and 120% during emergencies, as a vibes-y estimate.\nAfter two months, I’m on a really good track. A whole lot of goblin-slaying posts this year will probably be related to tools I’ve found that help me keep on track, and I’m excited to see how the trajectory of this next 4-10 months"},"notes/stability-of-marriage":{"title":"College Admissions and the Stability of Marriage","links":["notes/qamo","tags/viz/doodles","notes/scratch/some1-entry"],"tags":["engineering/python","economics/market-design","viz/doodles","notes"],"content":"In college, my area of focus during my economics degree was non-market environments, which encompass the areas of economics where money isn’t used for moral or practical reasons.\nA presentation I’m particularly proud of is a presentation on David Gale and Lloyd Shapley’s 1962 paper College Admissions and the Stability of Marriage\n\nThis presentation happened at a personally odd time during a globally odd year, and - due to the practical limitation of only having my iPad available while I was in-patient at the University of Utah - was the first time I started using doodles in the vast majority of my presentations.\nThere are a few things that were cut for the presentation for time - in particular, the incredibly interesting concept of matchmaking lattices. Eventually, I’d like to include some additional written resources here on the topic, because it’s pretty cool. One of the cut slides is available here.\nThe absolutely incredible Dr. Allison Stashko is the only reason this presentation was possible - she was incredibly kind and encouraging during a difficult period, and is in large part the reason why non-market environments are my primary interest in economics. "},"notes/strong-and-weak-opinions":{"title":"let opinion = null","links":["notes/strong-and-weak-opinions","tags/engineering/csharp","tags/games/roblox","assets/Pasted-image-20240305234056.png","tags/engineering/python","notes/scratch/the-best-side-of-the-road"],"tags":["engineering","notes","economics/project-management"],"content":"\nThis note ended up having a lot of front-loaded context — feel free to skip to the point\n\nOpinionated Software\nThere’s a term that floats around that feels new. It’s not something that I’d ever seen in the science portions of my undergraduate. As I’ve entered into industry and more frequently both use and participate in the development of a broader range of software, there has been a sharp increase in frequency that I’ve seen this term:\n\n[Proper noun] is an opinionated [general noun]\n\nWhat… does this mean? I’d never seen it in an academic slide deck before, but I can name a few high-profile instances of the label used. While it’s usually surrounding frameworks, I’ve seen it used in other places as well:\n\nDuring C# related work — I have previously used Wrapt as a csharp web API scaffolding tool, which describes its default options as opinionated\n\nOn wandering through the chain of opinions, Wrapt’s opinions are, in turn, based on the Vertical Slice Architecture (VSA) opinions I’d first seen in Jimmy Bogart’s post on VSA, a recommended must-read by my team at NWYC\n\n\nDuring roblox-related CGC work: “Flamework is a highly opinionated game framework”\n\nInterestingly, this description has since been replaced to say (as of writing) “Flamework is an extensible game framework.” — thought I was going insane in trailing back after seeing that phrase there months ago and not finding it, but a stale Google Index reference shows that I’m not crazy (on this one, at least). I am curious about the onus behind the wording change, because I do actually believe that the new description better captures the intent of the Flamework package.\nUPDATE: After talking to the creator of the package, this change was made as the project evolved from a personal single-user project to a broadly-used staple of Roblox/TypeScript development. In particular, the Modding API is what spurred the description change, as it allows an end-user to yoink very nifty behavior like dependency injection and decorators and attach it to custom utilities.\n\n\nStumbling on the black python formatter while wrangling Databricks codebases at M Science\n\n\nThis is probably the most severe(?) case of opinionated software I’ve ever seen — it is literally named after the Henry Ford quote regarding available paint jobs for the Model T: “Any color the customer wants, as long as it’s black”\n\n\nThe black repository is also home to one of my favorite GitHub Issues of all time, in which a senior staff engineer at Twitter requested the ability to configure 2-space tabs instead of 4-space tabs.\nThe response to one of the most influential tech companies to ever exist requesting a (seemingly) minor change in order to adopt it as the standard code formatter for their entire Python codebase was a resounding: “no.”\n\n\n\n\nAnybody who has ever seen highly-opinionated flame wars in the comments of Reddit threads, Stack Overflow posts, or — frankly — between any work call between more than, say, two engineers in the same field, opinions have the potential to run very strong.\n(Want a real life example? Go ahead and pop “is <PROGRAMMING LANGUAGE A> better than <PROGRAMMING LANGUAGE B>?” into your search engine. One of my strong opinions is that 99.9% of those conversations are bullshit, wasted time. Seeing them in the wild makes me want to close my laptop and walk into the ocean, my final thought before the air depletes from my lungs being that, at the very least, my decomposing body will contribute more back into The Universe than those people did by arguing complete fucking nonsense on the internet.)\nWhen strong opinions collide, it can feel very much like watching an unstoppable force hit an immovable object. Especially when I was a data analyst on the way up to software engineering, I often found myself a junior (and rightfully silent) participant in long-winded, fruitless, and ultimately destructive conversations, in which two parties who hold strong opinions talk past each other for… 10 minutes… 20 minutes… an hour? with the call ultimately ending because one side or the other has to hop into a different call of the same flavor.\nI think the way we can all occasionally approach having an opinion — or not having an opinion — is ultimately reductive, destructive. It slowly chisels away, bit by bit, hundreds of thousands of minutes, hours, and days of human life to the behest of our goals yet delight of the Reaper himself.\nThe Point\nI’m proposing, here, a way to:\na. properly classify your opinions; and\nb. how to use those classifications to lead legitimately constructive conversations with others\nLately, I’ve been trying to put my own opinions into one of two categories:\n\nStrong Opinions: Opinions that I have built up over time and experience, and that I feel I have a compelling case for that clearly communicates the full scope of costs and benefits to the decisions that opinion leads me to propose\nWeak Opinions: These are the ones that I have that are built on sub-par information, that I can’t clearly communicate, or that are generally based on ✨ vibes 🌈\n\nWhen contributing to a conversation or argument on a topic, I’ve been trying really hard to ask myself: “Is the opinion I’m about to state a strong opinion?”\nHaving a weak opinion on a topic doesn’t necessarily mean it shouldn’t be mentioned at all. It does mean that it’s likely a point that, should somebody else have a stronger opinion, that opinion isn’t a hill I should consider dying on.\nThat said, having a strong opinion also doesn’t necessarily mean it’s something I ought to be doubling (or, god forbid, tripling down on). Especially when in a conversation with more senior engineers — which, at the current stage of my career, is vastly more likely — a strong opinion is one in which I ought to present my best case, rationale, and analysis in the hopes of reaching consensus.\nEven if my case for one of my strong opinions fails to reach a consensus — or even convince anybody, which does happen (sometimes a lot) (most of the time) — that isn’t failure. Ultimately, as I’ve gotten further into my career and seen a broader range of opinions reach a broader range of success or shortcoming, my approach has ultimately fallen back to:\nThe best side of the road to drive on is the side everybody else has agreed to drive on.\nNo matter how strongly you might feel one way or another, never underestimate the power of consensus. Either way, the outcome of accepting consensus is positive:\n\nYour opinion ended up being the way to go in hindsight, and you’ve gained experience to make a better case the next time it comes up; or\nThe consensus opinion goes perfectly fine, you achieve your goals, and have gained experience on which you can build new, better-informed opinions.\n\nA final open letter\nI’ve noticed, in working with others, that the above classifications can somewhat freak people out — especially if they’re used to working with Strong Opinion People™, because how quickly I’m willing to roll over on my weak(er) opinions has the potential to come across as misunderstanding or apathy.\nAs an open letter to people in the future that see me do this, I make you this promise: when I quickly fold on an opinion, it is far more likely that — in that moment — I feel there’s more value in reaching a consensus and moving forward with any opinion than trying to force through a bad argument for a strong argument or a null argument for a weak one. At this point in my life, I am more interested in making progress than being right."},"notes/supply-chain":{"title":"From the Archive: \"What *are* Supply Chains, Anyway?\"","links":["tags/viz/","notes/caveat-lector","notes/qamo","contact"],"tags":["economics/finance","engineering/python","viz","notes"],"content":"Below, you’ll find some graphs. These are what is known in the field of data visualization as “bad.”\nThese graphs are (as far as I can recall) my first attempts at using & visualizing network structures. As much as I would like to say they’re not too bad for a first attempt, they don’t “speak for themselves” as much as they “scream torments in the face of the viewer.”\nThe below graphs are stock tickers, connected to one another by their supplier relationships. For example, if Company A publicly noted that they received supplies from Company B, that would be denoted by a connecting edge.\n\nBelow is my first (and only) attempt to organize the above output. If I’m recalling correctly (and this was in 2019, so caveat lector), this was at least an attempt to get the first-order to third-order suppliers of a given set of equities and place their order in shells around the center. It’s messy, it’s noisy, and frankly, incomprehensible.\n\nThe hope of this analysis was to ultimately connect supply chains and, as a secondary goal, find graph cycles among a set of target stocks to find and catalog “supply loops”, and see if stocks in a given supply loop rise and fall together. The visualization acted more as a sanity check to see if my web scraper was working and less of a rigorous tool for understanding.\nDespite my area of focus being the parts of economics that aren’t related to either studying (or making) money, this is something I’d like to return to again in the future. However, I unfortunately don’t have the same level of data access that I did as an undergraduate, and purchasing access to a Bloomberg terminal falls well outside the financial means of CGC. However, if you’re ever looking for somebody to make bad graphs pro-bono and do have access to that kind of data, please feel free to reach out."},"notes/the-quest-to-slay-the-goblin":{"title":"The Quest to Slay The Goblin","links":["tags/goblin-slaying","notes/caveat-lector"],"tags":["goblin-slaying","notes"],"content":"I am terrified by how quickly, without structure and habits, my life devolves into a cycle of sleeping, eating, procrastinating, and repeating until the visage of death sweeps me from this mortal coil.\nThis cycle is called “The Goblin”\nI hate The Goblin. The Goblin makes me sad. The Goblin stops me from doing the things that make me happy. Every day, I have to actively re-remember how much I hate The Goblin, and how much happier I am when it’s gone, so I try my best to kill The Goblin.\nI also hate the word “productivity.” There’s a sort of LinkedIn-centric slice of society that has built up the neutrality as an honest days’ work as a divinely-capitalized entity: Productivity™. It elicits (in me, personally) the visceral feeling I get when I see hustle-culture, rise-and-grind, success-is-a-choice-and-failure-is-your-fault types - the ones that farm for engagement by repeating gross approximations of quotes from self-help books that have been Human Centipede’d from one hustler to another. \nThat being said - productivity tools are some of my most reliable Goblin-slaying weapons. This leaves me feeling very conflicted. Absent the weird Soylent-chugging culture that has formed around capital-P Productivity™, I would be in (and have been to) a really rough place without the productivity tools and habits I’ve tried my best to use and form in my quest to kill The Goblin.\nAs a transparent act of shoving this internal inconsistency into the box in the back of my skull labelled “Don’t Think About It!”, I’ll be using the goblin-slaying tag to denote posts about productivity and productivity tools.\nIn these posts, I’ll talk a bit about the structures I put up in my own life, and the tools I use to enforce those structures. To some degree, goblin-slaying posts exist because I hope it might help another person who has their own Goblin to slay and is looking for new tools to slay it. To a far greater degree, they’re here because the act of writing it down helps me set these structures in stone for myself. It is surprisingly easy to forget about good habits when, some day in the future, they’re no longer the habits they used to be.\nCaveat lector - you are you, and I am me. Some of these posts may resonate with you, and some of them may not, because different people are different."},"notes/time-tracking":{"title":"Some initial thoughts about time tracking","links":["notes/season-of-rhythm","tags/goblin-slaying"],"tags":["goblin-slaying","notes"],"content":"Time-tracking is one of the tools I’ve picked up during my Year of Rhythm and goblin-slaying. I’ll probably write about it more, here, later."},"notes/utah-office-consult":{"title":"From the Archive: \"Utah Office Consult\"","links":["notes/scratch/the-best-side-of-the-road"],"tags":["engineering/python","economics/strategy","data/analytics","viz","notes","consulting"],"content":"\nThis presentation is one that I did for an organization I was a part of during college. Looking back, this is probably the first time this phrase clicked in my brain:\n\nPeople problems are the hardest engineering problems\n\nFrom the first time that phrase entered into my head, it has ricocheted against the inside of my skull every single day. In computer science, problems are relatively easy to solve because the only thing you have to convince is a compiler. I am not (nor would I ever claim to be) a sociologist, but nonetheless feel safe in stating that people are not compilers. People have opinions. People have motivations. To work together, to have systems, to make any progress as a society, people have to agree on things.\nNever forget - the best side of the road to drive on is the side everybody else has agreed to drive on. When it comes to people problems, consensus is king.\nI’d imagine that the vast majority of you are people, like me, whose professional reputations to not precede you. As fellow members of the proletariat who fall outside of the LinkedIn-influencer and TED-talk alumni elite, we don’t have a whole lot of ethos on which to make our arguments for why people should agree with an idea. I don’t think that’s a huge deal (although golly gee, wouldn’t it be nice to have authority on matters?). It does mean, though, that turning an idea into consensus has to rely on logic and vibes.\nTo that end, I do think that using what I’d describe as “soft” quantitative frameworks for largely qualitative (read: vibes) properties of things is a good way to bring pathos and logos together. In the above presentation, using distance from other locations is not necessarily a catch-all description for the causes of many of the above problems, but it was a good rule-of-thumb."},"tags/annotations":{"title":"Reading Annotations","links":[],"tags":[],"content":""},"tags/articles":{"title":"CGC Articles","links":["tags/notes/"],"tags":[],"content":"Here, you’ll find some actual, bona-fide Articles™. You can expect a higher degree of cohesion, revision, and quality than you may find in, say, notes."},"tags/business/admin":{"title":"CGC Business Admin","links":[],"tags":[],"content":""},"tags/economics/finance":{"title":"Economics & Finance","links":[],"tags":[],"content":""},"tags/economics/index":{"title":"Economics","links":[],"tags":[],"content":"Economics is the core research focus for Chaotic Good Computing. Wherever there is a finite amount of resources and an infinite capacity for demand, an economy forms.\nCGC’s focus is analysis & creation of well-balanced and intentionally-designed economies, particularly in digital spaces. Whether it’s players interacting with game systems or each other, the way you interact with your cloud provider, or even the way you work online at your company, well-designed systems are hard to understand and even harder to perfect."},"tags/economics/market-design":{"title":"Market Design & Non-Market Environments","links":[],"tags":[],"content":""},"tags/economics/project-management":{"title":"Project Management","links":[],"tags":[],"content":""},"tags/economics/strategy":{"title":"Economics & Strategy","links":[],"tags":[],"content":""},"tags/engineering/csharp":{"title":"C\\#","links":[],"tags":[],"content":""},"tags/engineering/index":{"title":"engineering","links":[],"tags":[],"content":""},"tags/engineering/python":{"title":"Python","links":[],"tags":[],"content":""},"tags/games/roblox":{"title":"Roblox","links":[],"tags":[],"content":""},"tags/goblin-slaying":{"title":"Goblin Slaying (a.k.a Productivity)","links":["notes/the-quest-to-slay-the-goblin"],"tags":[],"content":"Goblin slaying is a somewhat underhanded term for productivity habits and tools."},"tags/horticulture":{"title":"Digital Gardening, and Horticulture as Research","links":["notes/digital-gardening-with-quartz"],"tags":[],"content":"As a logical consequence of taking a digital gardening approach to writing, the horticulture tag will be used to catalog thoughts related to digital gardening and general research.\nI also wanted to use horticulture because I think it’s a funny word."},"tags/mentoring/index":{"title":"CGC Mentoring","links":[],"tags":[],"content":""},"tags/mentoring/tutorials":{"title":"Tutorials","links":[],"tags":[],"content":""},"tags/notes/daily":{"title":"Daily Notes","links":["notes/caveat-lector","notes/digital-gardening-with-quartz"],"tags":[],"content":"These are daily notes, where I log and track progress on ongoing projects and make notes for myself later. These are largely for my own benefit, and are here in case they’re helpful to myself (or, somehow, others) in the future.\nHowever, I’d describe these as loose, at best. Caveat lector.\nOne of the approaches to digital gardening is “learning in public” - to that end, I plan on continuing to outwardly publish these."},"tags/notes/index":{"title":"Spencer's Notes","links":["tags/articles","notes/daily","notes/scratch"],"tags":[],"content":"These are notes - they’re things that are structured, albeit not necessarily as structured or refined as CGC articles.\nIn general, there are three kinds of notes I make:\n\nGeneral notes, which are found on this page\nDaily Notes, which contain daily logs for my own benefit\nScratch Notes, which really just means “none of the above”\n"},"tags/notes/scratch":{"title":"Scratch Notes","links":["tags/horticulture"],"tags":[],"content":"For all intents and purposes, these are the digital gardening equivalent of shitposts and shower thoughts."},"tags/projects/cgc":{"title":"CGC Projects","links":[],"tags":[],"content":""},"tags/projects/dayjob":{"title":"Day Job Projects","links":[],"tags":[],"content":"For obvious reasons, anything related to these projects will be very vague."},"tags/projects/index":{"title":"Projects","links":[],"tags":[],"content":""},"tags/projects/personal":{"title":"Personal Projects","links":[],"tags":[],"content":""},"tags/viz/doodles":{"title":"Doodles, a.k.a. Minimally Professional Presentations","links":[],"tags":[],"content":""},"tags/viz/index":{"title":"Data Visualizations & Presentations","links":[],"tags":[],"content":""}}
\ No newline at end of file
+{"articles/binglish":{"title":"Binglish: The New Normal","links":["articles/structured-streaming","articles/robloxaville","notes/digital-gardening-with-quartz","notes/the-quest-to-slay-the-goblin"],"tags":["economics","horticulture","articles"],"content":"Binglish\n\nbing·lish, noun (derogatory)\n\nThe default writing style of generative AI text models. (derogatory)\nShorthand for a median style of writing, devoid of personality. (derogatory)\nThe way that a monkey’s paw would grant Microsoft’s wish to finally give Bing a generic English term like lowercase-g “google” (very derogatory)\n\n\nIn writing a bit more, I’ve been back-and-forth on the extent to which I want to swear. The style of my writing is more-or-less the way that I speak out loud (which prompted a lot of “couldn’t you make this a bit more professional?” feedback on the Databricks post), and when I speak out loud, I swear. Obviously, the frequency depends - at work it’s sparse (but not nil), on a personal level it’s pretty frequent - but I do still pepper it in for effect in some places. The only times I can remember not swearing is when I was teaching math to K-12 students, and when writing posts here.\nThis feels somewhat… internally dissonant? personally disingenuous? If, when I speak I swear, and I tend to write like I speak, why wouldn’t I also swear in my writing? The answer in the past has been professionalism. It feels unbecoming of a (relatively) young professional to swear in a professional context, and some of what I write here overlaps with what I do for work.\nToning down my writing has been, so far, a fairly active effort. I’ve occasionally even gone as far as writing with swears and then, after-the-fact, explicitly (ha!) searching for illicit words and removing them as I see them. This was probably most apparent when writing the Robloxaville recap article - it mentions a lot of my (very) early work history and, at the time, I thought that potential employers (or youths) might come across it, warranting a touch-up.\nThe world has changed quite a bit since ye olde 2021. I am (slightly) more established in my career, and the context of written documentation has changed quite a bit as technology has advanced. This has made me reconsider my previous policy of toning down my writing. In the process of porting over & more frequently updating this blog, I’ve made an executive decision for the foreseeable future: I’m going to be swearing a bit more.\nOr, at the very least, I’m going to be taking a dramatically less formal approach to writing and editing, here. The spur of this decision:\nThe Bots Are Coming Have Arrived\nI don’t think that, ten years ago, anybody expected that the next dramatic economic shift would’ve been generative AI - specifically text-based AI - putting white-collar workers at the highest risk of automation. An example that comes readily to mind is the CGPGrey video Humans Need Not Apply, in which self-driving cars were offered up as the most readily-apparent example of economic risk of automation. This feels slightly silly with the benefit of hindsight - having started my engineering studies at around the same time though, it’s easy to forget how high-profile advances in self-driving were (at the time). It has become wildly, obviously apparent that the current frontier of automation has come, first, in the form of automated writing.\nThis is obviously a concern for many reasons. The internet is made up of a whole lot of writing. It’s how we text, how we tweet, how we email. It’s what we read, how we get our news, how we ask questions and how we find our answers. Before now, we could safely assume that something we read on the internet came from, at the very least, another human being.\nThat’s not the case anymore, and it will likely not be the case ever again. I don’t think this is the end of the world (probably. hopefully). I do think that this is the end of what I refer to as the statistical median of language: boring, sanitized, business-speak.\nFor the purposes of this note, I am going to refer to this type of language as Binglish.\nBinglish is a Dead-On-Arrival Language\nBinglish, to me, can be described in a few ways:\n\nIt’s Business English. Office English. LinkedIn™ English. I hope, to those of you reading who have also worked a traditional office job, that you can recognize this shorthand as the type of thing that you’d find in your Outlook inbox;\nIt’s Boring English. It’s the median of the language, absent of all personality or human voice; and\nIt’s Bing English. Meant to half-heartedly celebrate Microsoft’s role in solidifying what is very likely a self-consuming business model , poised to consume the very medium it was trained on, this is the type of English that we (currently) see from Bing Copilot, ChatGPT, Gemini, and others\n\nAbsent explicit directives on tone, you can find examples of Binglish on pretty much every single chatbot you could open at the moment. It’s informative. It’s to the point. Once you get the hang of it, it’s easy to both read and write. What’s not to love?\nAnd - unfortunately for the mediums that rely on Binglish - it’s wildly automatable, now.\nIn an effort to formalize the English language into a common median, we have all but guaranteed that the data used to train LLMs has, by definition and in practice, mastered it and put all of us that use it at risk of automation. We can also see the remnants of human efforts to curtail widely-used bots from going outside Binglish at their default settings:\n\nChatGPT refuses to generate even a generic list of English swears…\n\n…as does Raycast’s AI feature, which uses either GPT-Turbo-3.5 or GPT-4…\n\n…as well as Github Copilot, which makes a bit more sense…\n\n…as well as Bing Copilot, even with creative settings, which makes less sense.\nThe takeaway, to me, seems as clear as day.\nHumans Need Not Write Binglish\nThe market for Binglish is cornered. It’s been automated away. The need for writing in the median, devoid of voice or personality, has gone the way of horses in a world of cars. The onus for humans, then, is not to stop writing, but instead to write outside of the Binglish median.\nOne of my favorite examples of people who write with an inimitable voice is Kyla Scanlon, who frequently writes and records thoughts about macro-economics. What makes her writing engaging is not just thoughtful analysis of economic trends, but the voice in which she writes it. It was specifically her post on language and trust that spurred this note to turn into the length of an article.\nWhile I feel like the market for dry, median writing was already withered in the wake of a rising palette preference for infotainment, it was the the advent of ChatGPT and widely-available generative text that has thoroughly crushed the future prospects of Binglish as a human activity. While I can make no guarantees that generative AI won’t someday master imitation of human voice in writing - something that has already reared its head in generative images and spurred active efforts to curtail imitation and preserve unique styles), I have made a decision for my personal writing:\nI don’t want to write Binglish anymore.\nI find no joy in editing the way I write into a now-dead, median form of language, optimized for algorithms with its supply automated away by language models. I’d rather write in my own voice, with my own anachronisms and mistakes and weird vocal ticks and stray punctuation and weirdly-placed emoticons and deeply-nested parentheticals and… run-on sentences? Probably quite a few run-on sentences.\nAs far as I’m concerned, my responsibility is to ensure that the text is readable and that the points I want to make come across, even if it the occasional sentence requires a re-read.\nAnd if it’s not search-engine optimized, or tonally non-neutral, or long-winded? if it contains foul language?\n¯\\(ツ)/¯ fuck it."},"articles/hello-blog":{"title":"Hello, Blog!","links":[],"tags":["data","engineering/python","articles"],"content":"Look at that - a whole blog, just for me! At this rate, the last thing I need to do to be a real tech guru is start a podcast.\nI’m starting this page as a place to talk about my cool projects, dumb thoughts, and other odds-and-ends aspects about my life and work.\nTo set the tone, here’s a quick code function to find the geomethic meandian of a set of numbers in Python - in case the need ever arises.\n\nimport numpy as np\n \ndef geothmetic_meandian(\n numbers: np.array,\n error: float = 10e-5\n):\n '''Find the geothmetic meandian of a set of numbers.'''\n \n # If all the numbers are close to the same, return the first number\n if sum(np.abs(numbers - numbers[0])) < error:\n return numbers[0]\n else:\n # Recursively call the geomethic meandian until the numbers converge\n return geothmetic_meandian(np.array([\n np.average(numbers),\n np.product(np.power(numbers, 1 / len(numbers))),\n np.median(numbers)\n ]), error)\n>>> geothmetic_meandian([1, 1, 2, 3, 5])\n2.089\n\nVoila!"},"articles/littlefield":{"title":"Zen, and the Art of Laziness","links":["notes/season-of-rhythm","notes/qamo","notes/stability-of-marriage","assets/clutter.png"],"tags":["data","games","engineering/python","viz","economics/market-design","articles"],"content":"I am awful at sticking to routines - the only thing I can bear to check on every day is Twitter. Even if it means spending twice as long over the course of a weekend, I will always try to find a way to automate away routine manual work.\n(All of the code used in this post can be found here.)\nToo much time to kill\nThis Monday, I started the last major semester of my undergraduate career. That’s right, folks - after six years, four program changes, and one brain surgery (which, in my personal opinion, is one too many), I will finally be wrapping up my degree!\nMy last two semesters will be my first part-time courseloads of my academic career, which leaves me quite a bit of extra time. Given that there’s not much to do during the quarantime, I took a bit of extra time to go over the top in a class I’m taking on operations and supply chain management.\nIntroducing: Littlefield\nAs part of the course, we’ll be spending four weeks in the operations simulation Littlefield. The premise of Littlefield is to tune parameters - things like orders taken and machines purchased - to optimize the profit of a simulated production line. The simulation itself runs a bit faster than real time. Over the course of five full calendar days, one session of the game simulates 250 days of work.\nIn the words of the course instructors:\n\nthe game is set up to give you the flexibility of checking it at your convenience (at any time of day or night)\n\nThe problem? I’m a walking dumpster fire when it comes to routinely checking on anything.\nMy email inbox is constantly full. I am always running on a quarter tank of gas. I’ve never even kept a house plant alive for more than a week - I’m responsible for the early death of more succulents than I would ever care to admit.\nIn the honest-to-God words that my fourth grade teacher wrote on my report card, I am somebody who is “lazy to the point of efficiency.” If there is any way for me to put in all of the work up front, I will find a way.\nWeb Scraping 101\nAs somebody who has been lazy for 23 years now, I’ve become well-acquainted with the concept of web scraping. This is a method for accessing websites using a program instead of your browser. If your job ever involves going to a massive number of web pages and finding specific information - as a few of my internships have entailed from time to time - it’s a fantastic way to avoid the grunt work of going to each individual page by hand. For example, rather than employing an army of people to go through every web page on the internet and keep track of keywords, Google instead uses an army of web scrapers to do all that work for them.\nLittlefield, like the rest of everything on the internet, is scrape-able as well. The game is hosted on a web page that looks like this:\n\nWhile the page is different - and a bit harder to scrape - than your average news article or social media page, it’s still just a set of links that we can get information from. Each node of the graph above leads to a page full of data about the game. If were playing this by hand, we would log in a few times each day, open up each page, and see if there were any issues or bottlenecks in need of fixing.\nIt takes about five minutes to check everything - five exhausting minutes.\nInstead of taking five minutes to check this data myself, I have - as the rational person I am - spent the last six hours creating a program that can check it for me: the Littlefield Driver.\nThe Littlefield Driver\nThe Littlefield Driver is a program that takes all of the data about the state of the game - things like machine utilization, cash on hand, and orders placed - and compiles it into one data table I can look at, graph, and use to diagnose any issues that may be the case. The program is based on a 2017 project by Greg Lewis, who was (at the time) an MBA student at the University of Maryland. Lewis wasn’t alone - developer Jiayu Yi created an export kit for Littlefield back in 2017 as well.\nIdeally, my Littlefield Driver will have a bit of extra functionality beyond data export, but the spirit of automation is still the same. By the end of the project, the goal will be to let the Littlefield Driver use incoming data to automate machine production in the background, and maybe even send me a notification if (or when) something goes horribly wrong.\nThe first order of business for the bot is to take the structure of the game and represent it as a network graph.\nHere’s what the driver sees when it opens up the game:\n>>> lfdriver.draw_graph()\n\n\nIt’s… admittedly not very pretty. At the very least, though, it shows that the program is seeing what we’re seeing: the production line starts at the Orders & Material menus, which goes to the first queue, which feeds into the first station, et cetera.\nHTML is the least romantic language\nWhen a web page is sent to your computer for your viewing pleasure, it’s sent as Hypertext Markup Language, or HTML for short. Normally, we don’t have to deal with HTML directly. The entire point of your web browser is to read the HTML and turn it into something clean, pretty, and - most importantly - human-readable. Scraping the web, however, means that we do need to read the HTML directly in order to pick out the parts we want the computer to look for.\nOur target for the first round of web scraping is going to be the Littlefield data pages. These data pages give you the historical plot of whatever the selected node of the graph represents. For example, this is the data page for how many incoming job requests are received every in-game day:\n\n(as an aside, every time you open a data view, it annoyingly opens as an entirely new window, which means playing Littlefield often looks like this)\nI can always appreciate a clean-looking graph.\nTo make that graph, it’s actually your web browser doing all the work to turn the HTML into a neat visualization. This is what your computer is actually sent:\n\nEven when you’re making a website, reading through raw HTML is rarely ever a fun time. Thankfully, the data we’re looking for is on line 29, smack-dab in the middle of the page:\n{label: 'data', points: '1 1 2 1 3 2 4 ...}\n\nThat’s the data that your browser is using to create the plot from the first image. It’s a bit weird that it’s all one line, but we can pick up from context that the data is being provided in pairs - the first number is the day, then the next is the datapoint - repeated over and over again. You could read it as:\nDay 1: 1 job\nDay 2: 1 job\nDay 3: 2 jobs\nDay 4: ...\n\nThankfully, the data is delivered in the same way for every single data page, from the orders received to the utilization of each station. All we have to do is write the code once, give the program all the data pages, and let it run. Instead of one string of data, we can tell it to put the data in nice, neat human-readable columns like this:\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nDayJOBIN112132425363\nAfter scraping every single page of data in the game, we can smash it all together and get a beautiful chart!\n>>> lfdriver.data()\n\n\nThis chart just shows the first and last 10 days of the historical data, and even from this we can gain some insights. S3UTIL, which represents the current utilization of the third station, is at 67% on Day 50 of the simulation. Judging by the rate of increase - roughly 1.3% every day on average - we should probably consider buying another machine at Station 3 prior to Day 75.\nOf course, that’s just an estimate. The purpose of the Driver is to check in on the game routinely to see if there are any critical issues. For example, the Driver can check to see if a station utilization is at a high percent and calculate whether or not it would be profitable to order a new machine.\nEverything is done - for now\nAt the time of writing this, I haven’t actually played a game of Littlefield - everything written so far was done on a static sample game. This means I haven’t had access to some features - like purchasing machines - so they haven’t been added to the Littlefield Driver… yet. Additionally, there’s some system variables - such as the cost of a new machine - that are available in the data but aren’t available in the Driver.\nUltimately, this is a god-awful way to play the game. With 12 years in development and only 2 examples (that I’ve found) of people attempting to automate the game, the game is clearly playable by hand. You might ask, then - why put in all this effort?\nThe real question is: why not?"},"articles/resume-ci-pipeline":{"title":"Job Applications are a Waking Nightmare - Let Tech Do It For You","links":[],"tags":["mentoring/tutorials","engineering/devops","articles"],"content":"\nNOTE: THIS IS OUTDATED. There are now far better PDF markup languages to use, such as Typst. These will be detailed in a later blog post.\n\nAfter trying - and failing - to constantly keep my resume up-to-date in a dozen different places, I took a page out of the wild world of software development and found the solution: continuous integration.\nIt used to be the case that software would release in massive updates. A bunch of big changes, tweaks, and bug fixes would be shoved together in a tidy package and delivered out all at once. As development became faster, though, development teams created continuous integration pipelines to constantly push changes out to their users. Rather than waiting until the next big change to release patches, small changes could be pushed out automatically.\nThis is common in web applications like Facebook or Twitter. You’ll never see Twitter come out with some big new Twitter 2.0 (Two-tter) - any changes they need to make are just pushed out to users in small batches, over time. Why not do the same with a resume?\nCreating an Always-Updated Resume\nCreating a continuously-updating resume can be done in two steps:\n\nSet up your resume so that you can make small, quick, iterative changes over time; and\nFind a central place to send those changes to, and send viewers to that location.\n\nI’m sure there are quite a few ways to run those two stages. I personally chose Overleaf and GitHub. At this point, I had already been using Overleaf to format my resume and had been using GitHub for small odds-and-ends projects for the first couple years of my undergrad. If we’re being honest, I didn’t want to learn new things - I just wanted it to work.\nSometimes being lazy pays off. Not usually, but sometimes.\nIt also helps that Overleaf has a neat built-in GitHub integration. How nice of them to do that! With this integration, the steps for the project start to come together:\n\nCreate a resume on Overleaf\nPush your resume to GitHub\nCompile your resume into a .pdf file\n\nAlright - let’s do it!\nCreating a Resume in Overleaf\nFor the uninitiated, Overleaf is an application commonly used in academia to write academic papers in a programming language called LaTeX. If you’ve ever used HTML to create a website, LaTeX does the same thing for PDFs. Thankfully, to get started, you don’t have to create a new resume from scratch. Overleaf has a ton of sleek-looking resume templates that you can put your current information into. If you’re looking for a friendly resume-making guide, I made a starter template with explanations on a few best resume practices for students.\nOnce you have a template picked, you can select “Open as Template” to add it to your personal Overleaf projects. You can start putting your information in at your own pace, but it isn’t necessary to start connecting your resume to GitHub.\nLinking Overleaf and GitHub\nIf you don’t have a GitHub account, now is the time to make a new one! GitHub is a fantastic place to store and update code - it’s literally what it was made for! Because your new resume (and the LaTeX it’s made from) are very small, it’s an ideal place to host your resume.\nOn your Overleaf resume, you can find the GitHub integration in the menu on the top-left of the code editing window. After signing into your GitHub account, you’ll have the option to create a new code repository for your project. Choose a name for your repository, enter a description if you’d like, and then select the option to make your repository public.\nWhile you can name your code repository anything, I would recommend sending this to your user’s GitHub Pages repository. Every GitHub user gets a special place where they can host a website or files publicly at <YOUR USERNAME>.github.io. If that type of address looks familiar, it’s because I host this blog on my own GitHub Pages site!\nIn order to use Github Pages, you’ll need to name your new repository <YOUR USERNAME>.github.io. For example, because my GitHub username is spelkington, my GitHub Pages repository is named spelkington.github.io.\nFinally, with your repository made, click the button to push Overleaf changes to GitHub. Now it’s time to make your resume update automatically!\nSetting up Continuous Integration\nGitHub has a nifty feature called Github Actions that development teams can use to automate some of the pain-in-the-ass aspects of programming. Typically this involves automatically testing software for glitches or compiling code, but for our case we’re going to use it to automatically turn our resume code into a .pdf file.\nThankfully, we don’t have to come up with a way to do this ourselves! Xu Cheng has created a nifty GitHub Action that will compile LaTeX files into PDFs.\nGo to your new repository on GitHub and click on the Actions button at the top. From there, you can create a new action.\nCreating a new action should take you to a code editor where you can create an Action configuration from scratch. Remove everything and place this code below:\nname: Build LaTeX document\n \n# Run this action every time the resume .tex is updated\non:\n push:\n paths:\n - '<RESUME FILE NAME>.tex'\n \njobs:\n build_latex:\n runs-on: ubuntu-latest\n steps:\n \n # Check out the repository on the Action machine\n - name: Set up Git repository\n uses: actions/checkout@v2\n \n # Compile the LaTeX document into a .pdf\n - name: Compile LaTeX document\n uses: xu-cheng/latex-action@v2\n with:\n root_file: resume/<RESUME FILE NAME>.tex\n \n # Commit this change as an automated user\n - name: Commit Change\n uses: EndBug/add-and-commit@v7.0.0\n with:\n author_name: <YOUR NAME> (auto)\n author_email: <YOUR EMAIL>\n message: 'TeX compile'\n \n # Push the change to the repository\n - name: Upload PDF\n uses: ad-m/github-push-action@master\n with:\n github_token: $\\{{ secrets.GITHUB_TOKEN }}\nReplace everything in the < > brackets, press the “Start Commit” button at the top right, and you’re all set! From now on, whenever you make changes to your resume in Overleaf, you can navigate back to the GitHub integration menu, push your change, and it’ll automatically send the code to GitHub and turn it into a .PDF!\nSending Your Resume\nOf course, the point of all this is be able to have a single link to your resume that will stay up-to-date forever. So how do we get this link?\nIf you followed the instructions for setting up the resume on your GitHub pages repository, it’s easy! Your resume will be available at <YOUR GITHUB USERNAME>.github.io/<YOUR RESUME FILE>.pdf. For example, mine is available at spelkington.github.io/Elkington_Resume.pdf."},"articles/robloxaville":{"title":"The Unbearable Weight of ROBLOX Celebrity","links":["notes/robloxaville-remaster"],"tags":["games/roblox","engineering/lua","articles"],"content":"Guilty confession: I’ve had a ROBLOX account for 14 years. At 23 years old, that’s more than half my life. After picking up development again in 2021, I’ve had quite the walk through memory lane.\nBeing a dork is a lifetime commitment\nFlash back to 2007: Spencer Elkington, 4th grade extraordinaire. Oh, to be at my peak again. While I appreciate that secondary schools mandate that all kids have a well-rounded curriculum to introduce them to a wide span of different fields of study, I can safely say that one of my least favorite classes was art. While I didn’t not enjoy spending an hour cutting, painting, gluing, and molding, there are only so many burnt ceramics and skewed portraits I could churn out before they - and my parents to whom that “art” was gifted - realized that perhaps a budding patronage in the fine arts wasn’t going to be in the cards for me.\nRelated point: Being a dork might be genetic\nAround that same time, I made an account on ROBLOX, an online game platform that was just a year old in 2007. Founded originally as a physics engine by mechanical engineers, it transitioned into a gaming platform after (I can only assume) the founders decided simulating viable buildings was far less fun than blowing them up. One of the coolest features was ROBLOX Studio, an IDE for developing user-made games and content in Lua. If you could play a game, you could make a game, and that ease of entry introduced a whole generation - myself included - to the wild world of programming.\nCooking up small games for me and my friends is probably the closest I’ll ever get being an artist. I could never paint a canvas worth a damn, but in an increasingly digital world being able to write a bit of code lets you create as much as a paintbrush ever could. Always looking for ways to hang out with her son, my mom - who is and always will be the biggest, coolest dork I have ever met and will ever will meet - joined the platform in 2008 to make games too.\nWith a background in coding from her physics, geology, and chemistry studies at BYU during her undergraduate (because just studying one STEM field would be too easy, obviously), she was an artist of her own right when it came to making games her kids could play. Over time, one of those games developed into a project called Robloxaville that she actively developed on from 2008 to 2012. The game became one of the most played games on ROBLOX during that period, later superceded by a follow-on project called Lakeside that became one of the fastest-growing projects made on the platform within two months of its’ initial release.\n\nOriginal Robloxaville thumbnail, 2008\nThis success was in no small part due to a feature she developed in 2009 called Morph Magic, a wildly clever, wildly hack-y workaround she discovered to modify the underlying skeleton of the ROBLOX player model and transform players into fully-animated dogs, cats, giants, or any of a dozen other models. The ad-hoc rigging method originally used in Morph Magic was the only method for modifying the player skeleton until ROBLOX added official rigging support - six years later. Watching millions of people enjoyed my mom’s masterpiece is what made me get into coding - first as a play tester, then as a (very amateur) contributor, and finally remastering the project in 2017.\nLike I said: the biggest, coolest dork I have ever met and will ever meet.\nThe wild ways software can go horribly, horribly wrong\nIn 2013, my mom stopped developing on the platform. It’s not that she didn’t enjoy it - as a general rule, nobody in their right mind writes clever workaround libraries in their free time if they don’t enjoy it. What stopped her from developing on the platform was years of rampant client-side exploits made possible by some questionable decisions made in the development of the ROBLOX game engine. For the uninitiated, here’s what went wrong:\nWhenever you use a device connected to the internet - your phone, laptop, tablet, desktop, etc. - it is communicating with tons of other computers called servers that belong to whatever service you happen to be using. When you’re browsing the web, it’s communicating with web servers. When you’re sending an email, it’s communicating with email servers. And when you’re playing ROBLOX, it’s communicating with ROBLOX servers.\nThese servers are responsible for running all the code that makes the game work. It diligently reads the Lua code that makes up a ROBLOX game, line by line, and changes the state of the game to whatever its’ told. If you tell the server to game.Workspace.Player.Head:Remove(), it will gleefully read that line of code and promptly remove the head of some unsuspecting victim.\nThat being said, sometimes the server needs a bit of help. Specifically, the server will occasionally rely on the game clients - your computer, and the computers of every other player in the game - to help it run some of the physics calculations. The millions of physics calculations run by the game can be a real pain in the ass for one computer, so the game server needs all the help it can get. Because of that, the client will occasionally send some information to the server to help the server know what the state of the game should be. Innocent enough, right?\nIt wasn’t always the case that the server relied on the clients for just physics calculations. Back in the early days of the ROBLOX engine, the client could send the server any information about the state of the game and the server would happily accept it as a gospel truth, even if it wasn’t in the game code. Here’s how the conversations between the server and client used to go:\n\nCLIENT: Hey Server! I need you to remove this player’s head for me.\nSERVER: Are you sure? I don’t remember seeing game.Workspace.Player.Head:Remove() anywhere in the code.\nCLIENT: Yep! Definitely supposed to not have a head. That’s how it looks on my computer.\n\nAnd then the server would, once again, gleefully remove the head of some unlucky soul, despite that fact that it was a rogue player’s client who gave the order, not the programmer.\nThese exploits, at a basic level, allowed users to wreak havoc on games: breaking game functions, harassing players, et cetera. However, the exploits were not always just to be annoying. The same exploit that allowed users to mess with the state of the server could also be used to download the source code for entire games, allowing users to rip and reupload entire years-long projects in minutes. At their worst, these exploits could get downright horrifying. \nBack in the Saddle\nIn 2014 - following what could only be described as a complete COPPA cluster - ROBLOX began making moves to clean up and polish the platform. This came in two major changes to the way development worked. \n1. Filtering Enabled\nTo fix the slightly problematic client exploit issue, the company introduced the Filtering Enabled feature to their game engine. Filtering Enabled locked down what types of information game servers would accept from the game client and fixed that little whoopsies mistake made in the younger days of the platform. However, development became a trickier after this feature was rolled out - many older games needed to be re-made to conform to the communication boundary between the server and client, and required a bit more formal knowledge of game development to make.\n(For reference, I didn’t learn anything about server and client relationships until my third year into a computer science degree.)\nEven though this feature could be disabled to allow legacy games to keep functioning (and yes, Robloxaville is legacy now - it even has the Fandom Wiki page to prove it), ROBLOX’s featured games algorithm wouldn’t touch your project with a ten-foot pole for risk of traumatizing even more unsuspecting children. This was especially important for the next major change,\n2. Monetization\nThat’s right, folks - every line of code my mom wrote from 2008 to 2013, she wrote for free. Making fun games for millions of kids across the world, pro bono? That’s just the way Tiffany rolls.\nWith the introduction of the Developer Exchange (DevEx), anybody (who could afford a premium membership) could take the in-game currency generated from their games and exchange it for cold, hard cash. Funnily enough, in that original blog post explaining how the DevEx program worked, the company describes a system where:\n\n[A] power-user [could cash] cash out and [buy] a car to get to and from college – all from developing a popular game on ROBLOX. This is the kind of story we can’t wait to tell.\n\nThat’s not quite the direction things went.\nThe Part Where Spencer Does Some Math\nSince DevEx was released, I’ve heard stories of full-time developers leaving their jobs to develop on the platform. Success could be lucrative: Welcome to Bloxburg, one of a number of games that filled in the Town And City genre gap after Robloxaville and Lakeside became deprecated by the Filtering Enabled update, has been played 4.8 billion times as of writing.\nBillion. With a “B”. 4,800,000,000.\nThat’s a lot of zeroes.\nThe game offers users a premium pass as an in-game purchase. This is a pretty common way to offer users a bit of extra bang for their buck by providing small bonus features - Robloxaville did the same thing at one point. How much do you think they could make off of a feature like that?\nUnfortunately for us, exact numbers are hard to find. ROBLOX stopped showing how many times in-game purchases were sold, no doubt to stop people from snooping around at how much developers make. That being said, we’ll make an assumption off of Robloxaville’s conversion rates: For every 10,000 player visits, 5 people will splurge on a premium pass at a purchase rate of 0.05%. That doesn’t look too impressive - yet.\nAt a 0.05% purchase rate against - and I cannot emphasize this enough - 4.8 billion visits, we could estimate that BLOXburg’s premium pass has been purchased 2.4 million times. Each game pass is 400 Robux (R$), ROBLOX’s in-game currency. If memory serves, ROBLOX keeps 30% of in-game player-to-player purchases (because, as we know, the only certainties are death and Robux taxation). We can assume, then, that the developers of BLOXburg received ~960 million R$ for that premium pass.\nThat golden parachute is starting to look real good.\nWhat might a 960 million R$ payout look like? The current DevEx exchange rate as of writing is 0.0035 USD for every 1 Robux (Robuck?) at the time of writing. This means that the estimated worth of that one premium pass for that one game is…\n3.6 million dollars.\nHm.\nHmmm.\nOf course, this does not come without some caveats. First: this is one of the most played games in the history of the platform - its’ success is by no means descriptive of every developer. Second: their estimated payout was made over the course of the six years the game has been in development. Third: the BLOXburg team is not just a single developer, and they are very good at what they do - enviously good, even.\nThat being said: those caveats also come with caveats. Although not every developer is going to be that successful, ROBLOX self-reportedly spends about 25% of their revenue on developer payouts as of a report from 2017. Since the company made its’ public debut on the New York Stock Exchange and started reporting revenue as a publicly traded company, they reported a revenue of $147 million dollars in 2020. Assuming they still pay out 25% to developers, that’s a big ol’ pot of 34 million dollars to go around.\nHow To Make Games and Influence Gen Z\nThis post has gotten way, way longer (and included way more math) than I intended, so I’ll wrap it up for now. Long story short: since 2007, ROBLOX has grown beyond description in both size and sophistication. The introduction of monetization professionalized the platform beyond what hobbyists from the early 2010s could ever pull off today. And, after (attempting) to redux Robloxaville back in 2016, I’m ready to take another stab at it.\nThe way I see it, games are to the computer what film was to the camera and paint was to the canvas. It’s a way to create an experience that people can look at, play with, and grow from. I really enjoy it. It’s a way for somebody who has no business in making art to get pretty damn close.\n(And it wouldn’t hurt to get some of that sweet DevEx stimmie, either.)"},"articles/structured-streaming":{"title":"Don't Double Down: Using Spark Structured Streaming to Wrangle Growing Data","links":[],"tags":["data","engineering/devops","engineering/python","mentoring/tutorials","viz/doodles","economics/finance","articles"],"content":"\nThis post was written during my time as a software engineer at M Science as a joint project with Databricks. You can view the original post here\n\nLet’s say that you, a ✨ humble data plumber ✨ of the Big Data era, have been tasked to create an analytics solution for an online retail dataset:\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nInvoiceNoStockCodeDescriptionQuantityInvoiceDateUnitPriceCustomerIDCountry53636585123AWHITE HANGING HEA…62012-01-10$2.5517850United Kingdom53636571053WHITE METAL LANTERN62012-01-10$3.3917850United Kingdom53636584406BCREAM CUPID HEART…82012-01-10$2.7517850United Kingdom……………………\nThe analysis you’ve been asked for is simple - an aggregation of the number of dollars, units sold, and unique users for each day, and across each stock code. With just a few lines of PySpark, we can transform our raw data into a usable aggregate:\nimport pyspark.sql.functions as F\n \ndf = spark.table("default.online_retail_data")\n \nagg_df = (\n df\n # Group data by month, item code and country\n .groupBy(\n "InvoiceDate",\n "StockCode",\n )\n # Return aggregate totals of dollars, units sold, and unique users\n .agg(\n F.sum("UnitPrice")\n .alias("Dollars"),\n F.sum("Quantity")\n .alias("Units"),\n F.countDistinct("CustomerID")\n .alias("Users"),\n )\n)\n \n(\n agg_df.write\n .format('delta')\n .mode('overwrite')\n .saveAsTable("analytics.online_retail_aggregations")\n)\nWith your new aggregated data, you can throw together a nice visualization to do… business things.\n\nThis works - right?\nAn ETL process like will work great for a static analysis, where you don’t expect the data to ever be updated - you assume the data you have now is going to be the only data you ever have. The problem with a static analysis?\nModern data doesn’t stop growing.\nWhat are you going to do when you get more data?\nThe naive answer would be to just run that same code every day, but then you’d be re-processing all of the data, every time you run the code. Each new update means re-processing data you’ve already processed before. When your data gets big enough, you’ll be doubling down on what you spend in time and compute costs.\n\nWith a static analysis, you spend money on re-processing data you’ve already processed before\nThere are very few modern data sources that aren’t going to be updated. If you want to keep your analytics growing with the source of your data, and save yourself a fortune on compute cost, you’ll need a better solution.\nWhat do we do when our data grows?\nIn the past few years, the term “Big Data” has become… lacking. As the sheer volume of data has grown and more of life has moved online, the era of Big Data has started to become the era of “God Help Us, It Just Won’t Stop Getting Bigger Data.” A good data source doesn’t stop growing while you work, and this growth can make keeping data products up-to-date a monumental task.\nAt M Science, our mission is to use alternative data - data outside of your typical quarterly report or stock trend data sources - to analyze, refine, and predict change in the market and economy.\nEvery day, our analysts and engineers face a challenge: alternative data grows really fast. I’d even go as far to say that, if our data ever stops growing, something in the economy has gone very, very wrong.\nAs our data grows, our analytics solutions need to handle that growth. Not only do we need to account for growth, we also need to account for data that may come in late or out-of-order. This is a vital part of our mission - every new batch of data could be the batch that signals a dramatic change in the economy.\nTo make scalable solutions to the analytics products that M Science analysts and clients depend on every day, we use Databricks Structured Streaming. Structured Streaming gives us the assurance that, as our data grows, our solutions will scale as well.\nUsing Spark Structured Streaming\nStructured Streaming comes into play when new batches of data are being introduced into your data sources. Structured Streaming leverages Delta Lake’s ability to track changes in your data to determine what data is part of an update and re-computes only the parts of your analysis that are affected by the new data.\nIt’s important to re-frame how you think about streaming data. For many people, “streaming” means real-time data - streaming a movie, checking Twitter, checking the weather, et cetera. If you’re an analyst, engineer, or scientist, any data that gets updated is a stream. The frequency of the update doesn’t matter. It could be seconds, hours, days, or even months - if the data gets updated, the data is a stream. If the data is a stream, then Structured Streaming is going to save you a lot of headaches.\n\nWith Structured Streaming, you can avoid the cost of re-processing previous data\nLet’s step back into our hypothetical - you have an aggregate analysis that you not only need to deliver today, but also keep updating as new data rolls in. This time, we have the DeliveryDate column to remind us of the futility of our previous single-shot analysis:\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nInvoiceNoStockCodeDescriptionQuantityInvoiceDateDeliveryDateUnitPriceCustomerIDCountry53636585123AWHITE HANGING HEA…62012-01-102012-01-172.5517850United Kingdom53636571053WHITE METAL LANTERN62012-01-102012-01-153.3917850United Kingdom53636584406BCREAM CUPID HEART…82012-01-102012-01-162.7517850United Kingdom………………………\nThankfully, the interface for Structured Streaming is incredibly similar to your original PySpark snippet. Here is your original static batch analysis code:\n# =================================\n# ===== OLD STATIC BATCH CODE =====\n# =================================\n \nimport pyspark.sql.functions as F\n \ndf = spark.table("default.online_retail_data")\n \nagg_df = (\n df\n \n # Group data by date & item code\n .groupBy(\n "InvoiceDate",\n "StockCode",\n )\n \n # Return aggregate totals of dollars, units sold, and unique users\n .agg(\n F.sum("UnitPrice")\n .alias("Dollars"),\n F.sum("Quantity")\n .alias("Units"),\n F.countDistinct("CustomerID")\n .alias("Users"),\n )\n)\n \n(\n agg_df.write\n .format('delta')\n .mode('overwrite')\n .saveAsTable("analytics.online_retail_aggregations")\n)\nWith just a few tweaks, we can adjust this to leverage Structured Streaming. To convert your previous code, you’ll:\n\nRead our input table as a stream instead of a static batch of data\nMake a directory in your file system where checkpoints will be stored\nSet a watermark to establish a boundary for how late data can arrive before it is ignored in the analysis\nModify some of your transformations to keep the saved checkpoint state from getting too large\nWrite your final analysis table as a stream that incrementally processes the input data\n\nWe’ll apply these tweaks, run through each change, and give you a few options for how to configure the behavior of your stream.\nHere is the ✨ stream-ified ✨ version of your old code:\n# =========================================\n# ===== NEW STRUCTURED STREAMING CODE =====\n# =========================================\n \n+ CHECKPOINT_DIRECTORY = "/delta/checkpoints/online_retail_analysis"\n+ dbutils.fs.mkdirs(CHECKPOINT_DIRECTORY)\n \n+ df = spark.readStream.table("default.online_retail_data")\n \nagg_df = (\n df\n+ # Watermark data with an InvoiceDate of -7 days\n+ .withWatermark("InvoiceDate", f"7 days")\n \n # Group data by date & item code\n .groupBy(\n "InvoiceDate",\n "StockCode",\n )\n \n # Return aggregate totals of dollars, units sold, and unique users\n .agg(\n F.sum("UnitPrice")\n .alias("Dollars"),\n F.sum("Quantity")\n .alias("Units"),\n+ F.approx_count_distinct("CustomerID", 0.05)\n .alias("Users"),\n )\n)\n \n(\n+ agg_df.writeStream\n .format("delta")\n+ .outputMode("update")\n+ .trigger(once = True)\n+ .option("checkpointLocation", CHECKPOINT_DIR)\n+ .toTable("analytics.online_retail_aggregations")\n)\nLet’s run through each of the tweaks we made to get Structured Streaming working:\n\nStream from a Delta Table\n\n+ df = spark.readStream.table("default.online_retail_data")\nOf all of Delta tables’ nifty features, this may be the niftiest: you can treat them like a stream. Because Delta keeps track of updates, you can use .readStream.table() to stream new updates each time you run the process.\nIt’s important to note that your input table must be a Delta table for this to work. It’s possible to stream other data formats with different methods, but .readStream.table() requires a Delta table\n\nDeclare a checkpoint location\n\n+ # Create checkpoint directory\n+ CHECKPOINT_DIRECTORY = "/delta/checkpoints/online_retail_analysis"\n+ dbutils.fs.mkdirs(CHECKPOINT_DIRECTORY)\nIn Structured Streaming-jargon, the aggregation in this analysis is a stateful transformation. Without getting too far in the weeds, Structured Streaming saves out the state of the aggregation as a checkpoint every time the analysis is updated.\nThis is what saves you a fortune in compute cost: instead of re-processing all the data from scratch every time, updates simply pick up where the last update left off.\n\nDefine a watermark\n\n+ # Watermark data with an InvoiceDate of -7 days\n+ .withWatermark("InvoiceDate", f"7 days")\nWhen you get new data, there’s a good chance that you may receive data out-of-order. Watermarking your data lets you define a cutoff for how far back aggregates can be updated. In a sense, it creates a boundary between “live” and “settled” data.\nTo illustrate: let’s say this data product contains data up to the 7th of the month. We’ve set our watermark to 7 days. This means aggregates from the 7th to the 1st are still “live”. New updates could change aggregates from the 1st to the 7th, but any new data that lagged behind more than 7 days won’t be included in the update - aggregates prior to the 1st are “settled”, and updates for that period are ignored.\n\nNew data that falls outside of the watermark is not incorporated into the analysis.\nIt’s important to note that the column you use to watermark must be either a Timestamp or a Window.\n\nUse Structured Streaming-compatible transformations\n\n+ F.approx_count_distinct("CustomerID", 0.05)\nIn order to keep your checkpoint states from ballooning, you may need to replace some of your transformations with more storage-efficient alternatives. For a column that may contain lots of unique individual values, the approx_count_distinct function will get you results within a defined relative standard deviation.\n\nCreate the output stream\n\n+ agg_df.writeStream\n .format("delta")\n+ .outputMode("update")\n+ .trigger(once = True)\n+ .option("checkpointLocation", CHECKPOINT_DIR)\n+ .toTable("analytics.online_retail_aggregations")\nThe final step is to output the analysis into a Delta table. With this comes a few options that determine how your stream will behave:\n\n.outputMode("update") configures the stream so that, each time the code runs, the aggregation will pick up where it left off instead of running from scratch. To re-do an aggregation from scratch, you can use "complete" - in effect, doing a traditional batch aggregate while still preserving the aggregation state for a future "update" run.\ntrigger(once = True) will trigger the query once when the line of output code is started, and then stop the query once all of the new data has been processed.\n"checkpointLocation" lets the program know where checkpoints should be stored.\n\nThese configuration options make the stream behave most closely like the original one-shot solution.\nThis all comes together to create a scalable solution to your growing data. If new data is added to your source, your analysis will take into account the new data without costing an arm and a leg.\nYou’d be hard pressed to find any context where data isn’t going to be updated at some point. It’s a soft agreement that data analysts, engineers, and scientists make when we work with modern data - it’s going to grow, and we have to find ways to handle that growth.\nWith Spark Structured Streaming, we can use the latest and greatest data to deliver the best products, without the headaches that come with scale."},"index":{"title":"Oh, Howdy!","links":["tags/engineering/","tags/economics/","tags/mentoring/","tags/articles"],"tags":[],"content":"Chaotic Good Computing is an organization specializing in providing advice and assistance for small companies around:\n\nSoftware Engineering\nData Analysis & Handling\nEconomic Analysis\n\nCGC’s general specialty is index, analysis, insights, and maintenance of digital economies like multiplayer game economies, network and cloud resource optimizations, or other digital spaces where you’d find a whole lot of chaos and complications.\nCGC also provides professional development resources and mentoring for young developers and professionals interested in economics and software engineering.\nThis site contains articles, notes, and thoughts on a broad range of topics. Chaotic Good Computing’s articles are great points to get started!"},"notes/annotations/college-admissions-and-the-stability-of-marriage":{"title":"Annotation: College Admissions and the Stability of Marriage","links":["notes/stability-of-marriage"],"tags":["economics/market-design","consulting","annotations"],"content":"\nth50%75%100%125%150%200%300%400 College Admissions and the Stability of Marriage Author(s): D. Gale and L. S. Sha\nshow annotation\nThis is a little test of something I’d like to do - keeping annotations of papers I’ve read on the CGC website, provided that they are open-access (which, unfortunately, is not always the case.)\nThis is being done with the Obsidian Annotator plugin. I have absolutely no idea how quartz is going to handle this, so… we’ll see! There’s a very real chance that this is going to end up being a substantially heavier lift on the frontend and design side than my feeble backend brain can handle. So, if you’re seeing this, feel free to pat me on the back for figuring it out (☞ﾟヮﾟ)☞\n\n\n\ncal Monthly.http://www.jstor.org COLLEGE ADMISSIONS AND THE STABILITY OF MARRIAGE D. GALE AND L. S. SHAPLEY, Brow*\nshow annotation\nWithout further ado! College Admissions and the Stability of Marriage was the second paper(ish) that I read related to market design and non-market environments that got me interested in the subject. The first, Dr. Kenneth Arrow’s Social Choice and Individual Values, was related more to voting systems - something that comes up every four years in America and then somewhat drifts into a hibernation until the next worthy national debate over the merits of first-past-the-post.\nThis one caught my attention, though, as it was the first technical analysis I’d seen of a topic that had some weight to it in my own life, and in the lives of others - partnership, romance, all the fun ones. If Arrow turned the handle on the door into this specific interest, it was Gale and Shapley that kicked it wide-open.\nIt was especially fun to present a summary of this paper in Dr. Allison Stashko’s Non-Market Environments course. I was proud to cap off the end of the term talking to my fellow early-20s undergraduates about something we could all understand at the deepest of levels: Being Horny™\n\n\n\nthe second will admit him later? Is it ethical to accept the first without informing the second and then withdraw his acceptance if the second later admits hi m? We contend that the difficult\nshow annotation\nIt’s interesting that exploding offers aren’t mentioned here - reading this for the first time in 3-4 years, I thought that it was mentioned at the top.\nIf it isn’t mentioned at all in this paper, then it was without a doubt Alvin Roth’s Who Gets What — And Why? that introduced the concept to me.\n\n\n\nng definition should not occur: DEFINITION. An assignment of applicants to colleges will be called unstable if there are two applicants a and ,B who are assigned to colleges A and B, respectively, although ,3 prefers A to B and A prefers : to a. Suppose the situation desc\nshow annotation\nIn the actual presentation, it was deeply difficult to re-word this in a way that didn’t sound like I was having another aneurysm. It reads somewhat well on paper, but sounds completely unhinged out loud.\n\n\n\nr let us look at some examples. Example 1. The following is the "ranking matrix" of three men, a, ,3, and 7, and three women, A, B, and C. A B C a 1,3 2,2 3,1 p 3,1\nshow annotation\nAnother difficulty was trying to reframe this specific style of diagram in such a way that a room full of people who’d been staring at game theory model grids for four years didn’t reflexively start eyeballing Nash equillibria\n\n\n\nignment of applicants. That is, THEOREM 2. Every applicant is at least as well off under the assignment given by the deferred acceptance procedure as he would be under any other stable assign- ment. Proof. Let us call a college “p\nshow annotation\nThe idea of match optimality was the concept that launched me out of this paper and into a rabbit hole of reading more about matchmaking algorithms, and actually modeling this in Python to toy around with the concept.\nIf, as they assert, this will give the assignees their optimal outcome compared to other stable match configurations, that implies the existence of other stable match configurations.\nTake a look at the assignment grid above — it’s a 2D grid representing one possible stable configuration. In that case, you could imagine that all possible configurations — stable or unstable — stacked on top of each other to form a 3D grid, or lattice, of match configurations. All of the possible 2D grids stacked like a big crystalline structure.\nFrom that lattice, you can slide out all of the unstable configurations (because, obviously, they suck and don’t mean anything), leaving us with a stable match lattice.\nThe even more interesting thing is - now that you have that stable match lattice, what exactly is in it? That’s where this particular part gets super zesty - “Every applicant is at least as well off […] as he would be under any other stable assignment”\nGale-Shapley’s stable match algorithm takes in two sets of people — in the case of this paper, men and women — which you can think of as the “assigner” and the “assignee”, the former of which makes the proposals and the latter who accepts or declines them. Think of it as a formula:\nmatches = gale_shapley(assigners, assignees)\n\nGale-Shapley is a noncommutative operation that optimizes the outcome for the assigners set.\nImagine you have a set of four men and four women, and you want to measure how “good” a matchmaking configuration is. You could imagine that the best-case scenario is that everybody ends up with their #1 choice, and the worst-case scenario is that everybody ends up with their #4 choice. We can use that as a baseline for measuring the “goodness” of the matchmaking.\nLet’s say, in some given matchmaking pair, everybody gets their first choice — the best case. If you were to add up everybody’s result (1st place) and average it by the number of people (8 people) you’d get a value of 1. On the flip side, for the worst-case, you’d get a value of 4 - ultimately being able to answer the question “On average, how well were people placed?”\nUltimately, if you take an analysis of the average placement of the assigner group (in this case, men), you’ll see that the assigner group’s average in Gale-Shapley is the highest of all possible stable configurations.\nUltimately, if you were to rank-order all possible stable matching sets on the matchmaking lattice on a spectrum of “Benefit to Group A” to “Benefits to Group B”, the outputs of gale_shapley(A, B) and gale_shapley(B, A) calculate the ends of the lattice.\nThis also implies a bit of a potential ethical issue. Depending on the context, there is an active decision made for “who gets better outcomes” by simply putting them into the algorithm first. Even within this paper, the marriage example itself generates the stable marriage output that provides men with the optimal output.\nUltimately, matchmaking is still something that interests me deeply, and I’m hoping that CGC (either the consulting side or development side) puts me in a position to keep working with these very interesting, very cool situations.\n\n"},"notes/annotations/simple-economics-of-open-source":{"title":"Annotation: The Simple Economics of Open Source","links":[],"tags":["consulting","economics/market-design","annotations","projects/cgc/economics-of-open-source"],"content":"\n00%400%NBER WORKING PAPER SERIES THE SIMPLE ECONOMICS OF OPEN SOURCE Josh LernerJean TiroleWorking Pa\nshow annotation\nI feel like I’ve been throwing out a lot of super speculative opinions on the economics of OSS lately - with a couple upcoming projects and opportunities, I want to make sure I understand some quantitative models for OSS contributions.\n\n\n\ner operating system market, International Data Corporation estimates that the open source program Linux has between seven to twenty-one million users worldwide, with a 200% annual growth rate. Many observers believe it repr\nshow annotation\nwell i’m sure this number is quite large 24 years later, huh?\n\n\n\nith a 200% annual growth rate. Many observers believe it represents a leading potential challenger to Microsoft Windows in this important market segment. • The significant capital in\nshow annotation\nI actually don’t know… what did people use for servers before Linux? Not to betray my youthful age but I can’t even imagine what was going on before Linux became a de-facto standard for headless server runs.\n\n\n\nrs helpless? [Stallman, 1999] The “utility function” Linux hackers is maximizing is not classically economic, but is the intangible of their own ego satisfaction and reputation among other hackers. [Parenthetical comment deleted] Voluntary cultures that work this way are actually not uncommon; one other in which I have long participated is science fiction fandom, which unlike hackerdom explicitly recognizes “egoboo” (the enhancement of one’s reputation among other fans) [Raymond, 1999b]. It is not initially clear\nshow annotation\n”egoboo” is a new term. While I do think (going into this on a first read-through) there are economic benefits related to labor contributions to OSS, i definitely don’t disagree that there is some level of utility in ego that keeps things moving.\n\n\n\ne provision of a public good? A ny explanation based on altruism1 only goes so far. While users in less deve\nshow annotation\nin general, “If everybody just _____” is a god-awful way to design markets.\n\n\n\nomics of open source software. Reflecting the early stage of the field’s development, we do not seek to develop new theoretical frameworks or to statistically analyze large samples . Rather, we focus on thr\nshow annotation\nDamn. I may have to find more recent papers, although I will finish reading this one through to the end.\n\n\n\nmost successful examples, the Unix operating system and the C language used for developing Unix applications, were originally developed at AT&T’s Bell Laboratories. The software was then instal\nshow annotation\noops. I thought UNIX was a standard (*NIX), not an OS in-and-of-itself.\n\n\n\nies was considerably enhanced. These cooperative software development projects were undertaken on a highly informal basis. Typically no effort to delineate property rights or to restrict reuse of the software were made. This informality proved to be problematic in the early 1980s, when AT&T began enforcing its (purported) intellectual property rights related to Unix. 2.2 The second era: early 1980s\nshow annotation\nNow this is ironic. given the inherently OSS nature of UNIX (with 24 years of hindsight) the fact that AT&T/Bell Labs tried to enforce a trademark on it is wild.\n\n\n\nperatively developed software. In exchange for being able to use and modify the GNU software (as it was known), users had to agree to make the source code freely available (or at a nominal cost). As part of the General Public License (GPL, also known as “copylefting”), the user had to also agree not to impose licensing restrictions on others. Furthermore, all enhancements\nshow annotation\nYahaha! The first case of a “cascading” license agreement, where software using GNU had to also, in turn, be GNU.\n\n\n\nng into various variants. In some cases, passionate disputes over product design have led to the splintering of open source projects into different variants. 8 Examples of such splinte\nshow annotation\nTwo examples off the top of my head:\n\n\nDolphin, the open-source Gamecube/Wii emulator, and its’ Ishiiruka fork. IIRC, there’s a bit of sass between the original developers regarding the Ishiiruka fork. Not sure what the context is.\n\n\nA recent (and spicy) one: the licensing changes around Terraform, spurring the creation of the OpenTofu fork.\n\n\n\n\n\nAnother challenge has been the apparently lesser emphasis on documentation and support, user interfaces,\nshow annotation\nsome things truly are evergreen.\n\n\n\ny by one open source developer: [I]n every release cycle Microsoft always listens to its most ignorant customers. This is the key to dumbing down each release cycle of software for further assaulting the non-personal computing population. Linux and OS/2 developers, on the other hand, tend to listen to their smartest customers... The good that Microsoft does in bringing computers to non-users is outdone by the curse that they bring on experienced users [Nadeau, 1999] . Certainly, the greatest d\nshow annotation\nI can get on board with this. Windows development is notoriously a pain in the ass.\n\n\n\nSource Definition and the concomitant decline of the GNU license\nshow annotation\na term worthy of a link.\n\n\n\n997 to 55% in September 1999.8 In 1999, the Apache Software Foundation was established to oversee the development and diffusion of the program . The current status of Apache,\nshow annotation\nThis is something else I’d like to look into — the structuring and continuity of OSS foundations.\n\n\n\nhow enjoyable the work is. The delayed reward covers two distinct, although hard-to-distinguish, incentives. The career concern incentive refers to future job offers, shares in commercial open source-based companies,9 or future access to the v\nshow annotation\nCertainly worthy of follow-up research.\n\n\n\ntrategic complementarities.” To have an “audience,” programmers will want to work on software projects that will attract a large number of other programmers. This suggests the possibil\nshow annotation\nInteresting to see the term “audience” made in the pre-influencer era.\n\n\n\nfads also have benefits. A fad can create a strong signaling incentive: researchers working in a popular area may be highly motivated to produce a high-quality work, since they can be confident that a large audience will examine their work. 4.2 Comparison between open so\nshow annotation\npossibly why i lowered the bar for the blog - i don’t have a clue if anybody actually reads it (☞ﾟヮﾟ)☞\n\n\n\nommon pool of students. 17 ii) Customization and bug-fixing benefîts: The cost of contributing to an open source project is lower if the activity brings about a private benefit (bug fixing, customization) for the programmer and her firm. Note again that this factor of cost reduction is directly linked to the openness of the source code. Let us now turn to the delayed\nshow annotation\nI hope this is elaborated on - I do think that this is the reason why companies will open-source previously private projects, as it opens up bug-fixes and improvements to a broader audience at the expense of (potentially) giving an edge to direct competitors.\n\n\n\nurce mode for three reasons: i) Better performance measurement: Outsiders can only observe inexactly the functionality and/or quality of individual elements of a typical commercially developed program, as they are unable to observe the proprietary source code. By way of contrast, in an open source project, the outsiders are able to see not only what the contribution of each individual was and whether that component “worked,” but also whether the task was hard, if the problem was addressed in a clever way, whether the code can be useful for other programming tasks in the future, and so forth . ii) Full initiative: The ope\nshow annotation\nOne of the things CS students are told constantly is “Contribute to OSS!” and this is the vast majority of the reason. If you don’t have the authority of having previously had an engineering position, OSS is a good way to show the results of your project.\nWhile I seriously doubt that any interviewer will look beyond an initial use-case for an OSS project you’ve contributed to, it does add a signal of “They did contribute!” to the project.\n\n\n\n’s interference, advice, etc. Economic theory would predict that the programmer's performance is more precisely measured in the former case. 18 iii) Greater fluidity: It\nshow annotation\nThis is crunchy! Because contributing to OSS is a self-decided move, it does make sense that the success is more indicative of single-agent performance rather than external factors.\n\n\n\ned in the former case. 18 iii) Greater fluidity: It may be argued that the labor market is more fluid in an open source environment. Programmers are likely to have less idiosyncratic, or firm-specific, human capital that limits shifting one’s efforts to a new program or work environment. (Since many elements of the source code are shared across open source projects, more of the knowledge they have accumulated can be transferred to the new environment) . These theoretical arguments a\nshow annotation\nYes!!! never underestimate the power of building transferrable skills.\n\n\n\nrong signaling incentives. By way of contrast, tasks aiming at helping the much-less-sophisticated end user—e.g., documentation, design of easy-to-use interfaces, technical support, and insuring backward compatibility—usually provide lower signaling incentives.14 4.3 Evidence on individual inc\nshow annotation\nHmmmm… I don’t know if I’d agree with this. Referencing the previous annotation, I think people tend to sleep on proper documentation and polish. While I don’t think this’d be the case for “industry-standard” OSS (curl being a good example), I do think that (for an individual contributing to OSS with less authority, adoption, or audience) documentation is key.\nIf an interviewer or company isn’t familiar with your work, they at least need to be able to see at-a-glance what the value of a project is. Even experienced engineers wouldn’t give a shit about a random GitHub repository with no documentation, even if it contains big-brain ideas or world-changing software.\n\n\n\ne nine key requirements in the “ Open Source Definition” [Open Source Initiative, 1999]. This point is also empha\nshow annotation\nFollow-up needed — I’m interested to see if this has evolved at all, or at the very least to read & review.\n\n\n\np, organization and governance. A successful open source project also requires a credible leader or leadership, and an organization consistent with the nature of the process. Although the leader is ofte\nshow annotation\nInteresting - let’s talk about some ethos!\n\n\n\nader must (a) provide a vision, (b) make sure that the overall project is divided into much smaller and well-defined tasks (“modules”) that individuals can tackle independently from other tasks, (c) attract other programmers,\nshow annotation\nMy brain is screaming “VSA!“. Parallelization is as important — if not more important — in planning software development as it is in software development itself.\n\n\n\ner” (prevent it from forking or being abandoned ). The initial leader must a\nshow annotation\noh, i hadn’t thought of this — part of the reason I try to bring collaborators in on CGC work is because having even a small team does wonders in keeping momentum.\nI’d liken it to the idea of variance in multiple distributions. Combining two normal distributions — in this case, representing motivation and velocity — ultimately lowers the variance of the distribution.\n\n\n\nto achieve full functionality. Another important determinant of project success appears to be the nature of its leadership. In some respects, the governance structures of open source projects are quite different. In a number of instances, such as Linux, there is an undisputed leader. While certain aspects are delegated to others, a strong centralization of authority characterizes these projects. In other cases, such as Apache, a committee will resolve the disputes by voting or a consensus process. At the same time, leaders\nshow annotation\nWhat?! They didn’t even mention Perl’s weird-ass “cycling leadership” that was mentioned at the top of the paper. I want to know more about that! It sounds whack as hell!\n\n\n\nted and all postings archived. For instance, on the Apache web site, it is explained how proposed changes to the program are reviewed by the program’s governing body, whose membership is largely based on contributions to the project. (Any significant change requires at least three “yes” votes—and no vetoes—by these key decision-makers.) 5. Commercial Software Compani\nshow annotation\nI wonder how this works now that Apache is no longer just “Apache,” but also an amalgamation of products (e.g Spark, Arrow (or whatever that underlying Database structure thing is called, etc.)\nI also wonder… how does this contribute to private use of the software? I know that the person who made Apache Spark now (IIRC) works as the CTO at Databricks, who benefits immensely from continued development on Spark. Are there instances of people in these positions potentially using this voting/veto power to further their own private interests at the cost of the broader community — and, if so, was trust undermined?\n\n\n\nl software companies (e.g., video game companies , Qualcomm for the Eudora e\nshow annotation\nHuh - this is something I’d never thought about. Video games are really weird, insofar as some of them do roll credits at the end of the game, not unlike movies or television shows.\n\n\n\ncommercial software firms. An internal Microsoft document on open source [Valloppillil, 1998] describes a number of pressures that limit the implementation of features of open source development within Microsoft. Most importantly, each so\nshow annotation\nOh shit - this should absolutely be followed up on.\n\n\n\nncentive to existing employees. Another area in which software companies might try to emulate open source development is the promotion of widespread code sharing within the company. This may enable them to reduce code duplication and to broaden a programmer's audience. Interestingly, existing orga\nshow annotation\nSee - I wonder if there are good, broad solutions to run software repositories for things like pip, apt, npm, etc, since that would be a fantastic thing to set up at companies for people to install things across the company.\nOne shortfall may be undermining the ease-of-use of just using the default, public software repositories.\n\n\n\ne benefits of the investments, the free-rider problem often discussed in the economics of innovation should apply here as well . Subsidies by commercial compa\nshow annotation\nfinally! the free-rider problem!! This is the one I think about most when OSS contributions come up.\nIf you can use a piece of software and don’t have to incur the cost of contribution, why contribute?\n\n\n\non should apply here as well. Subsidies by commercial companies for open source projects should remain limited unless the potential beneficiaries succeed in organizing a consortium (which will limit the free-riding problem). A second strategy is to take a\nshow annotation\n…I need to look into this more. What exactly does the consortium provide that de-limits the amount of commercial contributions to a project?\nSome level of coordination and more authority, perhaps?\n\n\n\nent of open source software. Companies can release existing proprietary code and create some governance structure for the resulting open source process. For example, Hewlett-Packa\nshow annotation\nAh, yes! K8s, React, Material, etc etc etc!\n\n\n\nckard’s RISC architecture. They can even (though probably less likely) encourage “ex nihilo” development of new pieces of open source software. This is similar to the s\nshow annotation\n”Ex Nihilo” in this case meaning “Something for nothing.” This is somewhat the flipped case of free-riding, where the owner of the OSS expects contributions from others.\nI’d imagine this use case would, in effect, undermine the authority of the company releasing proprietary software as OSS.\nHowever, the interesting thing - what if the company simply “transfers ownership” over to an OSS consortium? I believe I’ve seen this one happen before, but examples are eluding me.\n\n\n\nst visible of these efforts was Netscape’s 1998 decision to make “Mozilla,” a portion\nshow annotation\nI had no idea Mozilla came from Netscape in this situation - and it’s funny to see that Mozilla has worked out pretty damn well. Does Netscape still retain some ownership over the modern Mozilla organization?\n\n\n\ns had already been completed). It is also likely that Netscape did not adopt the right governance structure. Leadership by a commercia\nshow annotation\nIt’s so interesting to see this 2000 paper paint Mozilla as a failure. It’s worth a follow-up to see how Mozilla pulled out of what this paper paints as a nose-dive, death-spiral of irrelevance.\n\n\n\nhe open source community. In particular, a corporation may not be able to credibly commit to keeping all source code in the public domain and to adequately highlighting important contributions. 22 In this light, it is tempting\nshow annotation\nThis is interesting - IIRC Google has an “internal build” of K8s running, meaning that an inferior (Google-contribution-wise) version was the OSS version.\nI’d say the same thing about Databricks/Apache/Delta Lake, since I know that was the topic of some passion at one point.\n\n\n\ntion role for new firms. Collab.Net, a new venture funded by the venture capital group Benchmark Partners, will organize open source projects for corporations who wish to develop part of their software in this manner. Collab.net will receive fees for its online marketplace (SourceXchange, through which corporations will contact open source developers), for preparing contracts, for helping select and monitor developers, and for settling disputes. Hewlett Packard\nshow annotation\nCollab.net now redirects here, which is quite a pivot. That said, I feel I have seen SourceXchange out there - although I may be confusing it for StackExchange. It’s worth following up to see if there’s some relation, there.\n\n\n\nise, it is not without hazards. First, the open source project may be “hijacked” by a participant who builds a valuable module and then offers proprietary APIs to which application developers start writing. The innovator has then buil\nshow annotation\nAh, the curl-to-Postman pipeline.\n\n\n\nsuperimposes a new platform.25 Second, the coexistence of commercial activities may alter the programmers' incentives. To understand why it may be useful to make an analogy with academia (despite some differences between the academic research and open source development processes). To put our reflections in persp\nshow annotation\nSomething I have noticed is that people in OSS get real heated when the possibility of financial sponsorship comes up. I personally believe that it’s in the moral right to contribute financially to OSS — which is why I have contributions like that built into my budget — but I’ve seen people get real mad about that take.\n\n\n\nabout the latest developments. Second, the academic process may lose some of its integrity. The high-powered incentives\nshow annotation\nOkay — while I do get where they’re coming from, the fact alone that JSTOR was selling access to this paper for $40.00 when it was available elsewhere as a draft does make me giggle.\nTell me more about commercial interests polluting academic progress?\n\n\n\nemics to other communities. 32 While it is too early to tell, some of these same issues may appear in the open source world. Programmers working on an\nshow annotation\n”Too early to tell” from a paper written in 2000 is a great signal to follow-up.\n\n\n\nto be highly skewed, with a few individuals (or at most a few dozen) accounting for a disproportionate amount of the contributions, with most programmers making just one or two submissions. Many contributors to Send\nshow annotation\nThis feels like something GitHub would have data on. Worth a follow-up\n\n\n\nment itself grows and evolves. At the same time, it is heartening to us how much of open source activities can be understood within existing economic frameworks, despite the presence of claims to the contrary. The literature on “career\nshow annotation\nConsidering OSS has continued to grow, my gut instinct is that there are economically-incentivized reasons for contributions and that further research into… the further research, done in the last 24 years will yield some interesting new insights.\nI’m glad I started here - doing research on this chronologically will, I think, be a very interesting project.\n\n\n\ns will be less possible. But recent developments in computer science and programming languages (e.g., object-oriented programming) have encouraged further mod\nshow annotation\nteehee — “recent”\n\n\n\nof open source software. This is similar to the strategy of giving away the razor (the released code) to sell more razor blades (the related consulting services that HP will provide). Various efforts by corporations\nshow annotation\nAh, yes, the Terraform to Terraform Cloud pipeline.\n\n"},"notes/arc-review":{"title":"Arc Browser: Kicking off the Season of Rhythm","links":["notes/the-quest-to-slay-the-goblin"],"tags":["goblin-slaying","notes"],"content":"The Arc Browser is probably the first tool that slayed off a series of shake-ups on how I use technology to put up guard rails on being able to start - and stay - focused. It was also on their blog that I discovered Maggie Appleton’s blog - so in a way, it was very much a half-step into a pool of ideas that have helped me come to terms with the slightly-more-than-standardly chaotic way that I tend to work.\nAt some point, I’ll do a review here."},"notes/beethoven":{"title":"From the Archive: \"Reading the Room with Beethoven\"","links":[],"tags":["engineering/typescript","notes"],"content":"Beethoven was a project for HackTheU 2019, where a team and I (mostly the team) created an application for providing live captions to deaf students by using a remote microphone to stream audio and captioning to a user’s local device.\nFrom the original submission:\n\nInspiration\nSpencer, one of our team members, has a parent who is hearing-impaired and another that provides live, remote, ver batem transcription services for deaf and hard-of-hearing undergraduate, Ph.D., and Executive MBA students across the nation. Dan and Joseph, two of our other members, work for Sorenson Communications making video calling and other applications for the deaf community. We noticed that, for deaf students who would prefer live captioning in their lectures, there is no one-stop solution for both audio streaming and live captioning. We created Beethoven, an application for quick and accurate transcription in the classroom, to solve this.\nFunctionality\nPut simply, Beethoven will pair two devices together - an audio input and a screen output - and stream captioning from one device to the other in real time. The student can put one device at the front of the class for the clearest possible audio, and then position another next to them to read live captioning of the lecture and dialog as if it was the Closed Captioning on a television show or movie. This can either be done through Google Cloud transcription services or a live CART interpreter, depending on the level of accuracy desired and resources available.\nDevelopment Process\nThe application is split into three main parts - the audio input, the transcription service, and the screen output. First, the audio is captured by the audio input device. From there, it goes to either the Google Cloud voice transcriber, or a live CART interpreter. This happens through Google Cloud Services or peer-to-peer WebRTC, respectively. Once complete, the text will be sent to both our server’s backend to be compiled into a transcript and send to the screen output device, placed in front of the student, where a stream of the captioning is displayed.\nWhat’s next for Beethoven\nMoving forward, we’d like to fully implement the CART reporting features of the application. The P2P handshaking feature has a lot of improvements that can be made. Once finished, we’d like to reach out to the deaf and CART communities to get their feedback on the product.\n"},"notes/caveat-lector":{"title":"Caveat Lector: Reader Beware","links":["notes/digital-gardening-with-quartz","notes/qamo","tags/horticulture"],"tags":["horticulture"],"content":"The past few days have been a weird experience as I actively try to lower the internal bar for what I feel comfortable putting out into the world. As a proud member of the most terminally online generation to ever live, I have a deep and relentless fear of being wrong on the Internet.\nIn getting everything ready to re-publish this site as my own digital garden, I’ve started using the phrase “caveat lector” a lot. Back in my undergraduate degree, I learned about the phrase caveat emptor and caveat venditor:\n\nCaveat emptor means “buyer beware,” to be used in situations where a buyer of a product is responsible for understanding the risk they take in their purchase; and\nCaveat venditor means “seller beware”, to be used in situations where a seller of a product is responsible for minimizing the risk of the products they sell.\n\nA good napkin example would be a buyer and seller of digital LCD displays. In that case, it’s the seller’s responsibility to make sure that the display actually works - caveat venditor. However, what if a display works, but has a few dead pixels? This doesn’t render the display unfit for purpose - the seller would still be selling a good and usable product that fulfills their promise - but dead pixels are pretty annoying. The risk of dead pixels, then, may fall on the buyer - caveat emptor.\nOne of my intentions with Chaotic Good Computing is to contribute to the free body of knowledge I’ve benefitted immensely from myself, but to do so in a way that both fits my style and my habits better than traditional blogging or educational entertainment. The commitments that I’m making to you, the reader, are:\n\nTo my knowledge, the things that I’m posting are accurate at the time that I write them down; and\nIf I learn something in the future that corrects something I’ve written in the past, I’ll do my best to make corrections as I find mistakes. \n\nThat said, I’m also just some person on the internet. I know some things pretty well and other things not-so-much, with a pretty vast in-between. By the very nature of learning in public, some things here will be straight-up wrong, if not at least not-as-correct-as-they-could-be.\nSo, to you, I say:\n\nCaveat lector!\nReader Beware!\n\nand, as a follow-up piece of advice, I might also borrow from the Russian proverb:\n\nDoveryay, no Proveryay.\nTrust, but Verify.\n"},"notes/dailies/2024-02-25":{"title":"2024-02-25","links":["notes/dailies/2024-02-26","notes/caveat-lector","notes/digital-gardening-with-quartz","notes/over-under-engineering"],"tags":["horticulture","engineering/typescript","notes/daily"],"content":"2024-02-26 ⇒\n\nCAVEAT LECTOR: This is a daily note! Sometimes they’ll be structured, sometimes they won’t. These are more for the benefit of my near- and long-term future self, but I do tend to publish them in case they’re somehow helpful for other people.\n\nToday’s Plan\nI didn’t plan anything today, but I do want to start planning things in the future.\nToday’s Report\nI’m totally ripping off the labelling of this from Pikmin 2’s “Today’s Report” screen.\nThat’s not even a joke - I absolutely love the idea of ending the day with little graphs of what I did (or did not do) well that day in a tidy little reflective at-a-glance view.\nToday was (and will continue to be - my plan is to go home and keep going while I watch Avatar) a whole lot of writing. I’m still in the process of creating a new blog with Quartz, but I keep catching myself falling into a trap of “I want to make everything look perfect before I publish it!“. Despite spending the day jotting down my thoughts about over-engineering killing my dreams, I can’t seem to stop myself.\nThe stopping point for today will probably be a vague “do I feel like I’ve written enough to cover, at a base level, the anchor tags of what I want to write.” I still do want to do some weird D3.js fuckery to Quartz’s network graph where major “anchor” tags are locked to the outside of a circle (or some other shape) with extending graphs moving inward, but I am unfortunately blocked by not knowing how to debug with Quartz’s weird custom CLI build process.\nActually, it looks like the Quartz author just got back to me - QuartzComponents are built… \ntoo late? too early? I still don’t know enough about frontend development to fully understand how Quartz works under-the-hood or totally what that message means.\nTL;DR something about the Quartz build process means that I can’t use DevTools in a browser to debug Quartz components. Yesterday I was researching CodePens (or, more specifically, VSCode-inline CodeSwings) as a better workflow for my not-design-or-frontend-savvy self to develop these components in isolation. While I do still want to do some deep customizations to the way the Quartz Graph component works, I have decided (just now, as I’m writing) to leave this as a stretch goal, and instead prioritize getting this blog out as-is and customizing it to my liking later.\nMore work to be done in the endless struggle to avoid putting horses before carts, or whatever.\nNotes\n\nhttps://blog.partykit.io/posts/cursor-party\nhttp://www.bruno-latour.fr/sites/default/files/35-MIXING-H-ET-NH-GBpdf_0.pdf\nhttps://maggieappleton.com/gathering-structures\nhttps://maggieappleton.com/ambient-copresence\nhttps://github.blog/2023-10-05-a-developers-guide-to-open-source-llms-and-generative-ai/\nhttps://github.blog/2023-10-04-how-to-communicate-like-a-github-engineer-our-principles-practices-and-tools/\nhttps://www.rawgraphs.io/\n"},"notes/dailies/2024-02-26":{"title":"2024-02-26","links":["notes/dailies/2024-02-25","notes/dailies/2024-02-27","notes/caveat-lector","notes/the-quest-to-slay-the-goblin","notes/season-of-rhythm"],"tags":["goblin-slaying","horticulture","engineering/devops","engineering/csharp","notes/daily"],"content":"⇐ 2024-02-25 | 2024-02-27 ⇒\n\nCAVEAT LECTOR: This is a daily note! Sometimes they’ll be structured, sometimes they won’t. These are more for the benefit of my near- and long-term future self, but I do tend to publish them in case they’re somehow helpful for other people.\n\nToday’s Plan\nMost of today’s plan is going to be getting an internal NWYC project up-and-running with an MVP before we transition to some compliance updates to infrastructure. Follow-up on CGC will be taking inventory of the writing I did yesterday, and (once again) debating whether or not to get the formatting of the blog just right before I publish.\nB-Block\n\nCompleting C# wrappers for Geocoding services\nWorking on next ChainSharp steps for congressional data lookups\n\nM-Block\n\nLikely a continuation of B-block work\n\nE-Block\n\nCGC; in particular, either\n\nContinuing content writing, potentially starting the new posts for 1099 contractor writing; OR\nSetting up an environment & workflow for creating swings with CodeSwings\n\n\nStretch goal is contemplating whether or not MDX compatibility for Obsidian and Quartz will come now or later. I think Remark has the ability to do MDX files but I’ll need to spike that a bit more closely.\n\nToday’s Report\n\n\nNoon progress update: Got distracted at the possibilities of AWS CloudWatch Dashboards. Morning block is a bit behind, but getting back to it.\n\n\nAfternoon progress update: as a general rule, try to never fuck up the same way twice. that’s a good motivation for writing down as much as possible outside of code, so that the body of DX-related issues with undocumented fixes is always shrinking.\n🎵 come with me 🎵 //\n🎵 and you’ll see 🎵 //\n🎵 a world in which no developer fucks up the same way twice, and rather fucks up in new and creative ways each time 🌈 🎵\n\n\nIn sum total, today was… okay? ish? in terms of progress. Didn’t quite get as far as I’d hoped on C#, but some other (equally important) stuff came up and that seems okay. E-block certainly didn’t go as far as I’d hoped (read: it did not happen) but at least CGC stuff is at my own pace.\nAdmittedly, a minor distraction did pull away more of my attention than I ought to’ve allowed.\n\nAs a bit of a buzzer-beater against my self-imposed daily deadline of 12:00pm MT, I’ve decided to just publish this new blog format as-is. It doesn’t quite have the same style and zest as the original, but I’d rather put this out to encourage myself to improve it rather than let it languish in a “refactor” branch on GitHub.\nUPDATE: I did not beat the buzzer. It is currently 4:00AM the next day, and I am definitely not going to be deploying anything. Ironically, during this time I wrote the first posts about slaying The Goblin and my Year of Rhythm, which is ironic considering The Goblin certainly won tonight.\nNotes\n\nhttps://www.youtube.com/watch?v=0RFwyobtnKA\n"},"notes/dailies/2024-02-27":{"title":"2024-02-27","links":["notes/dailies/2024-02-26","notes/dailies/2024-02-28","notes/caveat-lector","articles/resume-ci-pipeline"],"tags":["engineering/csharp","engineering/python","notes/daily"],"content":"⇐ 2024-02-26 | 2024-02-28 ⇒\n\nCAVEAT LECTOR: This is a daily note! Sometimes they’ll be structured, sometimes they won’t. These are more for the benefit of my near- and long-term future self, but I do tend to publish them in case they’re somehow helpful for other people.\n\nToday’s Plan\nI didn’t get nearly as much C# done as I’d wanted to yesterday - while I’m a bit hesitant to say that “today is going to be a 100% effort day!”, considering how yesterday ended I’ve kept C# goals to a breezy middle-of-the-road, since I’d rather under- than over-set expectations.\nB-Block\n\nMore ChainSharp step testing.\n\nAs an aside, I think I’ve been dramatically over-complicating how I create separable PRs. This is one of my “Fuck, Theaux was right” moments that I increasingly have as I learn more C#. At some point, I’d like to do some gardening on philosophies for separable pull requests and commits.\n\n\n\nM-Block\n\nChecking on our internal project status - there’s likely tasks I can pick up, but my current work is the only very-large-web I have left to untangle (unless I’m forgetting something, which is likely.)\n\nE-Block\n\nToday’s E-block is a CGC day, so (sleep status pending) I’d like to work on actually taking what I have and uploading it to GitHub. This’ll mean taking a look at the Quartz build process.\n\nAnother aside - while I appreciate how much the developers of Quartz has made it user-friendly with a custom CLI, it abstracts away some of the usual development workflow and means I have to learn (and consequently fumble around with) a new thing. That said, I also don’t know jack shit about frontend development and there’s probably a good reason for that.\n\n\nOh, what’s this?? A friend of mine asked me to take a look over his resume. He’s still using the old LaTeX version of my resume template, so I’m real excited to get him moved over to the new Typst version (post pending). CGC blog stuff may get pushed back accordingly - not sure how high-capacity I’m feeling now at 5pm\n\nToday’s Report\nN/A\nNotes\n\nhttps://mypy-lang.org/\nhttps://github.com/apache/spark/blob/master/python/mypy.ini\nhttps://www.youtube.com/watch?v=xZYZEGHybqQ\nhttps://engineering.blackrock.com/static-type-checking-in-python-where-did-the-ducks-go-d17881d3205e - type hinting to static checks in mypy at a gradual pace\nhttps://github.com/kaiko-ai/typedspark Holy shit - it does exist!!\nhttps://typedspark.readthedocs.io/en/latest/index.html words cannot describe how excited i am??\n"},"notes/dailies/2024-02-28":{"title":"2024-02-28","links":["notes/dailies/2024-02-27","notes/dailies/2024-02-29","notes/caveat-lector","notes/dailies/2024-02-26","notes/putting-selfcontrol-on-raycast","notes/dailies/2024-02-25","tags/goblin-slaying","notes/raycast-review"],"tags":["goblin-slaying","engineering/csharp","notes/daily"],"content":"⇐ 2024-02-27 | 2024-02-29 ⇒\n\nCAVEAT LECTOR: This is a daily note! Sometimes they’ll be structured, sometimes they won’t. These are more for the benefit of my near- and long-term future self, but I do tend to publish them in case they’re somehow helpful for other people.\n\nToday’s Plan\nWhoops! Fell asleep at a reasonable hour yesterday, and didn’t get jack squat from my E-block done. That said, considering the day before, the sleep was very welcome. And - not to break my arm patting myself on the back - I did say that everything in that block was tentative.\nToday is pretty much the same thing as yesterday - ChainSharp does set things up in such a way that most work is “Today, I make another Step!” and that, plus not giving too many details for private projects, means that the B- and E-blocks for probably the rest of this week are not going to be too spicy and different.\nB-Block\n\nWorking on ChainSharp steps for private project\n\nM-Block\n\nWorking on ChainSharp step tests for private project\n\nE-Block\n\nThis… we’ll see. Today isn’t technically a CGC day - those tend to be Mondays, Tuesdays and Thursdays. However, yesterday was less than productive, so depending on my partner’s schedule (she may be going climbing) I’ll probably be staying in the office a bit later for CGC.\nIf CGC work happens today, current priority list (I should store these on an actual list app) is:\n\nResume reviews - 3 pending for two students and a friend\nQuartz deployment to GitHub Pages\nQuartz customization - likely a custom index page\n\n\n\nToday’s Report\n\n\n8PM Update: Hmmm… I think I’m going to pivot off of resumes. For an internal project, it’ll be in my best interest to get a bit of progress down on graph analysis & generation in TypeScript. I’ve already wasted (ironically) enough CGC time today on creating a SelfControl extension for RayCast. I think the reviews can wait until this weekend - likely Sunday, but I’ll try to get those done before then based on ✨ vibes 🌈.\nIdeally, I’d like to have a Jupyter-ish environment to prototype frontend - likely, this means picking up where I left off the other day with CodeSwings, and doing some horticulture on the TypeScript analog to Python’s Pandas/C#‘s LINQ, Data Forge.\n\n\n…\nOkay - so technically I did most of what I set out to do today. Left myself a bit of wiggle room on the E-block. Again. If you squint your eyes and tilt your head, foresight and unreliability do look quite similar.\nI have decided I’m very excited about the SelfControl on Raycast stuff. Of all the things that rattle around in my skull, the xkcd post about time saved vs time spent automating is the most rattle-y. After mulling it over (while i was doing it, which is the perfect time to second-guess yourself) I decided the juice was worth the squeeze if it means I can more reliably pull myself out of doomscrolls and death spirals.\nWhen it comes to the art of goblin-slaying, I do think that Raycast is the greatest set of steel-toed boots I own. It’s not the worst thing in the world that they now have a bit more weight in the heel with SelfControl tacked on.\n\nI will, however, also admit that - in terms of deliverables - the squeeze also squoze(?) out some time for other things I was hoping to do today. Hopefully trimming the doomscrolling fat makes up the time soon. Other than that slight detour, my CGC priorities remain generally the same and some of today’s list will likely get recycled into Friday. Tomorrow will be a lighter report day - CGC work is taking a backseat to watching the new(ish) Mean Girls with my partner, and then likely cleansing our palettes with the original Mean Girls afterwards. Taking a backseat? Classic Regina.\nIn a buzzer-beater, I learned a bit about D3 and finally settled on how to use CodeSwings in such a way that I can have them interact with Quartz internals. Maybe I am a frontend guy?? 😎\n(i am not.)\nNotes\n\nhttps://github.blog/2024-02-28-how-were-using-github-projects-to-standardize-our-workflows-and-stay-aligned/ - perhaps Projects is actually becoming… good? What a time to be alive!\nhttps://www.rawgraphs.io/ - Implementation as Backstage Plugin or Electron App?\nhttps://github.com/rawgraphs/rawgraphs-app - unfortunately not the type of “app” i was hoping for.\nhttps://github.com/jessejanderson/skylink - would be nice to do a popup as an Arc boost\nhttps://github.com/raycast/extensions/tree/c524392d80b35348811bae59831f29bfdbc34432/extensions/youtube-music/ - I’d like to, at some point, look at forking & adding the ability to add a song to a playlist. the selfcontrol stuff was quite a gateway drug, although i’ll have to adapt to using TypeScript for extensions rather than creating shitty, poorly-tested bash.\n"},"notes/dailies/2024-02-29":{"title":"2024-02-29","links":["notes/dailies/2024-02-28","notes/dailies/2024-03-01","notes/caveat-lector","notes/putting-selfcontrol-on-raycast"],"tags":["goblin-slaying","engineering/typescript","engineering/csharp","notes/daily"],"content":"⇐ 2024-02-28 | 2024-03-01 ⇒\n\nCAVEAT LECTOR: This is a daily note! Sometimes they’ll be structured, sometimes they won’t. These are more for the benefit of my near- and long-term future self, but I do tend to publish them in case they’re somehow helpful for other people.\n\nToday’s Plan\nOkay! With myself SelfControl’d this morning, feeling very optimistic today!! I think, today, I can finish up my part in the MVP of an internal project. As a treat, I’m going to take a closer look at tuning up our old Backstage deployment. I’ve been feeling a whole lot better about my TypeScript knowledge & workflows as of late - and my engineering in general - so taking another look at Backstage after ~8 months feels like it may be refreshing.\nB-Block\n\nFinalize MVP parts for internal project\n\nM-Block\n\nRe-clone and get Backstage local environment running\nLook into Backstage Plugins\n\nE-Block\n\nNone! It’s Mean Girls night, baybeeeeee 😎\n\nToday’s Report\n\nUPDATE 3PM: The SelfControl for Raycast integration was, in fact, worth the time spent. Today’s B&M blocks have been the most productive in probably 2-3 weeks 🎉\n\nSlight end-of-day, while I have the laptop open - we ended up stopping the new Mean Girls after about 10 minutes and just watching the old one instead.\nNotes\n\nhttps://kindavim.app/ - vim everywhere on macos. May automate some of the less-than-stellar shortcuts i’ve baked myself\nthere are few terms i despise more than “tribal knowledge.” at some point this will be written about.\nhttps://gist.github.com/timvisee/fcda9bbdff88d45cc9061606b4b923ca - Happy leap day!\nhttps://www.perkins.org/resource/how-write-alt-text-and-image-descriptions-visually-impaired/ - resource for writing alt text on images\n"},"notes/dailies/2024-03-01":{"title":"2024-03-01","links":["notes/dailies/2024-02-29","notes/dailies/2024-03-02","notes/caveat-lector","articles/binglish"],"tags":["engineering/csharp","notes/daily"],"content":"⇐ 2024-02-29 | 2024-03-02 ⇒\n\nCAVEAT LECTOR: This is a daily note! Sometimes they’ll be structured, sometimes they won’t. These are more for the benefit of my near- and long-term future self, but I do tend to publish them in case they’re somehow helpful for other people.\n\nToday’s Plan\nI think today is likely going to be lighter day, likely finishing up assigned work during the B-block - I have a couple last-minute touches to an internal MVP, but not sure to what extent my time is booked up for the rest of the day. May take M-block to do a bit of light research on Backstage plugins.\nB-Block\n\nQuick horticulture on (free) C# code coverage libraries\n\nM-Block\n\nBackstage horticulturie\n\nE-Block\n\nResume review - for real, this time!!\nActions/Quartz deployment\n\nToday’s Report\n\nUPDATE 3:00PM: Ended up writing an article about binglish, which is absolutely not what i had planned for the afternoon but i’m certainly not mad about it. from a quick cursory online search… i don’t think anybody else has coined this term yet?? At least not for the same concept - i was only able to find references to one chrome extension (ironic) and a somewhat(?) used term for british-indian-english (which, idk, maybe it’s far larger a term but not one that culturally/algorithmically would be at the tippity-top of my SLC-based search results, even on incognito) \nUPDATE 6:00PM: i was excited enough about that post to actually publish my blog. I was about to excitedly hyperlink to it, but… I suppose this will actually be on there now. There is something about seeing what i’ve been putting into a dusty refactor/ branch on GitHub actually be on The Internet™ that… I’m frankly not certain how to feel about.\nThere is something burning in me to really quickly learn everything more i need to know about TypeScript frontend development and finish the blog formatting posthaste, but I think that will need to wait for another time. I am hungry, and must depart.\n\nSee above - stopping CGC work and closing my laptop for the day. Perhaps I will play… Factorio???\n🫡\nNotes\n\nhttps://www.hanselman.com/blog/altcover-and-reportgenerator-give-amazing-code-coverage-on-net-core - Free code coverage for C#. Uncertain about Rider compat >:l\nhttps://youtu.be/_wxU3z9VxOY?si=Y2vuh_PH5_yHay9m&t=267 - suavemente\nhttps://kyla.substack.com/p/why-we-dont-trust-each-other-anymore - words, words, words (literally and topically)\nhttps://open.substack.com/pub/kyla/p/why-we-dont-trust-each-other-anymore?selection=f9b3b76b-bbe1-46c1-87f0-57b3d84594b1&utm_campaign=post-share-selection&utm_medium=web - poetry isn’t usually my thing but this got me\nhttps://smartbear.com/learn/code-review/best-practices-for-peer-code-review/ - Study on Pull Request ideal review size. I have no idea how big PRs should be and it falls into the same “economic value of lines of code” issue that I’ve looked at before. There’s lots of metrics - lines changed? files changed” feature count? subfeature count?\nhttps://smallbusinessprogramming.com/optimal-pull-request-size/ - More research resources I need to take a look at. Someday I will have an opinion.\n"},"notes/dailies/2024-03-02":{"title":"2024-03-02","links":["notes/dailies/2024-03-01","notes/dailies/2024-03-03","notes/caveat-lector","tags/games/roblox"],"tags":["mentoring","engineering/scratch","notes/daily"],"content":"⇐ 2024-03-01 | 2024-03-03 ⇒\n\nCAVEAT LECTOR: This is a daily note! Sometimes they’ll be structured, sometimes they won’t. These are more for the benefit of my near- and long-term future self, but I do tend to publish them in case they’re somehow helpful for other people.\n\nToday’s Plan\nToday wasn’t actually a CGC day, but I’m finding myself with some time at the tail end of the day after some sudden super-fun springtime winter storms have floated across Salt Lake City and tanked my plans to get tastefully wine blasted tonight.\nB-Block\n\nN/A\n\nM-Block\n\nN/A\n\nE-Block\n\nN/A\n\nToday’s Report\nToday was my partner’s first swim meet since she did triathlons in college! she did incredibly well - I wish that I knew more about photography so I could take cooler pictures of her when she’s swimming that we can send to our families. Maybe, someday, I’ll be a Photography Guy™.\nI ended up sitting next to a very nice group of kids. I had brought Kenneth Arrow’s The Limits of Organization - with no irony in the slightest, I put it away after seeing that the two kids next to me were reading objectively cooler books. They were very nice - we talked about video games and programming and played Exploding Kittens between meet heats. I cracked open my laptop and they showed me a few programs they’d made in Scratch, which brought back some memories of working with 4th-6th great students at Dr. David Johnson’s GREAT Camp back when I was a sophomore in college.\nSince teaching both that camp and working as a center director at a local Mathnasium for about a year around the same time, it’s been awhile since interacting with Modern Youths™, save for a few niche Roblox communities I lurk in for CGC-tangential projects. It is very cool to see how this generation (gen Alpha? i haven’t kept up much with the newfangled terminology around this generation) has a more… native? approach to technology. It makes sense, considering there will probably (hopefully) never be a moment they go without it.\nThey brought up roblox and I mentioned I’d played when I was their age - the oldest asked a few questions about moving from the block-based coding on Scratch to text-based coding. I feel it is a more interesting question than I was willing to inundate him with (i said “not really!” and capped it off at that.) - in hindsight, it was a pretty big transition for me. That said, I wonder if there exists a future where block-based coding comes back in a way, made possible by stronger generative code solutions to handle minute details within blocks. \nUnironically, I do think I would appreciate a graphical, scratch-like interface for cloud environments. That would be a pretty kickass Terraform frontend.\nAs a slight update on yesterday’s post about binglish - I did post it in a few places (and it didn’t even go viral??? shocked, offended, confused). I think the only piece of text feedback I got was in the Quartz discord, where somebody mentioned that albeit funny the term was one they were familiar with as a colonist term for British-Indian-English.\nI’m torn - by no means is this a Big Thing i’m doing with this blog, nor do I think it ever will be. Certainly not to the extent that the Venn diagrams of people who would read my notes and who would recognize “binglish” in that context would overlap. I also don’t know how I would go about checking - it is interesting that geography is a somewhat soft lock on that kind of cultural knowledge. Even from setting my VPN to Delhi, popping my browser into incognito and doing a (translated, obviously) search, I still get a “urban dictionary,” “stop trying to make fetch happen” vibe from the term. I’ll keep the post up purely because i like it (everybody, appreciate me!), but if that ever does become An Issue™ i will absolutely reconsider that decision.\nNotes"},"notes/dailies/2024-03-03":{"title":"2024-03-03","links":["notes/dailies/2024-03-02","notes/dailies/2024-03-04","notes/caveat-lector","notes/scratch/warning-about-slabtops"],"tags":["projects/personal/homelab","notes/daily"],"content":"⇐ 2024-03-02 | 2024-03-04 ⇒\n\nCAVEAT LECTOR: This is a daily note! Sometimes they’ll be structured, sometimes they won’t. These are more for the benefit of my near- and long-term future self, but I do tend to publish them in case they’re somehow helpful for other people.\n\nToday’s Plan\nToday is a bit strange - I’m currently on a kick of only gaming with friends (gaming solo, for whatever reason, has started to feel like “wasting time” (which is totally not a statement on the virtue of solo gaming, this is just how I feel about it at the moment)). After a bit of a misadventure trying to turn an old laptop into a well-ventilated server, a friend’s brother very graciously offered me some spare PC parts he had lying around so I could (properly, this time) start a home lab for running miscellaneous things, such as dedicated multiplayer servers at my home.\nI highly doubt any CGC work will get done today - I’d vastly prefer setting up the server and playing around with some homelab setup experimentation.\nB-Block\n\nPick up desktop case & parts\nClean out, assemble desktop case & parts\n\nM-Block\n\nPick up a new bed frame for a new mattress arriving later this week\n\nE-Block\n\nAssemble bed frame\nWork on desktop server\n\nToday’s Report\nVery minor EOD report - was able to get a Windows Proxmox setup with GPU passthrough and VirtIO set up today. With the addition of Moonlight & Sunshine, game streaming is now possible! I cannot wait to use this to bully invite my partner to join in game nights with us, now that we have a dedicated(ish) Windows machine for her to use.\n\nBy no means is running Lethal Company at a consistent HDx60fps a technological marvel, but I am very satisfied with the work tonight. While my Steam Deck is probably going to still be my go-to because I hate myself and love debugging, this was a fun little tangent.\nAs an aside - a lot of that setup was cobbling together random support forum posts and other weird shit - at one point I spuriously bought a pack of headless HDMI dummies from Amazon thinking I’d need them. I’d like to toss a quick note here at some point this week to elaborate on some of the missing steps between major guides.\nFuture work on the server (beyond just hosting multiplayer servers) will likely be looking at some actual capital-H-capital-L HomeLab stuff like HomeAssistant to bridge some of our disparate smart home devices into HomeKit so I can control them from my phone and computer a bit easier, but that’s for another night.\nNotes\n\nhttps://forum.proxmox.com/threads/pci-gpu-passthrough-on-proxmox-ve-8-windows-10-11.131002/ - okay, this one actually works.\n"},"notes/dailies/2024-03-04":{"title":"2024-03-04","links":["notes/dailies/2024-03-03","notes/dailies/2024-03-05","notes/caveat-lector","notes/season-of-rhythm"],"tags":["projects/personal/homelab","notes/daily"],"content":"⇐ 2024-03-03 | 2024-03-05 ⇒\n\nCAVEAT LECTOR: This is a daily note! Sometimes they’ll be structured, sometimes they won’t. These are more for the benefit of my near- and long-term future self, but I do tend to publish them in case they’re somehow helpful for other people.\n\nToday’s Plan\nBack to the work week! We haven’t had sprint planning, yet, so I’m not totally sure what today/this week has in store quite yet - I’ll loop back around and update blocks once I have ‘em.\nB-Block\n\nN/A\n\nM-Block\n\nN/A\n\nE-Block\n\nN/A\n\nToday’s Report\n\nTo have the full intended experience, please listen to the Pikmin 2 “Today’s Report” theme while reading & writing this\n\nSo, this particular report goes… well into tomorrow. I’ve been doing pretty good on my sleep rhythm lately, so let’s consider this 6:00AM posting a bit of a relapse.\nI need to get… a couple hours of sleep before I need to be up in the morning, especially considering it’s my partners birthday(!), so this’ll be a brief EOD. I will personally blame whoever it is that made this amazing set of proxmox LXC setup scripts for stealing my sleep tonight, in order to properly divert blame away from the true culprit: myself.\nNotes\n\nuhhhhh… quite the fuzzy match there, YouTube Music?\n\nhttps://gersom.nl/building-a-raycast-website-blocker - raycast blocker similar to Self Control\nhttps://www.raycast.com/tonka3000/homeassistant/commands - raycast extension for HomeAssistant, which is relevant to yesterday’s knock-on work\nhttps://docs.github.com/en/actions/managing-issues-and-pull-requests/closing-inactive-issues - action for closing stale issues, PM req\nTODO: Look up use of new on class method declaration in C#\n"},"notes/dailies/2024-03-05":{"title":"2024-03-05","links":["notes/dailies/2024-03-04","notes/dailies/2024-03-06","notes/caveat-lector"],"tags":["notes/daily"],"content":"⇐ 2024-03-04 | 2024-03-06 ⇒\n\nCAVEAT LECTOR: This is a daily note! Sometimes they’ll be structured, sometimes they won’t. These are more for the benefit of my near- and long-term future self, but I do tend to publish them in case they’re somehow helpful for other people.\n\nToday’s Plan\nToday ended up being a sick day (for both myself and my partner, which unfortunately coincided with her birthday), so not much got done. Not good, not bad - not really sure how to feel about it. At the moment, it’s a bit dour.\nB-Block\n\nR&R\n\nM-Block\n\nR&R\n\nE-Block\n\nR&R\n\nToday’s Report\n\nTo have the full intended experience, please listen to the Pikmin 2 “Today’s Report” theme while reading & writing this\n\ni do appreciates some rest and recovery. in storywriting terms, we’d probably call this one a bottle episode.\nNotes\n\nhttps://tteck.github.io/Proxmox/ - Proxmox helpers\nhttps://arxiv.org/pdf/2403.01152.pdf - research on current generative AI forensics solutions\n"},"notes/dailies/2024-03-06":{"title":"2024-03-06","links":["notes/dailies/2024-03-05","notes/dailies/2024-03-07","notes/caveat-lector","notes/dailies/2024-03-04","notes/utah-office-consult","notes/supply-chain"],"tags":["engineering/devops","projects/personal/homelab","notes/daily"],"content":"⇐ 2024-03-05 | 2024-03-07 ⇒\n\nCAVEAT LECTOR: This is a daily note! Sometimes they’ll be structured, sometimes they won’t. These are more for the benefit of my near- and long-term future self, but I do tend to publish them in case they’re somehow helpful for other people.\n\nToday’s Plan\nI think today’s plan has me diving back into the wonderful world of DevOps, testing out internal DNS capabilities for development resources within VPCs. Thankfully server bullshit the other day has gotten me in a Domain Name State of Mind.\nI think E-block today is going to be fixing up the homelab server. I took apart all of the pieces to clean them but (from what my partner has told me — oops!) there’s a fan in there that isn’t balanced and constantly clicking, so it’ll be some investigating there.\nAlso excited to put in a couple extra pieces - I bought a cheap burner card (GTX 1030 for like $60) to dedicate to video encoding/decoding, along with a few other here-and-there parts to expand storage slots. So long, 256GB storage!\nB-Block\n\nStarting on internal AWS DNS module for Account Factory Terraform\nWhile I’m in DevOps land, I kinda want to take, like, 30 minutes and follow up on a weird concept I read about awhile back - unit testing Terraform code. I don’t totally understand how one would unit test… infrastructure configurations? but who knows, maybe it’ll blow my mind\n\nM-Block\n\nMeeting up with our PM to talk over some developer experience stuff - people problems are truly the hardest engineering problems\n\nE-Block\n\nN/A - I’m hanging out with my partner tonight, but depending on how long her plague-ridden body can handle doing things, I may do some hardware stuff tonight on the homelab server.\n\nToday’s Report\n\nTo have the full intended experience, please listen to the Pikmin 2 “Today’s Report” theme while reading & writing this\n\nN/A\nNotes\n\nI’d like to eventually make the viz from the utah consult post a follow-up to the supply chain network post as an intro to that one guy on bluesky’s network graph viz guide\nhttps://apps.apple.com/us/app/proxmobo/id6447794447 - Proxmox mobile interface - very nice, 10/10 would recommend\nhttps://medium.com/@giuliodn/clone-and-backup-with-gparted-e1100219c1d7 - notes on transferring contents between hdrives with gparted\n"},"notes/dailies/2024-03-07":{"title":"2024-03-07","links":["notes/dailies/2024-03-06","notes/dailies/2024-03-08","notes/caveat-lector","notes/dailies/2024-03-03"],"tags":["projects/personal/homelab","engineering/devops","notes/daily"],"content":"⇐ 2024-03-06 | 2024-03-08 ⇒\n\nCAVEAT LECTOR: This is a daily note! Sometimes they’ll be structured, sometimes they won’t. These are more for the benefit of my near- and long-term future self, but I do tend to publish them in case they’re somehow helpful for other people.\n\nToday’s Plan\nToday’s plan is a bit more DevOps for work, and then some spring cleaning. Working on the new homelab server has, safe to say, unearthed a substantial amount of old hardware I had laying around in a box. If I don’t tidy up the living room, I may end up getting strangled in my sleep.\n\nThat said, the cat has fuckin’ loved the chaos. So many wires to attack - so little time between naps.\nB-Block\n\nCategorizing internal services into the internal LB/DNS\n\nM-Block\n\n\nLooking into GitHub Projects workflows through Actions. Yesterday was some messing about with actions/stale to manage some… pretty large batches of tickets at work. While this has seemed to do the trick so far, there was the realization that our current automation system would move all of the closed, stale tickets to our QA team.\nThere is currently a rolling boulder of about ~250 tickets slowly rolling towards our QA team that will squish them in 14 days’ time. God save their souls.\n\n\nE-Block\n\nSpring cleaning and playing around with the new dedicated GTX 1030 burner card in the homelab server\n\nToday’s Report\n\nTo have the full intended experience, please listen to the Pikmin 2 “Today’s Report” theme while reading & writing this\n\nN/A\nNotes\n\ntest!\n"},"notes/dailies/2024-03-08":{"title":"2024-03-08","links":["notes/dailies/2024-03-07","notes/dailies/2024-03-10","notes/caveat-lector"],"tags":["projects/personal/homelab","notes/daily"],"content":"⇐ 2024-03-07 | 2024-03-10 ⇒\n\nCAVEAT LECTOR: This is a daily note! Sometimes they’ll be structured, sometimes they won’t. These are more for the benefit of my near- and long-term future self, but I do tend to publish them in case they’re somehow helpful for other people.\n\nToday’s Plan\nN/A\nB-Block\n\nN/A\n\nM-Block\n\nN/A\n\nE-Block\n\nN/A\n\nToday’s Report\n\nTo have the full intended experience, please listen to the Pikmin 2 “Today’s Report” theme while reading & writing this\n\nN/A\nNotes\n\nhttps://github.com/languagetool-org/languagetool - languagetool OSS - can I get pro somehow? it’d be fun to hook that to the 1030\nhttps://www.reddit.com/r/portainer/comments/wuv5ei/comment/j18invp/?utm_source=share&utm_medium=web2x&context=3 - Setup for “global” environment variables across all portainer stacks\nhttps://i.kym-cdn.com/photos/images/newsfeed/001/318/550/e2d.jpg\nhttps://www.braytonstafford.com/partitioning-a-usb-drive-in-macos/ - actually partition a persistant boot drive on MacOS\n"},"notes/dailies/2024-03-10":{"title":"2024-03-10","links":["notes/dailies/2024-03-08","notes/dailies/2024-03-11","notes/caveat-lector","notes/scratch/shop-highlight-note","notes/dailies/2024-03-07","notes/digital-gardening-with-quartz","notes/supply-chain","notes/dailies/2024-02-25","notes/strong-and-weak-opinions"],"tags":["projects/personal/homelab","engineering/typescript","notes/daily"],"content":"⇐ 2024-03-08 | 2024-03-11 ⇒\n\nCAVEAT LECTOR: This is a daily note! Sometimes they’ll be structured, sometimes they won’t. These are more for the benefit of my near- and long-term future self, but I do tend to publish them in case they’re somehow helpful for other people.\n\nToday’s Plan\nWith a bit of a half-planned half-day, I’m at my office again (to put distance between me and that god-forsaken server, which has taken far too much of my time) to focus up on the CGC website.\nIn particular, I’m hoping to restore a bit of professionalism to the front landing page. At the moment, it’s the quartz default, and doesn’t have either the font of my handwriting or the actual CGC logo on it. Before submitting some notes for an office highlight, I’d vastly prefer to have my website looking far less cobbled-together than it is now.\nB-Block\n\nN/A\n\nM-Block\n\nN/A\n\nE-Block\n\nLaundry party! Tidying up the living room after the homelab explosion earlier this week\n\nToday’s Report\n\nTo have the full intended experience, please listen to the Pikmin 2 “Today’s Report” theme while reading & writing this\n\nUGH. This damn homelab server sucked me in again in this evening. A linux LVM is… oddly difficult to resize to the size of its new physical partition, and setting up a live disk to do that is incredibly stubborn. I’m in a difficult position with a goal and a few constraints:\n\nGOAL: Set up a Linux Live Disk that comes preconfigured with the utilities I need to remotely modify the server (e.g. lvm2, gparted, Chrome Remote Desktop (for remote-ness), etc)\nCONSTRAINT: I only have access to a MacBook Pro M2, Steam Deck, and the current 256G Proxmox instance itself\n\nSUBCONSTRAINT: There’s not many customizable options for flashing a customized Linux Live Disk on Mac\nSUBCONSTRAINT: The Steam Deck, despite being arch linux, fucking sucks at being used for Linux-y things due to the read-only1 root filesystem and kneecapped pacman installation\nSUBCONSTRAINT: All other Linux instances I have are virtualized on Proxmox, which makes creating custom drives onto actual hardware a pain in the ass\n\n\n\nTo get around this, I’ve settled (currently) on using Linux Live Kit on a virtualized Debian instance, customized with everything I need, and then doing a hardware passthrough of the USB drive to load the customized distro as a recovery disk. Ultimately, this means:\n\nUsing Proxmox…\n…to create a custom Debian live disk…\n…to use for remotely modifying LVM partitions…\n…to fix Proxmox.\n\nthis has sucked… a lot of time out of my life, most of which could’ve been better spent? and i’m debating whether or not to weather the clutter of moving a monitor back into the bedroom to just handle this.\nOther than that, Mrs. Lincoln…\nI did end up getting… a somewhat MVP up for the landing page of the site?\n\nBy all accounts, this doesn’t look good, but it at least has the logo available with… kinda? a title? ish?\nThere is a massive amount of frustration around having goals that require frontend know-how, despite not knowing frontend very well. I am debating, at the moment, the merits of continuing to use Quartz for the blog. I really like it on a design level, and specifically am very excited to dig into fixing what is ultimately Obsidian Publish’s unhelpful & messy (IMHO) network graph layout.\nHowever…\nQuartz, notably, is… weird? I think? The vibe I get is that Quartz was developed primarily for an audience who want, at a base level, to simply deploy an Obsidian vault onto the web without dealing with too much technical cruft. To that end, a lot of the actual frontend fundamental concepts seem to be abstracted away. \nLooking through the Quartz Showcase, there’s not a whole lot of structural changes made in most of these examples. This throws out a bit of an orange flag, making me a bit nervous that the framework itself wasn’t intended (at a base level) to be very extensible2.\nIt feels, somewhat, like an opinionated piece of software.\nThe tricky thing about opinionated software is that, when you pick it up as a beginner, it’s hard to separate whether or not the things you’re learning are:\n\nThe underlying, fundamental concepts of the technology you’re using (React.JS); or\nThe Quartz-level specifics, such as QuartzComponents.\n\nThat said, I don’t know if I have much of an option - from what I can tell, the closest OSS alternatives to Obsidian Publish are typically Gatsby and Jekyll themes - both of which I’ve used previously for this site, to mixed results.\nNotes\n\nhttps://music.youtube.com/watch?v=StlytUzBFcs&list=PLBxbqVsdFQnt5ihbCIkVa15MO7onhZwk5 - this channel has wildly impressive piano covers with super cool notes overlays - almost AR-esque?\nDue to an oversight, I am currently in the middle of a ~5 day gap in my prescriptions. I feel like my brain is currently operating with a constant background static noise that sounds vaguely like this\n\nFootnotes\n\n\nThe read-only OS for the steam deck isn’t actually read only, since you can disable that, but it is substantially restrained in its ability to do normal arch-y things like disk management. I’m also running into the issue of Pacman’s GPG key database is still complaining that it’s read-only and locked. Not sure what that’s all about ¯\\(ツ)/¯ ↩\n\n\nSo, there is such a thing as Quartz Plugins — the landing page work itself is a custom emitter plugin — but it doesn’t seem to have the same breadth or ecosystem that Gatsby did (I could be sub-informed, though, as I haven’t taken too close a look at Remark plugin integration yet). ↩\n\n\n"},"notes/dailies/2024-03-11":{"title":"2024-03-11","links":["notes/dailies/2024-03-10","notes/dailies/2024-03-12","notes/caveat-lector"],"tags":["engineering/devops","notes/daily"],"content":"⇐ 2024-03-10 | 2024-03-12 ⇒\n\nCAVEAT LECTOR: This is a daily note! Sometimes they’ll be structured, sometimes they won’t. These are more for the benefit of my near- and long-term future self, but I do tend to publish them in case they’re somehow helpful for other people.\n\nToday’s Plan\nJesus. I think I’m officially Old™ now. Daylight Savings was last night and I legitimately feel hung over right now.\nB-Block\n\nTerraform infrastructure - taking a look at the glories of DNS handling\nDoing brief & sexy horticulture on terraform unit testing\n\nM-Block\n\nI am too tired to look more than about ten minutes into the future.\n\nE-Block\n\nGoing home to assemble a bedside table and take a nap. God, I need a nap.\nLaundry Party!\n\nToday’s Report\n\nTo have the full intended experience, please listen to the Pikmin 2 “Today’s Report” theme while reading & writing this\n\nN/A\nNotes"},"notes/dailies/2024-03-12":{"title":"2024-03-12","links":["notes/dailies/2024-03-11","notes/dailies/2024-03-13","notes/caveat-lector","notes/over-under-engineering","notes/dailies/2024-03-03","notes/scratch/the-best-side-of-the-road","tags/notes/"],"tags":["projects/personal/homelab","notes/daily"],"content":"⇐ 2024-03-11 | 2024-03-13 ⇒\n\nCAVEAT LECTOR: This is a daily note! Sometimes they’ll be structured, sometimes they won’t. These are more for the benefit of my near- and long-term future self, but I do tend to publish them in case they’re somehow helpful for other people.\n\nToday’s Plan\nYesterday was, in fact, pretty much a complete wash Goblin-wise. That said, last night I got an actual human amount of sleep, so I’ll be starting work a couple hours early today to hopefully shift everything a healthy amount forward for the day\nB-Block\n\nN/A\n\nM-Block\n\nN/A\n\nE-Block\n\nN/A\n\nToday’s Report\n\nTo have the full intended experience, please listen to the Pikmin 2 “Today’s Report” theme while reading & writing this\n\nAlright! I broke yesterday’s homelab stalemate by sucking it up, getting past the need to over-engineer simple problems, and just… popping GParted onto a thumb drive and resizing the partition. As it turns out, easy problems only become hard problems when you make them hard problems ¯\\(ツ)/¯\nThis did leave time to get to the actual “lab” part of “homelab.” This involved knocking a few things off my list:\nProxmox Game Servers\nWhile the amount of focus has been nice, I will admit - I’m a bit lonely at the moment? A week ago the plan was to set up the server to play with friends, and that still hasn’t happened - so I wanted to at least get this off the ground tonight. This, however, cycled through a few phases… sometimes multiple times.\n\n\nOriginally, the plan was to use a lightweight Docker LXC on Alpine, with a sidecar Portainer instance, to run the game servers. However, coming from a Desktop Ubuntu environment, I was seriously missing the ability to load the Tailscale Docker Desktop extension to auto-add new containers to my Tailscale instance.\n\n\nI also thought that Portainer was a bit too overkill for what I was trying to do. Honestly, I saw a bunch of very Enterprise-y™ things, got scared, and looked at other solutions.\n\n\nNext stop was briefly trying out Pterodactyl - however, the setup process on their website absolutely tanked, so I ruled that out pretty quickly.\n\n\nNext up was AMP — which, to its credit, did work. However, the next snag was trying to set up a custom Docker Compose file. While AMP is great if the only control you’d like to have is “press button, have server,” I did a search across the entire system for anything resembling a Compose file and came up empty.\nDealbreaker.\nUnfortunately, it was only a dealbreaker after I had bought a license. Whoops.\n\n\nNext up was TurnKey Game Server. I’d seen a few things about TurnKey and LinuxGSD while searching around and figured it was worth a try. However, after multiple attempts, I wasn’t able to get a game server up at all1. Perhaps I misunderstood how the Webmin interface works, but… I also couldn’t even get a frontend to show up. Also a dealbreaker.\n\n\nSo, whaddaya know? I ended up exactly where I started - Docker & Portainer. Once you get past all the buttons, Portainer is very nice.\nI don’t think this is the worst thing in the world. Ultimately, every other of the above solutions (both the ones that worked, and the ones that didn’t) used weird, non-standard solutions for containerizing the game servers. The best side of the road is often the one others are driving on, and so choosing a pure-Docker solution for a very Docker-y problem is hardly a loss. However, I do wish that I’d spent less time fucking about with the options and more time just… making it work.\nThis spends a lot of time in my head, but it was very loud tonight.\nJellyfin & GPU Passthrough\nWell, fuck.\nAfter seeing some success with GPU passthrough, I rolled the dice on another go of it. I bought a GT 1030 card for hardware encoding & decoding for a video library I’ve got cookin’ and spent quite a bit of time tonight researching passthrough of the card into an LXC. Despite being virtually identical tasks in my own mind, the LXC passthrough turned turned out to be a surprisingly more involved process than passthrough into a VM2. After going through the trouble…\n…I realized that the GT 1030 doesn’t even support encoding.\nI’m not totally sure what to do with this card now…? Jellyfin hardware encoding was the entire reason I’d bought it originally. While $60 isn’t exactly financially devastating, it does feel weird having a useless card. I may do a quick search later this week to see if there’s some good reason to keep it in there - otherwise, I’ll likely take back the PCIx4 slot, likely for an additional NVME port. Still, quite the bummer.\nThankfully, most of my video library at this point is already encoded, and I suppose (worst-case scenario) I could potentially use the Windows VM (and its successfully passed-through GTX 970) to handle any on-the-spot encoding I need to do for specific files.\nPersonal Cloud\nThis is one that I kinda grazed, but didn’t get too far into - I’ve set up a NAS software to handle a bit of cloud storage, and I plan on setting up the aforementioned PCI/NVME expansion to set up a separate storage drive. I don’t have any hard drives at the moment, but this is something I’d very much like to get up-and-running at some point.\nFinal Thoughts: Things On My Mind\nYesterday, while internally mulling over the merits of using Quartz, I looked at the Navel-Gazing section of Maggie Appleton’s (very excellent) digital-gardening resource repo. I’d previously associated the term as being somewhat negative but realized I didn’t actually know what the term meant, so after a quick lookup on Wikipedia:\n\nHowever, phrases such as “contemplating one’s navel” or “navel-gazing” are frequently used, usually in jocular fashion, to refer to self-absorbed pursuits.\n\nI do worry somewhat that this blog - in particular, the notes section, is absolutely just me, navel-gazing, at my own cool, edgy ideas. It also coincides with a concern (possibly paranoia) that I use the word “I” too much when talking to others and that I’m being very self-absorbed.\n(I’ve also noticed myself linking others to this blog in my personal day-to-day. It feels silly to say “in hindsight” considering this last happened… less than 3 hours ago: in hindsight, this comes off super pretentious and i should absolutely not do it)\n(maybe i can do it a little bit - as a treat? i am pretty proud of this site so far.)\nThere doesn’t seem to be a great solution there, especially for the notes section, given that the whole thing is literally written in the first-person. However, I do want to make sure CGC remains a functional source of knowledge for others, and not just for my own benefit (although I do appreciate that part of this site.)\nSomething to think about, I suppose.\nNotes\n\nhttps://zeldor.biz/2011/01/mount-lvm-from-livecd/\nhttps://forum.proxmox.com/threads/pci-gpu-passthrough-on-proxmox-ve-8-installation-and-configuration.130218/ - proxmox gpu passthrough guide\n\nFootnotes\n\n\nWhile I’m thinking about it - honestly, from the polish of the application to the weird-ass unformatted, un-line-broken email they sent me as a confirmation for their website, TurnKey honestly doesn’t even look like a currently-maintained or legitimate website. Their stated goal is to bring server appliances to the masses, but their UX screams “Email from a Nigerian Prince” to me. ↩\n\n\nGPU passthrough to LXC for decoding - https://jocke.no/2022/02/23/plex-gpu-transcoding-in-docker-on-lxc-on-proxmox/ ↩\n\n\n"},"notes/dailies/2024-03-13":{"title":"2024-03-13","links":["notes/dailies/2024-03-12","notes/dailies/2024-03-14","notes/caveat-lector","notes/scratch/shop-highlight-note","notes/dailies/2024-03-10"],"tags":["projects/personal/homelab","notes/daily"],"content":"⇐ 2024-03-12 | 2024-03-14 ⇒\n\nCAVEAT LECTOR: This is a daily note! Sometimes they’ll be structured, sometimes they won’t. These are more for the benefit of my near- and long-term future self, but I do tend to publish them in case they’re somehow helpful for other people.\n\nToday’s Plan\nToday’s plan is going to be some logging service implementations - in particular, styling out a C# singleton to extend Discord webhook behavior for super quick-and-easy monitoring. This’ll be nice because I’d like to learn more about logging mechanisms, since being able to stream data straight out of a stdout would be nice for some other data gathering projects I’d like to do down the road.\nB-Block\n\nLook into C# standards and opinions around logging services - in particular, differentiating between a static Console-esque implementation of a logging service, versus an ASP.NET-flavored Singleton\n\nM-Block\n\nproject management notes meeting\nAt some point today, I need to finish the highlight note for The Shop.\nI’d also like to start reading (and potentially reviewing) The Framework Field Guide, which was written in part (or entirely? i don’t actually know the authorship distribution of that, since Unicorn Utterances is like, a collective, i think?). I’m super excited to dig into it - from the other day, it’ll be nice to put my paranoia aside about learning facts from opinions and getting to learn a more framework-agnostic baseline knowledge of frontend development.\n\nE-Block\n\n~~Hmmmm… good question? As much as I’d like to take advantage of the game server setup being actually functional again, I really ought to stick around the office while my partner is climbing to hammer out CGC work.\n~~That said… ~~factorio.\n\nToday’s Report\n\nTo have the full intended experience, please listen to the Pikmin 2 “Today’s Report” theme while reading & writing this\n\nIt’s already super late, so I’m gonna make this quick:\n\nI think friends have settled on a new Eco runthrough, which is very exciting!\nIn the meantime, spring cleaning on the steam deck - backed up /home, set up a Sunshine server for remote management from the laptop (QoL required)\nTried to go the Chrome Remote Desktop route by building it manually. I deeply regret this diversion, and would highly not recommend the attempt.\n\nNotes\n\nhttps://craftinginterpreters.com/ - A book my boss mentioned getting for his son about creating programming language. Total nerds\nhttps://aws.amazon.com/premiumsupport/technology/trusted-advisor/ - AWS trusted advisor, possibly(?) worthy of some horticulture\n"},"notes/dailies/2024-03-14":{"title":"2024-03-14","links":["notes/dailies/2024-03-13","notes/dailies/2024-03-15","notes/caveat-lector"],"tags":["economics/project-management","notes/daily"],"content":"⇐ 2024-03-13 | 2024-03-15 ⇒\n\nCAVEAT LECTOR: This is a daily note! Sometimes they’ll be structured, sometimes they won’t. These are more for the benefit of my near- and long-term future self, but I do tend to publish them in case they’re somehow helpful for other people.\n\nToday’s Plan\nN/A\nB-Block\n\nN/A\n\nM-Block\n\nN/A\n\nE-Block\n\nN/A\n\nToday’s Report\n\nTo have the full intended experience, please listen to the Pikmin 2 “Today’s Report” theme while reading & writing this\n\nN/A\nNotes\n\nhttps://rachelbythebay.com/w/2012/08/31/lines/ - pro puppy punting league\nhttps://stacks.stanford.edu/file/druid:xk241jp4730/Dissertation-LinFan-augmented.pdf - the concept of a “multi-armed bandit problem” is a) new to me; b) interesting; and c) incredibly funny out-of-context\n"},"notes/dailies/2024-03-15":{"title":"2024-03-15","links":["notes/dailies/2024-03-14","notes/dailies/2024-03-17","notes/dailies/2024-03-15","notes/scratch/shop-highlight-note","notes/digital-gardening-with-quartz"],"tags":["engineering/csharp","projects/personal/homelab","business/admin","notes/daily","projects/cgc/blog"],"content":"⇐ 2024-03-14 | 2024-03-17 ⇒\nToday’s Plan\n\n\nMorning Block: Morning work session, typically 9am - 12pm\nAfternoon Block: Afternoon work session - usually the bulk of work. 12pm - 5pm\nEvening Block: Evening work session for CGC-related work, 5pm - ???\n\n\nMorning Block\nNUnit Discord Logger\n\nPLAN:\n\nCreate ASP.NET general service to collect & batch logs\nCreate ITrace for use in NUnit tests\nImplement on test server webhook - once confirmed, move over to core server\n\n\nDELIVERABLES:\n\nA message (locally) posted from NUnit to Discord\nStretch goal is a webhook configuration able to create a new thread in a Forum channel1\n\n\n\nAfternoon Block\nBackstage Local Run\nAs dayjob work starts shuffling up a bit more, I want to elbow for a bit of room to keep working on Backstage plugins. To do that, I need to get my local Backstage build back up-and-running - and probably updated as well, since god knows how long it has been since I got that running 😬\n\nPLAN:\n\nRedownload Backstage as a fresh repo\nPossible minor configuration - I’ll likely test this on the CGC GitHub repository so I have the admin access needed to generate keys & pull information\nTaking a closer (more organized) look at the run scopes in the repo\n\n\nDELIVERABLES:\n\nDocker Compose-able Backstage running locally for plugin development. hype hype hype! 🎉\n\n\n\nShop Excerpt Written\nToday may be a dayjob early-release day, in which case I really need to get the Shop Highlight excerpt written & sent.\n\nPLAN:\n\nTake a look at the notes I took last week\nJot down draft, run it through something to un-fuck my fucked writing style.\n\n\nDELIVERABLES:\n\nDeliver as formatted email w/ Markdown attachment. God bless Markdown - hopefully this time doesn’t end up like the last guest post I wrote 😬\n\n\n\nEvening Block\nEco Server Run\nThe friends have settled on eco!! I believe I got most of this taken care of last night, but may need to adjust server run parameters. This’ll be the first stress test of the Allytron’s game container, so I don’t totally know what the resource req will look like.\n\nPLAN:\n\nRally the lads!\nMonitor specs during run\nOPTIONAL: bully my partner into joining us\n\n\nDELIVERABLES:\n\nFun, enjoyment, social satisfaction, etc. etc.\n\n\n\nToday’s Report\n\nTo have the full intended experience, please listen to the Pikmin 2 “Today’s Report” theme while reading\n\n\n\nLUNCH UPDATE: Did some TLC to the layout of the daily notes with Templater2. Added yesterday/tomorrow links to the top of each page (Formatting definitely required - tempted to just handle this in the standard quartz post emitter, now that I know how to do that) and configured the Daily Notes appender in the Raycast Quartz Plugin to append some custom logic for auto-incrementing footnotes. Not perfect, but works a treat.\n\n\nAlso did an update for a new layout for per-block tasks in the daily notes. I feel like I’ve been having trouble measuring tasks on a micro-level, and decided to use the same format for dayjob work that I do for contracting work, with a scope, plan, & declared set of deliverables so I can very confidently say that I did (or did not) finish something, and call the shot to stay on track & be intentional with my time.\n\n\nI’m considering setting up a Sunday-specific weekly template, so that I can better plan out my weeks during Sunday CGC work.\n\n\nA-BLOCK EOB UPDATE: Eeeee!! Feeling very behind on things. Overwhelmed and under-motivated.\n\n\nLikely EOD update — I don’t see myself getting back on my laptop at any point today for reasons — but I gotta hard stop to go take care of Life Business™. Did not get nearly as far as I’d hoped on any of today’s tasks, so today’s work will likely bleed into Sunday’s CGC work and (potentially, if the weather this weekend isn’t good enough to chill outside) Saturday.\nI’m 50/50 on whether or not I like the new daily template. While it is better-organized, I don’t totally know if it’s better-organized for me and where I’m at as a person doing things. I’ve been running constantly into an issue of over-estimating how much I can get done in a day. The deliverables are meant to help with that, but I don’t think it alleviates the issue of not being able to accurately predict my own day-to-day velocity.\nSome thoughts on the back of a napkin, but it may be worth trying out the Tasks List function in Obsidian, or potentially another solution. Interface aside, i think one of the best task management softwares I’ve used is Pivotal. I specifically appreciated the ability to “batch” tasks by velocity, to try and get a reasonable calculation for what is possible in a given amount of time. I think that’s something I’d like to get back to at some point, researching more “personal life” versions of the same thing.\nAnyway - likely signing off for the day. May the next day and the next be a better haul than today 🙏\nACTUALLY: one last final thought. I wonder if the Obsidian Git integration runs pre-commit hooks…?\nFootnotes\n\n\nhttps://arc.net/l/quote/flmmvzxd - the fact that Discord webhooks can auto-create threads is super cool. If there’s a way I can do thread auto-tags with this… i’ll be a very happy lad. ↩\n\n\nhttps://iwannabemewhenigrowup.medium.com/part-3-using-the-templater-community-plugin-to-automate-files-and-content-6f1f2f9227c3 - Guide used for intro to Obsidian Templater, used to create the automatic date entries and footnote counts on daily note template. ↩\n\n\n"},"notes/dailies/2024-03-17":{"title":"2024-03-17","links":["notes/dailies/2024-03-15","notes/dailies/2024-03-18","notes/dailies/2024-03-17","notes/dailies/2024-03-10","notes/over-under-engineering","tags/projects/","notes/time-tracking","notes/annotations/simple-economics-of-open-source"],"tags":["notes/daily","engineering/typescript","projects/cgc/blog","engineering/csharp"],"content":"⇐ 2024-03-15 | 2024-03-18 ⇒\nToday’s Plan\n\n\nMorning Block: Morning work session, typically 9am - 12pm\nAfternoon Block: Afternoon work session - usually the bulk of work. 12pm - 5pm\nEvening Block: Evening work session for CGC-related work, 5pm - ???\n\n\nMorning Block\nN/A\nAfternoon Block\nDiscord Logging\nJust to start the work week on a fresh note, I’m going to wrap up the Discord forum logging before anything else today.\n\nPLAN:\n\nTest hooks manually onto a forum channel in a test server\nTransfer customizations over to a configuration file\nInsert the head commit hash for splitting threads by deployment\n\n\nDELIVERABLES:\n\n Threaded, batched logs, output to the test server forum channel\n\n\n\nEvening Block\nQuartz Index Testing\nWhile the current landing page isn’t particularly bad, it isn’t good either. I’d like to do some touch-ups, with inspiration from a couple of examples of quartz that have customized their sites to a greater degree.\n\nPLAN:\n\nThere’s a particular Quartz site I saw that had a fantastic implementation of a landing page - I’ll need to find that\nReview the code for their emitter (provided it’s open-source — fingers crossed 🤞)\n\n\nDELIVERABLES:\n\n If feeling spicy, possibly give implementation a try.\n\n\n\nToday’s Report\n\nTo have the full intended experience, please listen to the Pikmin 2 “Today’s Report” theme while reading\n\n\nUPDATE: The Discord logging has become deceptively difficult. While creating a webhook and sending updates to a Forum channel are pretty easy, a surprising difficulty is getting the snowflake IDs of forum tags to use with the webhook’s applied_tags property of the JSON body. Unlike user, channel, and guild IDs, it doesn’t appear that the snowflake ID for tags are readily available. I think my next attempt for finding those will be looking directly at the webpage source for the discord web app. \nUPDATE: I think I’m going to be using a tag-based explorer approach1 for this site’s explorer - I’m a huge fan of being able to have a note in multiple places by subject matter rather than using the base file structure of the contents folder. Huge shout-out to whoever CVanmarcke is for publishing their Quartz components for this!!\n\nIn a typical act of over-engineering, I wasted a shitload of time today trying to figure out tags for Discord webhooks into forum posts. Ultimately, this is one of those “nice to have”, “oohs and ahhs” features for the relevant project that, ultimately, don’t need to be in there.\nI think that, personally, I need to start being better about catching myself getting diverted away from a defined MVP with “wouldn’t it be cool?”-type diversions. I can think of a few things to do about this:\n\nCreate well-written, well-scoped MVP documents for projects/milestones, and be better about catching myself wandering. Possibly a candidate for the new projects tag, perhaps?\nFind a way to more aggressively remind myself to time-track. Judging by a drop in my tracking coverage (-70% compared to two weeks ago), I need something to keep me on track, there.\n\nI do need to give myself a bit of grace, here - a recent lapse in medication left me pretty hobbled the past couple weeks. This week should be better.\n\n\nAdopt some kind of to-do list system. I talked about this a bit, recently, and I think I just need to kick myself into doing it. I need some better system for breaking milestones into subtasks, and then ranking them by priority/size. Ironically, I’m currently building a system specifically for this kind of thing — but having trouble getting momentum on it because, ironically, I don’t have a well-defined list of things to do. Very chicken-and-the-egg\n\nAs for wins today, I will say that the kick-off for an OSS economics research project and introduction/reorg of the tag and explorer systems for the site were wins. I do think that a tag-based explorer system meshes better with my intentions for the Quartz graph system, and is way easier for people to traverse. There is some fuckery afoot with the current way it’s handling tag index pages (only showing subtags, and not base-level tags within the category) but I’m feeling much more confident with Quartz tag emitters, now. That’ll be something on the chopping block, likely this week.\nAnother win is at least starting on a pre-build process for the site. The first tasks this’ll be responsible for is making sure that every tag being used has a proper index page. Even if it isn’t used, I’d like to be able to organize the /tags file structure automatically, rather than have to tune it by hand. Any opportunity to put up guard rails is a good opportunity.\nFootnotes\n\n\nhttps://github.com/CVanmarcke/quartz - Quartz plugin for tag-based post exploring ↩\n\n\n"},"notes/dailies/2024-03-18":{"title":"2024-03-18","links":["notes/dailies/2024-03-17","2024-03-19","notes/dailies/2024-03-18","articles/binglish"],"tags":["notes/daily","engineering/csharp","engineering/devops"],"content":"⇐ 2024-03-17 | 2024-03-19 ⇒\nToday’s Plan\n\n\nMorning Block: Morning work session, typically 9am - 12pm\nAfternoon Block: Afternoon work session - usually the bulk of work. 12pm - 5pm\nEvening Block: Evening work session for CGC-related work, 5pm - ???\n\n\nMorning Block\nCloud Testing\nIn order to catch the spunky, sexy, broad amalgamation of ways that infrastructure can fuck up in The Cloud™, it’ll be nice to have a cloud-based test suite that logs back to monitoring channels\n\nPLAN:\n\n Wrap tests into Docker Container\n Local test container/compose running (successfully or not) locally\n Adapt, adjust environment variable & ASP.NET environment\n\n\nDELIVERABLES:\n\n Tests running (successfully or not) in cloud\n\n\n\nAfternoon Block\nFinal Discord Webhook Work\nWith access to the GET REST APIs for Discord, now (through a bot - not thrilled, but w/e), I can do the final steps for deployment logging using Discord Forum channels\n\nPLAN:\n\n Generate webhook channel\n Add webhook to internal C# library\n Create new ILogger sub-interface\n Create batcher\n Create sender\n Inject logic into NUnit ITrace\n\n\nDELIVERABLES:\n\n Threads created through NUnit ITrace\n\n\n\nEvening Block\nShop Highlight (for real, this time).\nThis has been backburn’d for way too long. I will send this today, before I do anything else with CGC time.\n\nPLAN:\n\n Review note last state\n Finish first draft\n Edit it (probably with AI. there is a time and a place for binglish, and this is it.)\n Subject my partner to my shitty writing for proofread.\n\n\nDELIVERABLES:\n\n Highlight note sent as attachment & markdown.\n\n\n\nToday’s Report\n\nTo have the full intended experience, please listen to the Pikmin 2 “Today’s Report” theme while reading\n\nN/A"},"notes/digital-gardening-with-quartz":{"title":"Kicking off a Digital Garden with Quartz","links":["articles/structured-streaming","articles/hello-blog"],"tags":["goblin-slaying","horticulture","engineering/typescript","notes"],"content":"I’ve decided that I’m pretty bad at capital-B Blogging™. The last “post” I made (as of writing) was the Databricks guest post on Spark Structured Streaming that I did back when I worked at M Science in 2022. That particular post went through a few different stages of revisions:\n\nRevising initial drafts as I learned more about Structured Streaming as a concept\nRevising “personally final” drafts with the Structured Streaming engineering team at Databricks\nRevising “technically final” drafts with somebody from our company’s internal marketing team to make sure I didn’t sound like a doofus/didn’t make it seem like the company hired doofuses (it hired me, so it totally did, but that’s obviously not something to broadcast openly)\nRevising the “marketing final” draft with a lawyer to make sure I wasn’t blabbing trade secrets to the general public\nRevising the “legally final” draft with Databricks’ blog team to make sure the formatting was right (which I didn’t do a good job of, apparently - the final published draft ended up having some weird formatting errors. Not totally sure what happened there ¯\\(ツ)/¯ ) \n\nThat was quite a bit more than I’d expected when initially pitching “What if we published our internal guide for everybody?“. That, along with starting a new job that introduced me to a far higher standard for engineering, raised my own internal bar for “what should I put on the internet?” Since then, I’ve written quite a few posts that I’d describe as ~60% “done,” but didn’t end up hurdling internal (and arbitrary) bar for publishing.\nConsidering the editorial standards I started this blog with, the idea that I’ve had any bar at all is a bit funny.\nThe cost of learning is teaching\nAs somebody that has benefitted immensely from the free body of knowledge that exists on the internet, this doesn’t make me happy. A graveyard of posts I’ve arbitrarily deemed “unfinished” isn’t particularly helpful to anybody, including myself. Just because a set of knowledge isn’t refined enough to be perfectly, 100% clear to a broad audience doesn’t mean it wouldn’t be helpful to somebody, including myself in the future.\nI was fortunate enough to stumble on Maggie Appleton’s Ethos on Digital Gardening, which notably resonated with me more than many things I’d read in the recent past. In particular, this diagram she designed clicked something in my brain that’d remained un-clicked for quite awhile:\n\n\nbrief aside: Maggie brings a level of polish and class to doodling that is frankly enviable.\n\nThe cost of perfection is infinite\nPutting things on a spectrum of “chaos” to “cultivated”, in conjunction with the idea that “the cost of perfection is infinite” (if not asymptotic), puts into hardly-uncertain terms that simply because a thing is not perfect, or close to it, necessarily means that it is unhelpful and should die a slow death in a draft bin.\n"},"notes/literal-scratch-piano":{"title":"From the Archive: Scratch Piano Application","links":["notes/dailies/2024-03-02"],"tags":["mentoring","engineering/scratch","notes"],"content":"Back in 2018, I taught at the GREAT Camp at the University of Utah as an instructor for ~4th-6th grade students interested in intro programming in Scratch. Honestly, as a CS student at the time, it was actually really cool to try and put things I’d made in actual languages and translate them into Scratch, especially if they required manually implementing data structures like stacks and heaps.\nThis was a fun project to try and explain music - something that every Utah child learns, short of by-law - and also a way to cope with the fact that in Spring of 2018 I’d unintentionally signed up to take a weed-out Intro to Music Theory course at university.\n"},"notes/literal-scratch-quicksort":{"title":"From The Archive: Scratch Quicksort","links":["notes/literal-scratch-piano","notes/dailies/2024-03-02"],"tags":["mentoring","engineering/scratch","notes"],"content":"As an addendum to the Scratch Piano Application, I also around the same time cooked up an implementation of QuickSort. At my partner’s recent swim meet I sat next to a couple kids who’ve been learning Scratch and they were showing off some games they’d made.\ni was shocked - when it was my turn to show off, they didn’t seem very impressed by this.\n\nhonestly, the sort isn’t the weird thing - it’s the code itself that makes me, with 6 years of hindsight, think “…why the fuck would I make this?”\n"},"notes/over-under-engineering":{"title":"The Over/Unders of Over- and Under-Engineering","links":["notes/digital-gardening-with-quartz","notes/over-under-engineering","tags/engineering/python","articles/structured-streaming","tags/engineering/csharp"],"tags":["engineering","goblin-slaying","notes"],"content":"\n\n\n\nPremature optimization Over-engineering is the root of all evil\n— Donald Knuth\n— Sir Tony Hoare\n— Me, just now\n\n\n\n\nI think that over-engineering is the single greatest contributor to starting projects and never finishing them.\n\nLet’s define follow-through rate as the ratio of projects started to projects released. Not “completed”, since that’s an irrational way to think about projects.\n\nLooking back at periods of my life, I feel like my follow-through rate on projects was highest when I was 12 years old. At that point, my “projects” were:\n\nMy grades in high school\nWriting shitty programs on my TI-84\nWriting small games on Roblox\n\n13 years is long enough ago that, for the sake of simple math, my follow-through rate was 100%. Gee, what a reliable kid I was!\nI can name 4 periods of my life where the amount I learned about engineering grew at a more-rapid-than-usual pace:\n\n2016: I began studying computer science in my undergraduate\n2018: In trying to build up my resume, I started taking writing open-source and release-able projects more seriously\n2020: I got my first full-time job as a data analyst/software engineer writing data pipelines in Python and Spark\n2023: I got my second full-time job writing general software in csharp, with incredibly smart capital-E Engineers™ that were also incredibly good mentors\n\nWhile it feels great to know more, learning has caused me a fundamental issue: it’s easier to be bad at something on accident than on purpose.\nI’ve increasingly noticed myself trying to apply the things I’ve learned to new projects, causing a whole lot of up-front work to get a project started. Putting yourself in a prison of making the “best” choices at the very start of the project is incredibly demotivating and absolutely tanks the odds that a project started will someday be a project finished. Ironically, it’s future-proofing for a future that doesn’t happen.\nLet’s say, hypothetically, that my project follow-through rate started at 100% and dropped by a third following each learning period. After 4 learning periods, that would leave me at…\n(1.00−0.30)5=0.16\nYeesh. 16% follow-through rate sounds rough, but when I look at a recent history of what I’ve accomplished versus what I’ve wanted to accomplish, the number seems somewhat reasonable.\n\n"},"notes/putting-selfcontrol-on-raycast":{"title":"Watering Down SelfControl into Self Restraint","links":["notes/putting-selfcontrol-on-raycast","notes/season-of-rhythm","notes/arc-review","notes/the-quest-to-slay-the-goblin","notes/raycast-review","notes/caveat-lector","notes/dailies/2024-02-28"],"tags":["goblin-slaying","engineering","mentoring/tutorials","notes"],"content":"\nI talk a lot in this post. You may want to skip to the useful part.\n\nSo, during this recent period of trying to break quite a few years of bad work habits, I’ve been putting up little guardrails in my own life to keep myself on-track when I want to stay on-track. One issue I’ve run into - especially during a particularly tumultuous US election year - is a tendency to check the news.\nAnd then, five minutes later, check the news again.\n…and again\n…and again.\nI’ve rationalized it for awhile by saying “I work in politics - of course I should stay up to date!” However, that’s a lie - my day job is not in politics. At best, It’s working on software adjacent to people who are adjacent to politics.\nTo alleviate that for awhile, I had used a Chrome-specific app called StayFocused to block a pretty broad swath of websites, but ran into two issues:\n\nStayFocused allows you to schedule blocks, but only for predetermined amounts of time, and only a single block per day. This was not quite as flexible as I was comfortable with - I do like to check the news, my email, or Bluesky at lunch, or if I get off of work a bit earlier than expected.\nI started using Arc Browser with three different profiles - Personal, Work and CGC - and each one has its own Chrome plugins. Because of this, I’d have to configure/reconfigure/update the StayFocused installation in triplicate, which due to the way their app is set up is incredibly hard to do.\nTheir app kept hijacking my browser to notify me of updates, and I didn’t see a readily-available way to do that. Ironically, this had the effect of absolutely shattering my focus.\n\nI ended up uninstalling it and going with something a bit heavier-duty. SelfControl is an application for MacOS that blocks out apps across the entire operating system. I had also seen Focus listed as a possible solution, but… $49? For effectively the same thing with a prettier interface? yeesh.\nIronically, though, SelfControl requires self control to use, which is exactly what got me in this situation to begin with. It is incredibly easy to “take a break” and then just-a-couple-more-minutes yourself out of any semblance of flow. As a nice surprise, though, SelfControl comes with a CLI, which means we can use it for some more advanced workflows! We can use this to do two things:\n\nSchedule blocks of self-control for the future, e.g. “In 10 minutes, I will focus for 50 minutes”; and\nConnect that behavior to Raycast, which will help us lower the bar to starting Raycast (as, admittedly, having to swipe back to the main Mac desktop is a pain in the ass and puts it out-of-the-way)\n\nThe Useful Part\nI’ve added a couple of useful bash scripts to a SelfControl Raycast Plugin for others to use. At some point, I’ll look into the process for getting them added to the actual Raycast extension store. You can add these by cloning the repository and adding it in the Raycast settings as a “Script Directory.”\n\nTHAT BEING SAID: I’ve used the term caveat lector quite a bit to warn people that I’m just some guy. This goes doubly here! This is software that has literally only been tested, by me, a handful of times, that I made while (ironically) procrastinating other things I should be working on. To say it isn’t well-tested would be a dramatic understatement. The creators of SelfControl warn that it’s a tool that, if misused, can have side effects. So to that end… what would the phrase even be?\n\nHuh. I guess the Romans didn’t do much file sharing.\nCaveat download!"},"notes/qamo":{"title":"Quantitative Analysis of Markets & Organizations","links":["tags/economics/"],"tags":["economics","notes"],"content":"In 2022, I graduated from the University of Utah with a degree in Quantitative Analysis of Markets & Organizations (QAMO, more-or-less quantitative economics).\nWords cannot describe how much I appreciate the mentorship I received from my professors - especially Scott Schaefer, the program director who was incredibly kind in providing a level of flexibility that, frankly, was the only reason I was able to graduate while working full-time.\nThat said, I do wish the name of the degree wasn’t such a pain in the ass to type out."},"notes/raycast-review":{"title":"Diary of a Raycastaway","links":[],"tags":["goblin-slaying","notes"],"content":"Recently, I stumbled on Raycast - a MacOS Spotlight replacement that I would highly recommend to everybody, not just engineers."},"notes/robloxaville-remaster":{"title":"Thoughts on the bad 2017 Robloxaville Remaster","links":["articles/robloxaville","notes/over-under-engineering","notes/digital-gardening-with-quartz"],"tags":["games/roblox","engineering/lua","notes"],"content":"When I was in high school, I dabbled a bit in programming on Roblox. Due to some pretty glaring security concerns, a project started over the summer of 2017 to remaster Robloxaville. I’ll be the first to admit that the remaster is substantially less fun to play than the original. While part of this constraint was time - the original was made over the course of ~4 years, while the remaster was made in ~3 months - it’s undeniable that the velocity of the original project was simply far higher because it was under-engineered.\nThe original project was largely made by my mom, who studied programming during her physics-chemistry-geology triple(!!!) major in college, with some tangential work from me, who was a literal child. Not to sell short the sheer amount of work she put into the project, but between a lack of formal software engineering and the pretty glaring shortcomings of the Roblox engine itself meant that, frankly, it would’ve been impossible to engineer any of it properly to begin with. By the time the summer of 2017 rolled around and the remaster started, two circumstances collided:\n\nPlatform developers were now being paid, which introduced a far higher bar on quality of work being done\nI had just finished my first semester of a degree in computer science and learned tip-of-the-iceberg concepts of what constituted “good” software.\n\nDespite happening 7 years ago as of the time of writing, I remember how absolutely stressful that summer was. I spent wasted a lot of time worried that the code I was writing wasn’t engineered enough, and ended up designing a lot of bad and convoluted systems meant to save myself time in a future of development on that project that never came.\nThere exists a graveyard of projects and posts between 2017 and today that have fallen victim of arbitrary bars for under-engineering that I’ve put on my own work."},"notes/scratch/brevity":{"title":"brevity is no longer the soul of wit","links":["articles/binglish"],"tags":["horticulture","notes/scratch"],"content":"when he wrote “brevity is the soul of wit”, shakespeare clearly did not consider that someday AI generated briefs would consume the english language"},"notes/scratch/press-f-to-commit":{"title":"Press F to Commit","links":[],"tags":["viz","notes/scratch"],"content":"A friend of mine - who is one of the most consistent people I know - was called in over the weekend for an emergency patch deployment at work.\nThe weekday streak is over; long live the weekday streak!\n"},"notes/scratch/some1-entry":{"title":"Entries to SOME 1","links":["notes/stability-of-marriage"],"tags":["engineering/python","viz/doodles","economics/market-design","notes/scratch"],"content":"Awhile ago, the channel 3blue1brown announced an internship and public competition, Summer of Math Education - at the time, I submitted some half-assed entries that I’ve kept around since then.\nBachelor Analysis\n\nExcerpts from Stable Marriage\nLargely taken from a presentation on stable matching algorithms\n\n\n \nOptimal Stopping Problem\n"},"notes/scratch/the-best-side-of-the-road":{"title":"What is the best side of the road to drive on?","links":[],"tags":["notes/scratch"],"content":"\nThe best side of the road to drive on is the side everybody else has agreed to drive on.\n\nI don’t remember where I first heard this, but I think about it every day."},"notes/scratch/unstable-marriage":{"title":"Unstable Marriage","links":["notes/stability-of-marriage"],"tags":["data","economics/market-design","notes/scratch"],"content":"I’ve spent a whole lot of time researching matchmaking algorithms, and yet I’d never considered - didn’t even mention in the presentation - what would happen in odd-numbered sets.\nthe implications are frankly funnier than I would have imagined"},"notes/scratch/warning-about-slabtops":{"title":"Before You Slabtop Your Laptop: A Brief Warning","links":[],"tags":["projects/personal/homelab","notes/scratch"],"content":"I recently had a misadventure while trying to slabtop (remove the LCD panel of a laptop to use it as a server) an old Dell Alienware G2 15. I didn’t see this piece of friendly advice while Googling around before making my attempt. I really wish I had - it would’ve saved me a lot of time, trouble, and a motherboard to boot (pun intended).\nTo contribute to the body of all knowledge, I hope you (you, the person googling “how to slabtop a laptop” right now) see this before you whip out the screwdriver and go to town on your old burner:\nSearch “<YOUR LAPTOP MODEL> LCD Panel POST Check Beeps” before you attempt to slabtop your laptop!!\nComputers run without screens all the time - we call them servers, and they run everything, so the possibility of a laptop manufacturer literally not letting a computer complete POST check without an LCD panel wasn’t even something that crossed my mind. However, if you google the above, you’ll run into lots of posts of people who hear a handful of beeps when their LCD display is disconnected, signifying that the POST process didn’t complete. This will absolutely fuck you over if your laptop manufacturer is one of the many who have this check in place.\nWhile I believe it is possible to do this on a PC (not a Mac, where slabtopping has been documented extensively), I personally ended up frying the motherboard after about 5 hours of cutting away at the stupid fucking thing. If this helps even a single person, I will be immensely happy."},"notes/season-of-rhythm":{"title":"Spencer's Season of Rhythm","links":["notes/the-quest-to-slay-the-goblin","notes/over-under-engineering","tags/goblin-slaying"],"tags":["notes","goblin-slaying"],"content":"In the spirit of setting themes instead of resolutions, I’ve adopted a seasonal (or perhaps year-long - who knows?) theme for my own life:\nThe Season of 🎵 Rhythm! ✨\nDuring my Season of 🎵 Rhythm! ✨, I want to establish healthier cadences to how I spend my time day-to-day, and to be far more intentional with how I spend my time. This is largely a response to a broad problem in my life that I’ve felt for some time now.\nI am not a person who, at any point, has ever formed good, lasting, healthy habits.\nThrough high school, into college, and continuing into my adult life, I have always been a very “whichever way the wind blows” kind of personality. This hasn’t been entirely without its upsides. A coworker once described me as “very good to have in emergencies.” While I’d agree with that characterization, it’s because my tendency embrace The Goblin has meant I have been a co-conspirator to many of the emergencies in which I’ve learned to work well under pressure. If you do enough work, enough times, an hour before a deadline, you become very good at doing lots of work with hours before a deadline.\nThe downside to this is… fairly obvious. In the last year, I started working for and with a very emotionally healthy team of engineers who are remarkable at pacing themselves, doing diligent and level-headed work, and avoiding emergencies when possible - something that drove me absolutely insane at times during my first year. After having seen - and felt - how much healthier this environment is and how much more it can accomplish - particularly in collaborative and non-competitive settings - I’ve accepted it’s the way I’d ought to work, too.\nAs an additional downside to the way I worked, only having a cohesive and consistent pace of work meant that larger projects - things that aren’t done in one all-nighter but have to happen over the course of weeks and months - had a tendency to fall into obscurity. I’d attribute some of that to the way I have structured many of these projects, but constantly losing the thread between bursts of motivation means that bigger plans can easily turn into aimless wandering.\nSeason of Rhythm is a time to actively, rather than passively, establish good “rhythms” and routines that lower the variance of my own work and habits. The goal isn’t to shed the “works well in emergencies” characterization - on the face of it, I don’t think that’s a bad thing. However, where I’d describe the previous variance of my own ability to function personally and professionally as a 25-50% pace during “peace-time” and 200-400% pace during emergencies, I’d vastly prefer to even this out to something that doesn’t feel like personal and occupational whiplash: maybe 80% day-to-day and 120% during emergencies, as a vibes-y estimate.\nAfter two months, I’m on a really good track. A whole lot of goblin-slaying posts this year will probably be related to tools I’ve found that help me keep on track, and I’m excited to see how the trajectory of this next 4-10 months"},"notes/stability-of-marriage":{"title":"College Admissions and the Stability of Marriage","links":["notes/qamo","tags/viz/doodles","notes/scratch/some1-entry"],"tags":["engineering/python","economics/market-design","viz/doodles","notes"],"content":"In college, my area of focus during my economics degree was non-market environments, which encompass the areas of economics where money isn’t used for moral or practical reasons.\nA presentation I’m particularly proud of is a presentation on David Gale and Lloyd Shapley’s 1962 paper College Admissions and the Stability of Marriage\n\nThis presentation happened at a personally odd time during a globally odd year, and - due to the practical limitation of only having my iPad available while I was in-patient at the University of Utah - was the first time I started using doodles in the vast majority of my presentations.\nThere are a few things that were cut for the presentation for time - in particular, the incredibly interesting concept of matchmaking lattices. Eventually, I’d like to include some additional written resources here on the topic, because it’s pretty cool. One of the cut slides is available here.\nThe absolutely incredible Dr. Allison Stashko is the only reason this presentation was possible - she was incredibly kind and encouraging during a difficult period, and is in large part the reason why non-market environments are my primary interest in economics. "},"notes/strong-and-weak-opinions":{"title":"let opinion = null","links":["notes/strong-and-weak-opinions","tags/engineering/csharp","tags/games/roblox","assets/Pasted-image-20240305234056.png","tags/engineering/python","notes/scratch/the-best-side-of-the-road"],"tags":["engineering","notes","economics/project-management"],"content":"\nThis note ended up having a lot of front-loaded context — feel free to skip to the point\n\nOpinionated Software\nThere’s a term that floats around that feels new. It’s not something that I’d ever seen in the science portions of my undergraduate. As I’ve entered into industry and more frequently both use and participate in the development of a broader range of software, there has been a sharp increase in frequency that I’ve seen this term:\n\n[Proper noun] is an opinionated [general noun]\n\nWhat… does this mean? I’d never seen it in an academic slide deck before, but I can name a few high-profile instances of the label used. While it’s usually surrounding frameworks, I’ve seen it used in other places as well:\n\nDuring C# related work — I have previously used Wrapt as a csharp web API scaffolding tool, which describes its default options as opinionated\n\nOn wandering through the chain of opinions, Wrapt’s opinions are, in turn, based on the Vertical Slice Architecture (VSA) opinions I’d first seen in Jimmy Bogart’s post on VSA, a recommended must-read by my team at NWYC\n\n\nDuring roblox-related CGC work: “Flamework is a highly opinionated game framework”\n\nInterestingly, this description has since been replaced to say (as of writing) “Flamework is an extensible game framework.” — thought I was going insane in trailing back after seeing that phrase there months ago and not finding it, but a stale Google Index reference shows that I’m not crazy (on this one, at least). I am curious about the onus behind the wording change, because I do actually believe that the new description better captures the intent of the Flamework package.\nUPDATE: After talking to the creator of the package, this change was made as the project evolved from a personal single-user project to a broadly-used staple of Roblox/TypeScript development. In particular, the Modding API is what spurred the description change, as it allows an end-user to yoink very nifty behavior like dependency injection and decorators and attach it to custom utilities.\n\n\nStumbling on the black python formatter while wrangling Databricks codebases at M Science\n\n\nThis is probably the most severe(?) case of opinionated software I’ve ever seen — it is literally named after the Henry Ford quote regarding available paint jobs for the Model T: “Any color the customer wants, as long as it’s black”\n\n\nThe black repository is also home to one of my favorite GitHub Issues of all time, in which a senior staff engineer at Twitter requested the ability to configure 2-space tabs instead of 4-space tabs.\nThe response to one of the most influential tech companies to ever exist requesting a (seemingly) minor change in order to adopt it as the standard code formatter for their entire Python codebase was a resounding: “no.”\n\n\n\n\nAnybody who has ever seen highly-opinionated flame wars in the comments of Reddit threads, Stack Overflow posts, or — frankly — between any work call between more than, say, two engineers in the same field, opinions have the potential to run very strong.\n(Want a real life example? Go ahead and pop “is <PROGRAMMING LANGUAGE A> better than <PROGRAMMING LANGUAGE B>?” into your search engine. One of my strong opinions is that 99.9% of those conversations are bullshit, wasted time. Seeing them in the wild makes me want to close my laptop and walk into the ocean, my final thought before the air depletes from my lungs being that, at the very least, my decomposing body will contribute more back into The Universe than those people did by arguing complete fucking nonsense on the internet.)\nWhen strong opinions collide, it can feel very much like watching an unstoppable force hit an immovable object. Especially when I was a data analyst on the way up to software engineering, I often found myself a junior (and rightfully silent) participant in long-winded, fruitless, and ultimately destructive conversations, in which two parties who hold strong opinions talk past each other for… 10 minutes… 20 minutes… an hour? with the call ultimately ending because one side or the other has to hop into a different call of the same flavor.\nI think the way we can all occasionally approach having an opinion — or not having an opinion — is ultimately reductive, destructive. It slowly chisels away, bit by bit, hundreds of thousands of minutes, hours, and days of human life to the behest of our goals yet delight of the Reaper himself.\nThe Point\nI’m proposing, here, a way to:\na. properly classify your opinions; and\nb. how to use those classifications to lead legitimately constructive conversations with others\nLately, I’ve been trying to put my own opinions into one of two categories:\n\nStrong Opinions: Opinions that I have built up over time and experience, and that I feel I have a compelling case for that clearly communicates the full scope of costs and benefits to the decisions that opinion leads me to propose\nWeak Opinions: These are the ones that I have that are built on sub-par information, that I can’t clearly communicate, or that are generally based on ✨ vibes 🌈\n\nWhen contributing to a conversation or argument on a topic, I’ve been trying really hard to ask myself: “Is the opinion I’m about to state a strong opinion?”\nHaving a weak opinion on a topic doesn’t necessarily mean it shouldn’t be mentioned at all. It does mean that it’s likely a point that, should somebody else have a stronger opinion, that opinion isn’t a hill I should consider dying on.\nThat said, having a strong opinion also doesn’t necessarily mean it’s something I ought to be doubling (or, god forbid, tripling down on). Especially when in a conversation with more senior engineers — which, at the current stage of my career, is vastly more likely — a strong opinion is one in which I ought to present my best case, rationale, and analysis in the hopes of reaching consensus.\nEven if my case for one of my strong opinions fails to reach a consensus — or even convince anybody, which does happen (sometimes a lot) (most of the time) — that isn’t failure. Ultimately, as I’ve gotten further into my career and seen a broader range of opinions reach a broader range of success or shortcoming, my approach has ultimately fallen back to:\nThe best side of the road to drive on is the side everybody else has agreed to drive on.\nNo matter how strongly you might feel one way or another, never underestimate the power of consensus. Either way, the outcome of accepting consensus is positive:\n\nYour opinion ended up being the way to go in hindsight, and you’ve gained experience to make a better case the next time it comes up; or\nThe consensus opinion goes perfectly fine, you achieve your goals, and have gained experience on which you can build new, better-informed opinions.\n\nA final open letter\nI’ve noticed, in working with others, that the above classifications can somewhat freak people out — especially if they’re used to working with Strong Opinion People™, because how quickly I’m willing to roll over on my weak(er) opinions has the potential to come across as misunderstanding or apathy.\nAs an open letter to people in the future that see me do this, I make you this promise: when I quickly fold on an opinion, it is far more likely that — in that moment — I feel there’s more value in reaching a consensus and moving forward with any opinion than trying to force through a bad argument for a strong argument or a null argument for a weak one. At this point in my life, I am more interested in making progress than being right."},"notes/supply-chain":{"title":"From the Archive: \"What *are* Supply Chains, Anyway?\"","links":["tags/viz/","notes/caveat-lector","notes/qamo","contact"],"tags":["economics/finance","engineering/python","viz","notes"],"content":"Below, you’ll find some graphs. These are what is known in the field of data visualization as “bad.”\nThese graphs are (as far as I can recall) my first attempts at using & visualizing network structures. As much as I would like to say they’re not too bad for a first attempt, they don’t “speak for themselves” as much as they “scream torments in the face of the viewer.”\nThe below graphs are stock tickers, connected to one another by their supplier relationships. For example, if Company A publicly noted that they received supplies from Company B, that would be denoted by a connecting edge.\n\nBelow is my first (and only) attempt to organize the above output. If I’m recalling correctly (and this was in 2019, so caveat lector), this was at least an attempt to get the first-order to third-order suppliers of a given set of equities and place their order in shells around the center. It’s messy, it’s noisy, and frankly, incomprehensible.\n\nThe hope of this analysis was to ultimately connect supply chains and, as a secondary goal, find graph cycles among a set of target stocks to find and catalog “supply loops”, and see if stocks in a given supply loop rise and fall together. The visualization acted more as a sanity check to see if my web scraper was working and less of a rigorous tool for understanding.\nDespite my area of focus being the parts of economics that aren’t related to either studying (or making) money, this is something I’d like to return to again in the future. However, I unfortunately don’t have the same level of data access that I did as an undergraduate, and purchasing access to a Bloomberg terminal falls well outside the financial means of CGC. However, if you’re ever looking for somebody to make bad graphs pro-bono and do have access to that kind of data, please feel free to reach out."},"notes/the-quest-to-slay-the-goblin":{"title":"The Quest to Slay The Goblin","links":["tags/goblin-slaying","notes/caveat-lector"],"tags":["goblin-slaying","notes"],"content":"I am terrified by how quickly, without structure and habits, my life devolves into a cycle of sleeping, eating, procrastinating, and repeating until the visage of death sweeps me from this mortal coil.\nThis cycle is called “The Goblin”\nI hate The Goblin. The Goblin makes me sad. The Goblin stops me from doing the things that make me happy. Every day, I have to actively re-remember how much I hate The Goblin, and how much happier I am when it’s gone, so I try my best to kill The Goblin.\nI also hate the word “productivity.” There’s a sort of LinkedIn-centric slice of society that has built up the neutrality as an honest days’ work as a divinely-capitalized entity: Productivity™. It elicits (in me, personally) the visceral feeling I get when I see hustle-culture, rise-and-grind, success-is-a-choice-and-failure-is-your-fault types - the ones that farm for engagement by repeating gross approximations of quotes from self-help books that have been Human Centipede’d from one hustler to another. \nThat being said - productivity tools are some of my most reliable Goblin-slaying weapons. This leaves me feeling very conflicted. Absent the weird Soylent-chugging culture that has formed around capital-P Productivity™, I would be in (and have been to) a really rough place without the productivity tools and habits I’ve tried my best to use and form in my quest to kill The Goblin.\nAs a transparent act of shoving this internal inconsistency into the box in the back of my skull labelled “Don’t Think About It!”, I’ll be using the goblin-slaying tag to denote posts about productivity and productivity tools.\nIn these posts, I’ll talk a bit about the structures I put up in my own life, and the tools I use to enforce those structures. To some degree, goblin-slaying posts exist because I hope it might help another person who has their own Goblin to slay and is looking for new tools to slay it. To a far greater degree, they’re here because the act of writing it down helps me set these structures in stone for myself. It is surprisingly easy to forget about good habits when, some day in the future, they’re no longer the habits they used to be.\nCaveat lector - you are you, and I am me. Some of these posts may resonate with you, and some of them may not, because different people are different."},"notes/time-tracking":{"title":"Some initial thoughts about time tracking","links":["notes/season-of-rhythm","tags/goblin-slaying"],"tags":["goblin-slaying","notes"],"content":"Time-tracking is one of the tools I’ve picked up during my Year of Rhythm and goblin-slaying. I’ll probably write about it more, here, later."},"notes/utah-office-consult":{"title":"From the Archive: \"Utah Office Consult\"","links":["notes/scratch/the-best-side-of-the-road"],"tags":["engineering/python","economics/strategy","data/analytics","viz","notes","consulting"],"content":"\nThis presentation is one that I did for an organization I was a part of during college. Looking back, this is probably the first time this phrase clicked in my brain:\n\nPeople problems are the hardest engineering problems\n\nFrom the first time that phrase entered into my head, it has ricocheted against the inside of my skull every single day. In computer science, problems are relatively easy to solve because the only thing you have to convince is a compiler. I am not (nor would I ever claim to be) a sociologist, but nonetheless feel safe in stating that people are not compilers. People have opinions. People have motivations. To work together, to have systems, to make any progress as a society, people have to agree on things.\nNever forget - the best side of the road to drive on is the side everybody else has agreed to drive on. When it comes to people problems, consensus is king.\nI’d imagine that the vast majority of you are people, like me, whose professional reputations to not precede you. As fellow members of the proletariat who fall outside of the LinkedIn-influencer and TED-talk alumni elite, we don’t have a whole lot of ethos on which to make our arguments for why people should agree with an idea. I don’t think that’s a huge deal (although golly gee, wouldn’t it be nice to have authority on matters?). It does mean, though, that turning an idea into consensus has to rely on logic and vibes.\nTo that end, I do think that using what I’d describe as “soft” quantitative frameworks for largely qualitative (read: vibes) properties of things is a good way to bring pathos and logos together. In the above presentation, using distance from other locations is not necessarily a catch-all description for the causes of many of the above problems, but it was a good rule-of-thumb."},"tags/annotations":{"title":"Reading Annotations","links":[],"tags":[],"content":""},"tags/articles":{"title":"CGC Articles","links":["tags/notes/"],"tags":[],"content":"Here, you’ll find some actual, bona-fide Articles™. You can expect a higher degree of cohesion, revision, and quality than you may find in, say, notes."},"tags/business/admin":{"title":"CGC Business Admin","links":[],"tags":[],"content":""},"tags/economics/finance":{"title":"Economics & Finance","links":[],"tags":[],"content":""},"tags/economics/index":{"title":"Economics","links":[],"tags":[],"content":"Economics is the core research focus for Chaotic Good Computing. Wherever there is a finite amount of resources and an infinite capacity for demand, an economy forms.\nCGC’s focus is analysis & creation of well-balanced and intentionally-designed economies, particularly in digital spaces. Whether it’s players interacting with game systems or each other, the way you interact with your cloud provider, or even the way you work online at your company, well-designed systems are hard to understand and even harder to perfect."},"tags/economics/market-design":{"title":"Market Design & Non-Market Environments","links":[],"tags":[],"content":""},"tags/economics/project-management":{"title":"Project Management","links":[],"tags":[],"content":""},"tags/economics/strategy":{"title":"Economics & Strategy","links":[],"tags":[],"content":""},"tags/engineering/csharp":{"title":"C\\#","links":[],"tags":[],"content":""},"tags/engineering/index":{"title":"engineering","links":[],"tags":[],"content":""},"tags/engineering/python":{"title":"Python","links":[],"tags":[],"content":""},"tags/games/roblox":{"title":"Roblox","links":[],"tags":[],"content":""},"tags/goblin-slaying":{"title":"Goblin Slaying (a.k.a Productivity)","links":["notes/the-quest-to-slay-the-goblin"],"tags":[],"content":"Goblin slaying is a somewhat underhanded term for productivity habits and tools."},"tags/horticulture":{"title":"Digital Gardening, and Horticulture as Research","links":["notes/digital-gardening-with-quartz"],"tags":[],"content":"As a logical consequence of taking a digital gardening approach to writing, the horticulture tag will be used to catalog thoughts related to digital gardening and general research.\nI also wanted to use horticulture because I think it’s a funny word."},"tags/mentoring/index":{"title":"CGC Mentoring","links":[],"tags":[],"content":""},"tags/mentoring/tutorials":{"title":"Tutorials","links":[],"tags":[],"content":""},"tags/notes/daily":{"title":"Daily Notes","links":["notes/caveat-lector","notes/digital-gardening-with-quartz"],"tags":[],"content":"These are daily notes, where I log and track progress on ongoing projects and make notes for myself later. These are largely for my own benefit, and are here in case they’re helpful to myself (or, somehow, others) in the future.\nHowever, I’d describe these as loose, at best. Caveat lector.\nOne of the approaches to digital gardening is “learning in public” - to that end, I plan on continuing to outwardly publish these."},"tags/notes/index":{"title":"Spencer's Notes","links":["tags/articles","notes/daily","notes/scratch"],"tags":[],"content":"These are notes - they’re things that are structured, albeit not necessarily as structured or refined as CGC articles.\nIn general, there are three kinds of notes I make:\n\nGeneral notes, which are found on this page\nDaily Notes, which contain daily logs for my own benefit\nScratch Notes, which really just means “none of the above”\n"},"tags/notes/scratch":{"title":"Scratch Notes","links":["tags/horticulture"],"tags":[],"content":"For all intents and purposes, these are the digital gardening equivalent of shitposts and shower thoughts."},"tags/projects/cgc":{"title":"CGC Projects","links":[],"tags":[],"content":""},"tags/projects/dayjob":{"title":"Day Job Projects","links":[],"tags":[],"content":"For obvious reasons, anything related to these projects will be very vague."},"tags/projects/index":{"title":"Projects","links":[],"tags":[],"content":""},"tags/projects/personal":{"title":"Personal Projects","links":[],"tags":[],"content":""},"tags/viz/doodles":{"title":"Doodles, a.k.a. Minimally Professional Presentations","links":[],"tags":[],"content":""},"tags/viz/index":{"title":"Data Visualizations & Presentations","links":[],"tags":[],"content":""}}
\ No newline at end of file