diff --git a/common-content/en/module/tools/comparing-javascript-and-python/index.md b/common-content/en/module/tools/comparing-javascript-and-python/index.md new file mode 100644 index 000000000..aeef8151c --- /dev/null +++ b/common-content/en/module/tools/comparing-javascript-and-python/index.md @@ -0,0 +1,95 @@ ++++ +title = "Comparing JavaScript and Python" +headless = true +time = 40 +facilitation = false +emoji= "📖" +objectives = [ + "Identify and explain equivalences between JavaScript and Python", + "Compare and contrast differences between JavaScript and Python", + "Distinguish between essential and accidental complexity" +] ++++ + +JavaScript and Python have many things in common. + +Most differences are "cosmetic". Here are some examples of cosmetic differnces: +* Some functions and operators have different names. But often there are functions/operators which do exactly the same thing. +* JavaScript uses `{}` around blocks of code and we _choose_ if we indent code. Python uses `:` and indentation is required. +* In JavaScript we choose to name variables in `camelCase`, whereas in Python we choose to name variables in `snake_case`. In both langues we _could_ do either; this is called a {{}}A convention is something a group (maybe a team, or a company, or most users of a programming language) agree to do. It's not quite a rule - things _could_ work another way. But we agree one way we'll all do it anyway.

e.g. in Python you could name one variable `firstName` and another `middle_name` and another `LASTname`, but if everyone agrees to use `first_name` and `middle_name` and `last_name` it makes it a bit easier for everyone to read because they know what to expect.{{}}. + +Recall our "count containing words" JavaScript code. Now think about what it would look like in Python. + +```js +import { program } from "commander"; +import { promises as fs } from "node:fs"; +import process from "node:process"; + +program + .name("count-containing-words") + .description("Counts words in a file that contain a particular character") + .option("-c, --char ", "The character to search for", "e"); + +program.parse(); + +const argv = program.args; +if (argv.length != 1) { + console.error(`Expected exactly 1 argument (a path) to be passed but got ${argv.length}.`); + process.exit(1); +} +const path = argv[0]; +const char = program.opts().char; + +const content = await fs.readFile(path, "utf-8"); +const countOfWordsContainingChar = content + .split(" ") + .filter((word) => word.includes(char)) + .length; +console.log(countOfWordsContainingChar); +``` + +{{}} +===[[Exercise]]=== +Think about what we're doing in this code. + +Try to list the high-level ideas. This means describing in English what we're achieving, using sentences like like "Reading a file". + +We're not trying to think about the programming concepts we're doing here (we aren't talking about things like "Assigning a variable" or "An if statement"). Think about what a non-programmer would want to understand about our program. + +===[[Answer]]=== + +You may have slightly different answers, but the programme is doing roughly the following things: + +* Parsing command line flags - writing down what flags we expect to be passed, and reading values for them based on the actual command line. +* Validating the flags (i.e. checking that exactly one path was passed). +* Reading a file. +* Splitting the content of the file up into words. +* Counting how many of the words contained a particular character. +* Printing the count. + +{{}} + +These are the meaningful things we needed to do. To solve the same problem with Python, we'd still do all of these things. + +We did some other things in our code to make it work. For example, we imported some modules. To write this code in Python, we might need modules or we might not. Importing modules isn't one of our _goals_, it was just something we needed to do to help us. + +We split up things we need to do into two categories: essential and accidental. + +**Essential** means it is a core part of the problem. e.g. in order to count how many words are in a file, it is _essential_ that we read the file. + +**Accidental** means it isn't what we _care_ about doing, but we may need to do it anyway. e.g. importing the `process` module isn't _essential_ to our problem, but we needed to do it anyway so we could report errors. + +{{}} +Imagine we want to post a parcel, so we take the bus to the post office. + +_Essential_ to our goal is getting the parcel to someone who will deliver it. + +_Accidental_ to this, we took the bus. There may be ways we could achieve our essential goal without getting the bus. Maybe we could walk or cycle to the post office. Maybe we could arrange for someone from the post office to come to our home and collect the parcel. + +The accidental things we did were important - they helped us get our essential goal done. But we shouldn't get too attached to the accidental things - maybe we will replace them later. +{{}} + + +When we're thinking about how we use different languages, it's useful to think about what parts of our problem are _essential_ (we'll need to do them in any language), and which parts are _accidental_ (it's just something we had to do on the way to achieve our aim). + +Whether we write the JavaScript `someArray.length` or the Python `len(some_array)` isn't a big difference. Both lines do the same thing, they just express it differently. diff --git a/common-content/en/module/tools/convert-script-between-languages/index.md b/common-content/en/module/tools/convert-script-between-languages/index.md deleted file mode 100644 index 9f94143fe..000000000 --- a/common-content/en/module/tools/convert-script-between-languages/index.md +++ /dev/null @@ -1,11 +0,0 @@ -+++ -title = "Convert a script between languages" -headless = true -time = 30 -emoji= "📖" -[objectives] - 1="Identify and explain equivalences between JavaScript and Python" - 2="Identify and explain differences between JavaScript and Python" -+++ - -### Convert a script between languages diff --git a/common-content/en/module/tools/converted-program/index.md b/common-content/en/module/tools/converted-program/index.md new file mode 100644 index 000000000..187229668 --- /dev/null +++ b/common-content/en/module/tools/converted-program/index.md @@ -0,0 +1,35 @@ ++++ +title = "Putting it all together" +headless = true +time = 30 +facilitation = false +emoji= "📖" +hide_from_overview = true +objectives = [ +] ++++ + +Instead of calling `console.log`, in Python we call `print`. + +```python +import argparse + +parser = argparse.ArgumentParser( + prog="count-containing-words", + description="Counts words in a file that contain a particular character", +) + +parser.add_argument("-c", "--char", help="The character to search for", default="e") +parser.add_argument("path", help="The file to search") + +args = parser.parse_args() + +with open(args.path, "r") as f: + content = f.read() +count_of_words_containing_char = len([word for word in content.split(" ") if args.char in word]) +print(count_of_words_containing_char) +``` + +This looks similar to the JavaScript version. The shape is the same, but every line is a little bit different. + +Some programming languages are very different, as different as Mandarin and English. But JavaScript and Python are, essentially, quite similar, like Spanish and Portuguese. diff --git a/common-content/en/module/tools/converting-javascript-to-python/index.md b/common-content/en/module/tools/converting-javascript-to-python/index.md new file mode 100644 index 000000000..554e87f75 --- /dev/null +++ b/common-content/en/module/tools/converting-javascript-to-python/index.md @@ -0,0 +1,83 @@ ++++ +title = "Converting JavaScript to Python" +headless = true +time = 90 +facilitation = false +emoji= "📖" +objectives = [ + "Rewrite JavaScript code as Python" +] ++++ + +### Parsing command line flags + +In JavaScript, we wrote this code: + +```js +import { program } from "commander"; + +program + .name("count-containing-words") + .description("Counts words in a file that contain a particular character") + .option("-c, --char ", "The character to search for", "e"); + +program.parse(); + +const argv = program.args; +const path = argv[0]; +const char = program.opts().char; +``` + +Which of the following are _essential_ goals in this code, and which are _accidental_ goals? + +{{}} +[LABEL=Essential] Allow a user to pass a `-c` argument (defaulting to `e` if they don't). +[LABEL=Accidental] Made a `const` variable called `argv`. +[LABEL=Accidental] Import `program` from the `commander` library. +[LABEL=Essential] Allow a user to pass a path as a positional argument. +[LABEL=Accidental] Looked up element `0` in the `program.args` array. +[LABEL=Essential] Supply a nice `--help` implementation to help a user if they don't know how to use our tool. +[LABEL=Accidental] Use the commander library. +[LABEL=Accidental] Called the function `program.name()`. +{{}} + +If we want to work out how to do this in Python, we should focus on the essential goals. We may want to search for things like "Parse command line flags Python" and "Default argument values Python" because they get to the essential problems we're trying to solve. + +Searching Google for "Parse command line flags Python" brought us to [the Python argparse documentation](https://docs.python.org/3/library/argparse.html). The example code looks pretty similar to what we were doing in JavaScript. We can probably write something like: + +```python +import argparse + +parser = argparse.ArgumentParser( + prog="count-containing-words", + description="Counts words in a file that contain a particular character", +) + +parser.add_argument("-c", "--char", help="The character to search for", default="e") +parser.add_argument("path", help="The file to search") + +args = parser.parse_args() +``` + +There are some differences here. +* With commander we were calling functions on a global `program`, whereas with argparse we construct a new `ArgumentParser` which we use. +* `add_argument` takes separate parameters for the short (`-c`) and long (`--char`) forms of the option - `commander` expected them in one string. +* The Python version uses a lot of named arguments (e.g. `add_argument(...)` took `help=`, `default=`), whereas the JavaScript version (`option(...)`) used a lot of positional ones. +* The Python version handles positional arguments itself as arguments with names (`path`), whereas the JavaScript version just gives us an array of positional arguments and leaves us to understand them. + +### Validating command line flags + +In our JavaScript code, we needed to check that there was exactly one positional argument. + +We don't need to do this in our Python code. Because `argparse` treats positional arguments as arguments, it actually already errors if we pass no positional arguments, or more than one. + +So we can tick this essential requirement off our list. Different languages or libraries do things differently, and that's ok! + +> [!TIP] +> We don't need to convert every line. +> +> We're trying to convert _essential requirements_. + +{{}} +Identify all of the essential requirements from our JavaScript program, and finish implementing the Python version. +{{}} diff --git a/common-content/en/module/tools/counting-words/index.md b/common-content/en/module/tools/counting-words/index.md new file mode 100644 index 000000000..e542320ff --- /dev/null +++ b/common-content/en/module/tools/counting-words/index.md @@ -0,0 +1,44 @@ ++++ +title = "Counting words containing a character" +headless = true +time = 45 +facilitation = false +emoji= "📖" +hide_from_overview = true +objectives = [ +] ++++ + +In JavaScript we wrote: + +```js +content + .split(" ") + .filter((word) => word.includes(char)) + .length +``` + +What JavaScript calls arrays, Python calls lists. Arrays and lists are basically the same. + +Googling for "Python filter list" suggests there are two things we can use - a `filter` function, or something called a "list comprehension". Some people prefer one, other people prefer the other. + +Let's try out both approaches. We can do this in a standalone program, rather than in the whole word-counting program. This gives us a lot more control, and makes it easier for us to experiment. + +{{}} +Create a new file, `filter.py`. Start it with: + +```python +content = "this is a list of words" +char = "i" + +filtered = TODO + +print(filtered) +``` + +Now fill in the TODO. First, use a list comprehension. Run the file and make sure you get the expected output. + +Next, replace your list comprehension with some code that calls the global function `filter`. (`filter` takes a function, and it may be useful to know that `lambda` is a keyword for making an anonymous function in Python, similar to arrow functions in JavaScript). Run the file and make sure you get the expected output. +{{}} + +Now that we've learnt how to do the filtering, we can apply what we've learnt to the program we're converting. diff --git a/common-content/en/module/tools/first-nodejs-program/index.md b/common-content/en/module/tools/first-nodejs-program/index.md new file mode 100644 index 000000000..1e1a4a544 --- /dev/null +++ b/common-content/en/module/tools/first-nodejs-program/index.md @@ -0,0 +1,163 @@ ++++ +title = "Writing a NodeJS program" +headless = true +time = 60 +facilitation = false +emoji= "🛠️" +hide_from_overview = true +objectives = [ + "Write a zero-dependencies NodeJS program", +] ++++ + +Below we have a small NodeJS program. It is a bit like `wc`. It counts words in a file which contain the letter `e`. + +Our program accepts one command line argument - the path of the file to read and count. + +Our program's output to stdout is just the number of words which contain an e. + +Our program uses the same language (JavaScript) as we've written before, but uses some different APIs. + +```js +import process from "node:process"; +import { promises as fs } from "node:fs"; + +const argv = process.argv.slice(2); +if (argv.length != 1) { + console.error(`Expected exactly 1 argument (a path) to be passed but got ${argv.length}.`); + process.exit(1); +} +const path = argv[0]; + +const content = await fs.readFile(path, "utf-8"); +const countOfWordsContainingEs = content + .split(" ") + .filter((word) => word.includes("e")) + .length; +console.log(countOfWordsContainingEs); +``` + +Let's play computer with this program - line by line: + +```js +import process from "node:process"; +``` + +This `import` is loading some code from somewhere that isn't this file. + +We've seen `import` before. Here, instead of importing from a file we've written, we're importing the `process` module which is built into NodeJS. + +This is an example of the same language features (`import`) being used slightly differently (the `"node:"` is a special prefix to say "specially from node"). + +The `process` module is built into NodeJS for managing our {{}}The running instance of our program: the code, state, memory, and system resources.{{}}. We can use it to find out what arguments were passed to the process when it started, find out what user ran the process, exit the process, and more. + +```js +import { promises as fs } from "node:fs"; +``` + +We're importing another module. + +The `fs` module is built into NodeJS for interacting with the filesystem. + +This time, we're not importing the whole module. We are {{}}Destructuring is a form of variable assignment where we give variables values based on where we can find them structurally in another value. Examples:

We can write `const [first, second] = [3, 1];` to assign `first = 3` and `second = 1`.

We can write `const {name, age} = {name: "Amir", age: 34};` to assign `name = "Amir"` and `age = 34`.{{}}. The `node:fs` module exposes an object, and we are saying "import the `promises` property from the `fs` module, and bind it to the name `fs`". + +It is like writing `import { promises } from "node:fs"; const fs = promises;`. + +The `fs` module exposes two alternate APIs with functions with the same name (like `readFile`): +* The default `fs` module was written before `async`/`await` was added to JavaScript, and requires using callbacks, which can be annoying. +* `fs` has a submodule called `promises` which can be used with `async`/`await`, and is generally much more convenient. + +We want to use the `promises` submodule, because it's much more convenient for us to use `async`/`await`. But if we just wrote `import { promises } from "node:fs";`, we'd be binding the submodule to the name `promises`, and everywhere we used it we'd need to write `promises.readFile`. This is less clear than `fs.readFile`, because `promises` is a very general name. So we rename `promises` to `fs`, and can use it like `fs.readFile`. + +We are really doing this because we wish the `async`/`await` APIs were the default APIs exposed by the `fs` module, and this lets us pretend that they are in the rest of our code. + +{{}} +`fs` uses callbacks or promises because its operations are asynchronous. + +Why would interacting with the filesystem (e.g. reading a file) be an asynchronous operation? + +Explain on a Slack thread why you think this is. If you're not sure, ask about it on Slack. +{{}} + +```js +const argv = process.argv.slice(2); +``` + +We're getting the `argv` array from the `process` module, and slicing it. We can see in [the `process.argv` documentation](https://nodejs.org/api/process.html#processargv) that `process.argv[0]` will be the path to `node`, and `process.argv[1]` will be the path to this file. We don't care about those, so we'll skip them - as far as we're concerned the arguments start at index 2. + +Again, `Array.slice` is exactly the same as we know from JavaScript, but `process.argv` is a new API we can use to get the array we need. + +Play computer with the rest of the program - read each line, and explain what you think that line does. After you make your predictions, expand the explanations below and compare them to your predictions. + +
+ + +```js +if (argv.length != 1) { + console.error(`Expected exactly 1 argument (a path) to be passed but got ${argv.length}.`); + process.exit(1); +} +``` + +We always expect our program to be given exactly one argument. Here we check this using an `if` statement, just like we've seen before. + +`console.error` writes a message to stderr (which is where error messages should go). + +`process.exit` is a function which, when called, will stop our program running. Passing a non-zero number to it indicates that our program did not succeed. We can read more about it in the official NodeJS documentation for the `process` module. + +
+ +
+ + +```js +const path = argv[0]; +``` + + +Giving a useful name to our argument. +
+ +
+ + +```js +const content = await fs.readFile(path, "utf-8"); +``` + + +Reading the file at the path passed as an argument. We're using the `fs` module here from `node`, but everything else is just JavaScript - declaring a variable, using `await` because `fs.promises.readFile` is an `async` function, calling a function. + +You can read more about this in [the documentation for `fs.promises.readFile`](https://nodejs.org/api/fs.html#fspromisesreadfilepath-options). +
+ +
+ + +```js +const countOfWordsContainingEs = content + .split(" ") + .filter((word) => word.includes("e")) + .length; +``` + + +Just some regular JavaScript. Taking a string, splitting it into an array, filtering the array, searching strings to see if they contain any e characters, and getting the length of an array. +
+ +
+ + +```js +console.log(countOfWordsContainingEs); +``` + + +`console.log` in a NodeJS environment logs to stdout, so this outputs our result to stdout. +
+ +{{}} +Save the above program into a file. Run the file with `node`, and count how many words contain "e"s in a few different files. + +If you run into problems, ask for help. +{{}} diff --git a/common-content/en/module/tools/implement-single-use-data-analysis/index.md b/common-content/en/module/tools/implement-single-use-data-analysis/index.md deleted file mode 100644 index ac9517e88..000000000 --- a/common-content/en/module/tools/implement-single-use-data-analysis/index.md +++ /dev/null @@ -1,12 +0,0 @@ -+++ -title = "Implement a single-use data analysis program" -headless = true -time = 30 -emoji= "💻" -[objectives] - 1="Write a program to extract information from a JSON file" - 2="Identify the trade-offs between using existing shell tools and writing custom programs" - 3="Choose whether to use existing tools or write a custom program to solve a particular problem" -+++ - -### Implement a single-use data analysis program diff --git a/common-content/en/module/tools/implement-tools-in-nodejs/index.md b/common-content/en/module/tools/implement-tools-in-nodejs/index.md deleted file mode 100644 index 03d3c2528..000000000 --- a/common-content/en/module/tools/implement-tools-in-nodejs/index.md +++ /dev/null @@ -1,12 +0,0 @@ -+++ -title = "Implement tools in NodeJS" -headless = true -time = 30 -emoji= "💻" -[objectives] - 1="Implement cat in NodeJS" - 2="Implement wc in NodeJS" - 3="Implement ls in NodeJS" -+++ - -### Implement tools in NodeJS \ No newline at end of file diff --git a/common-content/en/module/tools/implement-tools-in-python/index.md b/common-content/en/module/tools/implement-tools-in-python/index.md deleted file mode 100644 index 97b322207..000000000 --- a/common-content/en/module/tools/implement-tools-in-python/index.md +++ /dev/null @@ -1,12 +0,0 @@ -+++ -title = "Implement tools in Python" -headless = true -time = 30 -emoji= "💻" -[objectives] - 1="Implement cat in Python" - 2="Implement wc in Python" - 3="Implement ls in Python" -+++ - -### Implement tools in Python \ No newline at end of file diff --git a/common-content/en/module/tools/installing-npm-dependencies/index.md b/common-content/en/module/tools/installing-npm-dependencies/index.md new file mode 100644 index 000000000..05400c290 --- /dev/null +++ b/common-content/en/module/tools/installing-npm-dependencies/index.md @@ -0,0 +1,136 @@ ++++ +title = "Installing dependencies with npm" +headless = true +time = 60 +facilitation = false +emoji= "📚" +objectives = [ + "Use a dependency in a NodeJS program", +] ++++ + +To use a library, we need to fetch the code we're going to use. When using NodeJS, we use a tool called `npm` for this. + +First we need a `package.json` file - this a file that `npm` will read to understand your project. This is the same as the `package.json` file you've seen when using `npm` in the past. + +Make this `package.json` file in the same directory as your e-word-counting program: + +```json +{ + "type": "module" +} +``` + +The `package.json` contains a JSON object with information about your project. For now, we're just telling `npm` that our project is a module - this means we are allowed to use `import` in our program. + +From a terminal which is `cd`'d to the same directory as your `package.json` file, run `npm install commander`. + +This command does two things: +1. Look in your `package.json` file - notice that now has a `dependencies` section listing `commander`. This means that if someone else downloads your program, they know they need to install `commander` to use it. +2. There's now a `node_modules` directory alongside your `package.json`. Inside that is a directory named `commander` which contains the code for the `commander` library. This means `node` now knows how to find the code when you try to import it. + +{{}} +Try running your program again. + +What has changed since the last time you tried to run it (and it didn't work)? + +What has changed since the last time you successfully ran it? +{{}} + +Now that we have `commander` installed, let's try using it in our program: + +```js +import { program } from "commander"; +import { promises as fs } from "node:fs"; +import process from "node:process"; + +program + .name("count-containing-words") + .description("Counts words in a file that contain a particular character") + .option("-c, --char ", "The character to search for", "e"); + +program.parse(); + +const argv = program.args; +if (argv.length != 1) { + console.error(`Expected exactly 1 argument (a path) to be passed but got ${argv.length}.`); + process.exit(1); +} +const path = argv[0]; +const char = program.opts().char; + +const content = await fs.readFile(path, "utf-8"); +const countOfWordsContainingChar = content + .split(" ") + .filter((word) => word.includes(char)) + .length; +console.log(countOfWordsContainingChar); +``` + +{{}} +Try running this program with the `--help` flag. + +What do you see? Where do you think this behaviour and text came from? + +We didn't have to write all the code for this functionality - `commander` did most of it for us. +{{}} + +{{}} +Try running the program with different values of the `-c` flag. Try also specifying some other flags, like `--count`. + +Make sure you understand how it's behaving, and why. +{{}} + +Let's run through what we changed: + +```js +program + .name("count-containing-words") + .description("Counts words in a file that contain a particular character") + .option("-c, --char ", "The character to search for", "e"); +``` + +We told `commander` information about our program. We gave it a name, a description, and told it that it should allow a user to pass a flag name `-c` (or equivalently `--char`), and use a default value of `-` for that flag if it's not specified. + +```js +program.parse(); +``` + +We asked `commander` to interpret the command line arguments our program was given, based on what options we wanted to allow. If it sees something it doesn't understand, it will error. + +```js +const argv = program.args; +``` + +Instead of asking NodeJS's process module for all of the program's arguments, we're asking `commander` to tell us "after you understood and removed all the flags, what arguments were left?" + +Then our `if` check about the number of arguments is exactly the same as before. + +```js +const char = program.opts().char; +``` + +We are getting the `char` flag that `commander` interpreted and storing it in a variable. + +```js +const countOfWordsContainingChar = content + .split(" ") + .filter((word) => word.includes(char)) + .length; +console.log(countOfWordsContainingChar); +``` + +We have renamed our `countOfWordsContainingEs` variable to `countOfWordsContainingChar` because we're no longer always looking for hyphens, and changed the `includes` call to look for the value of the `char` variable instead of always an `e`. + +We only needed to make a few small changes to get all of this new functionality: +* Support for accepting a new command line flag. +* `--help` support explaining how to use the program. +* Detection for if someone passes flags that aren't known, and warning them about this (and even suggesting what they maybe meant). + +We could have written all of this code ourselves. But using a library meant we could focus on what's unique about our problem, rather than spending time implementing flag parsing. + +This is a very common task in software development in the real world, joining together libraries (written by other people) to create some new unique solution. + +> [!NOTE] +> +> We also could have used [the builtin `util.parseArgs` function from NodeJS](https://nodejs.org/api/util.html#utilparseargsconfig) for most of this functionality, but it doesn't support `--help` like `commander` does. diff --git a/common-content/en/module/tools/learn-nodejs/index.md b/common-content/en/module/tools/learn-nodejs/index.md deleted file mode 100644 index 4801018bc..000000000 --- a/common-content/en/module/tools/learn-nodejs/index.md +++ /dev/null @@ -1,12 +0,0 @@ -+++ -title = "Learn NodeJS" -headless = true -time = 30 -emoji= "❓" -[objectives] - 1="Identify and explain why someone may use NodeJS" - 2="Write a zero-dependencies NodeJS program" - 3="Use a dependency in a NodeJS program" -+++ - -### Learn NodeJS \ No newline at end of file diff --git a/common-content/en/module/tools/learn-python/index.md b/common-content/en/module/tools/learn-python/index.md deleted file mode 100644 index 021457cda..000000000 --- a/common-content/en/module/tools/learn-python/index.md +++ /dev/null @@ -1,12 +0,0 @@ -+++ -title = "Learn Python" -headless = true -time = 30 -emoji= "❓" -[objectives] - 1="Identify and explain why someone may use Python" - 2="Write a zero-dependencies Python program" - 3="Use a dependency in a Python program" -+++ - -### Learn Python diff --git a/common-content/en/module/tools/nodejs/index.md b/common-content/en/module/tools/nodejs/index.md new file mode 100644 index 000000000..8c8be0425 --- /dev/null +++ b/common-content/en/module/tools/nodejs/index.md @@ -0,0 +1,33 @@ ++++ +title = "NodeJS" +headless = true +time = 20 +facilitation = false +emoji= "❓" +objectives = [ + "Identify and explain why someone may use NodeJS", +] ++++ + +We know that that JavaScript is an interpreted language. Running it needs some interpreter to read our lines of code and execute them. + +We've already seen that web browsers can run JavaScript. Web browsers provide a runtime environment for JavaScript. + +NodeJS is another runtime environment for running JavaScript. It allows us to run JavaScript files from a terminal. + +There are some similarities and differences between how NodeJS runs JavaScript, and how web browsers run JavaScript. For instance: +* Both support the same core language (e.g. defining variables, if statements, for loops, etc). +* Web browsers {{}}To expose an API means to provide functions or values to the programmer. Sometimes we expose these over the internet, using HTTP+JSON. Other times we expose them directly as symbols you can import into your program.{{}} extra APIs that can be used from JavaScript, e.g. the DOM. +* NodeJS exposes extra APIs that can be used from JavaScript, e.g. reading and writing files in the filesystem. +* Some APIs are implemented differently, e.g. if you call `console.log` in a web browser it will log to the web inspector console (hidden by default), whereas in NodeJS it will log to stdout (the default output of a program). + +People use NodeJS so that they can run code they've written in a terminal. Some example reasons: +* Because they want to use NodeJS's extra capabilities in their code (e.g. reading files). +* Because they want to use a JavaScript as part of a shell pipeline. +* Because they want their program to run for a long time on a server. + +You've already written JavaScript programs and run them in the NodeJS runtime environment - every time you run a command like `node index.js` or `npm test` you're running JavaScript with NodeJS. + +Most of the programs you wrote and ran like this in the Introduction to Programming course were short-lived experiments (learning a concept and trying it out), or tests. + +We're going to start thinking about writing programs _intended to be run like this_. diff --git a/common-content/en/module/tools/read-about-operating-systems-1/index.md b/common-content/en/module/tools/read-about-operating-systems-1/index.md deleted file mode 100644 index 99d9c0f10..000000000 --- a/common-content/en/module/tools/read-about-operating-systems-1/index.md +++ /dev/null @@ -1,10 +0,0 @@ -+++ -title = "Read about operating systems" -headless = true -time = 30 -emoji= "💻" -[objectives] - 1="TODO" -+++ - -### Operating systems \ No newline at end of file diff --git a/common-content/en/module/tools/read-about-operating-systems-2/index.md b/common-content/en/module/tools/read-about-operating-systems-2/index.md deleted file mode 100644 index a4a7429e9..000000000 --- a/common-content/en/module/tools/read-about-operating-systems-2/index.md +++ /dev/null @@ -1,10 +0,0 @@ -+++ -title = "Read about operating systems (2)" -headless = true -time = 30 -emoji= "💻" -[objectives] - 1="TODO" -+++ - -### Operating systems (2) diff --git a/common-content/en/module/tools/read-about-operating-systems/index.md b/common-content/en/module/tools/read-about-operating-systems/index.md new file mode 100644 index 000000000..11a938371 --- /dev/null +++ b/common-content/en/module/tools/read-about-operating-systems/index.md @@ -0,0 +1,49 @@ ++++ +title = "Operating systems" +headless = true +time = 120 +facilitation = false +emoji= "💻" +objectives = [ + "Define an operating system.", + "Describe what an kernel is.", + "Explain what a process is.", + "List what processes are created when running `ls | grep '[A-Z]'`.", + "Explain what a system call (syscall) is.", + "Give three examples of syscalls.", +] ++++ + +{{}} +Read chapter 10 of How Computers Really Work. + +Do every exercise listed in the chapters. + +You only need to do the projects listed below (though are welcome to try any others that you want!) + +Check you have achieved each learning objective listed on this page. +{{}} + +{{}} +Do project 23 from How Computers Really Work. + +You can do this on any {{}}Linux and macOS are both Unix-family operating systems. Windows is not.{{}} - you do not need a Raspberry Pi. +{{}} + +{{}} +Do project 20 from How Computers Really Work. + +You can do this on any Unix-family OS - you don't need a Raspberry Pi. + +Note: If you're on macOS, `ps -eH` doesn't exist. You can use `ps` or `ps aux` to get a list of processes. To get parent-child relationships, you'll need to install `pstree` using `brew` (`brew install pstree`), then run `pstree`. + +Note: If you're on macOS, process 1 will probably be `launchd` not `init`. +{{}} + +{{}} +If you're on a Linux machine, do projects 21, 22, and 24. + +If you're on macOS, pair up with someone who has a Linux machine to do these projects. + +Note: Several of these projects may not work inside Docker or virtual machines, you need to actually be using Linux. +{{}} diff --git a/common-content/en/module/tools/reading-a-file/index.md b/common-content/en/module/tools/reading-a-file/index.md new file mode 100644 index 000000000..143fd89f5 --- /dev/null +++ b/common-content/en/module/tools/reading-a-file/index.md @@ -0,0 +1,45 @@ ++++ +title = "Reading a file" +headless = true +time = 10 +facilitation = false +emoji= "📖" +hide_from_overview = true +objectives = [ +] ++++ + +In JavaScript we wrote: + +```js +import { promises as fs } from "node:fs"; + +const content = await fs.readFile(path, "utf-8"); +``` + +If we search Google for "Read file Python", we get an example which suggests we can write something like: + +```python +with open(args.path, "r") as f: + content = f.read() +``` + +Comparing these shows some interesting differences, particularly around {{}}Scope is where a variable can be accessed from.{{}}. + +### Scope + +In Python, we made our `content` variable in an indented block. + +In JavaScript this wouldn't have worked - in JavaScript when we declare a variable with `const` it only exists in the scope where it was defined. + +In Python, the `content` variable can be used for the rest of the function it's declared in. We call this {{}}Hoisting is where a variable is considered to exist at a broader scope than where it was declared.{{}}. + +### `with` blocks + +In Python, there's this `with` construct. Instead of writing `f = open(args.path, "r")` we wrote `with open(args.path, "r") as f:`. + +This has two interesting effects: + +One is that the variable we're declaring (`f`) doesn't get hoisted - it only exists within the `with` block. + +The other is that at the end of the `with` block, the file is closed. Not only does `f` stop existing at the end of the block, but some code also gets run to clean up the resources `f` was using. diff --git a/common-content/en/module/tools/single-use-data-analysis/index.md b/common-content/en/module/tools/single-use-data-analysis/index.md new file mode 100644 index 000000000..75f03336e --- /dev/null +++ b/common-content/en/module/tools/single-use-data-analysis/index.md @@ -0,0 +1,93 @@ ++++ +title = "Single-use data analysis programs" +headless = true +time = 60 +facilitation = false +emoji= "💻" +[objectives] + 1="Write a program to extract information from a JSON file" + 2="Identify the trade-offs between using existing shell tools and writing custom programs" + 3="Choose whether to use existing tools or write a custom program to solve a particular problem" ++++ + +We've seen two different ways of analysing some input to produce an output. + +Sometimes we can use, or combine, existing tools to get answers. For instance, we can count the words in a file with `wc`. + +Sometimes we can write custom tools when existing tools don't quite do what we want. For instance, we wrote a program to count specific words. + +When we want to answer some question, sometimes it's useful to write a program we may only use once. (Or we may re-use in the future.) + +It's not always obvious whether it's easier to try to use tools that already exist, or to write our own. + +> [!TIP] +> +> This is like in real life! Imagine if you had two differently sized bottles, and wanted to pour all of the liquid from one to the other without spilling any. +> +> You can imagine making the perfect tube that has exactly the right size connector at each end to connect to the bottles. +> +> Or maybe you already have a funnel that's about the right size - not perfect, but close enough, and you can probably use. +> +> But if you got a really wide or really narrow bottle, maybe that funnel wouldn't be good enough and you would need to make a custom solution. + +Sometimes the format of our data makes it easier or harder to use existing tools. + +Let's look at some sample data: + +```json +[ + { + "name": "Daniel", + "score": 100 + }, + { + "name": "Kristina", + "score": 120 + }, + { + "name": "Iulia", + "score": 95 + }, + { + "name": "Aleks", + "score": 190 + }, + { + "name": "Daniel", + "score": 80 + }, + { + "name": "Fatima", + "score": 110 + } +] +``` + +Here are a few questions we may want to answer about this data: +1. What was the name of the first person to play the game? +2. What was the name of the last person to play the game? +3. Who had the highest score? +4. The names of everyone who played the game directly after Daniel? + +We can probably answer all of these questions with `jq`. We can also definitely write a program to answer all of these questions for us. + +The first three are similarly hard to solve in `jq` or with a programming language. + +The last one is quite hard to solve in `jq`. + +{{}} +Solve all of the first three questions in both `jq` and your choice of JavaScript or Python. + +Which approach do you think is quicker to write? Which is easier to think about? +{{}} + +{{}} +Solve the fourth question in your choice of JavaScript or Python. + +Now spend no more than 20 minutes trying to solve it with `jq`. + +What do you think makes this harder to solve in `jq`? + +What {{}}A heuristic is a guideline. It's not an exact rule, but a "good enough" idea to guess what approach you should use to answer a question.{{}} can you think of about when to use existing tools vs writing your own? + +{{}} diff --git a/common-content/en/module/tools/splitting-a-string/index.md b/common-content/en/module/tools/splitting-a-string/index.md new file mode 100644 index 000000000..40d170453 --- /dev/null +++ b/common-content/en/module/tools/splitting-a-string/index.md @@ -0,0 +1,22 @@ ++++ +title = "Splitting the content of the file up into words" +headless = true +time = 5 +facilitation = false +emoji= "📖" +hide_from_overview = true +objectives = [ +] ++++ + +In JavaScript we wrote: + +```js +content.split(" ") +``` + +Googling for "Python split string" suggests we can write exactly the same code! + +```python +content.split(" ") +``` diff --git a/common-content/en/module/tools/using-npm-dependencies/index.md b/common-content/en/module/tools/using-npm-dependencies/index.md new file mode 100644 index 000000000..9349f5c9d --- /dev/null +++ b/common-content/en/module/tools/using-npm-dependencies/index.md @@ -0,0 +1,32 @@ ++++ +title = "Using dependencies from npm" +headless = true +time = 20 +facilitation = false +emoji= "📚" +objectives = [ + "Define a library", +] ++++ + +We've seen that we can use code that was built into NodeJS - we don't need to write everything ourselves. + +We can also use code that other people have written, which isn't built into NodeJS. You've probably seen this before, e.g. using `jest` for testing. + +This can be really useful - it means we can benefit from work others have already done, and focus on just solving the part of a problem which is unique to us. It's like making shell pipelines - instead of having to solve every problem from scratch, we can plug together different tools that other people have already made. + +Let's expand the functionality of our program. Rather than always searching for words containing the letter e, let's allow the user to specify what character they're searching for. + +This means we want to introduce a flag. And programs that accept flags, should also document themselves. One common convention is that if you run a program with the flag `--help`, it will tell you how to use it. + +But writing all of this code to parse flags, to output information about the flags, and so on, is a lot of work. + +So let's use a {{}}A library is a collection of code that we can use, but which isn't part of our project.{{}} for this. We will use a library called `commander`. + +{{}} +Add `import { program } from "commander";` to the top of your e-word-counting program. + +This line imports the `program` property from the object which is the `commander` library (using object destructuring). + +Try running your program. What happens? What does the output mean? +{{}} diff --git a/common-content/en/module/tools/using-python-dependencies/index.md b/common-content/en/module/tools/using-python-dependencies/index.md new file mode 100644 index 000000000..0d180282e --- /dev/null +++ b/common-content/en/module/tools/using-python-dependencies/index.md @@ -0,0 +1,84 @@ ++++ +title = "Using Python dependencies" +headless = true +time = 30 +facilitation = false +emoji= "📖" +objectives=[ + "Write and run a Python program which uses third-party dependencies.", +] ++++ + +Let's create a small program which uses a dependency. + +We're going to use the `cowsay` library to make a program which outputs a picture of a cow saying something. + +First let's create a Python file which tries to use `cowsay`: + +{{}} +It's important that you don't name your Python file the same as the name of a library you're trying to import. + +We can't call our file `cowsay.py` because we're going to try to import `cowsay`. + +We can call it `main.py` or `hello.py` or `cow.py`. Just not `cowsay.py`. + +In this example, we'll call it `cow.py`. +{{}} + +```python +import cowsay +``` + +Run `python3 cow.py`. It will trigger this error: + +``` +ModuleNotFoundError: No module named 'cowsay' +``` + +This is because we haven't installed cowsay yet. + +### Installing our dependency + +We will create a virtual environment, activate it, and install cowsay to it: + +```console +% python3 -m venv .venv +% . .venv/bin/activate +(.venv) % echo cowsay > requirements.txt +(.venv) % pip install -r requirements.txt +``` + +When we activate a virtual environment, its name gets shown before our terminal prompt. This is a useful reminder that we're in a virtual environment! + +### Running our program + +Run `python3 cow.py`. We don't get an error because we installed `cowsay` into our active virtual environment. + +Open a new terminal and run `python3 cow.py`. You will get an error again! This is because we haven't activated a virtual environment. + +Run `. .venv/bin/activate` and then `python3 cow.py`. It will start working again. + +Now we can finish our program - let's have the cow say the arguments back to the user (joining together the arguments with spaces). We need to use a slice to skip the first argument, which is our program name: + +```python +import cowsay +import sys + +cowsay.cow(" ".join(sys.argv[1:])) +``` + +Notice how `import cowsay` and `import sys` look the same - as long as we've installed dependencies, we can `import` them just like we can import things that are built into Python. + +```console +(.venv) % python3 cow.py Hello friend + ____________ +| Hello friend | + ============ + \ + \ + ^__^ + (oo)\_______ + (__)\ )\/\ + ||----w | + || || +``` diff --git a/common-content/en/module/tools/virtual-environments/index.md b/common-content/en/module/tools/virtual-environments/index.md new file mode 100644 index 000000000..3e82d54ff --- /dev/null +++ b/common-content/en/module/tools/virtual-environments/index.md @@ -0,0 +1,52 @@ ++++ +title = "Virtual environments" +headless = true +time = 30 +facilitation = false +emoji= "📖" +objectives=[ + "Create a virtual environment with some dependencies installed.", +] ++++ + +We often use libraries in Python. + +Python handles dependencies differently from JavaScript, but they have similarities. + +We've seen that in JavaScript we write down what dependencies we need in a `package.json` file, and when we run `npm install` they will get fetched into a folder called `node_modules`. + +In Python, we write down what dependencies we need in a file called `requirements.txt`. It doesn't contain JSON, it just contains a list of dependencies, one per line. + +### Virtual environments + +To install the dependencies, we need to make something called a virtual environment, where they will get installed to. + +{{}} +A virtual environment is like a `node_modules` folder - it contains all of the dependencies that are installed into it. + +When we run `node`, `node` _automatically_ looks for a `node_modules` folder to find dependencies in. + +When we run `python3`, `python3` _doesn't_ automatically look for a virtual environment. We need to _activate_ it - tell `python3` which virtual environment we want to use. + +There are trade-offs here: +* `node` uses a _convention_ to locate installed dependencies - we don't need to _do_ anything except put the folder in the right place, and it will automatically get used. +* `python3` uses _configuration_ to locate installed dependencies - we need to _configure_ which virtual environment it should use by activating it. + +One of the benefits of using configuration is that we could have different virtual environments with different versions of the same dependencies (e.g. to test a version upgrade), and we can switch between them. + +One of the drawbacks of using configuration is that we need to do the configuration - we need to explicitly activate the virtual environment for it to be used. +{{}} + +First we need to _create_ the virtual environment. We do this by running `python3 -m venv .venv`. This will create a virtual environment in a directory named `.venv`. We could actually create it anywhere, e.g. we could run `python3 -m venv /tmp/python_modules` to create it in a directory named `/tmp/python_modules`. We tend to just use a directory called `.venv` at the root of our project. + +> [!NOTE] +> +> This is another example of a convention - you could name your virtual environment anything, but if we all agree to call it `.venv` then we all know what this directory is when we see it. +> +> It also means we can write scripts, or `.gitignore` file entries assuming that's where the virtual environment will be. + +Next we need to _activate_ the virtual environment. We do this by running `. .venv/bin/activate` (yes, the command we're running is `.` with a path as an argument - the `.` is important). This will only activate the virtual environment for the terminal window we're in - if you're using more than one terminal window, you'll need to activate it in each of them. + +Finally we need to install our dependencies into the virtual environment. We do this by running `pip install -r requirements.txt`. This is saying "Please install all of the dependencies listed in `requirements.txt` into the currently active virtual environment". + +After we've done this, we should be able to `import` any installed dependencies into our Python code. This will work as long as we have activated the virtual environment in the terminal window where we're running our program. diff --git a/org-cyf-sdc/content/tools/sprints/3/_index.md b/org-cyf-sdc/content/tools/sprints/3/_index.md index a8f04873a..a789394ee 100644 --- a/org-cyf-sdc/content/tools/sprints/3/_index.md +++ b/org-cyf-sdc/content/tools/sprints/3/_index.md @@ -5,5 +5,5 @@ layout = 'sprint' emoji= '⏱️' menu_level = ['module'] weight = 2 -theme = "Implementing and testing shell tools in NodeJS, and Operating Systems concepts" +theme = "Implementing shell tools in NodeJS, and Operating Systems concepts" +++ diff --git a/org-cyf-sdc/content/tools/sprints/3/backlog/index.md b/org-cyf-sdc/content/tools/sprints/3/backlog/index.md index d23d2978e..34de6ed71 100644 --- a/org-cyf-sdc/content/tools/sprints/3/backlog/index.md +++ b/org-cyf-sdc/content/tools/sprints/3/backlog/index.md @@ -4,6 +4,6 @@ layout = 'backlog' emoji= '🥞' menu_level = ['sprint'] weight = 2 -backlog= 'Module-Template' -backlog_filter='📅 Sprint 1' +backlog= 'Module-Tools' +backlog_filter='📅 Sprint 3' +++ diff --git a/org-cyf-sdc/content/tools/sprints/3/prep/index.md b/org-cyf-sdc/content/tools/sprints/3/prep/index.md index dd667ed26..7ebe86829 100644 --- a/org-cyf-sdc/content/tools/sprints/3/prep/index.md +++ b/org-cyf-sdc/content/tools/sprints/3/prep/index.md @@ -6,15 +6,18 @@ emoji= '🧑🏾‍💻' menu_level = ['sprint'] weight = 1 [[blocks]] -name="Read about operating systems" -src="module/tools/read-about-operating-systems-1" +name="NodeJS" +src="module/tools/nodejs" +[[blocks]] +name="Writing a NodeJS program" +src="module/tools/first-nodejs-program" [[blocks]] -name="Learn NodeJS" -src="module/tools/learn-nodejs" +name="Using dependencies from npm" +src="module/tools/using-npm-dependencies" [[blocks]] -name="Implement and test CLI tools in NodeJS" -src="module/tools/implement-tools-in-nodejs" +name="Installing dependencies with npm" +src="module/tools/installing-npm-dependencies" [[blocks]] -name="Implement a single-use data analysis program" -src="module/tools/implement-single-use-data-analysis" +name="Read about operating systems" +src="module/tools/read-about-operating-systems" +++ diff --git a/org-cyf-sdc/content/tools/sprints/4/_index.md b/org-cyf-sdc/content/tools/sprints/4/_index.md index bc1557cd3..7c0ec2609 100644 --- a/org-cyf-sdc/content/tools/sprints/4/_index.md +++ b/org-cyf-sdc/content/tools/sprints/4/_index.md @@ -5,5 +5,5 @@ layout = 'sprint' emoji= '⏱️' menu_level = ['module'] weight = 2 -theme = "Implementing and testing shell tools in Python, and Operating Systems concepts" +theme = "Implementing shell tools in Python" +++ diff --git a/org-cyf-sdc/content/tools/sprints/4/backlog/index.md b/org-cyf-sdc/content/tools/sprints/4/backlog/index.md index d23d2978e..5eeb4563d 100644 --- a/org-cyf-sdc/content/tools/sprints/4/backlog/index.md +++ b/org-cyf-sdc/content/tools/sprints/4/backlog/index.md @@ -4,6 +4,6 @@ layout = 'backlog' emoji= '🥞' menu_level = ['sprint'] weight = 2 -backlog= 'Module-Template' -backlog_filter='📅 Sprint 1' +backlog= 'Module-Tools' +backlog_filter='📅 Sprint 4' +++ diff --git a/org-cyf-sdc/content/tools/sprints/4/prep/index.md b/org-cyf-sdc/content/tools/sprints/4/prep/index.md index b9c357aad..c9c9906c7 100644 --- a/org-cyf-sdc/content/tools/sprints/4/prep/index.md +++ b/org-cyf-sdc/content/tools/sprints/4/prep/index.md @@ -6,15 +6,30 @@ emoji= '🧑🏾‍💻' menu_level = ['sprint'] weight = 1 [[blocks]] -name="Read about operating systems" -src="module/tools/read-about-operating-systems-2" +name="Single-use data analysis programs" +src="module/tools/single-use-data-analysis" [[blocks]] -name="Learn Python" -src="module/tools/learn-python" +name="Comparing JavaScript and Python" +src="module/tools/comparing-javascript-and-python" [[blocks]] -name="Implement and test CLI tools in Python" -src="module/tools/implement-tools-in-python" +name="Converting a script from JavaScript to Python" +src="module/tools/converting-javascript-to-python" [[blocks]] -name="Convert a script between languages" -src="module/tools/convert-script-between-languages" +name="Reading a file" +src="module/tools/reading-a-file" +[[blocks]] +name="Splitting" +src="module/tools/splitting-a-string" +[[blocks]] +name="Counting words" +src="module/tools/counting-words" +[[blocks]] +name="Putting it all together" +src="module/tools/converted-program" +[[blocks]] +name = "Virtual environments" +src="module/tools/virtual-environments" +[[blocks]] +name = "Using python dependencies" +src="module/tools/using-python-dependencies" +++ diff --git a/org-cyf-sdc/content/tools/success/index.md b/org-cyf-sdc/content/tools/success/index.md index 465c0fbe2..99c6922aa 100644 --- a/org-cyf-sdc/content/tools/success/index.md +++ b/org-cyf-sdc/content/tools/success/index.md @@ -6,10 +6,10 @@ emoji= '✅' menu_level = ['module'] weight = 11 [[objectives]] -1="Translate requirements into high-level design outlines" -2="Break down solving problems into testable steps" -3="Ask questions systematically using a given formal language" -4="Demonstrate good citizenship on GitHub by participating in code review on PRs" -5="Solve at least 6 7kyu problems in Codewars" -6="Solve up to level 5 in the Bandit repeatedly" +1="Work with binary, hexadecimal, and decimal numbers" +2="Explain what a CPU is, what main memory is, and how they interact" +3="Describe an operating system, including the kernel, processes, and syscalls" +4="Process text files using standard command line tools, including in pipelines" +5="Write command line tools using JavaScript + NodeJS, including dependencies from npm" +6="Write command line tools using Python, including dependencies from pypi" +++