03 working with files and dirs_notes

Working with Files and Directories

Questions

• How can I create, copy, and delete files and directories?
• How can I edit files?

Objectives

• Create a directory hierarchy that matches a given diagram.
• Create files in that hierarchy using an editor or by copying and renaming existing files.
• Delete, copy and move specified files and/or directories.

Creating directories

Find out where we are

$ pwd

/home/<user>/Desktop/shell-lesson-data

Navigate to exercise-data/writing

$ cd exercise-data/writing/

List dir content

$ ls -F

haiku.txt LittleWomen.txt

Create a directory: mkdir [DIR]

$ mkdir thesis

List

$ ls -F

haiku.txt LittleWomen.txt thesis/

List content of new directory

$ ls thesis

Create multiple (nested) directories

$ mkdir -p ../project/data ../project/results

Exercise

Tasks

• Navigate to ~/Desktop/shell-lesson-data/exercise-data/writing/
• Create the directories ~/Desktop/shell-lesson-data/exercise-data/writing/thesis/, ~/Desktop/shell-lesson-data/exercise-data/project/data/, and ~/Desktop/shell-lesson-data/exercise-data/project/results/.
• Consult the ls help to find out how to list nested directories Recursively ;).
• List the content of ~/Desktop/shell-lesson-data/exercise-data/writing/ recursively.

Shell vs. File Explorer

• Open your systems graphical file explorer and create a new directory ~/Desktop/shell-lesson-data/exercise-data/writing/paper/ .
• Confirm in your shell that the directory is created and is empty.
• Remove the directory in the file explorer.
• Confirm that it’s gone in the shell.

Good names

• Letters, numbers, “.”, “-“, “_”
• parentheses, asterisk (“*”), ampersand (“&”), have special meaning and lead to unintended behaviour including data loss.
• Replace whitespace by “_” or “-” (but stick to one)
• No leading “-”
• Surround names with special characters with a pair of ’ ’ (single quotes).

Create a text file

Another great advantage of the shell is that it has a built-in editor (actually, more than one). Here, we use the nano editor. Built-in shell editors deal with plain text, period.

Starting nano

Navigate to the writing/thesis/ directory created above.

$ pwd

.../writing/thesis

$ nano draft.txt

Enter some text

• After nano started, you can directly start writing text.
• The two bottom lines list the keyboard shortcuts to trigger certain editor actions like saving the file (^O) or quitting the editor (^X).
• ^ maps to |CTRL| on english keyboards, |STRG| on german keyboards.

Exercise

Assignement

Another way to create a (empty) file is the touch [FILE] command

$ touch my_file.txt

• What did the touch command do? (List the directory).
• When might you want to create a file this way?

Solution

Remove files

The command

$ rm [FILE]

removes a file from disk (**ATTENTION** There is no such thing as a thrash bin in the shell!!)

Task

Remove the last created file.

Filenames

• A filename typically has two parts: The name proper and the filename extension. The extension is what follows

the last “.” of the filename. Example: thesis.txt: Filename “thesis”, extension “txt”.

• It is good practise to use certain extensions for certain filenames. Examples: “txt” for plain text, “csv” for comma separated values, “pdf” for portable document format files, “mp3” for music files, etc. .... But filename extensions in no way enforce any kind of behaviour: Nothing forbids naming a music file “title.txt” and opening and playing such a file in a music player is entirely valid. GUI file explorers on the other hand often associate extensions with apps, and double clicking a text file “music.mp3” will most likely cause problems in the music app.

Moving and renaming files and directories

• Rename:

  $ mv old new
      

  **ATTENTION**: mv will silently overwrite an existing file. Use

  $ mv -i old new
      

  to avoid unintended data loss.

Exercise

Task

• Rename the file draft.txt to quotes.txt.
• Confirm with ls.

Moving to directories

• If “new” in above example is a directory, “old” will be moved into that directory.

$ mv thesis/quotes.txt .

• Recall that “.” is the current directory.

Exercise

After running the following commands, Nelle realizes that she put the files

sucrose.dat and maltose.dat into the wrong folder. The files should have been placed in the raw folder.

$ ls -F
 analyzed/ raw/
$ ls -F analyzed
fructose.dat glucose.dat maltose.dat sucrose.dat
$ cd analyzed

Fill in the blanks to move these files to the raw/ folder (i.e. the one she forgot to put them in)

$ mv sucrose.dat maltose.dat ____/____

Solution

Copying files and directories: cp old new

• The cp command works much like mv
• It leaves the original file untouched.

Copy a file

$ cp quotes.txt thesis/quotations.txt
$ ls quotes.txt thesis/quotation.txt

Copy a directory (recursively)

$ cp -r thesis thesis_backup
$ ls thesis thesis_backup

Exercises

Task

Suppose that you created a plain-text file in your current directory to contain a list of the statistical tests you will need to do to analyze your data, and named it statstics.txt After creating and saving this file you realize you misspelled the filename! You want to correct the mistake, which of the following commands could you use to do so?

1. cp statstics.txt statistics.txt
2. mv statstics.txt statistics.txt
3. mv statstics.txt .
4. cp statstics.txt .

Solution

Removing files and directories

• Let’s tidy up the writing directory by removing the file quotes.txt

$ rm quotes.txt

Confirm with

$ ls quotes.txt

**ATTENTION**: rm removes stuff forever! Use rm i to safeguard.

Exercises

Task

How can you remove a directory Recursively?

Solution

Task

For this exercise, you can test the commands in the shell-lesson-data/exercise-data/ directory.

In the example below, what does cp do when given several filenames and a directory name?

$ mkdir backup
$ cp creatures/minotaur.dat creatures/unicorn.dat backup/

In the example below, what does cp do when given three or more file names?

$ cd creatures
$ ls -F

basilisk.dat  minotaur.dat  unicorn.dat

$ cp minotaur.dat unicorn.dat basilisk.dat

Solution

Wildcards

• Wildcards are placeholders for one or multiple characters.
  • ”*”: wildcard for zero or more charaters
  • ”?”: wildcard for exactly one character

Examples

The directory shell-lesson-data/exercise-data/alkanes contains 6 files named by various chemicals (alkanes).

• *.pdb -> any file with the extension “.pdb”
• *thane.pdb -> ethane.pdb, methane.pdb
• *ethane.pdb -> ethane.pdb, methane.pdb
• p*.pdb -> pentane.pdb, propane.pdb
• ???ane.pdb -> cubane.pdb, ethane.pdb, octane.pdb

Exercise

Task

When run in the alkanes directory, which ls command(s) will produce this output?

ethane.pdb methane.pdb

1. ls *t*ane.pdb
2. ls *t?ne.*
3. ls *t??ne.pdb
4. ls ethane.*

Solution

Task

Sam has a directory containing calibration data, datasets, and descriptions of the datasets:

├── 2015-10-23-calibration.txt ├── 2015-10-23-dataset1.txt ├── 2015-10-23-dataset2.txt ├── 2015-10-23-dataset_overview.txt ├── 2015-10-26-calibration.txt ├── 2015-10-26-dataset1.txt ├── 2015-10-26-dataset2.txt ├── 2015-10-26-dataset_overview.txt ├── 2015-11-23-calibration.txt ├── 2015-11-23-dataset1.txt ├── 2015-11-23-dataset2.txt ├── 2015-11-23-dataset_overview.txt ├── backup │ ├── calibration │ └── datasets └── send_to_bob ├── all_datasets_created_on_a_23rd └── all_november_files Task

Before heading off to another field trip, she wants to back up her data and send some datasets to her colleague Bob. Sam uses the following commands to get the job done:

$ cp *dataset* backup/datasets
$ cp ____calibration____ backup/calibration
$ cp 2015-____-____ send_to_bob/all_november_files/
$ cp ____ send_to_bob/all_datasets_created_on_a_23rd/

Help Sam by filling in the blanks.

The resulting directory structure should look like this

. ├── 2015-10-23-calibration.txt ├── 2015-10-23-dataset1.txt ├── 2015-10-23-dataset2.txt ├── 2015-10-23-dataset_overview.txt ├── 2015-10-26-calibration.txt ├── 2015-10-26-dataset1.txt ├── 2015-10-26-dataset2.txt ├── 2015-10-26-dataset_overview.txt ├── 2015-11-23-calibration.txt ├── 2015-11-23-dataset1.txt ├── 2015-11-23-dataset2.txt ├── 2015-11-23-dataset_overview.txt ├── backup │ ├── calibration │ │ ├── 2015-10-23-calibration.txt │ │ ├── 2015-10-26-calibration.txt │ │ └── 2015-11-23-calibration.txt │ └── datasets │ ├── 2015-10-23-dataset1.txt │ ├── 2015-10-23-dataset2.txt │ ├── 2015-10-23-dataset_overview.txt │ ├── 2015-10-26-dataset1.txt │ ├── 2015-10-26-dataset2.txt │ ├── 2015-10-26-dataset_overview.txt │ ├── 2015-11-23-dataset1.txt │ ├── 2015-11-23-dataset2.txt │ └── 2015-11-23-dataset_overview.txt └── send_to_bob ├── all_datasets_created_on_a_23rd │ ├── 2015-10-23-dataset1.txt │ ├── 2015-10-23-dataset2.txt │ ├── 2015-10-23-dataset_overview.txt │ ├── 2015-11-23-dataset1.txt │ ├── 2015-11-23-dataset2.txt │ └── 2015-11-23-dataset_overview.txt └── all_november_files ├── 2015-11-23-calibration.txt ├── 2015-11-23-dataset1.txt ├── 2015-11-23-dataset2.txt └── 2015-11-23-dataset_overview.txt

Solution

Task

Jamie is working on a project, and she sees that her files aren’t very well organized:

$ ls -F

analyzed/  fructose.dat    raw/   sucrose.dat

The fructose.dat and sucrose.dat files contain output from her data analysis. What command(s) covered in this lesson does she need to run so that the commands below will produce the output shown?

$ ls -F

analyzed/   raw/

$ ls analyzed

fructose.dat    sucrose.dat

Solution

Reproduce a folder structure

You’re starting a new experiment and would like to duplicate the directory structure from your previous experiment so you can add new data.

Assume that the previous experiment is in a folder called 2016-05-18/, which contains a data folder that in turn contains folders named raw and processed that contain data files. The goal is to copy the folder structure of the 2016-05-18/ folder into a folder called 2016-05-20/ so that your final directory structure looks like this:

2016-05-20/ └── data ├── processed └── raw

Which of the following set of commands would achieve this objective? What would the other commands do?

1.

$ mkdir 2016-05-20
$ mkdir 2016-05-20/data
$ mkdir 2016-05-20/data/processed
$ mkdir 2016-05-20/data/raw

2.

$ mkdir 2016-05-20
$ cd 2016-05-20
$ mkdir data
$ cd data
$ mkdir raw processed

3.

$ mkdir 2016-05-20/data/raw
$ mkdir 2016-05-20/data/processed

4.

$ mkdir -p 2016-05-20/data/raw
$ mkdir -p 2016-05-20/data/processed

5.

$ mkdir 2016-05-20
$ cd 2016-05-20
$ mkdir data
$ mkdir raw processed

Solution

Take home

• cp [old] [new] copies a file.
• mkdir [path] creates a new directory.
• mv [old] [new] moves (renames) a file or directory.
• rm [path] removes (deletes) a file.
• ”*” matches zero or more characters in a filename, so *.txt matches all files ending in .txt.
• ”?” matches any single character in a filename, so ?.txt matches a.txt but not any.txt.
• Use of the Control key may be described in many ways, including Ctrl-X, Control-X, and ^X.
• The shell does not have a trash bin: once something is deleted, it’s really gone.
• Most files’ names are something.extension. The extension isn’t required, and doesn’t guarantee anything, but is normally used to indicate the type of data in the file.
• Depending on the type of work you do, you may need a more powerful text editor than Nano.