forked from swcarpentry/shell-novice
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy path06-script.html
1144 lines (1101 loc) · 70 KB
/
06-script.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<!DOCTYPE html>
<!-- START: inst/pkgdown/templates/layout.html --><!-- Generated by pkgdown: do not edit by hand --><html lang="en" data-bs-theme="auto"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="utf-8"><title>The Unix Shell: Shell Scripts</title><meta name="viewport" content="width=device-width, initial-scale=1"><script src="assets/themetoggle.js"></script><link rel="stylesheet" type="text/css" href="assets/styles.css"><script src="assets/scripts.js" type="text/javascript"></script><!-- mathjax --><script type="text/x-mathjax-config">
MathJax.Hub.Config({
config: ["MMLorHTML.js"],
jax: ["input/TeX","input/MathML","output/HTML-CSS","output/NativeMML", "output/PreviewHTML"],
extensions: ["tex2jax.js","mml2jax.js","MathMenu.js","MathZoom.js", "fast-preview.js", "AssistiveMML.js", "a11y/accessibility-menu.js"],
TeX: {
extensions: ["AMSmath.js","AMSsymbols.js","noErrors.js","noUndefined.js"]
},
tex2jax: {
inlineMath: [['\\(', '\\)']],
displayMath: [ ['$$','$$'], ['\\[', '\\]'] ],
processEscapes: true
}
});
</script><script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js" integrity="sha256-nvJJv9wWKEm88qvoQl9ekL2J+k/RWIsaSScxxlsrv8k=" crossorigin="anonymous"></script><!-- Responsive Favicon for The Carpentries --><link rel="apple-touch-icon" sizes="180x180" href="apple-touch-icon.png"><link rel="icon" type="image/png" sizes="32x32" href="favicon-32x32.png"><link rel="icon" type="image/png" sizes="16x16" href="favicon-16x16.png"><link rel="manifest" href="site.webmanifest"><link rel="mask-icon" href="safari-pinned-tab.svg" color="#5bbad5"><meta name="msapplication-TileColor" content="#da532c"><meta name="theme-color" media="(prefers-color-scheme: light)" content="white"><meta name="theme-color" media="(prefers-color-scheme: dark)" content="black"></head><body>
<header id="top" class="navbar navbar-expand-md top-nav software"><svg xmlns="http://www.w3.org/2000/svg" class="d-none"><symbol id="check2" viewbox="0 0 16 16"><path d="M13.854 3.646a.5.5 0 0 1 0 .708l-7 7a.5.5 0 0 1-.708 0l-3.5-3.5a.5.5 0 1 1 .708-.708L6.5 10.293l6.646-6.647a.5.5 0 0 1 .708 0z"></path></symbol><symbol id="circle-half" viewbox="0 0 16 16"><path d="M8 15A7 7 0 1 0 8 1v14zm0 1A8 8 0 1 1 8 0a8 8 0 0 1 0 16z"></path></symbol><symbol id="moon-stars-fill" viewbox="0 0 16 16"><path d="M6 .278a.768.768 0 0 1 .08.858 7.208 7.208 0 0 0-.878 3.46c0 4.021 3.278 7.277 7.318 7.277.527 0 1.04-.055 1.533-.16a.787.787 0 0 1 .81.316.733.733 0 0 1-.031.893A8.349 8.349 0 0 1 8.344 16C3.734 16 0 12.286 0 7.71 0 4.266 2.114 1.312 5.124.06A.752.752 0 0 1 6 .278z"></path><path d="M10.794 3.148a.217.217 0 0 1 .412 0l.387 1.162c.173.518.579.924 1.097 1.097l1.162.387a.217.217 0 0 1 0 .412l-1.162.387a1.734 1.734 0 0 0-1.097 1.097l-.387 1.162a.217.217 0 0 1-.412 0l-.387-1.162A1.734 1.734 0 0 0 9.31 6.593l-1.162-.387a.217.217 0 0 1 0-.412l1.162-.387a1.734 1.734 0 0 0 1.097-1.097l.387-1.162zM13.863.099a.145.145 0 0 1 .274 0l.258.774c.115.346.386.617.732.732l.774.258a.145.145 0 0 1 0 .274l-.774.258a1.156 1.156 0 0 0-.732.732l-.258.774a.145.145 0 0 1-.274 0l-.258-.774a1.156 1.156 0 0 0-.732-.732l-.774-.258a.145.145 0 0 1 0-.274l.774-.258c.346-.115.617-.386.732-.732L13.863.1z"></path></symbol><symbol id="sun-fill" viewbox="0 0 16 16"><path d="M8 12a4 4 0 1 0 0-8 4 4 0 0 0 0 8zM8 0a.5.5 0 0 1 .5.5v2a.5.5 0 0 1-1 0v-2A.5.5 0 0 1 8 0zm0 13a.5.5 0 0 1 .5.5v2a.5.5 0 0 1-1 0v-2A.5.5 0 0 1 8 13zm8-5a.5.5 0 0 1-.5.5h-2a.5.5 0 0 1 0-1h2a.5.5 0 0 1 .5.5zM3 8a.5.5 0 0 1-.5.5h-2a.5.5 0 0 1 0-1h2A.5.5 0 0 1 3 8zm10.657-5.657a.5.5 0 0 1 0 .707l-1.414 1.415a.5.5 0 1 1-.707-.708l1.414-1.414a.5.5 0 0 1 .707 0zm-9.193 9.193a.5.5 0 0 1 0 .707L3.05 13.657a.5.5 0 0 1-.707-.707l1.414-1.414a.5.5 0 0 1 .707 0zm9.193 2.121a.5.5 0 0 1-.707 0l-1.414-1.414a.5.5 0 0 1 .707-.707l1.414 1.414a.5.5 0 0 1 0 .707zM4.464 4.465a.5.5 0 0 1-.707 0L2.343 3.05a.5.5 0 1 1 .707-.707l1.414 1.414a.5.5 0 0 1 0 .708z"></path></symbol></svg><a class="visually-hidden-focusable skip-link" href="#main-content">Skip to main content</a>
<div class="container-fluid top-nav-container">
<div class="col-md-8">
<div class="large-logo">
<img id="swc-logo" alt="Software Carpentry" src="assets/images/software-logo.svg"></div>
</div>
<div class="selector-container">
<div id="theme-selector">
<li class="nav-item dropdown" id="theme-button-list">
<button class="btn btn-link nav-link px-0 px-lg-2 dropdown-toggle d-flex align-items-center" id="bd-theme" type="button" aria-expanded="false" data-bs-toggle="dropdown" data-bs-display="static" aria-label="Toggle theme (auto)">
<svg class="bi my-1 theme-icon-active"><use href="#circle-half"></use></svg><i data-feather="chevron-down"></i>
</button>
<ul class="dropdown-menu dropdown-menu-end" aria-labelledby="bd-theme-text"><li>
<button type="button" class="btn dropdown-item d-flex align-items-center" data-bs-theme-value="light" aria-pressed="false">
<svg class="bi me-2 theme-icon"><use href="#sun-fill"></use></svg>
Light
<svg class="bi ms-auto d-none"><use href="#check2"></use></svg></button>
</li>
<li>
<button type="button" class="btn dropdown-item d-flex align-items-center" data-bs-theme-value="dark" aria-pressed="false">
<svg class="bi me-2 theme-icon"><use href="#moon-stars-fill"></use></svg>
Dark
<svg class="bi ms-auto d-none"><use href="#check2"></use></svg></button>
</li>
<li>
<button type="button" class="btn dropdown-item d-flex align-items-center active" data-bs-theme-value="auto" aria-pressed="true">
<svg class="bi me-2 theme-icon"><use href="#circle-half"></use></svg>
Auto
<svg class="bi ms-auto d-none"><use href="#check2"></use></svg></button>
</li>
</ul></li>
</div>
<div class="dropdown" id="instructor-dropdown">
<button class="btn btn-secondary dropdown-toggle bordered-button" type="button" id="dropdownMenu1" data-bs-toggle="dropdown" aria-expanded="false">
<i aria-hidden="true" class="icon" data-feather="eye"></i> Learner View <i data-feather="chevron-down"></i>
</button>
<ul class="dropdown-menu" aria-labelledby="dropdownMenu1"><li><button class="dropdown-item" type="button" onclick="window.location.href='instructor/06-script.html';">Instructor View</button></li>
</ul></div>
</div>
</div>
<hr></header><nav class="navbar navbar-expand-xl bottom-nav software" aria-label="Main Navigation"><div class="container-fluid nav-container">
<button class="navbar-toggler" type="button" data-bs-toggle="collapse" data-bs-target="#navbarSupportedContent" aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="Toggle Navigation">
<span class="navbar-toggler-icon"></span>
<span class="menu-title">Menu</span>
</button>
<div class="nav-logo">
<img class="small-logo" alt="Software Carpentry" src="assets/images/software-logo-sm.svg"></div>
<div class="lesson-title-md">
The Unix Shell
</div>
<div class="search-icon-sm">
<!-- TODO: do not show until we have search
<i role="img" aria-label="Search the All In One page" data-feather="search"></i>
-->
</div>
<div class="desktop-nav">
<ul class="navbar-nav me-auto mb-2 mb-lg-0"><li class="nav-item">
<span class="lesson-title">
The Unix Shell
</span>
</li>
<li class="nav-item">
<a class="nav-link" href="key-points.html">Key Points</a>
</li>
<li class="nav-item">
<a class="nav-link" href="reference.html#glossary">Glossary</a>
</li>
<li class="nav-item">
<a class="nav-link" href="profiles.html">Learner Profiles</a>
</li>
<li class="nav-item dropdown">
<button class="nav-link dropdown-toggle" id="navbarDropdown" data-bs-toggle="dropdown" aria-expanded="false">
More <i data-feather="chevron-down"></i>
</button>
<ul class="dropdown-menu" aria-labelledby="navbarDropdown"><li><a class="dropdown-item" href="discuss.html">Discussion</a></li><li><a class="dropdown-item" href="reference.html">Summary of Basic Commands</a></li>
</ul></li>
</ul></div>
<!--
<form class="d-flex col-md-2 search-form">
<fieldset disabled>
<input class="form-control me-2 searchbox" type="search" placeholder="" aria-label="">
<button class="btn btn-outline-success tablet-search-button" type="submit">
<i class="search-icon" data-feather="search" role="img" aria-label="Search the All In One page"></i>
</button>
</fieldset>
</form>
-->
<a id="search-button" class="btn btn-primary" href="aio.html" role="button" aria-label="Search the All In One page">Search the All In One page</a>
</div><!--/div.container-fluid -->
</nav><div class="col-md-12 mobile-title">
The Unix Shell
</div>
<aside class="col-md-12 lesson-progress"><div style="width: 67%" class="percentage">
67%
</div>
<div class="progress software">
<div class="progress-bar software" role="progressbar" style="width: 67%" aria-valuenow="67" aria-label="Lesson Progress" aria-valuemin="0" aria-valuemax="100">
</div>
</div>
</aside><div class="container">
<div class="row">
<!-- START: inst/pkgdown/templates/navbar.html -->
<div id="sidebar-col" class="col-lg-4">
<div id="sidebar" class="sidebar">
<nav aria-labelledby="flush-headingEleven"><button role="button" aria-label="close menu" alt="close menu" aria-expanded="true" aria-controls="sidebar" class="collapse-toggle" data-collapse="Collapse " data-episodes="Episodes ">
<i class="search-icon" data-feather="x" role="img"></i>
</button>
<div class="sidebar-inner">
<div class="row mobile-row" id="theme-row-mobile">
<div class="col" id="theme-selector">
<li class="nav-item dropdown" id="theme-button-list">
<button class="btn btn-link nav-link px-0 px-lg-2 dropdown-toggle d-flex align-items-center" id="bd-theme" type="button" aria-expanded="false" data-bs-toggle="dropdown" data-bs-display="static" aria-label="Toggle theme (auto)">
<svg class="bi my-1 theme-icon-active"><use href="#circle-half"></use></svg><span class="d-lg-none ms-1" id="bd-theme-text">Toggle Theme</span>
</button>
<ul class="dropdown-menu dropdown-menu-right" aria-labelledby="bd-theme-text"><li>
<button type="button" class="btn dropdown-item d-flex align-items-center" data-bs-theme-value="light" aria-pressed="false">
<svg class="bi me-2 theme-icon"><use href="#sun-fill"></use></svg>
Light
<svg class="bi ms-auto d-none"><use href="#check2"></use></svg></button>
</li>
<li>
<button type="button" class="btn dropdown-item d-flex align-items-center" data-bs-theme-value="dark" aria-pressed="false">
<svg class="bi me-2 theme-icon"><use href="#moon-stars-fill"></use></svg>
Dark
<svg class="bi ms-auto d-none"><use href="#check2"></use></svg></button>
</li>
<li>
<button type="button" class="btn dropdown-item d-flex align-items-center active" data-bs-theme-value="auto" aria-pressed="true">
<svg class="bi me-2 theme-icon"><use href="#circle-half"></use></svg>
Auto
<svg class="bi ms-auto d-none"><use href="#check2"></use></svg></button>
</li>
</ul></li>
</div>
</div>
<div class="row mobile-row">
<div class="col">
<div class="sidenav-view-selector">
<div class="accordion accordion-flush" id="accordionFlush9">
<div class="accordion-item">
<h2 class="accordion-header" id="flush-headingNine">
<button class="accordion-button collapsed" id="instructor" type="button" data-bs-toggle="collapse" data-bs-target="#flush-collapseNine" aria-expanded="false" aria-controls="flush-collapseNine">
<i id="eye" aria-hidden="true" class="icon" data-feather="eye"></i> Learner View
</button>
</h2>
<div id="flush-collapseNine" class="accordion-collapse collapse" aria-labelledby="flush-headingNine" data-bs-parent="#accordionFlush2">
<div class="accordion-body">
<a href="instructor/06-script.html">Instructor View</a>
</div>
</div>
</div><!--/div.accordion-item-->
</div><!--/div.accordion-flush-->
</div><!--div.sidenav-view-selector -->
</div><!--/div.col -->
<hr></div><!--/div.mobile-row -->
<div class="accordion accordion-flush" id="accordionFlush11">
<div class="accordion-item">
<button id="chapters" class="accordion-button show" type="button" data-bs-toggle="collapse" data-bs-target="#flush-collapseEleven" aria-expanded="false" aria-controls="flush-collapseEleven">
<h2 class="accordion-header chapters" id="flush-headingEleven">
EPISODES
</h2>
</button>
<div id="flush-collapseEleven" class="accordion-collapse show collapse" aria-labelledby="flush-headingEleven" data-bs-parent="#accordionFlush11">
<div class="accordion-body">
<div class="accordion accordion-flush" id="accordionFlush1">
<div class="accordion-item">
<div class="accordion-header" id="flush-heading1">
<a href="index.html">Summary and Setup</a>
</div><!--/div.accordion-header-->
</div><!--/div.accordion-item-->
</div><!--/div.accordion-flush-->
<div class="accordion accordion-flush" id="accordionFlush2">
<div class="accordion-item">
<div class="accordion-header" id="flush-heading2">
<a href="01-intro.html">1. Introducing the Shell</a>
</div><!--/div.accordion-header-->
</div><!--/div.accordion-item-->
</div><!--/div.accordion-flush-->
<div class="accordion accordion-flush" id="accordionFlush3">
<div class="accordion-item">
<div class="accordion-header" id="flush-heading3">
<a href="02-filedir.html">2. Navigating Files and Directories</a>
</div><!--/div.accordion-header-->
</div><!--/div.accordion-item-->
</div><!--/div.accordion-flush-->
<div class="accordion accordion-flush" id="accordionFlush4">
<div class="accordion-item">
<div class="accordion-header" id="flush-heading4">
<a href="03-create.html">3. Working With Files and Directories</a>
</div><!--/div.accordion-header-->
</div><!--/div.accordion-item-->
</div><!--/div.accordion-flush-->
<div class="accordion accordion-flush" id="accordionFlush5">
<div class="accordion-item">
<div class="accordion-header" id="flush-heading5">
<a href="04-pipefilter.html">4. Pipes and Filters</a>
</div><!--/div.accordion-header-->
</div><!--/div.accordion-item-->
</div><!--/div.accordion-flush-->
<div class="accordion accordion-flush" id="accordionFlush6">
<div class="accordion-item">
<div class="accordion-header" id="flush-heading6">
<a href="05-loop.html">5. Loops</a>
</div><!--/div.accordion-header-->
</div><!--/div.accordion-item-->
</div><!--/div.accordion-flush-->
<div class="accordion accordion-flush" id="accordionFlushcurrent">
<div class="accordion-item">
<div class="accordion-header" id="flush-headingcurrent">
<button class="accordion-button" type="button" data-bs-toggle="collapse" data-bs-target="#flush-collapsecurrent" aria-expanded="true" aria-controls="flush-collapsecurrent">
<span class="visually-hidden">Current Chapter</span>
<span class="current-chapter">
6. Shell Scripts
</span>
</button>
</div><!--/div.accordion-header-->
<div id="flush-collapsecurrent" class="accordion-collapse collapse show" aria-labelledby="flush-headingcurrent" data-bs-parent="#accordionFlushcurrent">
<div class="accordion-body">
<ul><li><a href="#nelles-pipeline-creating-a-script">Nelle’s Pipeline: Creating a Script</a></li>
</ul></div><!--/div.accordion-body-->
</div><!--/div.accordion-collapse-->
</div><!--/div.accordion-item-->
</div><!--/div.accordion-flush-->
<div class="accordion accordion-flush" id="accordionFlush8">
<div class="accordion-item">
<div class="accordion-header" id="flush-heading8">
<a href="07-find.html">7. Finding Things</a>
</div><!--/div.accordion-header-->
</div><!--/div.accordion-item-->
</div><!--/div.accordion-flush-->
</div>
</div>
</div>
<hr class="half-width"><div class="accordion accordion-flush lesson-resources" id="accordionFlush12">
<div class="accordion-item">
<h2 class="accordion-header" id="flush-headingTwelve">
<button class="accordion-button collapsed" id="lesson-resources" type="button" data-bs-toggle="collapse" data-bs-target="#flush-collapseTwelve" aria-expanded="false" aria-controls="flush-collapseTwelve">
RESOURCES
</button>
</h2>
<div id="flush-collapseTwelve" class="accordion-collapse collapse" aria-labelledby="flush-headingTwelve" data-bs-parent="#accordionFlush12">
<div class="accordion-body">
<ul><li>
<a href="key-points.html">Key Points</a>
</li>
<li>
<a href="reference.html#glossary">Glossary</a>
</li>
<li>
<a href="profiles.html">Learner Profiles</a>
</li>
<li><a href="discuss.html">Discussion</a></li><li><a href="reference.html">Summary of Basic Commands</a></li>
</ul></div>
</div>
</div>
</div>
<hr class="half-width lesson-resources"><a href="aio.html">See all in one page</a>
<hr class="d-none d-sm-block d-md-none"><div class="d-grid gap-1">
</div>
</div><!-- /div.accordion -->
</div><!-- /div.sidebar-inner -->
</nav></div><!-- /div.sidebar -->
</div><!-- /div.sidebar-col -->
<!-- END: inst/pkgdown/templates/navbar.html-->
<!-- START: inst/pkgdown/templates/content-instructor.html -->
<div class="col-xl-8 col-lg-12 primary-content">
<nav class="lesson-content mx-md-4" aria-label="Previous and Next Chapter"><!-- content for small screens --><div class="d-block d-sm-block d-md-none">
<a class="chapter-link" href="05-loop.html"><i aria-hidden="true" class="small-arrow" data-feather="arrow-left"></i>Previous</a>
<a class="chapter-link float-end" href="07-find.html">Next<i aria-hidden="true" class="small-arrow" data-feather="arrow-right"></i></a>
</div>
<!-- content for large screens -->
<div class="d-none d-sm-none d-md-block">
<a class="chapter-link" href="05-loop.html" rel="prev">
<i aria-hidden="true" class="small-arrow" data-feather="arrow-left"></i>
Previous: Loops
</a>
<a class="chapter-link float-end" href="07-find.html" rel="next">
Next: Finding Things...
<i aria-hidden="true" class="small-arrow" data-feather="arrow-right"></i>
</a>
</div>
<hr></nav><main id="main-content" class="main-content"><div class="container lesson-content">
<h1>Shell Scripts</h1>
<p>Last updated on 2023-08-05 |
<a href="https://github.com/swcarpentry/shell-novice/edit/main/episodes/06-script.md" class="external-link">Edit this page <i aria-hidden="true" data-feather="edit"></i></a></p>
<div class="text-end">
<button role="button" aria-pressed="false" tabindex="0" id="expand-code" class="pull-right" data-expand="Expand All Solutions " data-collapse="Collapse All Solutions "> Expand All Solutions <i aria-hidden="true" data-feather="plus"></i></button>
</div>
<div class="overview card">
<h2 class="card-header">Overview</h2>
<div class="row g-0">
<div class="col-md-4">
<div class="card-body">
<div class="inner">
<h3 class="card-title">Questions</h3>
<ul><li>How can I save and re-use commands?</li>
</ul></div>
</div>
</div>
<div class="col-md-8">
<div class="card-body">
<div class="inner bordered">
<h3 class="card-title">Objectives</h3>
<ul><li>Write a shell script that runs a command or series of commands for a
fixed set of files.</li>
<li>Run a shell script from the command line.</li>
<li>Write a shell script that operates on a set of files defined by the
user on the command line.</li>
<li>Create pipelines that include shell scripts you, and others, have
written.</li>
</ul></div>
</div>
</div>
</div>
</div>
<p>We are finally ready to see what makes the shell such a powerful
programming environment. We are going to take the commands we repeat
frequently and save them in files so that we can re-run all those
operations again later by typing a single command. For historical
reasons, a bunch of commands saved in a file is usually called a
<strong>shell script</strong>, but make no mistake — these are actually
small programs.</p>
<p>Not only will writing shell scripts make your work faster, but also
you won’t have to retype the same commands over and over again. It will
also make it more accurate (fewer chances for typos) and more
reproducible. If you come back to your work later (or if someone else
finds your work and wants to build on it), you will be able to reproduce
the same results simply by running your script, rather than having to
remember or retype a long list of commands.</p>
<p>Let’s start by going back to <code>alkanes/</code> and creating a new
file, <code>middle.sh</code> which will become our shell script:</p>
<div class="codewrapper sourceCode" id="cb1">
<h3 class="code-label">BASH<i aria-hidden="true" data-feather="chevron-left"></i><i aria-hidden="true" data-feather="chevron-right"></i>
</h3>
<pre class="sourceCode bash" tabindex="0"><code class="sourceCode bash"><span id="cb1-1"><a href="#cb1-1" tabindex="-1"></a><span class="ex">$</span> cd alkanes</span>
<span id="cb1-2"><a href="#cb1-2" tabindex="-1"></a><span class="ex">$</span> nano middle.sh</span></code></pre>
</div>
<p>The command <code>nano middle.sh</code> opens the file
<code>middle.sh</code> within the text editor ‘nano’ (which runs within
the shell). If the file does not exist, it will be created. We can use
the text editor to directly edit the file by inserting the following
line:</p>
<pre class="source"><code>head -n 15 octane.pdb | tail -n 5</code></pre>
<p>This is a variation on the pipe we constructed earlier, which selects
lines 11-15 of the file <code>octane.pdb</code>. Remember, we are
<em>not</em> running it as a command just yet; we are only incorporating
the commands in a file.</p>
<p>Then we save the file (<code>Ctrl-O</code> in nano) and exit the text
editor (<code>Ctrl-X</code> in nano). Check that the directory
<code>alkanes</code> now contains a file called
<code>middle.sh</code>.</p>
<p>Once we have saved the file, we can ask the shell to execute the
commands it contains. Our shell is called <code>bash</code>, so we run
the following command:</p>
<div class="codewrapper sourceCode" id="cb3">
<h3 class="code-label">BASH<i aria-hidden="true" data-feather="chevron-left"></i><i aria-hidden="true" data-feather="chevron-right"></i>
</h3>
<pre class="sourceCode bash" tabindex="0"><code class="sourceCode bash"><span id="cb3-1"><a href="#cb3-1" tabindex="-1"></a><span class="ex">$</span> bash middle.sh</span></code></pre>
</div>
<div class="codewrapper">
<h3 class="code-label">OUTPUT<i aria-hidden="true" data-feather="chevron-left"></i><i aria-hidden="true" data-feather="chevron-right"></i>
</h3>
<pre class="output" tabindex="0"><code>ATOM 9 H 1 -4.502 0.681 0.785 1.00 0.00
ATOM 10 H 1 -5.254 -0.243 -0.537 1.00 0.00
ATOM 11 H 1 -4.357 1.252 -0.895 1.00 0.00
ATOM 12 H 1 -3.009 -0.741 -1.467 1.00 0.00
ATOM 13 H 1 -3.172 -1.337 0.206 1.00 0.00</code></pre>
</div>
<p>Sure enough, our script’s output is exactly what we would get if we
ran that pipeline directly.</p>
<div id="text-vs.-whatever" class="callout">
<div class="callout-square">
<i class="callout-icon" data-feather="bell"></i>
</div>
<div id="text-vs.-whatever" class="callout-inner">
<h3 class="callout-title">Text vs. Whatever</h3>
<div class="callout-content">
<p>We usually call programs like Microsoft Word or LibreOffice Writer
“text editors”, but we need to be a bit more careful when it comes to
programming. By default, Microsoft Word uses <code>.docx</code> files to
store not only text, but also formatting information about fonts,
headings, and so on. This extra information isn’t stored as characters
and doesn’t mean anything to tools like <code>head</code>, which expects
input files to contain nothing but the letters, digits, and punctuation
on a standard computer keyboard. When editing programs, therefore, you
must either use a plain text editor or be careful to save files as plain
text.</p>
</div>
</div>
</div>
<p>What if we want to select lines from an arbitrary file? We could edit
<code>middle.sh</code> each time to change the filename, but that would
probably take longer than typing the command out again in the shell and
executing it with a new file name. Instead, let’s edit
<code>middle.sh</code> and make it more versatile:</p>
<div class="codewrapper sourceCode" id="cb5">
<h3 class="code-label">BASH<i aria-hidden="true" data-feather="chevron-left"></i><i aria-hidden="true" data-feather="chevron-right"></i>
</h3>
<pre class="sourceCode bash" tabindex="0"><code class="sourceCode bash"><span id="cb5-1"><a href="#cb5-1" tabindex="-1"></a><span class="ex">$</span> nano middle.sh</span></code></pre>
</div>
<p>Now, within “nano”, replace the text <code>octane.pdb</code> with the
special variable called <code>$1</code>:</p>
<pre class="source"><code>head -n 15 "$1" | tail -n 5</code></pre>
<p>Inside a shell script, <code>$1</code> means ‘the first filename (or
other argument) on the command line’. We can now run our script like
this:</p>
<div class="codewrapper sourceCode" id="cb7">
<h3 class="code-label">BASH<i aria-hidden="true" data-feather="chevron-left"></i><i aria-hidden="true" data-feather="chevron-right"></i>
</h3>
<pre class="sourceCode bash" tabindex="0"><code class="sourceCode bash"><span id="cb7-1"><a href="#cb7-1" tabindex="-1"></a><span class="ex">$</span> bash middle.sh octane.pdb</span></code></pre>
</div>
<div class="codewrapper">
<h3 class="code-label">OUTPUT<i aria-hidden="true" data-feather="chevron-left"></i><i aria-hidden="true" data-feather="chevron-right"></i>
</h3>
<pre class="output" tabindex="0"><code>ATOM 9 H 1 -4.502 0.681 0.785 1.00 0.00
ATOM 10 H 1 -5.254 -0.243 -0.537 1.00 0.00
ATOM 11 H 1 -4.357 1.252 -0.895 1.00 0.00
ATOM 12 H 1 -3.009 -0.741 -1.467 1.00 0.00
ATOM 13 H 1 -3.172 -1.337 0.206 1.00 0.00</code></pre>
</div>
<p>or on a different file like this:</p>
<div class="codewrapper sourceCode" id="cb9">
<h3 class="code-label">BASH<i aria-hidden="true" data-feather="chevron-left"></i><i aria-hidden="true" data-feather="chevron-right"></i>
</h3>
<pre class="sourceCode bash" tabindex="0"><code class="sourceCode bash"><span id="cb9-1"><a href="#cb9-1" tabindex="-1"></a><span class="ex">$</span> bash middle.sh pentane.pdb</span></code></pre>
</div>
<div class="codewrapper">
<h3 class="code-label">OUTPUT<i aria-hidden="true" data-feather="chevron-left"></i><i aria-hidden="true" data-feather="chevron-right"></i>
</h3>
<pre class="output" tabindex="0"><code>ATOM 9 H 1 1.324 0.350 -1.332 1.00 0.00
ATOM 10 H 1 1.271 1.378 0.122 1.00 0.00
ATOM 11 H 1 -0.074 -0.384 1.288 1.00 0.00
ATOM 12 H 1 -0.048 -1.362 -0.205 1.00 0.00
ATOM 13 H 1 -1.183 0.500 -1.412 1.00 0.00</code></pre>
</div>
<div id="double-quotes-around-arguments" class="callout">
<div class="callout-square">
<i class="callout-icon" data-feather="bell"></i>
</div>
<div id="double-quotes-around-arguments" class="callout-inner">
<h3 class="callout-title">Double-Quotes Around Arguments</h3>
<div class="callout-content">
<p>For the same reason that we put the loop variable inside
double-quotes, in case the filename happens to contain any spaces, we
surround <code>$1</code> with double-quotes.</p>
</div>
</div>
</div>
<p>Currently, we need to edit <code>middle.sh</code> each time we want
to adjust the range of lines that is returned. Let’s fix that by
configuring our script to instead use three command-line arguments.
After the first command-line argument (<code>$1</code>), each additional
argument that we provide will be accessible via the special variables
<code>$1</code>, <code>$2</code>, <code>$3</code>, which refer to the
first, second, third command-line arguments, respectively.</p>
<p>Knowing this, we can use additional arguments to define the range of
lines to be passed to <code>head</code> and <code>tail</code>
respectively:</p>
<div class="codewrapper sourceCode" id="cb11">
<h3 class="code-label">BASH<i aria-hidden="true" data-feather="chevron-left"></i><i aria-hidden="true" data-feather="chevron-right"></i>
</h3>
<pre class="sourceCode bash" tabindex="0"><code class="sourceCode bash"><span id="cb11-1"><a href="#cb11-1" tabindex="-1"></a><span class="ex">$</span> nano middle.sh</span></code></pre>
</div>
<pre class="source"><code>head -n "$2" "$1" | tail -n "$3"</code></pre>
<p>We can now run:</p>
<div class="codewrapper sourceCode" id="cb13">
<h3 class="code-label">BASH<i aria-hidden="true" data-feather="chevron-left"></i><i aria-hidden="true" data-feather="chevron-right"></i>
</h3>
<pre class="sourceCode bash" tabindex="0"><code class="sourceCode bash"><span id="cb13-1"><a href="#cb13-1" tabindex="-1"></a><span class="ex">$</span> bash middle.sh pentane.pdb 15 5</span></code></pre>
</div>
<div class="codewrapper">
<h3 class="code-label">OUTPUT<i aria-hidden="true" data-feather="chevron-left"></i><i aria-hidden="true" data-feather="chevron-right"></i>
</h3>
<pre class="output" tabindex="0"><code>ATOM 9 H 1 1.324 0.350 -1.332 1.00 0.00
ATOM 10 H 1 1.271 1.378 0.122 1.00 0.00
ATOM 11 H 1 -0.074 -0.384 1.288 1.00 0.00
ATOM 12 H 1 -0.048 -1.362 -0.205 1.00 0.00
ATOM 13 H 1 -1.183 0.500 -1.412 1.00 0.00</code></pre>
</div>
<p>By changing the arguments to our command, we can change our script’s
behaviour:</p>
<div class="codewrapper sourceCode" id="cb15">
<h3 class="code-label">BASH<i aria-hidden="true" data-feather="chevron-left"></i><i aria-hidden="true" data-feather="chevron-right"></i>
</h3>
<pre class="sourceCode bash" tabindex="0"><code class="sourceCode bash"><span id="cb15-1"><a href="#cb15-1" tabindex="-1"></a><span class="ex">$</span> bash middle.sh pentane.pdb 20 5</span></code></pre>
</div>
<div class="codewrapper">
<h3 class="code-label">OUTPUT<i aria-hidden="true" data-feather="chevron-left"></i><i aria-hidden="true" data-feather="chevron-right"></i>
</h3>
<pre class="output" tabindex="0"><code>ATOM 14 H 1 -1.259 1.420 0.112 1.00 0.00
ATOM 15 H 1 -2.608 -0.407 1.130 1.00 0.00
ATOM 16 H 1 -2.540 -1.303 -0.404 1.00 0.00
ATOM 17 H 1 -3.393 0.254 -0.321 1.00 0.00
TER 18 1</code></pre>
</div>
<p>This works, but it may take the next person who reads
<code>middle.sh</code> a moment to figure out what it does. We can
improve our script by adding some <strong>comments</strong> at the
top:</p>
<div class="codewrapper sourceCode" id="cb17">
<h3 class="code-label">BASH<i aria-hidden="true" data-feather="chevron-left"></i><i aria-hidden="true" data-feather="chevron-right"></i>
</h3>
<pre class="sourceCode bash" tabindex="0"><code class="sourceCode bash"><span id="cb17-1"><a href="#cb17-1" tabindex="-1"></a><span class="ex">$</span> nano middle.sh</span></code></pre>
</div>
<pre class="source"><code># Select lines from the middle of a file.
# Usage: bash middle.sh filename end_line num_lines
head -n "$2" "$1" | tail -n "$3"</code></pre>
<p>A comment starts with a <code>#</code> character and runs to the end
of the line. The computer ignores comments, but they’re invaluable for
helping people (including your future self) understand and use scripts.
The only caveat is that each time you modify the script, you should
check that the comment is still accurate. An explanation that sends the
reader in the wrong direction is worse than none at all.</p>
<p>What if we want to process many files in a single pipeline? For
example, if we want to sort our <code>.pdb</code> files by length, we
would type:</p>
<div class="codewrapper sourceCode" id="cb19">
<h3 class="code-label">BASH<i aria-hidden="true" data-feather="chevron-left"></i><i aria-hidden="true" data-feather="chevron-right"></i>
</h3>
<pre class="sourceCode bash" tabindex="0"><code class="sourceCode bash"><span id="cb19-1"><a href="#cb19-1" tabindex="-1"></a><span class="ex">$</span> wc <span class="at">-l</span> <span class="pp">*</span>.pdb <span class="kw">|</span> <span class="fu">sort</span> <span class="at">-n</span></span></code></pre>
</div>
<p>because <code>wc -l</code> lists the number of lines in the files
(recall that <code>wc</code> stands for ‘word count’, adding the
<code>-l</code> option means ‘count lines’ instead) and
<code>sort -n</code> sorts things numerically. We could put this in a
file, but then it would only ever sort a list of <code>.pdb</code> files
in the current directory. If we want to be able to get a sorted list of
other kinds of files, we need a way to get all those names into the
script. We can’t use <code>$1</code>, <code>$2</code>, and so on because
we don’t know how many files there are. Instead, we use the special
variable <code>$@</code>, which means, ‘All of the command-line
arguments to the shell script’. We also should put <code>$@</code>
inside double-quotes to handle the case of arguments containing spaces
(<code>"$@"</code> is special syntax and is equivalent to
<code>"$1"</code> <code>"$2"</code> …).</p>
<p>Here’s an example:</p>
<div class="codewrapper sourceCode" id="cb20">
<h3 class="code-label">BASH<i aria-hidden="true" data-feather="chevron-left"></i><i aria-hidden="true" data-feather="chevron-right"></i>
</h3>
<pre class="sourceCode bash" tabindex="0"><code class="sourceCode bash"><span id="cb20-1"><a href="#cb20-1" tabindex="-1"></a><span class="ex">$</span> nano sorted.sh</span></code></pre>
</div>
<pre class="source"><code># Sort files by their length.
# Usage: bash sorted.sh one_or_more_filenames
wc -l "$@" | sort -n</code></pre>
<div class="codewrapper sourceCode" id="cb22">
<h3 class="code-label">BASH<i aria-hidden="true" data-feather="chevron-left"></i><i aria-hidden="true" data-feather="chevron-right"></i>
</h3>
<pre class="sourceCode bash" tabindex="0"><code class="sourceCode bash"><span id="cb22-1"><a href="#cb22-1" tabindex="-1"></a><span class="ex">$</span> bash sorted.sh <span class="pp">*</span>.pdb ../creatures/<span class="pp">*</span>.dat</span></code></pre>
</div>
<div class="codewrapper">
<h3 class="code-label">OUTPUT<i aria-hidden="true" data-feather="chevron-left"></i><i aria-hidden="true" data-feather="chevron-right"></i>
</h3>
<pre class="output" tabindex="0"><code>9 methane.pdb
12 ethane.pdb
15 propane.pdb
20 cubane.pdb
21 pentane.pdb
30 octane.pdb
163 ../creatures/basilisk.dat
163 ../creatures/minotaur.dat
163 ../creatures/unicorn.dat
596 total</code></pre>
</div>
<div id="list-unique-species" class="callout challenge">
<div class="callout-square">
<i class="callout-icon" data-feather="zap"></i>
</div>
<div id="list-unique-species" class="callout-inner">
<h3 class="callout-title">List Unique Species</h3>
<div class="callout-content">
<p>Leah has several hundred data files, each of which is formatted like
this:</p>
<pre class="source"><code>2013-11-05,deer,5
2013-11-05,rabbit,22
2013-11-05,raccoon,7
2013-11-06,rabbit,19
2013-11-06,deer,2
2013-11-06,fox,1
2013-11-07,rabbit,18
2013-11-07,bear,1</code></pre>
<p>An example of this type of file is given in
<code>shell-lesson-data/exercise-data/animal-counts/animals.csv</code>.</p>
<p>We can use the command
<code>cut -d , -f 2 animals.csv | sort | uniq</code> to produce the
unique species in <code>animals.csv</code>. In order to avoid having to
type out this series of commands every time, a scientist may choose to
write a shell script instead.</p>
<p>Write a shell script called <code>species.sh</code> that takes any
number of filenames as command-line arguments and uses a variation of
the above command to print a list of the unique species appearing in
each of those files separately.</p>
</div>
</div>
</div>
<div id="accordionSolution1" class="accordion challenge-accordion accordion-flush">
<div class="accordion-item">
<button class="accordion-button solution-button collapsed" type="button" data-bs-toggle="collapse" data-bs-target="#collapseSolution1" aria-expanded="false" aria-controls="collapseSolution1">
<h4 class="accordion-header" id="headingSolution1"> Show me the solution </h4>
</button>
<div id="collapseSolution1" class="accordion-collapse collapse" aria-labelledby="headingSolution1" data-bs-parent="#accordionSolution1">
<div class="accordion-body">
<div class="codewrapper sourceCode" id="cb25">
<h3 class="code-label">BASH<i aria-hidden="true" data-feather="chevron-left"></i><i aria-hidden="true" data-feather="chevron-right"></i>
</h3>
<pre class="sourceCode bash" tabindex="0"><code class="sourceCode bash"><span id="cb25-1"><a href="#cb25-1" tabindex="-1"></a><span class="co"># Script to find unique species in csv files where species is the second data field</span></span>
<span id="cb25-2"><a href="#cb25-2" tabindex="-1"></a><span class="co"># This script accepts any number of file names as command line arguments</span></span>
<span id="cb25-3"><a href="#cb25-3" tabindex="-1"></a></span>
<span id="cb25-4"><a href="#cb25-4" tabindex="-1"></a><span class="co"># Loop over all files</span></span>
<span id="cb25-5"><a href="#cb25-5" tabindex="-1"></a><span class="cf">for</span> file <span class="kw">in</span> <span class="va">$@</span></span>
<span id="cb25-6"><a href="#cb25-6" tabindex="-1"></a><span class="cf">do</span></span>
<span id="cb25-7"><a href="#cb25-7" tabindex="-1"></a> <span class="bu">echo</span> <span class="st">"Unique species in </span><span class="va">$file</span><span class="st">:"</span></span>
<span id="cb25-8"><a href="#cb25-8" tabindex="-1"></a> <span class="co"># Extract species names</span></span>
<span id="cb25-9"><a href="#cb25-9" tabindex="-1"></a> <span class="fu">cut</span> <span class="at">-d</span> , <span class="at">-f</span> 2 <span class="va">$file</span> <span class="kw">|</span> <span class="fu">sort</span> <span class="kw">|</span> <span class="fu">uniq</span></span>
<span id="cb25-10"><a href="#cb25-10" tabindex="-1"></a><span class="cf">done</span></span></code></pre>
</div>
</div>
</div>
</div>
</div>
<p>Suppose we have just run a series of commands that did something
useful — for example, creating a graph we’d like to use in a paper. We’d
like to be able to re-create the graph later if we need to, so we want
to save the commands in a file. Instead of typing them in again (and
potentially getting them wrong) we can do this:</p>
<div class="codewrapper sourceCode" id="cb26">
<h3 class="code-label">BASH<i aria-hidden="true" data-feather="chevron-left"></i><i aria-hidden="true" data-feather="chevron-right"></i>
</h3>
<pre class="sourceCode bash" tabindex="0"><code class="sourceCode bash"><span id="cb26-1"><a href="#cb26-1" tabindex="-1"></a><span class="ex">$</span> history <span class="kw">|</span> <span class="fu">tail</span> <span class="at">-n</span> 5 <span class="op">></span> redo-figure-3.sh</span></code></pre>
</div>
<p>The file <code>redo-figure-3.sh</code> now contains:</p>
<pre class="source"><code>297 bash goostats.sh NENE01729B.txt stats-NENE01729B.txt
298 bash goodiff.sh stats-NENE01729B.txt /data/validated/01729.txt > 01729-differences.txt
299 cut -d ',' -f 2-3 01729-differences.txt > 01729-time-series.txt
300 ygraph --format scatter --color bw --borders none 01729-time-series.txt figure-3.png
301 history | tail -n 5 > redo-figure-3.sh</code></pre>
<p>After a moment’s work in an editor to remove the serial numbers on
the commands, and to remove the final line where we called the
<code>history</code> command, we have a completely accurate record of
how we created that figure.</p>
<div id="why-record-commands-in-the-history-before-running-them" class="callout challenge">
<div class="callout-square">
<i class="callout-icon" data-feather="zap"></i>
</div>
<div id="why-record-commands-in-the-history-before-running-them" class="callout-inner">
<h3 class="callout-title">Why Record Commands in the History Before Running Them?</h3>
<div class="callout-content">
<p>If you run the command:</p>
<div class="codewrapper sourceCode" id="cb28">
<h3 class="code-label">BASH<i aria-hidden="true" data-feather="chevron-left"></i><i aria-hidden="true" data-feather="chevron-right"></i>
</h3>
<pre class="sourceCode bash" tabindex="0"><code class="sourceCode bash"><span id="cb28-1"><a href="#cb28-1" tabindex="-1"></a><span class="ex">$</span> history <span class="kw">|</span> <span class="fu">tail</span> <span class="at">-n</span> 5 <span class="op">></span> recent.sh</span></code></pre>
</div>
<p>the last command in the file is the <code>history</code> command
itself, i.e., the shell has added <code>history</code> to the command
log before actually running it. In fact, the shell <em>always</em> adds
commands to the log before running them. Why do you think it does
this?</p>
</div>
</div>
</div>
<div id="accordionSolution2" class="accordion challenge-accordion accordion-flush">
<div class="accordion-item">
<button class="accordion-button solution-button collapsed" type="button" data-bs-toggle="collapse" data-bs-target="#collapseSolution2" aria-expanded="false" aria-controls="collapseSolution2">
<h4 class="accordion-header" id="headingSolution2"> Show me the solution </h4>
</button>
<div id="collapseSolution2" class="accordion-collapse collapse" aria-labelledby="headingSolution2" data-bs-parent="#accordionSolution2">
<div class="accordion-body">
<p>If a command causes something to crash or hang, it might be useful to
know what that command was, in order to investigate the problem. Were
the command only be recorded after running it, we would not have a
record of the last command run in the event of a crash.</p>
</div>
</div>
</div>
</div>
<p>In practice, most people develop shell scripts by running commands at
the shell prompt a few times to make sure they’re doing the right thing,
then saving them in a file for re-use. This style of work allows people
to recycle what they discover about their data and their workflow with
one call to <code>history</code> and a bit of editing to clean up the
output and save it as a shell script.</p>
<section><h2 class="section-heading" id="nelles-pipeline-creating-a-script">Nelle’s Pipeline: Creating a Script<a class="anchor" aria-label="anchor" href="#nelles-pipeline-creating-a-script"></a></h2>
<hr class="half-width"><p>Nelle’s supervisor insisted that all her analytics must be
reproducible. The easiest way to capture all the steps is in a
script.</p>
<p>First we return to Nelle’s project directory:</p>
<div class="codewrapper sourceCode" id="cb29">
<h3 class="code-label">BASH<i aria-hidden="true" data-feather="chevron-left"></i><i aria-hidden="true" data-feather="chevron-right"></i>
</h3>
<pre class="sourceCode bash" tabindex="0"><code class="sourceCode bash"><span id="cb29-1"><a href="#cb29-1" tabindex="-1"></a><span class="ex">$</span> cd ../../north-pacific-gyre/</span></code></pre>
</div>
<p>She creates a file using <code>nano</code> …</p>
<div class="codewrapper sourceCode" id="cb30">
<h3 class="code-label">BASH<i aria-hidden="true" data-feather="chevron-left"></i><i aria-hidden="true" data-feather="chevron-right"></i>
</h3>
<pre class="sourceCode bash" tabindex="0"><code class="sourceCode bash"><span id="cb30-1"><a href="#cb30-1" tabindex="-1"></a><span class="ex">$</span> nano do-stats.sh</span></code></pre>
</div>
<p>…which contains the following:</p>
<div class="codewrapper sourceCode" id="cb31">
<h3 class="code-label">BASH<i aria-hidden="true" data-feather="chevron-left"></i><i aria-hidden="true" data-feather="chevron-right"></i>
</h3>
<pre class="sourceCode bash" tabindex="0"><code class="sourceCode bash"><span id="cb31-1"><a href="#cb31-1" tabindex="-1"></a><span class="co"># Calculate stats for data files.</span></span>
<span id="cb31-2"><a href="#cb31-2" tabindex="-1"></a><span class="cf">for</span> datafile <span class="kw">in</span> <span class="st">"</span><span class="va">$@</span><span class="st">"</span></span>
<span id="cb31-3"><a href="#cb31-3" tabindex="-1"></a><span class="cf">do</span></span>
<span id="cb31-4"><a href="#cb31-4" tabindex="-1"></a> <span class="bu">echo</span> <span class="va">$datafile</span></span>
<span id="cb31-5"><a href="#cb31-5" tabindex="-1"></a> <span class="fu">bash</span> goostats.sh <span class="va">$datafile</span> stats-<span class="va">$datafile</span></span>
<span id="cb31-6"><a href="#cb31-6" tabindex="-1"></a><span class="cf">done</span></span></code></pre>
</div>
<p>She saves this in a file called <code>do-stats.sh</code> so that she
can now re-do the first stage of her analysis by typing:</p>
<div class="codewrapper sourceCode" id="cb32">
<h3 class="code-label">BASH<i aria-hidden="true" data-feather="chevron-left"></i><i aria-hidden="true" data-feather="chevron-right"></i>
</h3>
<pre class="sourceCode bash" tabindex="0"><code class="sourceCode bash"><span id="cb32-1"><a href="#cb32-1" tabindex="-1"></a><span class="ex">$</span> bash do-stats.sh NENE<span class="pp">*</span>A.txt NENE<span class="pp">*</span>B.txt</span></code></pre>
</div>
<p>She can also do this:</p>
<div class="codewrapper sourceCode" id="cb33">
<h3 class="code-label">BASH<i aria-hidden="true" data-feather="chevron-left"></i><i aria-hidden="true" data-feather="chevron-right"></i>
</h3>
<pre class="sourceCode bash" tabindex="0"><code class="sourceCode bash"><span id="cb33-1"><a href="#cb33-1" tabindex="-1"></a><span class="ex">$</span> bash do-stats.sh NENE<span class="pp">*</span>A.txt NENE<span class="pp">*</span>B.txt <span class="kw">|</span> <span class="fu">wc</span> <span class="at">-l</span></span></code></pre>
</div>
<p>so that the output is just the number of files processed rather than
the names of the files that were processed.</p>
<p>One thing to note about Nelle’s script is that it lets the person
running it decide what files to process. She could have written it
as:</p>
<div class="codewrapper sourceCode" id="cb34">
<h3 class="code-label">BASH<i aria-hidden="true" data-feather="chevron-left"></i><i aria-hidden="true" data-feather="chevron-right"></i>
</h3>
<pre class="sourceCode bash" tabindex="0"><code class="sourceCode bash"><span id="cb34-1"><a href="#cb34-1" tabindex="-1"></a><span class="co"># Calculate stats for Site A and Site B data files.</span></span>
<span id="cb34-2"><a href="#cb34-2" tabindex="-1"></a><span class="cf">for</span> datafile <span class="kw">in</span> NENE<span class="pp">*</span>A.txt NENE<span class="pp">*</span>B.txt</span>
<span id="cb34-3"><a href="#cb34-3" tabindex="-1"></a><span class="cf">do</span></span>
<span id="cb34-4"><a href="#cb34-4" tabindex="-1"></a> <span class="bu">echo</span> <span class="va">$datafile</span></span>
<span id="cb34-5"><a href="#cb34-5" tabindex="-1"></a> <span class="fu">bash</span> goostats.sh <span class="va">$datafile</span> stats-<span class="va">$datafile</span></span>
<span id="cb34-6"><a href="#cb34-6" tabindex="-1"></a><span class="cf">done</span></span></code></pre>
</div>
<p>The advantage is that this always selects the right files: she
doesn’t have to remember to exclude the ‘Z’ files. The disadvantage is
that it <em>always</em> selects just those files — she can’t run it on
all files (including the ‘Z’ files), or on the ‘G’ or ‘H’ files her
colleagues in Antarctica are producing, without editing the script. If
she wanted to be more adventurous, she could modify her script to check
for command-line arguments, and use <code>NENE*A.txt NENE*B.txt</code>
if none were provided. Of course, this introduces another tradeoff
between flexibility and complexity.</p>
<div id="variables-in-shell-scripts" class="callout challenge">
<div class="callout-square">
<i class="callout-icon" data-feather="zap"></i>
</div>
<div id="variables-in-shell-scripts" class="callout-inner">
<h3 class="callout-title">Variables in Shell Scripts</h3>
<div class="callout-content">
<p>In the <code>alkanes</code> directory, imagine you have a shell
script called <code>script.sh</code> containing the following
commands:</p>
<div class="codewrapper sourceCode" id="cb35">
<h3 class="code-label">BASH<i aria-hidden="true" data-feather="chevron-left"></i><i aria-hidden="true" data-feather="chevron-right"></i>
</h3>
<pre class="sourceCode bash" tabindex="0"><code class="sourceCode bash"><span id="cb35-1"><a href="#cb35-1" tabindex="-1"></a><span class="fu">head</span> <span class="at">-n</span> <span class="va">$2</span> <span class="va">$1</span></span>
<span id="cb35-2"><a href="#cb35-2" tabindex="-1"></a><span class="fu">tail</span> <span class="at">-n</span> <span class="va">$3</span> <span class="va">$1</span></span></code></pre>
</div>
<p>While you are in the <code>alkanes</code> directory, you type the
following command:</p>
<div class="codewrapper sourceCode" id="cb36">
<h3 class="code-label">BASH<i aria-hidden="true" data-feather="chevron-left"></i><i aria-hidden="true" data-feather="chevron-right"></i>
</h3>
<pre class="sourceCode bash" tabindex="0"><code class="sourceCode bash"><span id="cb36-1"><a href="#cb36-1" tabindex="-1"></a><span class="ex">$</span> bash script.sh <span class="st">'*.pdb'</span> 1 1</span></code></pre>
</div>
<p>Which of the following outputs would you expect to see?</p>
<ol style="list-style-type: decimal"><li>All of the lines between the first and the last lines of each file
ending in <code>.pdb</code> in the <code>alkanes</code> directory</li>
<li>The first and the last line of each file ending in <code>.pdb</code>
in the <code>alkanes</code> directory</li>
<li>The first and the last line of each file in the <code>alkanes</code>
directory</li>
<li>An error because of the quotes around <code>*.pdb</code>
</li>
</ol></div>
</div>
</div>
<div id="accordionSolution3" class="accordion challenge-accordion accordion-flush">
<div class="accordion-item">
<button class="accordion-button solution-button collapsed" type="button" data-bs-toggle="collapse" data-bs-target="#collapseSolution3" aria-expanded="false" aria-controls="collapseSolution3">
<h4 class="accordion-header" id="headingSolution3"> Show me the solution </h4>
</button>
<div id="collapseSolution3" class="accordion-collapse collapse" aria-labelledby="headingSolution3" data-bs-parent="#accordionSolution3">
<div class="accordion-body">
<p>The correct answer is 2.</p>
<p>The special variables <code>$1</code>, <code>$2</code> and
<code>$3</code> represent the command line arguments given to the
script, such that the commands run are:</p>
<div class="codewrapper sourceCode" id="cb37">
<h3 class="code-label">BASH<i aria-hidden="true" data-feather="chevron-left"></i><i aria-hidden="true" data-feather="chevron-right"></i>
</h3>
<pre class="sourceCode bash" tabindex="0"><code class="sourceCode bash"><span id="cb37-1"><a href="#cb37-1" tabindex="-1"></a><span class="ex">$</span> head <span class="at">-n</span> 1 cubane.pdb ethane.pdb octane.pdb pentane.pdb propane.pdb</span>
<span id="cb37-2"><a href="#cb37-2" tabindex="-1"></a><span class="ex">$</span> tail <span class="at">-n</span> 1 cubane.pdb ethane.pdb octane.pdb pentane.pdb propane.pdb</span></code></pre>
</div>
<p>The shell does not expand <code>'*.pdb'</code> because it is enclosed
by quote marks. As such, the first argument to the script is
<code>'*.pdb'</code> which gets expanded within the script by
<code>head</code> and <code>tail</code>.</p>
</div>
</div>
</div>
</div>
<div id="find-the-longest-file-with-a-given-extension" class="callout challenge">
<div class="callout-square">
<i class="callout-icon" data-feather="zap"></i>
</div>
<div id="find-the-longest-file-with-a-given-extension" class="callout-inner">
<h3 class="callout-title">Find the Longest File With a Given Extension</h3>
<div class="callout-content">
<p>Write a shell script called <code>longest.sh</code> that takes the
name of a directory and a filename extension as its arguments, and
prints out the name of the file with the most lines in that directory
with that extension. For example:</p>
<div class="codewrapper sourceCode" id="cb38">
<h3 class="code-label">BASH<i aria-hidden="true" data-feather="chevron-left"></i><i aria-hidden="true" data-feather="chevron-right"></i>
</h3>
<pre class="sourceCode bash" tabindex="0"><code class="sourceCode bash"><span id="cb38-1"><a href="#cb38-1" tabindex="-1"></a><span class="ex">$</span> bash longest.sh shell-lesson-data/exercise-data/alkanes pdb</span></code></pre>
</div>
<p>would print the name of the <code>.pdb</code> file in
<code>shell-lesson-data/exercise-data/alkanes</code> that has the most
lines.</p>
<p>Feel free to test your script on another directory e.g.</p>
<div class="codewrapper sourceCode" id="cb39">
<h3 class="code-label">BASH<i aria-hidden="true" data-feather="chevron-left"></i><i aria-hidden="true" data-feather="chevron-right"></i>
</h3>
<pre class="sourceCode bash" tabindex="0"><code class="sourceCode bash"><span id="cb39-1"><a href="#cb39-1" tabindex="-1"></a><span class="ex">$</span> bash longest.sh shell-lesson-data/exercise-data/writing txt</span></code></pre>
</div>
</div>
</div>
</div>
<div id="accordionSolution4" class="accordion challenge-accordion accordion-flush">
<div class="accordion-item">
<button class="accordion-button solution-button collapsed" type="button" data-bs-toggle="collapse" data-bs-target="#collapseSolution4" aria-expanded="false" aria-controls="collapseSolution4">
<h4 class="accordion-header" id="headingSolution4"> Show me the solution </h4>
</button>
<div id="collapseSolution4" class="accordion-collapse collapse" aria-labelledby="headingSolution4" data-bs-parent="#accordionSolution4">
<div class="accordion-body">
<div class="codewrapper sourceCode" id="cb40">
<h3 class="code-label">BASH<i aria-hidden="true" data-feather="chevron-left"></i><i aria-hidden="true" data-feather="chevron-right"></i>
</h3>
<pre class="sourceCode bash" tabindex="0"><code class="sourceCode bash"><span id="cb40-1"><a href="#cb40-1" tabindex="-1"></a><span class="co"># Shell script which takes two arguments:</span></span>
<span id="cb40-2"><a href="#cb40-2" tabindex="-1"></a><span class="co"># 1. a directory name</span></span>
<span id="cb40-3"><a href="#cb40-3" tabindex="-1"></a><span class="co"># 2. a file extension</span></span>
<span id="cb40-4"><a href="#cb40-4" tabindex="-1"></a><span class="co"># and prints the name of the file in that directory</span></span>
<span id="cb40-5"><a href="#cb40-5" tabindex="-1"></a><span class="co"># with the most lines which matches the file extension.</span></span>
<span id="cb40-6"><a href="#cb40-6" tabindex="-1"></a></span>
<span id="cb40-7"><a href="#cb40-7" tabindex="-1"></a><span class="fu">wc</span> <span class="at">-l</span> <span class="va">$1</span>/<span class="pp">*</span>.<span class="va">$2</span> <span class="kw">|</span> <span class="fu">sort</span> <span class="at">-n</span> <span class="kw">|</span> <span class="fu">tail</span> <span class="at">-n</span> 2 <span class="kw">|</span> <span class="fu">head</span> <span class="at">-n</span> 1</span></code></pre>
</div>
<p>The first part of the pipeline, <code>wc -l $1/*.$2 | sort -n</code>,
counts the lines in each file and sorts them numerically (largest last).
When there’s more than one file, <code>wc</code> also outputs a final
summary line, giving the total number of lines across <em>all</em>
files. We use <code>tail -n 2 | head -n 1</code> to throw away this last
line.</p>
<p>With <code>wc -l $1/*.$2 | sort -n | tail -n 1</code> we’ll see the
final summary line: we can build our pipeline up in pieces to be sure we
understand the output.</p>
</div>
</div>
</div>
</div>
<div id="script-reading-comprehension" class="callout challenge">
<div class="callout-square">
<i class="callout-icon" data-feather="zap"></i>
</div>
<div id="script-reading-comprehension" class="callout-inner">
<h3 class="callout-title">Script Reading Comprehension</h3>
<div class="callout-content">
<p>For this question, consider the
<code>shell-lesson-data/exercise-data/alkanes</code> directory once
again. This contains a number of <code>.pdb</code> files in addition to
any other files you may have created. Explain what each of the following
three scripts would do when run as <code>bash script1.sh *.pdb</code>,
<code>bash script2.sh *.pdb</code>, and
<code>bash script3.sh *.pdb</code> respectively.</p>
<div class="codewrapper sourceCode" id="cb41">
<h3 class="code-label">BASH<i aria-hidden="true" data-feather="chevron-left"></i><i aria-hidden="true" data-feather="chevron-right"></i>
</h3>
<pre class="sourceCode bash" tabindex="0"><code class="sourceCode bash"><span id="cb41-1"><a href="#cb41-1" tabindex="-1"></a><span class="co"># Script 1</span></span>
<span id="cb41-2"><a href="#cb41-2" tabindex="-1"></a><span class="bu">echo</span> <span class="pp">*</span>.<span class="pp">*</span></span></code></pre>
</div>
<div class="codewrapper sourceCode" id="cb42">
<h3 class="code-label">BASH<i aria-hidden="true" data-feather="chevron-left"></i><i aria-hidden="true" data-feather="chevron-right"></i>
</h3>
<pre class="sourceCode bash" tabindex="0"><code class="sourceCode bash"><span id="cb42-1"><a href="#cb42-1" tabindex="-1"></a><span class="co"># Script 2</span></span>
<span id="cb42-2"><a href="#cb42-2" tabindex="-1"></a><span class="cf">for</span> filename <span class="kw">in</span> <span class="va">$1</span> <span class="va">$2</span> <span class="va">$3</span></span>
<span id="cb42-3"><a href="#cb42-3" tabindex="-1"></a><span class="cf">do</span></span>
<span id="cb42-4"><a href="#cb42-4" tabindex="-1"></a> <span class="fu">cat</span> <span class="va">$filename</span></span>
<span id="cb42-5"><a href="#cb42-5" tabindex="-1"></a><span class="cf">done</span></span></code></pre>
</div>
<div class="codewrapper sourceCode" id="cb43">
<h3 class="code-label">BASH<i aria-hidden="true" data-feather="chevron-left"></i><i aria-hidden="true" data-feather="chevron-right"></i>
</h3>
<pre class="sourceCode bash" tabindex="0"><code class="sourceCode bash"><span id="cb43-1"><a href="#cb43-1" tabindex="-1"></a><span class="co"># Script 3</span></span>
<span id="cb43-2"><a href="#cb43-2" tabindex="-1"></a><span class="bu">echo</span> <span class="va">$@</span>.pdb</span></code></pre>
</div>
</div>
</div>
</div>
<div id="accordionSolution5" class="accordion challenge-accordion accordion-flush">
<div class="accordion-item">
<button class="accordion-button solution-button collapsed" type="button" data-bs-toggle="collapse" data-bs-target="#collapseSolution5" aria-expanded="false" aria-controls="collapseSolution5">
<h4 class="accordion-header" id="headingSolution5"> Solutions </h4>
</button>
<div id="collapseSolution5" class="accordion-collapse collapse" aria-labelledby="headingSolution5" data-bs-parent="#accordionSolution5">
<div class="accordion-body">
<p>In each case, the shell expands the wildcard in <code>*.pdb</code>
before passing the resulting list of file names as arguments to the
script.</p>
<p>Script 1 would print out a list of all files containing a dot in
their name. The arguments passed to the script are not actually used
anywhere in the script.</p>
<p>Script 2 would print the contents of the first 3 files with a
<code>.pdb</code> file extension. <code>$1</code>, <code>$2</code>, and
<code>$3</code> refer to the first, second, and third argument
respectively.</p>
<p>Script 3 would print all the arguments to the script (i.e. all the
<code>.pdb</code> files), followed by <code>.pdb</code>. <code>$@</code>
refers to <em>all</em> the arguments given to a shell script.</p>
<div class="codewrapper">
<h3 class="code-label">OUTPUT<i aria-hidden="true" data-feather="chevron-left"></i><i aria-hidden="true" data-feather="chevron-right"></i>
</h3>
<pre class="output" tabindex="0"><code>cubane.pdb ethane.pdb methane.pdb octane.pdb pentane.pdb propane.pdb.pdb</code></pre>
</div>
</div>
</div>
</div>
</div>
<div id="debugging-scripts" class="callout challenge">
<div class="callout-square">
<i class="callout-icon" data-feather="zap"></i>
</div>
<div id="debugging-scripts" class="callout-inner">
<h3 class="callout-title">Debugging Scripts</h3>
<div class="callout-content">
<p>Suppose you have saved the following script in a file called
<code>do-errors.sh</code> in Nelle’s <code>north-pacific-gyre</code>
directory:</p>
<div class="codewrapper sourceCode" id="cb45">
<h3 class="code-label">BASH<i aria-hidden="true" data-feather="chevron-left"></i><i aria-hidden="true" data-feather="chevron-right"></i>
</h3>
<pre class="sourceCode bash" tabindex="0"><code class="sourceCode bash"><span id="cb45-1"><a href="#cb45-1" tabindex="-1"></a><span class="co"># Calculate stats for data files.</span></span>
<span id="cb45-2"><a href="#cb45-2" tabindex="-1"></a><span class="cf">for</span> datafile <span class="kw">in</span> <span class="st">"</span><span class="va">$@</span><span class="st">"</span></span>
<span id="cb45-3"><a href="#cb45-3" tabindex="-1"></a><span class="cf">do</span></span>
<span id="cb45-4"><a href="#cb45-4" tabindex="-1"></a> <span class="bu">echo</span> <span class="va">$datfile</span></span>
<span id="cb45-5"><a href="#cb45-5" tabindex="-1"></a> <span class="fu">bash</span> goostats.sh <span class="va">$datafile</span> stats-<span class="va">$datafile</span></span>