CA644, System Software
Table of Contents
CA644, System Software
Dr. Niall McMahon
2022-09-13
If you print these slides, think about using two pages per sheet although don't worry too much about it!
Dr. Niall McMahon
Drawing on previous work by:
Dr. Michael Scriney
Dr. Long Cheng
And sources credited in the references.
Autumn 2022.
We will talk more about the history and development of Unix and Linux as well as its structure soon. First, let's get started with using it.
There are a few different ways to run Linux when you're on campus:
The School of Computing's Termcast deployment allows browser-based access to a Unix/Linux environment on a physical server in the school - it's a browser-based terminal and Linux shell. You can get access at termcast.computing.dcu.ie. Log in with your School of Computing credentials, i.e. your student.computing.dcu.ie account.
You can get help with termcast here. All of this was implemented by Stephen Blott.
A terminal is a program that processes input from the keyboard and provides output, or feedback, from the operating system. The concept of a terminal harks back to the old days when large main-frame machines were accessed using standalone physical consoles; these consoles were referred to as computer terminals.
In Unix-like systems these days, the terminal is a program that opens allowing the user to type commands and read output.
From the GNU (Unix-like) OS documentation, "a shell is simply a macro processor that executes commands. The term macro processor means functionality where text and symbols are expanded to create larger expressions. A Unix shell is both a command interpreter and a programming language".
The shell is launched when a terminal is opened and the functionality of both the terminal and the shell provide access to the operating system.
Again, from the GNU OS documentation, "[s]hells may be used interactively or non-interactively. In interactive mode, they accept input typed from the keyboard. When executing non-interactively, shells execute commands read from a file".
Other programs, for example, Graphical User Interfaces (GUIs), that mediate between the user and the system are usually not called shells but are equivalent systems.
There are many Unix shells but the most common is called Bash. This is based on the original Bourne Shell. Bash is an acronym for "Bourne Again Shell".
The default prompt is a dollar sign followed by the Bash version number, e.g. bash-5.10$, or in the DCU environment, username@machine_name:~>
Apple moved its operating system default shell to Z Shell (zsh) from Bash a few years ago. Both MacOS and Linux are derived from Unix and so both inherited shells and the functionality is very similar. For much of the shell work, you can use zsh. Some things will be different, for example the .bashrc initialisation file doesn't exist in zsh; you'll find the .zshrc file instead. You can read about Apple's move to zsh here. However, you can enable Bash on Apple machines - you can find out how to do this here.
As a note here, there is an accepted notation for describing commands. It is worth looking at some references, for example Google's note about document command-line syntax and reading the Bash documentation carefully. The output from Unix help files may be difficult to understand otherwise.
Shell commands are programs that allow you to control your machine and data effectively. Commands help with things like:
$ mysqldump -u root -p '123456’ mydb > /tmp/mydb.backup
$ gzip -9 /tmp/mydb.backup
$ scp /tmp/mydb.backup username@remote_ip:/backup/mydb/
These commands create a copy of a SQL, usually pronounced Sequel, database belonging to user "root" and upload it to a remote computer.
Some commands are built-in - called builtins and others can be installed later.
Shell commands can be input interactively at the prompt.
The shell has an associated current directory. This is the default starting point when locating files.
Commands are used by entering their names. As an example:
Typing ls and pressing Enter will run a program that lists the contents of the current directory. This sequence of actions is usually denoted,
$ ls
When you enter a command, the shell will check if the command is a builtin. If it's not, it will search its search path, i.e. all the directories that may contain the program. More about this later but the default directory for programs is the bin directory.
Flags are options that modify the behaviour of the command, e.g.
$ ls
Lists the files and directories (folders) in the current directory.
$ ls -l
Lists all information about the files and directories (folders) in the current directory. This is specified by the -l flag.
$ ls -al
Lists all files and directories (folders) in the current directory, including hidden files. This is specified by the -al flags.
The --help flag provides information about a command, e.g. ls --help.
Arguments are typed after the basic command and flags to further refine the command. They often refer to a file location. For example,
$ ls -l *.txt
In this case, the argument *.txt refines the list command to limit itself to all information about text files only in the current directory.
You can find out what folder you are currently in by using the pwd, or the print working directory, command:
$ pwd
cd means "change directory",
$ cd .
Change to the current working directory (.).
$ cd ..
Change to the directory above the current working directory (..).
ls ../../
List the contents of two directories above me.
$ cd ~
Change to the home directory (~). The home directory is the default base location for a user after logging in. A /home directory is created for each user on the system.
ls ~
List the contents of my home directory.
You can also use wildcards, e.g.
? matches a single character. For example, we could list myfil?.txt. This would return all files beginning with myfil and ending in .txt with any one character in between.
* matches all characters. For example, ls *.txt will list all text files in the current directory.
[] matches any one from a specified range of characters. For example, if a directory contains files called data.txt date.txt dats.txt, the command ls dat[ae].txt will return data.txt and date.txt.
Variables are created using <variable_name>=value, accessed using $<variable_name>.
By convention, variable names should be all uppercase. So VARIABLE_NAME, not variable_name.
Create a variable called NAME that stores your name. This line sets the variable:
$ NAME="Niall"
The echo command prints out the value, i.e.
$ echo $NAME
Note that the syntax is important. Accessing variables required an extra $ before the variable name.
When assigning a variable, you cannot have any spaces:
These are wrong:
$ NAME = Niall
$ NAME= Niall
$ NAME =Niall
This is right:
$ NAME=Niall
You can define variables like this:
$ var=value
$ var='value'
$ var="value"
Quote marks are not needed on the right in Bash scripts although there are situations where you might use them.
You can use variables like this:
echo $var
echo ${var}
Variable values can be locked by setting the variable as "read only", so in the following example:
$ myName=Niall
$ readonly myName
$ myName="New Name"
We will get an error after the third line. We can't change a readonly value. You cannot reset a readonly variable.
However, you can always reset regular variables:
$ NAME=Niall
$ unset NAME
$ NAME="Mary"
We can assign the results of commands to variables, so for example:
$ var=$(ls)
$ echo $var
You ought to see a list of files and folders in your directory.
When you type in a command, e.g. $ ls, shell checks if there is a function with that name (functions are user specified command groups), if not then if there is a builtin with that name and if not then if there exists an executable program in the locations specified by the PATH variable.
The PATH variable stores a list of possible directories and files.
When you enter a command, the PATH variable is checked. To see the locations specified in your PATH variable, type:
$ echo $PATH
Shell scripts can support one-dimensional arrays only, i.e. lists. For example, to create arrays, all with five things:
$ nums_1=(1 2 3 4 5)
$ nums_2=("A" "B" "C" "D" "E")
$ nums_3=(1 "B" 3 "4" "E")
You can easily add new terms, i.e. the length is flexible:
$ nums[6]="F"
$ nums[7]="G"
To get a particular element of an array:
$ ${array_name[index]}
To get the length of an array:
$ ${#array_name[@]}
To print all elements of an array requires a little scripting.
Commands, variables and control flows are the building blocks of shell scripts
Create a file called echo_1.sh using the touch command.
(The default usage of touch is to create or open and save a file without altering its contents. This has the effect of updating the file's access or modification date.)
So:
$ touch echo_1.sh
Use the Vi text editor to modify the file. Type:
$ vi echo_1.sh
Type i to insert text. On the first line write,
#!/bin/sh
This tells the operating system what interpreter to use, in this case Bash. sh is shorthand for the Bourne/Bash shell. Other possibilities include #!/usr/bin/python, i.e. Python.
Press Enter. On the second line write,
echo Hello World
The contents of echo_1.sh will look like this:
#!/bin/sh
echo Hello World
Press Esc to escape out of the text editing mode and type :wq to write and quit the file.
The editor that we're using us vi, or visual editor. This is a standard text editor that is shipped with Unix/Linux systems that has been around since the 1970s. It was designed for writing C programs.
Vim, which stands for "vi improved", was an upgraded version of vi developed in the 1980s. In general, vim is shipped by default, so that in most Linux distributions, typing the vi will in fact launch vim.
Some distribution include a stripped down version of vim and you may need to manually upgrade if you want the full version.
In general, it's safer to type vim <filename> rather than vi <filename>.
Vi uses the hjkl keys on the keyboard for navigation and the original implementation assigns letters to the arrow keys. Newer and more complete versions of vim have a more intuitive layout, with arrow keys working as you'd expect these days.
As an additional note, when you launch the program, you enter in "command" mode. To edit, you must type i for "insert" mode. If you get lost, hit esc to get back to "command" mode.
In this week's lecture, we created a file using touch example.txt. We then used vi example.txt to edit it.
You can also go straight to creating with vi by typing vi example.txt.
If you type vi with no argument, i.e. no filename, you will not be allowed save to disk. You must specify the file whe launching vi/vim.
You can find an overview of vi functionality in different places, including here (Colorado State), here (Indiana University).
You will need to make the script executable. When you create a file with touch, it is not executable by default. This is a feature of Unix-like operating systems. To make the file executable, use the chmod command:
$ chmod +x echo_1.sh
Here the flag x makes the file executable. The filename is the argument. chmod is read as "change mode".
To run the script, type:
$ ./echo_1.sh
Note that the file location is explicitly defined using ./
Pay attention to whitespace - the shell is sensitive to this. Whitespace errors are the most common cause of shell script bugs.
Comments are written as # My comment; the space after the # is important.
We can create and assign values to variables within shell scripts.
Create a script called variable_1.sh. The script will create a variable called NAME and assign it the value "Niall". The script will print out "Hello Niall" when it's run.
$ touch variable_1.sh
$ chmod +x ./variable_1.sh
$ vi variable_1.sh
Script content:
#!/bin/sh
NAME="Niall"
echo $NAME
If you run a script with arguments, e.g.
$ ./myscript.sh arg_one arg_two ...
The arguments will be passed into your script as variables. Within the script,
$0 is the name of the script.
$1 is the first argument.
$2 is the second.
$@ refers to the entire list of arguments.
$# refers to the number of arguments passed.
Write a script called myargs.sh that prints out the first argument passed into it.
We can also as the user to input text by using the read command; in interactive mode, this is:
$ read myvar
$ read –p "prompt message" myvar
will first print out a message to the user.
Control will then be passed over to the user and wait for them to enter a value. In this example, the entered value is stored in a variable called myvar.
my_name.sh
"What is your name? "
"Hello <name>".
If statements begin with if and end with fi.
The syntax is, remembering the importance of spaces:
if [ condition ]; then
# run code
fi
In order to compare two strings $a and $b, we can write a script containing:
if [ "$a" = "$b" ]; then
# This is a comment; the code goes here.
fi
Remember spaces: "$a" = "$b" is different to "$a"="$b"
In order to compare two integers $a and $b, we use "–eq" instead of "=":
if [ "$a" -eq "$b" ]; then
# run code
fi
| Operator | String | Int |
|---|---|---|
| Equals | = and == | -eq |
| Not equal | != | -ne |
| Greater than (dictionary position for string) | < | -gt |
| Less than (dictionary position for string) | < | -lt |
| Greater than or equal | -ge |
|
| Less than or equal | -le |
|
| Is null (empty string) | -z "string" | |
| Is not null (not empty string) | -n "string" |
If/else control works as follows:
if [ condition ]; then
# run code
else
# run different code
fi
As you'd expect.
The read example above, my_name.sh, has a problem. The user does not have to enter a name at all and can return an empty value by pressing Enter. This result in a null string. This will cause problems!
The code [ -z "$var" ] checks if the variable var is empty or not.
Update the my_name.sh, script to make this fix using an if/else block. Make a new copy of my_name.sh and call it my_name_2.sh.
Use the cp command to copy the file:
$ cp my_name.sh my_name-2.sh
Use Vi to edit the new file; update the script so that if the user enters an empty string it prints a warning message. Otherwise it ought to print out Hello <name> as before.
An else / if statement is written as elif. It works like this:
if [ condition ]; then
# run code
elif [ another_condition ]; then
# run this code instead
else
# run different code
fi
Make a copy of the previously created file my_name_2.sh and call it my_name_3.sh
Change the code so that:
["$a" = "$b"] checks if two strings are the same
https://kernel.org/. The Linux kernel archive; the official Linux kernel website. Linux was written by Linus Torvalds.
Debian. The Debian free Linux distribution.
Linux on ChromeOS. Overview from Google.
Raspberry Pi OS. Overview.
MacOS Terminal User Guide. Apple.
Cygwin. Get that Linux feeling - on Windows.
Bash, the GNU Project's shell.
Bash. The official Bash reference manual. Most of your questions will have answers here.
Zsh. The default macOS shell.
Linux/BSD command line wizardry: Learn to think in sed, awk, and grep. Jim Salter at Ars Technica. This is a very nice overview of shell.
Command line wizardry, part two: Variables and loops in Bash. Part 2 of Jim Salter's nice review on Ars Technica.
Linux Essentials. Very complete "cheat sheet" put together by Thomas Girke at the University of California Riverside.
Linux Shell Commands. Another good summary page put together at Cardiff University's School of Computer Science & Informatics.
UNIX Introduction . Good overview from M. Stonebank in the University of Surrey.
MIT OpenCourseWare Notes on Operating System Engineering by Professor Kaashoek.
UNIX Commands. Summary Unix commands put together at North Carolina State University.
Gentoo's bash guide. Nice section about string and integer comparisons.
Google Developer Documentation Style Guide, Document Command-line Syntax. Useful overview of command notation syntax.
© Copyright 2022. Please contact Niall McMahon for more.