Linux Operating System

Overview

Using computers effectively requires understanding the fundamentals of operating systems. This chapter covers the fundamentals of the Linux operating system and Linux distributions. A Linux distribution contains the Linux kernel, package management system, GNU tools and libraries, window system and manager, and desktop environment. We provide instructions for installing and setting up the popular Ubuntu distribution on the Ubuntu Desktop page.

The operating system is a layer between the hardware and applications that operate on them. The main features of an operating system are process management, interrupts, memory management, file system, device drivers, networking, security, and input/output. We can access these features via an interface called shell. The most common types of shells are command line interfaces (CLI) and graphical user interfaces (GUI).

Command line interface allows users to interact with the computer via text-based commands. Programmers and power users favor command lines because they need to use the computer programmatically and access all its functionality. However, becoming proficient at using the command line requires plenty of practice and understanding of the operating system.

Alternatively, a graphical user interface allows interaction via the mouse by clicking elements. It is more intuitive and easier to use and does not require learning commands but doesn’t allow programmatic use, and it only exposes a part of the computer functionality. Hence specific tasks can be slow or impossible to perform on a graphical user interface.

Command Line Interface

In the early days of computers, the command line was the only way to interact with the computer. These days command line is useful for using a computer programmatically, installing applications, and accessing remote servers. Also, it common for instructions and tutorials to use the command line to describe their actions. We will cover the basic usage of a popular command line shell called Bash.

Command Syntax

Commands are programs that expose a command line interface. When we execute a command, it runs the program. Commands may accept values as arguments and options that modify their behavior. The standard shell command syntax is as follows:

command [options]... [arguments]...

A command may accept options with or without arguments in short-form or long-form, or both. The short-form and long-form aliases one another, and additionally, the long-form may have multiple aliases.

We can write a short option as -o <opt> where we express the option name as one dash followed by a single character, and the argument <opt> is separated by space. Similarly, we can write a long option as --option=<opt> where we express the option name as two dashes followed by a string of characters, and the argument <opt> is separated by an equal sign. An option without argument is a flag written as -f or --flag. A flag corresponds to true if given and false otherwise. For example, we can express the same command in multiple different ways as below.

command -o <opt> -f <argument>
command --option=<opt> --flag <argument>
command -f -o <opt> <argument>
command -flag --option=<opt> <argument>
command <argument> -o <opt> -f
command <argument> --option=<opt> --flag

We can also concatenate multiple short flags such that -x -y -z is equivalent to -xyz.

command -x -y -z <arg>
command -xyz <arg>

Manual

We can read the reference manual of command with the man command.

man command

We can access the documentation of command with the --help option.

command --help

Superuser

We can also run commands as the superuser by using sudo before the command.

sudo command [options]... [arguments]...

We should be careful when executing the commands as a superuser since it has the privileges to modify all parts of the computer including even removing the whole operating system.

File System

Files, Directories and Paths

Understanding the file system is essential for using a computer. The file system allows us to store data in files and group files into collections called directories. Directories can also contain other directories, forming a tree-like directory hierarchy. Each file consists of content and metadata. The content is the actual data held in the file, and the metadata stores additional information such as the file size, file type, access permissions, access date, and modification date. The access permissions for the owner, groups, and other users define whether they can read the file, write to it, or execute it as a program.

Each file and directory has a name given as a string. A string refers to a sequence of characters. Names are case-sensitive in Linux but not in Windows. Hence, we should avoid relying on case-sensitive names to differentiate files and folders. Filenames usually consist of basename and extension separated by a dot, such as image.jpg or text.txt. We can use extensions for indicating file types – extension names are conventions and not strictly defined or enforced by operating systems. A file without a basename is hidden file, and we can use the shortcut Ctrl + h to show and hide them.

We can specify a unique location in the file system using a path. A path points to a file or directory from another directory following the directory hierarchy. In Linux, we can write paths as strings by concatenating names with the slash / as a delimiter. The starting directory is usually the root, home, or working directory. In Linux, everything is located under the root directory denoted by /. Also, each user has their home directory /home/<user>/, aliased with the tilde ~ character and the $HOME variable. Furthermore, a working directory is the directory where the operating system executed the program. An absolute path begins from the root directory, and a relative path starts from the working directory denoted by a single dot ./. We can denote the parent directory relative to the working directory with two dots ../.

For example, /home/<user>/Documents/file.txt points to a text file called file.txt in the user’s documents directory. Alternatively, we could write ~/Documents/file.txt or $HOME/Documents/file.txt. If a program that runs from our home directory points to the file, we can use ./Documents/file.txt.

Symbolic Links

Symbolic links offer a method for manipulating paths in the file system. We may refer to them as symlinks or soft links as compared to regular hard links. A symbolic link is a reference to another file or directory, even to one that does not exist. The reference can be an absolute or relative path. Symbolic links allow reordering the directory hierarchy without moving files. However, symbolic links affect the pathname resolution because they change the directory tree structure into a directed graph. Hence, there can be multiple paths to a file or directory when we use symbolic links.

Text Files

Text files are human-readable files that contain natural and formal language, and we can open, read and modify them using any text editor. A text file is a simple, easy-to-use, general-purpose file format representing many types of files, including source code, configuration files, and data files. For example, you might encounter .txt plain text files, .jl Julia code files and .yml YAML data-serialization files. Formal languages, such as code and data files, have predefined syntax, which machines interpret to read data or execute code.

Binary Files

Binary files are machine-readable files stored in binary format which require specific software for opening and modifying them. Binary files are specialized file formats that can offer more efficient encoding for large files such as media files like images .jpg, .png, video .mp4, and audio .mp3.

Executable Files

An executable file is either a text file containing a program code that runs on an interpreter or a binary file containing machine instructions that execute a program. Software vendors often distribute compiled programs as executable files, often without a file extension in Linux and as .exe in Windows.

Package Management System

We can install applications either using a package manager or installing them manually. A package manager is a software that automates installing, upgrading, configuring, and removing applications. Package managers make it easier to maintain up-to-date versions of applications. Ubuntu has two package managers; the newer Snapcraft and older Advanced Package Tool (APT).

When possible, we should use a package manager instead of manually installing applications. However, some applications are not available via a package manager, or the available application version is old. In these cases, we have to resort to manual installation from a Debian Package or Tarball.

Snapcraft

We will primarily use Snapcraft, the current default package manager for Ubuntu. We usually refer to Snapcraft simply as Snap. Snap manages and maintains packages automatically in the background. It has a security model of least privilege and containerizes its application. We can also publish our applications on Snapcraft. Also, we will use Snapcraft to install applications later in the book.

Snap has a command-line interface called snap. We can install applications using the install argument with superuser privileges.

sudo snap install <pkg>

Snap installs applications to ~/snap/ directory. We can list all installed applications using the list argument.

snap list

Due to Snap’s security model, we sometimes need to manage interfaces manually. For example, if we want to give Snap package rights to write files to a disk other than $HOME, we need to connect the package to removable-media.

sudo snap connect <pkg>:removable-media

The Snap documentation discusses all of its different features and commands.

Sometimes Snap’s security model does not give enough privileges for applications, which may hinder their capabilities. In that case, we may have to use APT or manual installation instead.

Advanced Package Tool

Advanced Package Tool (APT) is an older package manager for Ubuntu. It installs packages to the /usr/bin/ directory. It works with the apt command using the install argument and superuser privileges.

sudo apt install <pkg>

A common problem with APT is that the maintained package versions are often old, which we should prefer Snap or install the application manually.

Debian Packages

We also come across Debian package files in the format <pkg>.deb. Double-clicking them opens the Ubuntu software center, where we can click install to install the package.

Tarball

Application vendors usually distribute binary executables as Gzip compressed tarball file, <pkg>.tar.gz. After obtaining a tarball, we must extract it to the directory where we store manually installed applications. For example, you can create ~/software/ directory to store the applications. We can remove the tarball after extraction. You can perform these step either using the graphical interface or the following commands:

mkdir ~/software/<pkg>
tar -xzvf <pkg>.tar.gz --directory ~/software/<pkg>
rm <pkg>.tar.gz

Next, we must make the binaries findable to other applications and the command-line by extending the $PATH variable with the directory path that contains the package’s binary executables. By opening the ~/.bashrc configuration file and appending it with the export below, the operating system will extend the path automatically on system startup.

export PATH="$HOME/software/<pkg>/<path-to-binaries>:$PATH"

Now, you should be able to use the application. We should also notice that installing tarball does not require superuser privileges.