Advance Bash Scripting Guide

What Is Bash Scripting?

Scripting in Linux refers to the process of writing and executing scripts or programs in a scripting language to automate tasks or perform system administration tasks. A script is a plain text file that contains a series of commands and instructions that can be executed by the interpreter for that scripting language.

There are many scripting languages available in Linux, such as Bash, Python, Perl, Ruby, and others. Each language has its syntax, features, and capabilities that make it suitable for different tasks and applications.

Bash or Bourne-Again SHell is the default shell on most Linux distributions and is used for its powerful features and flexibility. Let's understand some of the fundamental concepts of scripting using bash.

Bash scripts in Linux are used to automate tasks. It gives users, the flavor of a programming language inside the Linux terminal. With the help of Bash scripts, we can use concepts such as variables, loops, conditional statements, and functions in the terminal itself. This allows users to execute a series of commands and programs in a specific sequence, define and use variables to store data and perform calculations, and control the flow of a script using control structures such as conditional statements and loops.

These scripting concepts can be implemented in various situations such as in automating tasks, performing system administration, and creating custom command line tools. We will cover various bash concepts and understand how they are used to automate various tasks further in this advanced bash scripting guide.

Loops in Bash

Loops in Bash are control structures, which are used to automate repetitive tasks in the Linux command-line interface. These are exactly like loops in any other programming or scripting language and allow us to execute a block of code repeatedly until a certain condition is met.

There are three types of loops, used in bash scripting, the "for" loop, the "while" loop, and the "until" loop. Let's go through each of these loops, their use cases, and syntax and understand how they work in the Linux command line interface.

For Loop

The "for" loop is used to traverse through a list of items and execute a specific set of commands defined by the user for each item in the list. The syntax of the "for" loop is as follows-

Here, variable is the variable name that will store each item of the list as the loop iterates, and list is the specific list that the loop will iterate. The commands represent the set of commands which will be executed for each item of the list.

While Loop

The "while" loop is used to execute a specified set of commands, as long as a certain condition is true. The syntax of the "while" loop is as follows-

Here, condition represents the condition that has to be true for the loop to work. This condition is checked before each iteration of the loop, and if false the loop is terminated. If the condition is true, a set of specified commands will be executed.

Until Loop

The "until" loop is similar to a while loop, but unlike a while loop it executes a specific set of commands until the specified condition is matched. It checks the condition before every iteration and only if the condition is false, it executes the commands. The loop is terminated when the condition becomes true. Here, is the syntax of the until loop-

Similar to the "while" loop, the condition here, is an expression that is evaluated before every iteration. If the condition is false, the until loop executes the specified commands, and is terminated when the condition becomes true.

How Loops Can Be Used in Bash Scripts to Automate Repetitive Tasks?

We can automate numerous tasks by using loops in Bash scripts. These tasks may range from simple file management to complex network configuration. Let us see a few instances in which we can use loops to automate various system administration tasks-

File Management

Using loops in Bash scripts, managing files and directories becomes much more efficient. For instance, a loop can be used to copy or move multiple files from one directory to another.

By using the above script, we can copy all the files from the "/source/directory" to the "/destination/directory".

System Backup

Repetitive system backups reduce the risk of data loss and ensure that the system can be restored quickly in case of a disaster. Loops can be used to automate the backup process. For instance, we can use a while loop to monitor a directory and perform incremental backups after a specified interval of time.

The above script monitors the "/source/directory" for new files and performs incremental backups to the "/backup/directory" every hour using the rsync command. The sleep command is used to pause the script for the next 3600 seconds or 60 minutes, once it has been executed.

System Monitoring

System administrators need to monitor various system metrics, such as CPU usage, memory usage, and disk space. Loops can be used to monitor these metrics and take action accordingly. For example, we can use a while loop to monitor the memory usage and disk space and append them to separate log files every 60 minutes.

The above script monitors the memory usage using the free -m command and the disk space using the df -h command and appends both outputs to separate log files. Finally, the sleep command is used to pause the script for 60 minutes.

Network Configuration

We can also use loops to `automate network configuration tasks. For instance, we can add multiple users to a remote server using a for loop.

This above script uses a for loop to simply iterate through the list of users, and add them to a remote server using the ssh command.

Now that we have a better understanding of loops, let us learn about conditional statements in Bash and how we can implement them within loops.

How to Use Conditional Statements with Loops?

Conditional statements give our scripts, the essential capability of decision-making. These are used to test a condition and execute a specific set of commands based on it. These are similar to the conditional statements used in any other programming or scripting language, with slight differences in the syntax. Let us go through the different conditional statements used in Bash and understand how we can use them with loops.

if Statement

The if statement as it seems checks a condition and executes a set of commands if the condition is true.

Syntax:

If commands in Bash are very similar to the ones you might have used in other programming or scripting languages. Just in this case, you must provide the commands to be executed after then and need to close the if statement with fi, which is the exact opposite of it.

else statement

The else statement is used with the if statement and is just as simple as it sounds. Using an else condition with if means that, if the given condition in the if statement is not true, the commands provided in the else statement will be executed.Syntax:

Just like the if commands, when using the else commands with the if commands we need to use fi to close the if-else statement.

Elif Statements

The elif statements are referred to as else-if statements in other programming languages. This is like using another if statement after an else statement. The elif condition is only checked if the previous condition is false.

Syntax:

The if and elif statements can also be clubbed with the else statement, which in turn will execute a particular set of commands when any of the provided conditions are not true.

case Statement

In Bash, the case statement is used to test a variable or condition against multiple patterns or cases and it executes a specific set of commands based on the pattern, which is matched.

Syntax

The case statement is used to eliminate the problem of writing multiple if statements when various cases are to be checked. It is closed with esac, i.e. is the exact opposite of the case.

Let us see a few examples of how we can use various conditional statements with loops in our Bash scripts.

Example 1:

For this example, let us see a script that iterates over a set of integers and determines if they are even or odd.

Output

In the above script, we have iterated over the integers0 - 5using for loop. Using conditional statements such as if, elif, and else we could determine if each of those numbers were even or odd.

Let us see another example using a case statement. Example 2: For this example, let a see a script that greets the user in a language based on the input.

Output:

In the above script, we use case statements to greet the user in Spanish, English, or Hindi based on the input. If anything else is selected, the user is prompted with a message to select anything between the given choices.

Once the user selects one of the correct choices, the user will be greeted in the language chosen and the while loop will break.

Optimizing Bash Loops

When dealing with large datasets or time-consuming operations, sometimes using loops can be slow and inefficient. Although we can use a few methods to optimize the performance of loops in bash scripts. Here are some tips for optimizing bash loops-

• Using the right loop construct: Depending on the requirement of the task, using the right loop construct can make a big difference. For instance, when the exact number of iterations is known, a "for" loop may be more appropriate than a "while" or "until" loop. Whereas, if a task has to be done until a specific condition is true, an "until" loop might be more efficient than a "while" loop.
• Avoid calling external commands inside a loop: External commands like grep, awk, and sed can be resource-intensive and tend to slow down the execution of loops. Whenever possible, built-in Bash commands or shell expansions should be preferred, while performing operations on data inside a loop. This helps in speeding up the loop.
• Use arrays to store data: When a large number of data items are to be processed, consider using arrays to store the data. These can be very useful when optimizing loops as it allows users to store and manipulate multiple values at once. For example, instead of iterating over a list of files with a for loop, instead, the list of files can be stored in an array, and operations can be performed on the entire array at once.
• Use parallel processing: If your system has multiple processors or cores, consider using parallel processing to execute multiple tasks at the same time. Bash provides several tools for parallel processing, such as xargs, parallel, and gnu parallel.
• Implement the break and continue statements: The break and continue statements are used to exit or skip iterations of a loop. By using these statements strategically, the number of unnecessary iterations can be reduced, which will improve the performance of the script.
• Minimize the number of loops: Try to minimize the number of loops in your script by combining operations or using built-in commands that can operate on multiple files at once. For example, instead of iterating over a directory with a for loop and running a command on each file, a single command like find or args can be used to perform the same operation on all files in the directory.

Common Errors That Can Occur When Using Loops in Bash Scripts

When working with loops in Bash scripts, several common errors can occur. Some of the most common ones are listed below-

• Infinite loops: One of the most common errors is creating an infinite loop, where the loop condition is never met and the loop continues to execute indefinitely. This can happen if the loop condition is not properly defined or if the loop variables are not being updated correctly. One can avoid this error, by making sure that the loop condition is valid and the variables are being updated properly. This will make sure that the loop will terminate eventually.
• Syntax errors: Another common type of error is syntax errors, such as missing parentheses, quotes, or other special characters. These errors can prevent the script from running or cause unexpected behavior.
• Incorrect variable scoping: When variables are being used in the loop, it's important to ensure that the variables are properly scoped. Common errors include using global variables when local variables are needed or overwriting the values of variables that are used outside the loop. This can be avoided by making sure that, the variables used inloopsare properly scoped and that any values that need to be used outside the loop are properly passed or stored.
• Array indexing errors: When using arrays in loops, it is important to ensure that the array is indexed properly. If not indexed properly, it could lead to errors such as skipping an iteration or executing an extra iteration, referencing the wrong index, or using an uninitialized variable.
• Incorrect loop construct: Implementing an appropriate loop construct is essential for the task to be successfully fulfilled. Choosing an inappropriate loop construct for a specific task can lead to unexpected behaviors or errors. To avoid this, a general understanding of the different loops, their differences, and an understanding of when to use each type of loop is required.

Let dive further into this advanced bash scripting guide and understand how functions in bash work.

What Are Bash Functions?

Functions in Bash provide a way to group a set of commands into a single reusable block, which can be called multiple times from different parts of the script or other scripts. Using functions makes the script more modular and organized, reducing command duplication and making it easier to read and maintain.

In addition to the reusability of commands, functions have several uses in Linux. They can be used to handle errors and exceptions more efficiently by wrapping code blocks in a function and returning an error code if something goes wrong. They can also be used to create custom commands, which can be called from the command line, making it easier to perform complex tasks.

Functions in bash can be declared in the following ways-

Syntax 1

Here, function_name represents the name of the function which is followed by parentheses, which are followed by curly braces which signify the body of the function, within which the commands to be executed are specified. The single-line version of the above function declaration will include a semicolon after each command within the curly braces.

Syntax 2

Here, the function keyword is used before the name of the function to declare the function followed by parentheses and curly braces. The commands to be executed are written within the curly braces similar to the previous syntax. The single-line version of this function declaration will also include a semicolon after each command within the curly braces.

In Bash, when a function is declared with the function keyword, using parentheses is optional.

How to Use Bash Functions?

To use functions in Bash we need to create a script using any editor of choice. In this example, we will be using the nano editor.

Let us create a script named "greetings_in_bravos.sh"-

Creating A Function in Bash

In this example, we are going to create a function that will display two messages when called.

In the above example, we created a function named "greetings_of_bravos" which, when executed will display the greetings, used in the city of Bravos. The function is created using the function keyword followed by curly braces which signify the body of the function. The body of the function contains two echo commands which are used to display the greetings used in Bravos. Lastly, the function is called simply by typing the function name. This function can be called multiple times, making the group of commands used inside it reusable. To save the script press Ctrl+X followed by y and Enter.

By default, Bash scripts only have read and write permissions. Now to execute these, we need to make them executable first. We can do this by using the chmod command followed by +x, before the name of the script.

This will make the script executable, and we can run our script by simply typing the name of the script preceded by ./.

Output:

Functions in Bash can be very useful when used with arguments. This allows it to work with various variables and inputs, allowing us to program a script that can help us make the script interactive and automate certain tasks. We will be learning about function arguments and variables further in this article.

Bash Function Variables

In this section, we are going to understand how variables are used in Bash and how we can use variables within functions to manipulate data in a script. But for that, we first need to understand what a variable is and what are its properties.

What Are Variables in Bash?

In the most basic sense, a variable is a container that is used to store data for later use in a script. The variable is assigned a name, and this name represents any value that the variable holds, making the values stored, accessible.

In Bash, variables can be assigned values of any data type. These are case-sensitive and are provided with both read and write access. Variables in Bash can also be expanded into scripts by using the $ sign before the variable name and running it.

Another important feature of Bash scripting is variable scoping. Variable scoping allows us to scope a variable as local or global. This comes in handy when using functions, and is an essential concept in Bash scripting. We will be understanding variable scoping in functions along with examples further.

Variable Scoping in Functions

The scope can be defined as the position in the script in which a variable exists. In Bash scripting, variables can either have global or local scope. Global Variables Generally, a variable that is declared outside a function is defined as a global variable. But, in the case of Bash scripts, any variable declared is by default a global variable. These can be modified from anywhere in the script.

Local Variables The variables that are defined and used only within the function are called local variables. These variables are created when a function is called and are destroyed when the function ends. Local variables in Bash are created using the local keyword followed by the variable name.

Let us understand both of these scopes with an example.

Example:

to gain a better understanding of how local and global variables work, let us create a script named "variable_scope.sh",

Now, In this "variable_scope.sh" script, we will declare some variables and a function.

In the above script, three global variables are declared and assigned the value "1". A function named "boolean" is created which when called, will change the values of all three variables to "0" and display it. Lastly, after the function execution is finished, the final values of the three global variables are displayed.

To run this script, we need to save the script and make it executable first. We can do this by using the chmod command as shown in the previous section.

Output

In the above output, we can see that the final value of one of the global variables differs from the initial value. The final value of the variable "var3" is changed after function execution since it was not defined using the "local" keyword in the function. The other two variables "var1" and "var2" were defined using the local keyword in the function which made sure that these were temporary instances and would no longer exist once the function execution is finished.

For this reason, using local variables is generally recommended when writing functions. This will prevent naming conflicts and unintended updating of variables.

Bash Function Arguments

Like most other programming or scripting languages, Bash functions can also work with arguments. Arguments in Bash are referenced within the function using positional parameters. These are denoted by the "$" sign followed by the argument number. For example, the first, second, and third parameters would be referenced by $1, " $2", and "$3" respectively.

The arguments can be passed by simply calling the function followed by the arguments separating each with spaces.

Example:

To understand the concept better let us see an example where we shall use function arguments. Let us create a script named "function_arguments.sh".

In this script, we will create a function that takes two values as arguments, performs addition, and then displays the result.

To run the above script, we need to save it and make it executable. When executed, this script will display the following.

Output

In the above output, we can see that the "add" function is called multiple times with different arguments.

In some cases, we might not know the number of arguments that are to be passed. In those cases, the "$@" variable is used. It is a special variable that represents all the arguments passed to a function.

Let us say, we create a function that displays all the arguments. If we do not know the number of arguments there might be, we can use the "$@" variable.

When executed, this function will display all the arguments passed during the call.

Bash Function Return

Similar to other programming and scripting languages, in Bash, we can also return values or outputs to the calling part of the script or function using the "return" command followed by the value to be returned. In Bash scripting, the return value is assigned to the $? variable. Thus to access the return value of a function, we should use the $? variable. Let us see how the $? variable works with an example. In This example we will be working on a script named "example. sh".

Example

Output

In the above example, a function is created that returns the sum of two values passed as arguments. The return value is accessed using the "$?" variable and is displayed on the terminal.