# How to write a for-loop in Bash
by Seth Kenlon

A common reason for learning the UNIX shell is to unlock the power of botch processing. If you want to perform some set of actions on many files, one of the ways to do that is by constructing a command that iterates over those files. In programming terminology, this is called *execution control* and one of the most common examples of it is the **for** loop.

A **for** loop is a recipe detailing what actions you want your computer to take *for* each data object (such as a file) you specify.


## The classic for loop

An easy for loop to try is one that lists a group of files. This isn't a useful loop on its own, but it's a safe way to prove to yourself that you have the ability to handle each file in a directory individually. First, create a simple test environment by creating a directory, and placing some copies of some files into it. Any file will do, initially, but later examples require graphic files (such as JPEG, PNG, or similar). You can create the folder and copy files into it using a file manager, or in the terminal:

   $ mkdir example
       $ cp ~/Pictures/vacation/*.{png,jpg} example

Change directory to your new folder, and then list the files in it to confirm that your test environment is what you expect:

   $ cd example
       $ ls -1
       cat.jpg
       design_maori.png
       otago.jpg
       waterfall.png

To loop through each file individually in a loop, the syntax is: create a variable (``f`` for file, for example). Then define the data set that the variable is to cycle through. In this case, cycle through all files in the current directory using the ``*`` wildcard character (the ``*`` wildcard matches *everything*). Then terminate this introductory clause with a semi-colon (``;``).

   $ for f in * ;

Depending on your own preference, you may optionally press **Return** here. The shell won't try to execute the loop until it is syntactically complete.

Next, define what you want to happen with each iteration of the loop. For simplicity, use the ``file`` command to get a little bit of data about each file, represented by the ``f`` variable (but prepended with a ``$`` to tell the shell to swap out the value of the variable for whatever the variable currently contains):

       do file $f ;

Terminate the clause with another semi-colon, and then close the loop:

       done

Press **Return** to start the shell cycling through *everything* in the current directory. The for loop assigns each file, one by one, to the variable ``f``, and then runs the command you proscribed:

   $ for f in * ; do
       > file $f ;
       > done
       cat.jpg: JPEG image data, EXIF standard 2.2
       design_maori.png: PNG image data, 4608 x 2592, 8-bit/color RGB, non-interlaced
       otago.jpg: JPEG image data, EXIF standard 2.2
       waterfall.png: PNG image data, 4608 x 2592, 8-bit/color RGB, non-interlaced

You may also have written it this way:

   $ for f in *; do file $f; done
       cat.jpg: JPEG image data, EXIF standard 2.2
       design_maori.png: PNG image data, 4608 x 2592, 8-bit/color RGB, non-interlaced
       otago.jpg: JPEG image data, EXIF standard 2.2
       waterfall.png: PNG image data, 4608 x 2592, 8-bit/color RGB, non-interlaced

Both the multi-line and single-line formats are the same to your shell and produce the exact same results.


### A practical example

Here's a practical example of how a loop can be useful for everyday computing. Assume you have a collection of vacation photos you want to send to friends. Your photos are very large, making them too large to email and inconvenient to upload to your [photo sharing service](http://nextcloud.com). You want to create smaller web-versions of your photos, but you have a hundred photos and don't want to spend the time reducing each photo, one by one.

First, install the **ImageMagick** command using your package manager on Linux, BSD, or Mac. For instance, on Fedora and RHEL:

   $ sudo dnf install ImageMagick

On Ubuntu or Debian:

   $ sudo apt install ImageMagick

On BSD, use **ports** or [pkgsrc](http://pkgsrc.org). On Mac, use [Homebrew](http://brew.sh) or [Mac Ports](https://www.macports.org).

Once you have ImageMagick installed, you have a set of new commands designed to operate on photos.

Create a destination directory for the files you're about to create:

   $ mkdir tmp

To reduce each photo to 33% of its original size, try this loop:

   $ for f in * ; do convert $f -scale 33% tmp/$f ; done

After the files have been processed, look in the ``tmp`` folder to see your scaled photos.

Any number of commands can be used within a loop, so if you have a complex actions you need to perform on a batch of files, you can place your whole workflow between the ``do`` and ``done`` statements of a **for** loop. For example, suppose you want to copy each processed photo straight to a shared photo directory on your web host, and then remove the copy of the photo from your local system:

   $ for f in * ; do
         convert $f -scale 33% tmp/$f
         scp -i seth_web tmp/$f [email protected]:~/public_html ;
         trash tmp/$f
         done

For each file processed by the **for** loop, your computer is automatically running three commands. That means if you process just 10 photos in this way, you save yourself 30 commands and probably at least as many minutes.


## Limiting your loop

A loop doesn't always have to look at every file, or even at files at all. For example, you might want to process only the JPEG files in your example directory:

   $ for f in *.jpg ; do convert $f -scale 33% tmp/$f ; done
       $ ls -m tmp
       cat.jpg, otago.jpg

Or you may not need to process files, but you need to repeat some action a specific number of times. A **for** loop's variable is defined by whatever data you provide it, so you can create a loop that iterates over numbers instead of files:

   $ for n in {0..4}; do echo $n ; done
       0
       1
       2
       3
       4

## More looping

You now know enough to create your own loops. Until you're comfortable with looping, use them on *copies* of the files you want to process, and as often as possible use commands with safeguards built in to prevent you from clobbering your own data. This prevents you from making irreparable mistakes, like accidentally renaming a whole directory of files to the same name, each overwriting the other.

For advanced **for** loop topics, read on.


## Not all shells are Bash

The **for** key word is built into the Bash shell. Many other similar shells also use the same key word and syntax, but some shells, like Tcsh, use a different key word, like ``foreach``, instead.

In Tcsh, the syntax is similar in spirit, but more strict than Bash. In the code sample, do not type the string  ``foreach?`` in lines 2 and 3. It is a secondary prompt alerting you that you are still in the process of building your loop.

   $ foreach f (*)
   foreach? file $f
   foreach? end
   cat.jpg: JPEG image data, EXIF standard 2.2
       design_maori.png: PNG image data, 4608 x 2592, 8-bit/color RGB, non-interlaced
       otago.jpg: JPEG image data, EXIF standard 2.2
       waterfall.png: PNG image data, 4608 x 2592, 8-bit/color RGB, non-interlaced

In Tcsh, both ``foreach`` and ``end`` must appear alone on separate lines, so you cannot create a **for** loop on one line, as you can do with Bash and similar shells..


## For loops with the find command

In theory, you could find a shell that doesn't provide a **for** loop function, or you may just prefer to use a different command with added features.

An alternate way to implement a **for** loop is with the ``find`` command, which offers several ways to define the scope of which files to include in your loop, as well as options for [parallel](https://opensource.com/article/18/5/gnu-parallel) processing.

The ``find`` command is meant to help you find files on your hard drives. Its syntax is simple: you provide the path of the location you want to search and ``find``, by default, finds all files and directories:

   $ find .
       .
       ./cat.jpg
       ./design_maori.png
       ./otago.jpg
       ./waterfall.png

You can filter the search results by adding some portion of the name, or the type of file you're searching for, and so on:

   $ find . -name "*jpg"
       ./cat.jpg
       ./otago.jpg

The great thing about ``find`` is that each file it finds can be fed into loop using the ``-exec`` flag. For instance, to scale down only the PNG photos in your example directory:

   $ find . -name "*png" -exec convert {} -scale 33% tmp/{} \;
       $ ls -m tmp
       design_maori.png, waterfall.png

In the ``-exec`` clause, the bracket characters ``{}`` stand in for whatever item ``find`` is currently processing (in other words, whatever file ending in ``png`` that has been located, one at a time). The ``-exec`` clause must be terminated with a semi-colon, but Bash usually tries to use the semi-colon for itself, so you "escape" the semi-colon with a back-slash (``\;``) so that ``find`` knows to treat that semi-colon as its terminating character.

The ``find`` command is very good at what it does, and in fact can be too good sometimes. For instance, if you use it again to find PNG files for some other photo process, you get a few errors:

   $ find . -name "*png" -exec convert {} -flip -flop tmp/{} \;
       convert: unable to open image `tmp/./tmp/design_maori.png':
       No such file or directory @ error/blob.c/OpenBlob/2643.
       ...

It seems that ``find`` has located the PNG files not only in your current directory (``.``) but also those that you already processed and placed into your ``tmp`` subdirectory. In some cases, you may want ``find`` to search the current directory plus all other directories within it (and all directories in *those*). It can be a powerful recursive processing tool, especially in complex file structures (like directories of music artists containing directories of albums filled with music files), but you can limit this with the ``-maxdepth`` option.

To find only PNG files in the current directory (excluding subdirectories):

   $ find . -maxdepth 1 -name "*png"

To find and process files in the current directory plus one level of suddirectories, increment the maximum depth by 1:

   $ find . -maxdepth 2 -name "*png"

The default is to descend into all subdirectories.

## Looping for fun and profit

The more you use loops, the more time and effort you save, and the bigger the tasks you're able to tackle. You're just one user, but with a well-thought out loop, you can make your computer do the hard work.

You can and should treat looping like any other command, keeping it close at hand for when you need to repeat a single action or two on several files. However, it's also a legitimate gateway to serious programming, so if you have a complex task to accomplish on any number of files, take a moment out of your day to plan out your workflow. If you can achieve your goal on one file, then wrapping that repeatable process in a **for** loop is relatively simple, and the only "programming" required of you is an understanding of how variables work and enough organization to separate unprocessed from processed files. With a little practice, you can move from a Linux user to a Linux user who knows how to write a loop, so get out there and make your computer work for you!

You are viewing proxied material from sdf.org. The copyright of proxied material belongs to its original authors. Any comments or complaints in relation to proxied material should be directed to the original authors of the content concerned. Please see the disclaimer for more details.