Real-Time Script Control and GUI Interfaces in Bash

While working on a script to batch-process items in parallel, I found the need to provide an interface that would let me control how the script was running, as well as provide a simple graphical overview of what was going on in the background.

You can do both at once using a combination of while loops and the dialog utility (or an equivalent called whiptail, depending on your distro). Dialog is a simple utility that allows wizard-like interactions over the command line. If run in a loop, dialog can be "refreshed" to display info as a script runs. This wiki page is a good starting point for learning the dialog command.

Painting a GUI

Every time you run dialog, you are essentially re-drawing the screen from scratch using the arguments and widgets you define in the command. The easiest way to get a refresh-able GUI is to put your dialog command into a function, and then call it in a while loop.

Below is a basic dialog GUI that displays data from a list of items at /path/to/processed_list:

scriptstatus="RUNNING"

function paint_ui {

  dialog --backtitle "item processor 1.0 - p to pause, r to resume, q to quit" \
  --begin 3 3 --title "Processed: `wc -l < /path/to/processed_list`" --infobox "`cat /path/to/processed_list`" 0 0 \
  --and-widget --title "Status" --begin 3 50 --infobox "${scriptstatus}" 0 0

}

• The back slashes are to break up the command and make it more readable
• --backtitle lets you set a string of text to use in the background
• --begin 3 3 sets the y and x coordinates of each widget, respectively
• --infobox is a good widget type if you just want to show data without any user input
• --and-widget lets you add additional widgets to the dialog window, such as progress bars, other info boxes, etc. 
• The 0 0 is the height and width of a widget (0 means auto-size). If setting a custom size, the default behavior is to wrap text until it runs out of vertical space, then cut off anything extra. You can't be 0 for one dimension and custom for another- it's either all auto ( 0 0 ), or custom values for both.
• You can use variables, expansion and simple subshells in the title and text field arguments.
• Rather than cat-ting all content into an infobox, you can also use tools like head, tail, grep or tac to control how much data to display, and in what order.

The result will look something like this:

Real-time Script Control

This is one area where you can kill two birds with one stone- your input method can also act as a way to refresh the GUI.

See below, again assuming a list of items at /path/to/processed_list:

scriptstatus="RUNNING"
pausevar="n"

function paint_ui {

  dialog --backtitle "item processor 1.0 - p to pause, r to resume, q to quit" \
  --begin 3 3 --title "Processed: `wc -l < /path/to/processed_list`" --infobox "`cat /path/to/processed_list`" 0 0 \
  --and-widget --title "Status" --begin 3 50 --infobox "${scriptstatus}" 0 0
}

function example_function {
  # doing important stuff here
  return 0
}


while true; do

  #redraw GUI

  paint_ui

  # If not paused, run example_function

  [ "$pausevar" == "n" ] && example_function
  
  # Read for input at a 5 second timeout (results in 5 second 'refresh' rate)

  while read -s -n 1 -t 5 command; do
    case $command in
    p|P)
     scriptstatus="PAUSED"
     pausevar="y"
     break
     ;;
    r|R)
     scriptstatus="RESUMED"
     pausevar="n"
     break
     ;;
    q|Q)
     # Breaks out of both while loops, effectively stopping the GUI.
     break 2
     ;;
    esac
  done

done #outer loop

exit 0

Some notes on the above:

• The idea is to run two while loops: an outer loop that runs forever and redraws the GUI, and an inner while read loop to check for input at a 5-second timeout interval.
• The read statement's timeout value of -t 5 effectively acts as our "refresh rate" for the GUI. As soon as the read statement times out (or when we break out of it), the inner while loop ends, and the outer while loop will start from the top: it runs a GUI refresh, then kicks off the inner while read again for 5 seconds. 
• When the user selects q for quit, we break out of both while loops at once with a break 2 , and the script proceeds to the exit 0. For the other input options, we issue a regular break to get out of the inner while read loop early, so that the GUI will refresh as soon as you enter your input.
• It is a good idea to add a "status" infobox and text string to your GUI, so that you can give visual feedback on any input you send.
• Another cool use-case: you can have a command or function run at each refresh of the while loop, but change how it runs based on user input (in this case we stop running the example_function every refresh if the user hits pause).

Cable Management for Home Wiring

After moving into our new house, I found myself in serious need of a wired LAN to handle the gadgets in our living room. I can no longer get away with running cable under the rugs and furniture, and my attempt to go all-wireless resulted in tears of shame and buffering. Time to hit the crawlspace and lay down some wire!

I only need ethernet between two rooms for now- but I wanted to run them in a way I could expand on later. Bonus points if it is easy to install in a cramped crawlspace.

The solution:

Vinyl rain gutters!

For cable management, they have a lot of advantages:

• Readily available at home improvement stores, in 10-foot lengths
• Easy to work with- just screw in the brackets, and everything else snaps together
• You can snap in more sections as you extend into other rooms
• No need to fish cables through conduit, or risk pinching them with a staple gun

Note that this is for low-voltage cables only- I'm sure the NEC has lots to say about the proper way to run electrical wiring. 

Parallel Processing in Bash - Some Fun Tricks

So you need to perform a task or run a command across a medium-to-large number of hosts, in some kind of controlled fashion. The typical options are:

• Grinding through a list of entries one-by-one with a script
• Using tools like GNU parallel or xargs, to process multiple entries at a time
• Using a vendor-provided API, or else an orchestration tool of some kind

The first is simple, but slow. The second is more efficient, but leaves you open to race conditions and clobbering- especially if your parallel jobs need to write into the same files. The third is ideal, but it presumes that you have those orchestration tools in place (and the knowledge to use those tools).

In our case, we needed something better than option 1, more controllable than option 2... and there was no option 3 at the time :-P

Background Processes

The quick-and-dirty way to parallelize a process in bash, is to run it in the background- done by putting an ampersand after the command. You can also do this with functions inside your bash script. Below is a simple reboot script:

function reboot_host {
  server=$1
  ssh $server "reboot"
  sleep 60
} 

# Feed our server list into a while loop, and ask to reboot for each server.

while read -r hostname; do
read -n 1 -p "Reboot host ${hostname}? [y/n]: " choice

  case $choice in
    y|Y)
    reboot_host ${hostname} &
    ;;
    n|N)
    echo "Stopped on ${hostname}"
    exit 1
    ;;
    *)
    exit 1
    ;;
  esac

done < /path/to/server_list

The above would let you kick off background instances of the reboot_host function as fast as you could mash the 'y' key, until you have read through your server list.

File Locking

In some cases, you may want to maintain a log file, or a list of successful/failed/remaining items. So now you have to worry about multiple background processes clobbering with each other, when they try to write into the same set of files.

One Bash-friendly way to avoid this clobbering, is to use flock: a utility that provides a kernel-supported method for file locking. The idea is to assign a file handle (or descriptor) to a file, and then have your running script or function use flock to claim a lock on that file descriptor. If other jobs also use flock to claim access to a file, they will wait for any existing flocks to be released, before continuing their activities. It looks like this in action:

# Open or create a file descriptor into file /path/to/server_list (can be any number but 0, 1, or 2)
exec 200>/path/to/server_list || echo "ERROR: descriptor not opened!"

# Acquire an exclusive lock on descriptor 200 (-e 200), or wait until descriptor is available for up to 30 seconds (-w 30)
flock -w 30 -e 200 

# any commands to run when you get the lock...
  echo "$servername" >> /path/to/server_list
  command1
  command2
  command3

#release lock on file descriptor 200 when done
flock -u 200  

In the above example, you have to explicitly call flock twice: once to get the lock, and a second time to release it. A more elegant method is to use a subshell () that runs when you open the file descriptor. In this case you don't have to worry about releasing the flock manually- it will release automatically at the end of the subshell, when the file descriptor closes. The usual Bash restrictions apply when working with subshells.

function process_item {

  # take item as argument
  item=$1

  (
    flock -w 30 -e 200 || echo "ERROR: flock attempt failed!"
    echo -e "$item" >> /path/to/processed_list
    sed -i "/^${item}$/d" /path/to/remaining_list
    sleep 10

  ) 200>/path/to/lockfile

}

process_item item01 &
process_item item02 &

Above, item02 should take 10 seconds to appear on the lists, since item01 is keeping the flock open on the lockfile while sleeping.

Some important things to note:

• You can use the optional -w flag to set a timeout on any flock attempts, in case something goes wrong. Then you don't worry about a rogue flock attempt hanging forever.
• To control write access, you don't have to put a flock on the same descriptor/file that you are writing into. In the second example, function process_item uses a separate lockfile to keep track of when it is allowed to write into several other files. 
• Running flock with the -n flag makes for a non-blocking flock, meaning it will fail with a return code instead of waiting for the descriptor to be available. Useful if you know you only want once instance of a process opened, no matter what. The default is a blocking flock, which will wait for access.
• Flocks work against file descriptors, and not on the files directly- it only has significance to other jobs using flock on that descriptor. It doesn't magically protect the file from modification by other processes.