## Automating Tasks with Shell Scripts ## Joe Read Very few people realize this, but there's only a small number of ISP's in the world that actually bring in a profit. There are a few different types of ISP out in the market right now, there's the new startups trying to live past the two year rule, there's the semi-successful ISP that's doing well enough to be bought out by Verio, and then there's the local ISP that was the first in a given area to offer internet access and is therefore the one making money. Unfortunately, we don't all have the luxury of having been the first to come up with the idea to offer internet access in a certain area, so we have to win our credibility based on quality of service and timely customer response. I come from the first type of ISP mentioned above, a small ISP which actually made it past two years but eventually threw in the towel after the third. Most of my time was spent with low-budget equipment and high management demands that put me in a lot of stress for time. Any sysadmin out there that's been under similar circumstances knows more about business then they care to, and yet understands how important it is to manage your time carefully to allow for emergency situations and new management requests with sudden deadlines. Hopefully, most of these sysadmins have learned the same lesson I have: automation of your servers is important. Here are a few tricks I'd like to share with you. Automation is rather simple for anyone who's done any sort of shell scripting. In fact, every automation script I write is in bash and covers all the bases. For the sake of example, lets cover the steps involved with writing an automation script to add a virtual host in apache. Step 1: Figure out what information you need and name your variables. For the sake of time, I'll tell you what you need. When it comes to adding virtual hosts, you need the virtual host's: username, password, real name, e-mail address, domain name, and the ip address (that the vhost's www.domain.com points to). In your script, this information should be read from the command-line so that you could pass this information to your script from another script you end up writing in the future that, lets say, got the information off a web page or database. With this variable information in mind, here's how the beginning of your bash automation script should look: ------yourscript.sh------ #!/bin/bash username=$1 # $1 means the first command-line argument password=$2 realname=$3 emailaddr=$4 domain=$5 ip=$6 ------------------------- Step 2: Add the user account and create the home directory To automate this process, you need some sort of command line tool that will add a user account such as pw, (see "An Intro to User Account Management with PW" by Nathan Underwood, The FreeBSD 'zine, Issue #2). The problem with pw is that it doesn't allow you to define a password on the command line. Because of this, I've made available a replacement for the adduser program that ships with FreeBSD. The filename is adduser.c, and you can get it from /dl/scripts/adduser.c. Once you have this program, you then want to customize the skeleton home directory it uses (typically /usr/share/skel) which is a directory that is copied to create the home directory of any new user on the system. Typically, you want to set the permissions on this directory so that "other" and "group" don't have read permissions to it, and you may want to set the .rhosts file read-only to the user to avoid security issues in the future. Others may wish to create a public_html directory here too. I typically do not, instead I have my script create a "virtual_html-www.domain.com" directory after the skeleton directory is copied over. This is entirely up to you. After you have everything ready for the user to be added, you need to read up on the command-line parameters for your adduser program, then add the appropriate lines to your code making use of the variables we defined in Step 1: ------yourscript.sh------ /usr/sbin/pw useradd $username -c "$realname" -m ------------------------- As you can see, we've just created a user with the username and realname defined from the command line parameters sent to our script. Step 3: Bind the IP address to the network card This step is something I used to do manually even though I had a script to do everything else for me. I eventually wised up realizing that it was just one extra step I was doing that I could automate. While many people are nervous about having their network settings modified automatically, I don't mind as long as I know what's being modified and the code that gets executed I've run myself many times without incident. This process is actually two-fold: first we call ifconfig and bind the IP to the NIC on-the-fly, then second we add an entry to /etc/rc.conf to bring this IP address up after any reboot. Doing both of these requires your script to know which network device to use. Some may just hard-code the device name into the routine, or purists like myself may add a variable at the top of the script with the device name that can be changed at some point in the future should you get a different brand of NIC in your system. Because we have to add an ifconfig_device_alias to rc.conf, we're going to have to come up with a unique number for this alias due to the nature of the rc.conf parsing script. Each alias entry in rc.conf is numbered like "ifconfig_device_alias0" and "ifconfig_device_alias1", etc., based on order that it gets added. Because of this, we're going to first have to find a number that's not in use and make that the number for the alias we're about to add. Sorry if this seems confusing, I've come up with a loop that will do this for you, so please be my guest and just cut-and-paste the code. ------yourscript.sh------ device="vx0" /sbin/ifconfig vx0 $device $ip netmask 255.255.255.0 loop=0 count="no" while [ "$count" = "no" ]; do testcmd=`grep ifconfig_${device}_alias${loop} /etc/rc.conf` if [ -z "$testcmd" ]; then count=$loop else loop=`expr $loop + 1` fi done echo ifconfig_$device_alias${count}="inet $ip netmask 255.255.255.0" >> /etc/rc.conf ------------------------- Confused yet? First we defined the variable $device to "vx0", which is the name of our ethernet device. Then we ran ifconfig on that device to add the alias that our variable $ip is set to (which is actually defined by the user on a command-line parameter). Second, we ran a loop that basically checks rc.conf for a number, if it finds it it increases the number until it doesn't find it, then uses the first number it doesn't find as the alias number. Then, we output the value of these variables in a format that the startup-script parsing engine will recognize to add that alias every time it boots up. Pat yourself on the back if you get it. Step 4: Add your virtual host to httpd.conf This is probably the easiest of all steps, since all we're doing is adding text to a configuration file, and telling the program who's configuration file we're modifying to restart. Again, purists like myself may wish to put a variable at the top of their program pointing to the path of the httpd.conf file, since the location of this file has been known to change with later versions of apache. This is entirely up to the coder. As I mentioned earlier, you need to decide where to put the HTML files for this domain. Some people prefer ~user/public_html/, I prefer ~user/virtual_html-www.domain.com. We'll make this a variable of it's own and you can modify it to suit your needs. You also need to decide where to put the log files for this domain, we'll also make this a variable for your customization. ------yourscript.sh------ config="/usr/local/apache/etc/httpd.conf" # These ONLY work once the user has been added, otherwise it won't be # able to parse the ~username! documentroot="~$username/virtual_html-www.$domain" logdir="~$username/logs" mkdir $documentroot echo "" >> $config echo "ServerAdmin $email" >> $config echo "DocumentRoot $documentroot" >> $config echo "ServerName www.$domain" >> $config echo "ErrorLog $logdir/error_log" >> $config echo "TransferLog $logdir/access_log" >> $config echo "" >> $config /usr/bin/killall -HUP httpd ------------------------- Step 5: Miscellaneous At this point, you should have a working script to add a user, create their virtual host, and add that virtual hosts IP address all in under 10 seconds. Since you still have values for each of the variables, you might as well make use of them, for example, to send out a confirmation e-mail to the customer letting them know that "creation of the virtual host www.$domain has been completed", and they need to "use the username $username and password $password" to upload their web site. Or, you could use this data to pass onto another script that will make their frontpage extensions, or you could modify step 4 to create and "ScriptAlias" a cgi-bin directory for them... The possibilities are endless. Step 6: Run the Script Now that you have your script completed, lets take a look at what actually happens when you run it. Remember that we need to pass to it command-line parameters to assign values to our variables. Alot of times, people will try "program --help" or just "program" to get a list of these variables, so what I tend to do is output a usage message and halt the program if not all variables are defined on the command line. Here's an example of that (this would go right under your variable assignments for the command-line arguments): ------yourscript.sh------ if [ -z "$6" ]; then echo Usage: $0 username password \"real name\" email@address.com domain i.p.add.ress echo echo Notice: make sure that the users real name is in quotes, and the domain echo is NOT preceded by www. exit 0; fi ------------------------- (In case you're wondering, $0 is replaced by the filename of your program) What we've done here is display a message anytime someone enters less then 6 command line arguments. So, if someone types just yourscript.sh, or yourscript.sh --help, they'll get the same message. Now that the users know what to do with it, they type in the proper syntax and end up with a completed virtual host! Epilogue: I hope I've instilled in you a desire to increase your efficiency through automation. There are many different types of scripts you can make using the same principal ideas in this script. I tend to write scripts for every function done by an administrator, such as adding nameserver entries, adding frontpage extensions, making personal web space, checking our radius logs for the last login of a certain user.. You may invest more time then management sees as necessary in writing programs that "do the same thing you already do", but in reality you're writing programs to "do the work for you" to give you a chance to focus on other things. Even if you didn't understand the code examples in this article, I urge you to start writing your own code to automate tasks, because in reality, all you're doing is taking the keystrokes that are the same for every different customer and putting them in a file, then throwing in variables in the appropriate places that you can define the meaning of in one central location: the command line. In doing this, you'll pick up on the details of bash shell (or whatever shell you decide to use) quickly enough that your code might soon rival that of perl scripts written by people that get paid way to much for them. --Joe Next Issue: I would like to invite anyone interested in attempting their own automation scripts to e-mail them to me for review. All *unique* and *clean-code* automation scripts will be accepted for posting on the FreeBSD 'zine automation archive (coming soon), and each month a feature will be done on the most imaginative script in our up-coming automation feature article. A more advanced and complete version of the script I originally wrote and based this article on will be present on this archive next issue, along with other scripts that might help ease the daily administrators job. Please send submissions to . $Id: isp.txt,v 1.1 2000/02/16 08:07:44 jim Exp $