How to Automate System Administration: The Basics

Last updated: 8 Feb 2007

Overview
Overview of Windows and Linux scripting and scheduling tools; hands-on experience with automating common tasks; testing scripts


Workshop Duration

2 hrs.


Student Prerequisites

how to administer a computer


Student Provides

  1. nothing


Lab Provides

  1. computer with Windows XP and Linux installed

  2. itadmin and ituser accounts on both operating systems


Preparation

  1. Windows XP

    1. Make sure Perl and module AdminMisc are installed

    2. Make sure AutoIT is installed

  2. Linux

    1. Make sure Perl is available


Delivery

  1. Command line vs. GUI

    Type Advantages Disadvantages
    Command line
    • fast to type
    • composable (one command's output is another's input)
    • scriptable (can easily group together in file, check conditions, do loops)
    • can parameterize effect (use in different situations)
    • output and input can be redirected or thrown away
    • easily adaptable to disabilities related to sight
    • requires very little bandwidth over network
    • requires minimal RAM
    • must remember commands
    • must look up various options
    • very prone to errors in typing
    • inconsistent options among similar commands
    GUI
    • visual, graphical interface
    • consistent menu system (mostly) to remind and reduce cognitive load
    • support for some disabilities
    • more easily adaptable to real-world interfaces (e.g., play button)
    • multimedia, not just text
    • drag and drop to perform file manipulation
    • click to start
    • not easily scriptable
    • not composable
    • can be slower to navigate (must find right window)
    • applications are often not parameterizable
    • only control over input and output is what application provides
    • graphics are difficult to convey to the blind
    • not easy to document how to do something (must show screenshots), due to graphics
    • requires a lot of bandwidth over the network
    • requires a lot of RAM

  2. Other information... from the first workshop
      Automation:
      A. Series of manual tasks -> one task, possibly parameterized
    
      B. Task done periodically or at a particular time by person ->
      periodically executed by computer
    
      Scripting: writing utillity programs; focus on speed of coding, ease of
      use by peers, re-use of OS and other cmds and their output; simplicity in
      code and data storage/retrieval
    
      Scheduling: do something at a specified time, perhaps repeatedly; often
      use a script the thing to do
    
      When to automate:
      - tedious task
      - error-prone tasks
      - promote consistency
      - increase reliability
      - conditionally do task
      - repeatedly do task
      - many steps in task
      - many tasks (combine tasks)
    
      How to automate?
      1. Start simple
    
      2. Know how to do it manually
      A. Do it manually
      B. Record the subtasks
    
      3. Review subtasks
      A. Is there an existing cmd/cmds that can perform function of subtask?
      B. What is input of subtask?
      C. What is output of subtask?
      D. What changes in system due to subtask?
    
      4. Code and test incrementally
    
      Tricks of the trade
      - return codes: check them; provide them in your scripts
      - provide help on usage
      - args: options vs. required; validate
      - use cmds vs. Code
      -- unless performance problem
      -- cmds require no maintenance, are more understandable, are usually
      documented, are well-tested
      - use stdin,stdout,stderr
      -- allows chaining/piping/composing commands
      -- cleaner output
      

  3. Common admin tasks

    1. User Management

    2. Service & Process Management

    3. Networking & Security

    4. Backup & Restore

  4. Windows XP

    1. Scripting

      1. Shell Scripts and Batch Files

      2. Access to OS API

      3. Specialized Tools

    2. Scheduled Tasks

      1. Command line

      2. GUI

    3. Tracing Execution

      1. Language features

      2. Logging

  5. Linux

    1. Shell Scripts

      1. Bash

    2. Access to OS API

      1. Perl

    3. Cron jobs

    4. Tracing Execution

      1. Language features

      2. Logging

Hands-on Time

  1. Windows XP

    1. Login as itadmin

    2. make a temporary directory and go to it
            mkdir c:\temp
            cd /d c:\temp
            
    3. Some commands useful in batch files

      1. output something
                echo hi
                
      2. save a value in an environment variable
                set greeting=hi
                echo %greeting%
                
      3. do something conditionally (on file/folder non-existence)
                if not exist c:\temp\uwnetid mkdir c:\temp\uwnetid
                
      4. direct output to file
                echo 1 hi >C:\temp\uwnetid\greeting
                type C:\temp\uwnetid\greeting
                
      5. append output to file
                echo 2 bye >>C:\temp\uwnetid\greeting
                type C:\temp\uwnetid\greeting
                
      6. get input from a file
                more <C:\temp\uwnetid\greeting
                
      7. use output from one command as input to another ("piping")
                type C:\temp\uwnetid\greeting | findstr bye
                
      8. do something repeatedly
                for /l %n in (1,1,5) do @echo %n >c:\temp\uwnetid\test.%n
                
    4. Simple batch file: do for every file that matches input argument

      1. edit the batch file (a.k.a. script)

        It will dump the contents of all of the files provided in the argument list.

                  notepad doforfiles.cmd
                  
      2. Add these lines
                  @echo off
                  set filename=C:\temp\uwnetid\files
                  dir /b %1 >%filename%
        
                  for /f %%l in (%filename%) do @type %%l
                  
        @echo off prevents tracing, making the output a lot more readable.
        dir /b means to just list the file names, one per line -- no other information.
        %1 is the contents of the first argument passed to the script.
        The doubling of percent signs is REQUIRED for the for variable when inside a batch file.

      3. save the file

      4. Run the script
                  doforfiles C:\temp\uwnetid\test.*
                  
      5. Look at the file of files:
                  type C:\temp\uwnetid\files
                  
        We don't need that file, so it should be deleted afterwards.

      6. improve the script by showing which file name, and deleting the temporary file
                  notepad doforfiles.cmd
                  
      7. Replace the for line with this:
                  for /f %%l in (%filename%) do @(echo %%l & type %%l)
                  
        Here the body of the for loop does two commands, within parentheses to group them and separated by the ampersand sign "&", with the tracing of the command execution suppressed (the at sign "@").

      8. Add this line to the end of the file
                  del /f /q %filename%
                  
        Delete the file without asking for permission "/f" and without outputting any message "/q".

      9. save the file

      10. Run the modified script
                  doforfiles C:\temp\uwnetid\test.*
                  
      11. Beware: Calling another script inside a script

        Use, for example:

                  call nextscript.cmd
                  
        If you don't use call, but instead simply type the script name, it will transfer control to the other script, never returning to the calling script.

      12. There are other "gotchas", not listed here.

    5. Sample system administration task using batch files

      Add 20 new users and make a new home directory for each

      1. Save a list of the user names and place them in a file called C:\temp\uwnetid\new_users.txt, one user per line (here are commands and contents to copy and paste):
                  notepad C:\temp\uwnetid\new_users.txt
        
                  testuser1
                  testuser2
                  testuser3
                  testuser5
                  testuser6
                  testuser7
                  testuser8
                  testuser9
                  testuser10
                  testuser11
                  testuser12
                  testuser13
                  testuser14
                  testuser15
                  testuser16
                  testuser17
                  testuser18
                  testuser19
                  testuser20
                  
        Note that since the user names are so consistent, there is an easier method. But in the real world, a file of user names is very handy, as the names will vary considerably.

      2. create a new script
                  notepad create_users.cmd
                  
      3. Add these lines
                  for /f %%l in (%1) do @net user %%l std3926pw /add
        
                  REM Create their home directories, making sure other users are denied
                  REM any permissions and that this user is granted change permission.
        
                  mkdir C:\home
                  for /f %%l in (%1) do @(mkdir C:\home\%%l & cacls c:\home\%%l /e /r Users /g %%l:C)
                  
        They will all receive the same password... not generally a good idea. I thought you might be able to get a random number with this instead of the std3926pw password above:
                  std%RANDOM%pw
                  
        but it did not work -- gave the same random number -- so it must be an overly simplistic random number generator. To fix this using Windows batch files is beyond the scope of this workshop.

      4. save the file

      5. Run the script
                  create_users C:\temp\uwnetid\new_users.txt
                  

    6. Perl scripts

      1. Here is a one-line perl script to change the contents of all of these files as follows: if the file contains a 1 or a 3, substitute the string "BBB":
                  rem get to the directory
                  cd /d C:\temp\uwnetid
        
                  rem verify contents before
                  for /f "usebackq" %l in (`dir /b test.?`) do @type %l
        
                  rem do the command
                  C:\perl\bin\perl -i.bak -p -e "s/1|3/BBB/;" test.1 test.2 test.3 test.4 test.5
        
                  rem to verify:
                  for /f "usebackq" %l in (`dir /b test.?`) do @type %l
                  
        That will take some explaining, both for the verification and the Perl steps:

        • for command

          The /f option allows some additional options expressed in quotes. The usebackq option means to allow back-quote or back-tick characters around another command to indicate to execute it, and return the results at that spot. Here, the dir /b test.? command is being executed, and it is supposed to list all files that have the filename of "test." followed by exactly one character. That should list test.1, test.2, etc. -- but we don't want them listed to the console. Instead, we want to use the output as input to the for command, much like a pipe. Each iteration of the loop reads another line of output from the dir command, and with that line (which is the file name), we ask to type out the contents of the files.

        • perl command

          This is called an in-place edit of multiple files. What we do is:

          • tell perl to make a backup copy of each file with -i.bak, which appends ".bak" to the end of each file name Perl reads. This keeps the original file around in case the editing does not work as expected.

          • -p means to loop over all of the input, as if the Perl statements (in quotes) were within a while loop, and print the line at the end of the loop as well.

          • -e tells Perl to run the Perl statements as the program, as modified by the options.

          • The Perl statement
            s/1|3/BBB/;
                          
            says to search the current line for a 1 or a 3, and if found, replace it with the string BBB.

          • The list of files make up the list of files to edit.

        So what you can see is that even a simple line of Perl code can do a lot of work.

      2. Here is a Perl script that would do the same thing (from the perlrun manual page):
                  $extension = ".bak";
                  LINE: while (<>) {
                    if ($ARGV ne $oldargv) {
                      if ($extension !~ /\*/) {
                        $backup = $ARGV . $extension;
                      }
                      else {
                        ($backup = $extension) =~ s/\*/$ARGV/g;
                      }
                      rename($ARGV, $backup);
                      open(ARGVOUT, ">$ARGV");
                      select(ARGVOUT);
                      $oldargv = $ARGV;
                    }
                    s/1|3/BBB/;
                  } continue {
                    print;  # this prints to original filename
                  }
                  select(STDOUT);
                  
        I won't try to explain all of that. It is meant to illustrate that in this case, Perl can be written concisely.

      3. Sample system administration task using API:

        Detect removable volumes (based on code by former UWT student Peter Paquette):

        • notepad find_devices.pl

        • Add these lines
                      use Win32::AdminMisc;
          
                      my @drives = Win32::AdminMisc::GetDrives();
                      my @userMedia;
          
                      #------------------------------------------------------------------------------
                      # For each drive on the system test to see if REMOVABLE or CDROM drives have
                      #   media present. This can be determined by checking to see if they have a
                      #   volume number. However, this check will fail if the media has not been
                      #   formatted (e.g. a blank CD-R).
                      #------------------------------------------------------------------------------
                      $sent_header = 0;
          
                      foreach my $drive (@drives)
                      {
                        my $type = Win32::AdminMisc::GetDriveType($drive);
                        (my $volume) = Win32::AdminMisc::GetVolumeInfo($drive);
          
                        if(($type == DRIVE_REMOVABLE || $type == DRIVE_CDROM) && $volume eq "Volume")
                        {
                          if (! $sent_header)
                          {
                            print "\nYou have these devices attached to the computer:\n";
                            $sent_header = 1;
                          }
          
                          #---------------------------------------------------------------------------
                          # For each drive: get its type and if it is REMOVEABLE or a CDROM print it
                          #   get its type
                          #---------------------------------------------------------------------------
          
                          my $type = Win32::AdminMisc::GetDriveType($drive);
                          print "$drive ";
          
                          $vol_type = "unknown";
          
                          if($type == DRIVE_REMOVABLE)
                          {
                            if($drive eq "A:\\")
                            {
                              $vol_type = "floppy disk";
                            }
                            else
                            {
                              $vol_type = "removeable media";
                            }
                          }
                          if($type == DRIVE_CDROM)
                          {
                            $vol_type = "optical (CD\\DVD) media";
                          }
          
                          print "$vol_type\n";
                        }
                      }
          
                      
          This uses the AdminMisc Perl module from Roth.net, which provides access to various Windows APIs, allowing Perl scripts to call Windows functions.

        • Save the file

        • Run the script (you will need to first put a CD, DVD, or floppy in a drive, or plug in a USB drive):
                      C:\perl\bin\perl find_devices.pl
                      
      4. Specialized Tools

        1. Windows Resource Kits

          Many useful command-line tools are only available from these kits, which in the past were parts of Microsoft books you could buy, but more recently are also downloadable from Microsoft.

        2. Systinternals Toolkits (now part of Microsoft)

          These are utility tools written by some Windows programming experts and consultants -- very good quality and very powerful. They are not all commands, and some are not scriptable, so they aren't all aids to automating tasks.

        3. AutoIT

          One of the better scripting tools out there is AutoIT, because it allows scripting of window and keyboard events. Here is a brief example, where we will change the workgroup name:

          • notepad C:\temp\uwnetid\change_workgroup.au3

          • Add these lines
                        ; Open the System Properties window
            
                        Send( "#{PAUSE}" )
                        Sleep(500) ; Give it a little time to pop up completely
            
                        $rc = WinWaitActive( "System Properties", "", 60 )
                        if $rc == 0 then error( "Timeout Error: waiting for 'System Properties'" )
            
                        WinActivate("System Properties") ; Make sure it is the active window
            
                        Sleep(200) ; Give it a little time to pop up completely
            
                        Send( "{RIGHT}" )
            
                        Sleep(200) ; Give it a little time to pop up completely
            
            
                        ; Click the 'Properties...' or 'Change...' button
                        Send( "!c" )
            
                        $MenuTitle = "Computer Name Changes";
            
                        $rc = WinWaitActive( $MenuTitle, "", 5 )
                        if $rc == 0 then error( "Timeout Error: opening 'Network Identification' --> '" & $MenuTitle & "'")
            
                        WinActivate($MenuTitle) ; Make sure it is the active window
            
                        ; Go to the workgroup field
            
                        Send ( "!W{TAB}" );
                        Sleep(200) ; Give it a little time to move
            
                        ; Provide a new workgroup name
            
                        Send ("MYGROUP")
                          
            Here you see a lot of keyboard events and waiting for windows to appear:
            • The pound sign "#" is shorthand for "press and hold the Windows key", and combined with the Pause key, that is a default "hotkey" to open the system properties window.

              If you don't know about hotkeys, it can be difficult to open some of the windows.

            • Then we wait 60 seconds for the window to pop up; if it doesn't, we issue an error message.

            • If it does, we activate the window, since keystrokes will work only if sent to the right window.

            • The Right arrow button ("{RIGHT}") is pressed to move the window tab to "Computer Name", and

            • then the shorthand for the Alt key, "!" is used with the "c" key to get to and click on the Change button.

            • We move to the workgroup field the same way, tabbing to the text box that follows, then entering the value that we want -- "MYGROUP".

            • Subsequent steps would send the Enter key ("{ENTER}"), click on OK buttons to close screens and ultimately restart the computer (changing the workgroup name requires a restart).

          • Save the file

          • To run it as a command, click on the icon "Compile Script to .exe", browse to or enter as Source:
                          C:\temp\uwnetid\change_workgroup.au3
                          
          • Enter this as the destination:
                          C:\temp\uwnetid\change_workgroup.exe
                          
          • click Convert

          • Run the command:
                          c:\temp\uwnetid\change_workgroup
                          

    7. Scheduled tasks

      Use the GUI, or the command schtasks. We use the latter since it is easier to explain.

      Whenever you want something to be done periodically, from every minute to every day, week, month, at startup or shutdown, or other time, you need to schedule a task to do it. Common tasks are backups (but the ntbackup wizard provides its own scheduling), time synchronization, cleanups of temporary directories, and restarts.

      1. Look at the scheduled tasks
                  schtasks
                  
      2. Look at what you can do with the schtasks command
                  schtasks /?
                  
      3. Create a task called "Nagger" that runs every minute, starting a new process that dumps the contents of "C:\temp\uwnetid\test.1":
                  schtasks /create /sc minute /mo 1 /tn Nagger /tr "cmd /c start """nag""" """cmd /k type C:\temp\uwnetid\test.1""""
                  
      4. Delete a task
                  schtasks /delete /tn Nagger
                  
      5. You can also end a running task, run a task now, get detailed info on a task, or change an option on an existing task.

        All schtasks subcommands can be executed on a remote system where you know the computer name, privileged user name and password.

        Tasks are stored in C:\windows\tasks as .job files, which can be copied to other computers (user names and passwords might need to be changed).

      6. Schedule a daily cleaning at 1:00 am of just the test backup files in the C:\temp\uwnetid directory
                  schtasks /create /sc daily /st 01:00:00 /tn "Clean Test Backups" /tr "cmd /c del c:\temp\uwnetid\test.*.bak"
        
                  rem verify that files exist now
                  dir c:\temp\uwnetid\test.*.bak
        
                  rem trust that it will run at 1:00 am, but test the command now
                  schtasks /run /tn "Clean Test Backups"
        
                  rem Verify that they are gone
                  dir c:\temp\uwnetid\test.*.bak
                  

    8. Tracing execution

      1. Batch files

        If you don't include

                  @ECHO OFF
                  
        as the first line, batch processing will automatically trace the execution.

      2. Perl debug mode

        You can invoke Perl with the -d option; e.g.:

                  perl -d find_devices.pl
                  
        to invoke the debugger.

  2. Linux

    1. Restart the computer

      • Choose to boot up Fedora Core 5 instead of Windows.

      • Press:
                Ctrl-Alt-F4
                
        to get a login prompt.

    2. Login as itadmin

      1. output something
                echo hi
                
      2. save a value in an environment variable
                export greeting=hi
                echo $greeting
                
      3. do something conditionally (on file/folder non-existence)
                if [ ! -e /tmp/uwnetid ]; then mkdir /tmp/uwnetid;  fi
                
      4. direct output to file
                echo 1 hi >/tmp/uwnetid/greeting
                cat /tmp/uwnetid/greeting
                
      5. append output to file
                echo 2 bye >>/tmp/uwnetid/greeting
                cat /tmp/uwnetid/greeting
                
      6. get input from a file
                more 
                
      7. use output from one command as input to another ("piping")
                cat /tmp/uwnetid/greeting | grep bye
                
      8. do something repeatedly
                for (( i=1 ; i <= 5 ; i++ )) ; do echo $i >/tmp/uwnetid/test.$i; done
                
    3. Simple bash script: do for every file that matches input argument

      1. edit the bash script

        It will dump the contents of all of the files provided in the argument list.

                  vi doforfiles
                  
      2. Enter a capital "A" to add lines:
                  A
                  
      3. Add these lines
                  #!/bin/bash
                  export filename=/tmp/uwnetid/files
                  ls -1 $1 >$filename
        
                  for file in `cat $filename` ; do cat $file ; done
                  
        • #!/bin/bash tells the system this is a bash script, and what command interprets the statements inside

        • ls -1 means to just list the file names, one per line -- no other information.

        • $1 is the contents of the first argument passed to the script.

        • `cat $filename` says to dump the names in the file to the for list.

      4. Press Esc to exit insert mode

      5. Save the file -- enter:
                  :wq
                  
      6. Run the script
                  bash doforfiles "/tmp/uwnetid/test.*"
                  
      7. Look at the file of files:
                  cat /tmp/uwnetid/files
                  
        We don't need that file, so it should be deleted afterwards.

      8. improve the script by showing which file name, and deleting the temporary file
                  vi doforfiles
                  
      9. Replace the for line with this:
                  for file in `cat $filename` ; do (echo $file & cat $file) ; done
                  
        Here the body of the for loop does two commands, within parentheses to group them and separated by the ampersand sign "&".

      10. Add this line to the end of the file
                  rm -f $filename
                  
        Delete the file without asking for permission "-f".

      11. save the file

      12. Run the modified script
                  bash doforfiles "/tmp/uwnetid/test.*"
                  
      13. Let it run without specifying the shell interpreter command, marking as executable:
                  chmod +x doforfiles
                  
        and run it as:
                  doforfiles "/tmp/uwnetid/test.*"
                  
      14. You didn't actually need a script or a temporary file for this:
                  for file in `ls -1 /tmp/uwnetid/test.*` ; do cat $file ; done
                  

    4. Sample system administration task using batch files

      Let's backup the /home/itadmin directory early in the morning of every day using the tar command.

      1. Test the command to create a tape archive:
                  tar -cvpzf /tmp/uwnetid_backup.tar /home/itadmin
                  
        tar is the tape archive command, and it used to be almost exclusively used to make archives of files onto tapes. Nowadays, it is probably used more often to create archives on disk.
        • -c means to create the archive

        • -v indicates to output a lot of information about the files as they are added to the archive

        • -p indicates to preserve permissions (so restoring files can keep the same file permissions)

        • -z means to use gzip to compress the data to save space

        • -f flags that the archive file name comes next

        • archive filename is whatever you want it to be

        • dirs is a list of directories and/or files to add to the archive

        Now that we know the command will work to create the archive, we verify that it has added the files by checking the table of contents.

      2. Check the table of contents for the archive:
                  tar -tvf /tmp/uwnetid_backup.tar
                  
      3. Put the command in a script called run_backup
                  vi run_backup
                  
      4. Add the lines, first enter:
                  A
                  
      5. Then enter the lines:
                  #!/bin/sh
                  /bin/tar -cvpzf /tmp/uwnetid_backup.tar /home/itadmin
                  
      6. Exit insert mode; press the Esc key.

      7. Save the file; enter:
                  :wq
                  
      8. Make sure the file is executable:
                  chmod u+x run_backup
                  
      9. Make sure it runs:
                  ./run_backup
                  
      10. Schedule the task to run with the cron facility

        There are two ways to do it -- automatically or manually

        • Automatically -- move it to /etc/cron.daily, which, if you look at /etc/crontab, will run at 4:20 am every day.
                      mv run_backup /etc/cron.daily
                      
        • Manually -- set up a cron job
          • Use crontab to edit your cron table entry
                          export EDITOR=/usr/bin/vim
                          crontab -e
                          
            crontab needs an editor to use; setting the EDITOR environment variable allows crontab to use that one. Now you are in vi.

          • Get ready to enter lines:
                          A
                          
          • Enter the specification for 4:00 am every day:
                          0 4 * * * /home/itadmin/run_backup
                          
          • Get out of insert mode; press the Esc key

          • Save the file
                          :wq
                          
          • You'll only be able to check the results the next day.

    5. Et Cetera

      There can be many other similarities in automating tasks between Windows and Linux, but one can also emphasize the differences. Here are just a few minor pointers:

      • Use bash -x before a bash script to trace execution.

      • Perl was written in a Unix environment, and is well-integrated with the Unix system calls.

      • Composing or chaining together simple commands, via the pipe and other symbols, allows complex and powerful processing to be done from the command line.

      • Network firewall rules are often scripted to make it easier to specify and change the behavior of the firewall, as opposed to writing a configuration file.

      • expect is a tool similar in spirit to AutoIT, but does not work with the GUI.


Cleanup

  1. Clone disks with master image