White dot for spacing only
The Dice Project


Created by Lindsey Brown 19/02/2008

archiving-procedures.html,v 1.58 2011/11/15 10:29:22 lmb Exp

This procedure should be considered draft/incomplete while this line is here

Account Archiving

This is a guide to the procedures of archiving accounts. The policy is available at:- Account Archiving

Roger's decription of the process is available here

ptn121 and ptn140 on Sphinx are the archiving partitions for NFS accounts.
/group/support/archiving is the archiving partition for AFS accounts.

Selection

This process is the same for AFS and NFS accounts.

The accounts to be archived are selected using the script old_accounts, one of the account management tools. This script checks to see which Informatics computing accounts (specifically those recorded in LDAP) do not correspond to current (or recently current) staff or students doing Informatics degrees. These accounts are archived, ie their homedir, mail (not students) and web files are copied to an archive partition and then written to tape when the partition is full.

The script external_accounts checks to see which Informatics computing accounts (specifically those recorded in LDAP) correspond to old non-Informatics students accounts (i.e students currently doing a non-Informatics degree who are not registered on any Informatics courses this session or last session). These accounts are sent a warning email and then, if there is no reason why they should be saved, are deleted.

The script warn_account_deletion can be used to send email to the list of users produced by old_accounts.

Software for Archiving

The following scripts are available on sphinx for the copying to the archive partition, the writing of the files to tape and the restoring of files from tape.
/usr/local/sbin/archive-rsync                      uses the least full of the archiving partitions
/usr/local/sbin/archive-rsync-ptn -p ptn140        gives the choice of archiving partitions
/usr/local/sbin/archive-rsync-ptn.NEW              fixes a few bugs
/usr/local/sbin/archive-tape-write
/usr/local/sbin/archive-tape-access
The tape-write script has been modified to work from amaranthine (or any linux machine)
/disk/archiving/scripts/archive-tape-write.Ross  -v -device /dev/st0 -source /partition/ptn140

Moving Home Directory to Archive Partition

NFS

Do not fill the partition more than 95% full.

The script /usr/local/sbin/archive-rsync copies the user's files to the archive partition. It also copies mail and web directories, producing the following output:-

bash-2.05# ls -l TEST/ptn041
total 20
drwxr-xr-x  91 woodsar  staff       7680 Feb 18  2006 woodsar-archive
drwx------   3 woodsar  daemon       512 Oct 15  2004 woodsar-archive.Mail
drwxr-xr-x   4 root     48           512 Nov  8  2004 woodsar-archive.Web
It accepts a file of users, (comma-separated) list of one or more users, or prompts for user(s) if none given. It does not delete files from the original location or update ldap.

AFS

This command creates an archive file. It does not remove anything.

vos dump -v -id user.foo -time 0 -file /group/support/archiving/foo-archive
Homepages can be rsync'd.
for i in `cat users`; do rsync -a -x -v homepages.inf.ed.ac.uk::homepages/homepages/$i . ; done;

Writing to Tape

On Sphinx
bash-2.05# script ArchiveTape003.out
Script started, file is ArchiveTape003.out
sh-2.05#  /usr/local/sbin/archive-tape-write -v -s /partition/ptn140

=>  Source "/partition/ptn140" is OK...

=>  Dumping location "/partition/ptn140" without further checks...

--  main::DumpLocation: Ready to run: "cd /partition/ptn140; /usr/sfw/bin/gtar -b 200 -cf /dev/rmt/0ub ."
    Continue? (y/n)[n]: y

=>  Verifying write to /dev/rmt/0ub & creating log...

>>  Create logfile: continue? (y/n)[n]: y

=>  Checking archive dump and creating log...

    Log file is /var/log/ArchiveDumpLogs/ArchiveTape003

sh-2.05# exit

On Amaranthine

Roger's original script has been modified so that the partition being archived can be varied, as can the tape device. These are given as command arguments. The modified script is /disk/archiving/scripts/archive-tape-write.Ross.
eg archive-tape-write.Ross -v -s /partition/ptn121 -device /dev/st0

Log Files

On Sphinx
cd /var/log/ArchiveDumpLogs/
bash-2.05# ls -l
total 8792616
-rw-r--r--   1 root     other    551219349 Nov  7  2006 ArchiveTape001
-rw-r--r--   1 root     other    551219349 Nov 10  2006 ArchiveTape002
-rw-r--r--   1 root     other    574013950 Jan 17  2007 ArchiveTape003
-rw-r--r--   1 root     other       1227 Jan 17  2007 ArchiveTape003.out
-rw-r--r--   1 root     other    576105272 Jan 26  2007 ArchiveTape004
-rw-r--r--   1 root     other        970 Jan 29  2007 ArchiveTape004.out
-rw-r--r--   1 root     other        853 Jan 26  2007 ArchiveTape004.out-1
-rw-r--r--   1 root     other    576105272 Jan 30  2007 ArchiveTape005
-rw-r--r--   1 root     other       1092 Jan 30  2007 ArchiveTape005.out
-rw-r--r--   1 root     other    224305349 Apr  9  2007 ArchiveTape006
-rw-r--r--   1 root     other       1133 Apr 10  2007 ArchiveTape006.out
-rw-r--r--   1 root     other    224305349 Apr 11  2007 ArchiveTape007
-rw-r--r--   1 root     other       1093 Apr 11  2007 ArchiveTape007.out
-rw-r--r--   1 root     other    611078242 Jun 18 18:42 ArchiveTape008
-rw-r--r--   1 root     other        588 Jun 19 13:57 ArchiveTape008.out
-rw-r--r--   1 root     other    611083962 Jun 19 21:48 ArchiveTape009
-rw-r--r--   1 root     other        432 Jun 20 10:02 ArchiveTape009.out
-rw-r--r--   1 root     other          4 Jun 19 21:48 Count

On Amaranthine
/disk/archiving/Logs
[amaranthine+]lmb: ls -l
total 8790152
-rw-r--r-- 1 lmb  people 551219349 Nov  7  2006 ArchiveTape001
-rw-r--r-- 1 lmb  people 551219349 Nov 10  2006 ArchiveTape002
-rw-r--r-- 1 lmb  people 574013950 Jan 17  2007 ArchiveTape003
-rw-r--r-- 1 lmb  people      1227 Jan 17  2007 ArchiveTape003.out
-rw-r--r-- 1 lmb  people 576105272 Jan 26  2007 ArchiveTape004
-rw-r--r-- 1 lmb  people       970 Jan 29  2007 ArchiveTape004.out
-rw-r--r-- 1 lmb  people 576105272 Jan 30  2007 ArchiveTape005
-rw-r--r-- 1 lmb  people      1092 Jan 30  2007 ArchiveTape005.out
-rw-r--r-- 1 lmb  people 224305349 Apr  9  2007 ArchiveTape006
-rw-r--r-- 1 lmb  people      1133 Apr 10  2007 ArchiveTape006.out
-rw-r--r-- 1 lmb  people 224305349 Apr 11  2007 ArchiveTape007
-rw-r--r-- 1 lmb  people      1093 Apr 11  2007 ArchiveTape007.out
-rw-r--r-- 1 lmb  people 611078242 Jun 18  2007 ArchiveTape008
-rw-r--r-- 1 lmb  people       588 Jun 19  2007 ArchiveTape008.out
-rw-r--r-- 1 lmb  people 611083962 Jun 19  2007 ArchiveTape009
-rw-r--r-- 1 lmb  people       432 Jun 20  2007 ArchiveTape009.out
-rw-r--r-- 1 lmb  people 615095730 Feb 25  2008 ArchiveTape010
-rw-r--r-- 1 lmb  people       795 Feb 26  2008 ArchiveTape010.out
-rw-r--r-- 1 lmb  people 615095730 Feb 26  2008 ArchiveTape011
-rw-r--r-- 1 lmb  people       493 Feb 27  2008 ArchiveTape011.out
-rw-r--r-- 1 lmb  people 491630892 Jun  9  2008 ArchiveTape012
-rw-r--r-- 1 lmb  people       438 Jun 10  2008 ArchiveTape012.out
-rw-r--r-- 1 lmb  people 491630892 Jun 10  2008 ArchiveTape013
-rw-r--r-- 1 lmb  people       506 Jun 10  2008 ArchiveTape013.out
-rw-r--r-- 1 root root   846348412 May  7 11:27 ArchiveTape014
-rw-r--r-- 1 root root       65536 May  7 11:38 ArchiveTape014.out
-rw-r--r-- 1 root root   846348412 May  8 10:45 ArchiveTape015
-rw-r--r-- 1 root root       65640 May  8 11:27 ArchiveTape015.out
-rw-r--r-- 1 root root           0 Sep 30 15:44 ArchiveTape016.out
-rw-r--r-- 1 lmb  people         4 Sep 30 15:42 Count

Restoring from Tape

On sphinx
/usr/sfw/bin/gtar -b 200 -xvf /dev/rmt/0ub ptn111/a1oshkar-archive

On amaranthine
tar -b 200 -xvf /dev/st0 ./ptn011/s0344566-archive


 : Units : User_support : Procedures 

Mini Informatics Logo - Link to Main Informatics Page
Please contact us with any comments or corrections.
Unless explicitly stated otherwise, all material is copyright The University of Edinburgh
Spacing Line