SEO

Hello Fellow Slackers,

I am using Slackware 12.0 and I have several directories that I would
like to backup incrementally every day to an external USB hard drive..
As per the tar manual, tar --update is not a good option for this.

What's the best way to do this? Everything I've found on the net is for
people looking to clone or back up an entire system. I just want to
preserve my few precious data directories, all on my /home partition.

I've also tried using /usr/sbin/backup included with tar, but that is
specifically designed to control tape drives, and hasn't worked yet with
this HD.

Thanks for any suggestions,
Joel
--
Joel J. Adamson
Biostatistician
Pediatric Psychopharmacology Research Unit
Massachusetts General Hospital
Boston, MA 02114
(617) 643-1432
(303) 880-3109

Joel J. Adamson wrote:
> What's the best way to do this? Everything I've found on the net is for
> people looking to clone or back up an entire system. I just want to
> preserve my few precious data directories, all on my /home partition.

rsync i think would be the standard utility. you may also check out unison
<http://www.cis.upenn.edu/~bcpierce/unison/>. unison is really meant for
two-way synchronisation, so in actual fact more than what you want, but it
has a GUI (which is optional: you can also run it from the command line or
in a script) which may make it a bit easier to use. (you can set up the
synchronisation through the GUI, then run the CLI version every day through
cron, for example.)


--
Joost Kremers joostkremers@yahoo.com
Selbst in die Unterwelt dringt durch Spalten Licht
EN:SiS(9)

On 2007-12-11, Joel J. Adamson <jadamson@partners.org> wrote:
>
> I am using Slackware 12.0 and I have several directories that I would
> like to backup incrementally every day to an external USB hard drive..
> As per the tar manual, tar --update is not a good option for this.
>
> What's the best way to do this? Everything I've found on the net is for
> people looking to clone or back up an entire system. I just want to
> preserve my few precious data directories, all on my /home partition.

I use a Perl script called dobackup.pl for this purpose:

http://www.webcon.ca/opensource/dobackup/

It's pretty easy to set up and run from cron, and (for your needs) easy
to configure to back up only certain subdirectories of your filesystems.

--keith

--
kkeller-usenet@wombat.san-francisco.ca.us
(try just my userid to email me)
AOLSFAQ=http://www.therockgarden.ca/aolsfaq.txt
see X- headers for PGP signature information

On Tue, 11 Dec 2007 16:52:16 -0500, jadamson@partners.org (Joel J. Adamson) wrote:

>Hello Fellow Slackers,
>
>I am using Slackware 12.0 and I have several directories that I would
>like to backup incrementally every day to an external USB hard drive..
>As per the tar manual, tar --update is not a good option for this.
>
>What's the best way to do this? Everything I've found on the net is for
>people looking to clone or back up an entire system. I just want to
>preserve my few precious data directories, all on my /home partition.

I use this script: http://bugsplatter.mine.nu/misc/backup-manager for
a while now to keep a regular rsync snapshot backup of selected areas.

Perhaps you could roll-your-own based on this?

Grant.
--
http://bugsplatter.mine.nu/

On Wed, 11 Dec 2007, Joost Kremers wrote:

>
> Joel J. Adamson wrote:
>> What's the best way to do this? Everything I've found on the net is for
>> people looking to clone or back up an entire system. I just want to
>> preserve my few precious data directories, all on my /home partition.
>
> rsync i think would be the standard utility.


I use rsync to fully sync all machines weekly, I do a 'rolling 7 day'
daily and a weekly TAR of /etc /home and a few other critical dirs
depending on what server/desktop it is and config files such as linux
src's .config, sendmail.mc and so on, so backing up is a cp -a
of the disk and extract the most recent tar of configs and home etc ...

If you have databases, I do a 'rolling 24 hour by 7 days' (yup 168
backups) mysqldump backup [ warning: requires huge disks unless your
DB's are small... ] I also do a daily tar backup of the
/var/lib/mysql/mysql directory, all the sqldumps in the world wont help
you much if you screwup your mysql access Well ok its easy enough to
recover, but I find it faster to do it this way... I also keep a copy of
all this on the database server as well (saves times and hassle by
avoiding logging into and copying the dump to the db server then loading
it, ok, so it might only save 15 seconds, but those seconds can mean
missed business for customers.

Although rsync is fantastic and has saved my ass on more times that I
care to remember with failed disks, it only takes you to go away for the
weekend, and miss one 'run' (or 2 if you use the backup option) and you
can't recover as the runs that occured when you weren't there, has synced
the corrupted file(s) or deleted the missing one(s).

All backup scripts are in perl and use Mail::Sendmail to let me know if
somthing fails.



--
Cheers
Res

mysql> update auth set Framed-IP-Address='127.0.0.127' where user= 'troll';
~# radzap troll

Hallo, Joel,

Du meintest am 11.12.07:

> I am using Slackware 12.0 and I have several directories that I would
> like to backup incrementally every day to an external USB hard
> drive.. As per the tar manual, tar --update is not a good option for
> this.

> What's the best way to do this?

http://arktur.de/Wiki/Zusatzprogramme:rsnapshot
http://www.rsnapshot.org/

Viele Gruesse
Helmut

"Ubuntu" - an African word, meaning "Slackware is too hard for me".

On 2007-12-12, Helmut Hullen wrote:

> Viele Gruesse
> Helmut
>
> "Ubuntu" - an African word, meaning "Slackware is too hard for me".

If you're gonna use my signature, you should at least learn how to
properly use a signature. Hint: "delimiter".


--
"Ubuntu" - an African word meaning "Slackware is too hard for me".

On Tue, 11 Dec 2007 16:52:16 -0500, Joel J. Adamson wrote:

> Hello Fellow Slackers,
>
> I am using Slackware 12.0 and I have several directories that I would
> like to backup incrementally every day to an external USB hard drive..
> As per the tar manual, tar --update is not a good option for this.
>
> What's the best way to do this? Everything I've found on the net is for
> people looking to clone or back up an entire system. I just want to
> preserve my few precious data directories, all on my /home partition.
>
> I've also tried using /usr/sbin/backup included with tar, but that is
> specifically designed to control tape drives, and hasn't worked yet with
> this HD.
>
> Thanks for any suggestions,
> Joel
>
I have been using a home-grown system to backup directories. It has been a
good choice for me because it concentrates most of the backup effort on
actual file changes, and ignores a large set of static files. The
usual caveats should be stated in advance, though. Backing up does not
lend itselt to a one-size-fits-all approach. Pick the method which
fits your needs. IME, it never hurts to know the fundamentals of the
tools which you use, and to practice using them (both backing up and
restoring) whatever method is selected.

The method that is explained below sets up a simple "CVS" system for the
directory targetted for backup. This method allows any previous saved
state to be restored. For example, it's easy to retrieve files saved last
Thursday, or last July. This method may be overkill for you specific
application. If so, then another tool which might be used to do something
very similar is rsync, secifically, using rsync's "--only-write-batch"
mode.

Here is an overview of the method that I use:

1. Create a full snapshot of the directory to be backed up using the
method of your choice; IMO, tar works fine. If you need extended
attributes (acl and attr), then save them separately (man getfacl, man
getfattr)

2. Schedule incremental daily backup jobs with "at" or as a cron job at
the appropriate time interval for your requirements. When this backup job
runs it obtains a current listing of the target directory, and compares
this new listing to the listing as obtained by the last backup job. This
comparison yields four reports:

report 1: files which match exactly with the previous day.
report 2: files which have changed
report 3: files which have been deleted
report 4: files which have been added

Files in report 2 and report 4 should be backed up. Again, tar works fine
for this job using the "-T" switch. Also, a compression stage (gzip,
bzip2) can significantly reduce the size of incremental jobs.

Along with the archive, save the following:
a. The directory listing when the incremental backup was saved
b. File selections in this backup. This makes it easier to lookup what
has been backed up (and which versions are available for restore.)
c. File extended attributes. Some features may not be captured by tar, or
other archive tools.

Warning: "locked" and active files and databases may require special
consideration in order for them to be saved in a consistent state.

3. Review daily backup log. "at" jobs will send email back to the
user. Review the message to verify that all files which were selected for
backup could be opened and were written into the daily backup job.

4. Repeat to step 2. To start over with a new clean backup set, repeat to
step 1.

There are various options for restoring. The simplest is to restore in
reverse chronological order (use the "-k" switch with tar.)

Notice: this method yields a "superset" of all files which were backed up
in the past. This is good from a "CVS" point of view, but can have some
side effects, also. For example, the space necessary to perform a restore
operation can be much larger than the space in use, especially if the
directory structure has been rearranged. Another side effect is that a
restore operation is not complete until the "superset" (as restored) is
"pruned" to account for file deletions on the interval.

Notice: daily incremental backups can be restored separately to generate
larger relative backups. For example, daily backups can be recombined into
weekly, or monthly relative backups. This can simplify the restore
operation, while potentially sacrificing some granularity.


--
Douglas Mayne

On Tue, 11 Dec 2007 16:52:16 -0500, Joel J. Adamson wrote:

> Hello Fellow Slackers,
>
> I am using Slackware 12.0 and I have several directories that I would
> like to backup incrementally every day to an external USB hard drive..
> As per the tar manual, tar --update is not a good option for this.
>
> What's the best way to do this? Everything I've found on the net is for
> people looking to clone or back up an entire system. I just want to
> preserve my few precious data directories, all on my /home partition.
>
> I've also tried using /usr/sbin/backup included with tar, but that is
> specifically designed to control tape drives, and hasn't worked yet with
> this HD.
>

Here is a link to a nice article on using rsync to do backups, including
incremental backups.

http://www.mikerubel.org/computers/r...ots/index.html

Cheers,
Mike

Joel J. Adamson <jadamson@partners.org> wrote:
> Hello Fellow Slackers,

> I am using Slackware 12.0 and I have several directories that I would
> like to backup incrementally every day to an external USB hard drive..
> As per the tar manual, tar --update is not a good option for this.

[snip]

Joel,

It sounds to me like you want to copy the directories to a location on
your USB hard drive.

So make a destination directory on the USB hard drive and mount the USB
hard drive. For example, your destination directory when mounted could
be

/mnt/backups/071214aa

The directory /mnt/backups is the mount point, and the dir 071214aa is
the dir that you made on the USB hard drive. The name 071214aa means
"the aa backup (or first backup) on 12-14-07". The second backup on
that date would be "ab" and so on, however I don't know if you are
likely to do more than one in one day.

Then use (probably as root):

cp -Rpdi path-to-source-directory /mnt/backups/071214aa

repeat for each source directory that you wish to backup

Here's what the options -Rpdi do

-R Copy directories recursively. This gets all the files *and*
subdirectories, and so on.

-p Preserve the original files' owner, group, permissions.

-d Copy symbolic links as symbolic links rather than copying
the files that they point to.

-i Prompts you if anything already exists at the target with the
same name. This is not going to happen if the target dir is
new, but it is a nice habbit to use -i. You might someday
avoid overwriting something this way.

The idea is that you can do backups quite adequately (despite all the
complex advice you received from others) by doing a simple copy. Each
time (each new backup date) use a new target directory.

After a while you can start deleting the old target directories. For
example, to delete /mnt/backups/071214aa do this (probably as root):

rm -R /mnt/backups/071214aa

If you backups are really large, then you may want to look at the data
you are backing up. Have you mixed short files that change frequently
with longer files (like sound, video or picture files) that never
change? I keep sound, video picture and various other kinds of files
in separate directories. Because the files are large, they fill up a
lot of space in a hurry. Because they don't ever change there is no
need to back them up repeatedly. So I occasionally "archive" them by
burning them to a cd, then I delete the dirs from my hard drive. Or
you could just make the dirs read only.

The idea is that it pays to curtail the size of the directories that you
are backing up. If you do this effectively then it is practical to
backup everything each time by copying the whole directory as I showed
above. A copy of a directory has the benefit that it is easy to refer
to later.

Or maybe your directories are small enough that size isn't an issue.

Do you write bash scripts? A little scripting to make some tools for
your own use would make the above use of mount, cp and rm easier and
more consistant.

-Joe Rosevear