Gossamer Forum: General: Perl Programming: Simple backup script....

Veteran / Moderator (18441 posts)

Jun 6, 2002, 7:14 AM

Post #1 of 40

Shortcut

Simple backup script....

Ok, it took me a while to work out how to do all this...but I thought I would share it with you lot. To start with I had a look around for a decent script that would backup my SQL databases and sites, but I couldn't find any. This is what I came up with;

Code:
#!/usr/bin/perl 
##################################### 

use CGI::Carp qw(fatalsToBrowser); 
use Time::HiRes; 

#se tthis to your emailo address that you want alerting on... 
$webmaster = 'webmaster@ace-installer.com'; 

# start the timer... 
$start_time = time; 

# backup SQL stuff... 
`mysqldump -uusername -ppassword ace_clipart > /home/dumps/ace_clipart.sql`; 
`mysqldump -uusername -ppassword ace_installer_shop > /home/dumps/ace_installer_shop.sql`; 
`mysqldump -uusername -ppassword ace_templates > /home/dumps/ace_templates.sql`; 

# backup all home directory... 
`tar -cf installer.tar /home/ace-installer.com/*`; 
`gzip -c installer.tar > /home/dumps/installer.tar.gz`; 
`tar -cf clipart.tar /home/ace-clipart.com/*`; 
`gzip -c clipart.tar > /home/dumps/clipart.tar.gz`; 
`tar -cf wwwtemplates.tar /home/wwwtemplates.com/*`; 
`gzip -c wwwtemplates.tar > /home/dumps/wwwtemplates.tar.gz`; 
`tar -cf templates.tar /home/templates.ws/*`; 
`gzip -c templates.tar > /home/dumps/templates.tar.gz`; 
`tar -cf 123carlinks.tar /home/123carlinks.com/*`; 
`gzip -c 123carlinks.tar > /home/dumps/123carlinks.tar.gz`; 
`tar -cf acetemplates.tar /home/ace-templates.com/*`; 
`gzip -c acetemplates.tar > /home/dumps/acetemplates.tar.gz`; 

$end_time = time; 

$taken_time = $end_time - $start_time; 

open(MAIL,"|/usr/sbin/sendmail -t"); 
print MAIL "To: $webmaster \n"; 
print MAIL "From: $webmaster \n"; 
print MAIL "Reply-to: $webmaster \n"; 
print MAIL "Subject: RE Server Backup... \n\n"; 
print MAIL "The server has been backed up...it took $taken_time to backup the sites and MySQL databases...\n"; 
print MAIL "\n \n Thanks"; 
print MAIL "\n"; 
print MAIL "A.J.Newby \n"; 
print MAIL "Ace Installer \n"; 
close(MAIL);

Obviously you can hack this script to your hearts conent. You may want to even back up the whol e/home/ directory in one tar.gz file...its completly up to you. I just thought i would share it with you guys cos I'm nice like that Tongue

Andy (mod)
andy@ultranerds.co.uk

Jun 6, 2002, 7:16 AM

Veteran (19537 posts)

Jun 6, 2002, 7:16 AM

Post #2 of 40

Shortcut

Re: [Andy] Simple backup script.... In reply to

All those gzip commands could be eradicated by adding a z onto the tar command.

...and then the rest of the tars could be removed by adding all /path/* into one command (I think)

Last edited by:

Paul: Jun 6, 2002, 7:18 AM

Jun 6, 2002, 7:22 AM

Veteran / Moderator (18441 posts)

Jun 6, 2002, 7:22 AM

Post #3 of 40

Shortcut

Re: [Paul] Simple backup script.... In reply to

Yes Paul..thanks for your vote of confidence in my coding Wink

Andy (mod)
andy@ultranerds.co.uk

Jun 6, 2002, 7:30 AM

Veteran (19537 posts)

Jun 6, 2002, 7:30 AM

Post #4 of 40

Shortcut

Re: [Andy] Simple backup script.... In reply to

Its got nothing to do with that...it was a tip.

If you want to use 6 system commands more that you need to when you could just add a z then thats up to you.

Last edited by:

Paul: Jun 6, 2002, 7:30 AM

Jun 6, 2002, 9:05 AM

Crapshoot70

User (130 posts)

Jun 6, 2002, 9:05 AM

Post #5 of 40

Shortcut

Re: [Paul] Simple backup script.... In reply to

Yeah OK...Where do I learn to do THAT.

excuse the sarcasm, I just had a rough morning, and I prefer to take it out on others rather than deplete what little ounce of sanity I have left...

Actually I know how to program with perl, it's just that my knowledge only goes to the extent of the DBI module and the basic programming structure... Where can I go to learn the more advanced procedures?

Crapshoot70 (this isn't my real name)
--
I don't like anyone...but everyone LOVES me Unsure

--
AKA...The Post Pirate Pirate

Last edited by:

Crapshoot70: Jun 6, 2002, 9:07 AM

Jun 6, 2002, 9:22 AM

Veteran / Moderator (18441 posts)

Jun 6, 2002, 9:22 AM

Post #6 of 40

Shortcut

Re: [Crapshoot70] Simple backup script.... In reply to

That script isn't advanced really..its just a load of system commands, that are executed via a Perl script Tongue

Andy (mod)
andy@ultranerds.co.uk

Jun 12, 2002, 11:40 PM

Veteran / Moderator (18441 posts)

Jun 12, 2002, 11:40 PM

Post #7 of 40

Shortcut

Re: [Andy] Simple backup script.... In reply to

MMm..this is weird!

Quote:

tar: Removing leading `/' from member names
tar: z: Wrote only 0 of 10240 bytes
tar: Error is not recoverable: exiting now
Backed up Ace-Installer
tar: Removing leading `/' from member names
tar: z: Wrote only 0 of 10240 bytes
tar: Error is not recoverable: exiting now
Backed up Ace-Clipart
tar: Removing leading `/' from member names
tar: z: Wrote only 0 of 10240 bytes
tar: Error is not recoverable: exiting now
Backed up WWWTemplates
tar: Removing leading `/' from member names
tar: z: Wrote only 0 of 10240 bytes
tar: Error is not recoverable: exiting now
Backed up Templates.ws
tar: Removing leading `/' from member names
tar: z: Wrote only 0 of 10240 bytes
tar: Error is not recoverable: exiting now
Backed up 123CarLinks
tar: Removing leading `/' from member names
tar: z: Wrote only 0 of 10240 bytes
tar: Error is not recoverable: exiting now
Backed up Ace-Templates

Thats the code im using;

Code:
# backup all home directory... 
`tar -cfz installer.tar /home/ace-installer.com/*`; 
#`gzip -c installer.tar > /home/dumps/installer.tar.gz`; 
print "Backed up Ace-Installer\n"; 
`tar -cfz clipart.tar /home/ace-clipart.com/*`; 
#`gzip -c clipart.tar > /home/dumps/clipart.tar.gz`; 
print "Backed up Ace-Clipart\n"; 
`tar -cfz wwwtemplates.tar /home/wwwtemplates.com/*`; 
#`gzip -c wwwtemplates.tar > /home/dumps/wwwtemplates.tar.gz`; 
print "Backed up WWWTemplates\n"; 
`tar -cfz templates.tar /home/templates.ws/*`; 
#`gzip -c templates.tar > /home/dumps/templates.tar.gz`; 
print "Backed up Templates.ws\n"; 
`tar -cfz 123carlinks.tar /home/123carlinks.com/*`; 
#`gzip -c 123carlinks.tar > /home/dumps/123carlinks.tar.gz`; 
print "Backed up 123CarLinks\n"; 
`tar -cfz acetemplates.tar /home/ace-templates.com/*`; 
#`gzip -c acetemplates.tar > /home/dumps/acetemplates.tar.gz`; 
print "Backed up Ace-Templates\n";

Seems weird that it worked a couple of days ago, but not now Unimpressed

Andy (mod)
andy@ultranerds.co.uk

Jun 13, 2002, 2:12 AM

Veteran (19537 posts)

Jun 13, 2002, 2:12 AM

Post #8 of 40

Shortcut

Re: [Andy] Simple backup script.... In reply to

Well you'll want to start by adding P to the tar command.

Jun 13, 2002, 3:39 AM

Veteran / Moderator (18441 posts)

Jun 13, 2002, 3:39 AM

Post #9 of 40

Shortcut

Re: [Paul] Simple backup script.... In reply to

What, like;

>tar -cfzP installer.tar /home/ace-installer.com/*

That is still giving me the error.

tar: Removing leading `/' from member names
tar: zP: Wrote only 0 of 10240 bytes
tar: Error is not recoverable: exiting now

Any more ideas Unsure

Thanks

Andy (mod)
andy@ultranerds.co.uk

Jun 13, 2002, 3:46 AM

Veteran / Moderator (4108 posts)

Jun 13, 2002, 3:46 AM

Post #10 of 40

Shortcut

Re: [Andy] Simple backup script.... In reply to

Yes. Make life easier for yourself.

Use Archive::Tar:

http://search.cpan.org/...hive-Tar-0.22/Tar.pm

- wil

Jun 13, 2002, 4:01 AM

Veteran (19537 posts)

Jun 13, 2002, 4:01 AM

Post #11 of 40

Shortcut

Re: [Wil] Simple backup script.... In reply to

There is such a thing as over using modules you know ;)

If you want to use Archive::Tar you'll probably need to install Compress::ZLib too.

Jun 13, 2002, 4:03 AM

Veteran / Moderator (4108 posts)

Jun 13, 2002, 4:03 AM

Post #12 of 40

Shortcut

Re: [Paul] Simple backup script.... In reply to

Never! It makes life a hundred times easier.

And if you install Archive::Tar using CPAN++ it will automatically install any dependencies for you.

- wil

Jun 13, 2002, 7:08 AM

Veteran (19537 posts)

Jun 13, 2002, 7:08 AM

Post #13 of 40

Shortcut

Re: [Wil] Simple backup script.... In reply to

Yeah it can make life easier but you _can_ over use them.

I can just imagine your scripts...you probably "require" about 10 modules to print "Hello World" Tongue

Jun 13, 2002, 8:21 AM

SeanP

User (456 posts)

Jun 13, 2002, 8:21 AM

Post #14 of 40

Shortcut

Re: [Andy] Simple backup script.... In reply to

Good idea... I haven't automated my backups yet, but I sure need to do it... Question:

Is there an easy to name the backup files according to the date? For example, when using:

`mysqldump -uusername -ppassword database > database.sql`;

and

`tar -cpsvzf www.tar.gz /www`;

is there an easy way of renaming the files to a name with the date in it like:

database061302.sql

and

www061302.tar.gz

Sean

Last edited by:

SeanP: Jun 13, 2002, 8:21 AM

Jun 13, 2002, 8:24 AM

Veteran / Moderator (18441 posts)

Jun 13, 2002, 8:24 AM

Post #15 of 40

Shortcut

Re: [SeanP] Simple backup script.... In reply to

Sure is. Just grab the date() (not sure of the format)...and then call it something like;

$filename = $todays_date . "databasename.sql";

Andy (mod)
andy@ultranerds.co.uk

Jun 13, 2002, 12:38 PM

Veteran / Moderator (4108 posts)

Jun 13, 2002, 12:38 PM

Post #16 of 40

Shortcut

Re: [Paul] Simple backup script.... In reply to

No. Not really.

In most of my stable, production scripts I even turn off use strict; to speed up my programs.

And I hardly ever use CGI.pm anymore as it's so slow I'm sure I could code some BASIC to run faster.

However, you try writing something to replace XML::Twig.

Ah.. Exactly ;-)

- wil

Jun 13, 2002, 1:06 PM

Veteran (19537 posts)

Jun 13, 2002, 1:06 PM

Post #17 of 40

Shortcut

Re: [Wil] Simple backup script.... In reply to

>>In most of my stable, production scripts I even turn off use strict; to speed up my programs. <<

I don't think thats a good idea myself.

>>However, you try writing something to replace XML::Twig.<<

Thats the point I'm trying to make (and probably failing to do).

Modules are great, however there is no point in using one when you can do the same task yourself without the module with a relatively small amount of code...(the mkpath thing springs to mind).....remember every "use" at the top of your script is compiled right at the beginning so if you "use" 5 modules then you have to wait for them all to compile before anything useful starts happening....then if you have "use" within a module being "use"d ...well you get the picture.

Maybe I'm thinking too much from my own perspective as someone who has enough knowledge to write my own code...I can see that modules would be a great help to those wanting greater functionality in their scripts without writing the code themselves.

Last edited by:

Paul: Jun 13, 2002, 1:07 PM

Jun 13, 2002, 5:13 PM

Veteran / Moderator (4108 posts)

Jun 13, 2002, 5:13 PM

Post #18 of 40

Shortcut

Re: [Paul] Simple backup script.... In reply to

I'm not going to even start off on this argument with you.

Well, maybe when I have more energy another day. I'll give it a go on Sunday.

- wil

Jun 13, 2002, 9:54 PM

SeanP

User (456 posts)

Jun 13, 2002, 9:54 PM

Post #19 of 40

Shortcut

Re: [Andy] Simple backup script.... In reply to

In Reply To:

Sure is. Just grab the date() (not sure of the format)...and then call it something like;

$filename = $todays_date . "databasename.sql";

How about this:

Code:
#!/usr/bin/perl 
##################################### 

use CGI::Carp qw(fatalsToBrowser); 
use Time::HiRes; 

# email address to send alert... 
$webmaster = 'email@mydomain.com'; 

# start the timer... 
$start_time = time; 

# backup database... 
`mysqldump -uUSERNAME -pPASSWORD database > /backup/database.sql`; 

# compress database backup... 
`tar -cspzf /backup/database.tar.gz /backup/database.sql`; 
`mv /backup/database.tar.gz /backup/database\`date +%m%d%Y\`.tar.gz`; 
`rm /backup/database.sql`; 

# backup web files... 
`tar -cpszf /backup/www.tar.gz /www`; 
`mv /backup/www.tar.gz /backup/www\`date +%m%d%Y\`.tar.gz`; 

$taken_time = time - $start_time; 

open(MAIL,"|/usr/sbin/sendmail -t"); 
print MAIL "To: $webmaster \n"; 
print MAIL "From: $webmaster \n"; 
print MAIL "Reply-to: $webmaster \n"; 
print MAIL "Subject: RE: Server Backup... \n\n"; 
print MAIL "The server has been backed up...  It took $taken_time seconds to backup the database and web files...\n"; 
close(MAIL);

I removed the $end_time variable and edited the $taken_time variable. I also added a tar command to compress the database dump, and added the date to the file names. That way, you can set this as a cron job and have a backup every night. Cool little scipt Andy!

Sean

Jun 14, 2002, 1:38 AM

Veteran (19537 posts)

Jun 14, 2002, 1:38 AM

Post #20 of 40

Shortcut

Re: [SeanP] Simple backup script.... In reply to

Are you sure that mv command works....it looks like you have a command within a command?

Last edited by:

Paul: Jun 14, 2002, 1:41 AM

Jun 14, 2002, 1:43 AM

Veteran / Moderator (18441 posts)

Jun 14, 2002, 1:43 AM

Post #21 of 40

Shortcut

Re: [SeanP] Simple backup script.... In reply to

Yeah..but I don't think your date stuff will work. You will need to assign the date() before the command, and then use something like $date to insert it into the Shell command part. I could be wrong, but it would just seem logical Wink

Andy (mod)
andy@ultranerds.co.uk

Jun 14, 2002, 7:39 AM

SeanP

User (456 posts)

Jun 14, 2002, 7:39 AM

Post #22 of 40

Shortcut

Re: [Andy] Simple backup script.... In reply to

It works. I'm running it now. As long as you escape the backticks to the date command with a "\", you can execute that mv command. I have it implemented on two servers now and setup a cron job to run them every night. Works great.

Sean

Last edited by:

SeanP: Jun 14, 2002, 7:40 AM

Jun 19, 2002, 9:10 AM

Veteran / Moderator (4108 posts)

Jun 19, 2002, 9:10 AM

Post #23 of 40

Shortcut

Re: [Paul] Simple backup script.... In reply to

In Reply To:

Modules are great, however there is no point in using one when you can do the same task yourself without the module with a relatively small amount of code...(the mkpath thing springs to mind).....remember every "use" at the top of your script is compiled right at the beginning so if you "use" 5 modules then you have to wait for them all to compile before anything useful starts happening....then if you have "use" within a module being "use"d ...well you get the picture.

Heh. Code I am working on now... the first few lines look like this:

Code:
use strict; 
use Carp; 
use XML::Twig; 
use LWP::Simple; 
use Text::CSV_XS; 
use MIME::Lite; 
use DBI; 
use IO::File; 
use Getopt::Std;

:-)

- wil

Jun 20, 2002, 12:49 AM