Gossamer Forum
Home : Products : DBMan : Installation :

delete records after a specific time?

(Page 1 of 3)
> >
Quote Reply
delete records after a specific time?
Hello!

How can I delete records after a specific time (e.g. 14 days) automaticly?
Every tip welcome!

Markus
Quote Reply
Re: delete records after a specific time? In reply to
Hi Markus

I donīt think you can do it automatically from the script itself. Somebody would have to click on something for something to happen. If you are using UNIX you might be able to do it by using a cron job that runs every day (I belive there is also some program for NT that simulates the cron feature).

The problem I see with using cron jobs is that in order to access the delete functions in the script you will have to go though the login and you canīt do that automatically (as far as I know). You would either have to disable the removal of the login file after x amount of time (i.e: Account never expire), disable the whole login procedure, or write some sort of mod that allows some specific account whose login never expires.

After you go though login, it is just a matter of sending the list of the records to delete. Not so easy, but possible

cheers
-JO
Quote Reply
Re: delete records after a specific time? In reply to
I did a little mod that still requires the user to click a link in order to delete, but might be helpful.

First, there's a new line in sub main of db.cgi:

elsif ($in{'auto_delete'}) { if ($per_admin) { &auto_delete; } else { &html_unauth; } }

Then I added a new subroutine -- sub auto_delete

Code:
sub auto_delete {
$days = 30;
$time = time() - ($days * 86400);

my ($sec, $min, $hour, $day, $mon, $year, $dweek, $dyear, $daylight) = localtime($time);
my (@months) = qw!Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec!;
($day < 10) and ($day = "0$day");
$year = $year + 1900;
$in{'Date-lt'} = $day-$months[$mon]-$year";

open(FILE, "< $db_file_name") or &cgierr("can't open $db_file_name: $!");
$count++ while <FILE>;
close FILE;

$in{'mh'} = $count;

my ($status,@hits) = &query("mod");
for (0 .. $db_total_hits - 1) {
%tmp = &array_to_hash($_, @hits);
$in{$tmp{$db_key}} = "delete";
}

&delete_records;
}

The link to call the subroutine is

print qq|<a href="$db_script_link_url&auto_delete=1">
Delete old records</a>
|;

You could, if you wanted, have it delete whenever you log on. At the beginning of html_home, before &html_print headers, just include a line

&auto_delete;

and it will run the subroutine every time you go to the html_home page. You'd probably want to do something about the delete_success and delete_failure subroutines, since you probably wouldn't want to see them every time.




------------------
JPD







[This message has been edited by JPDeni (edited March 24, 1999).]
Quote Reply
Re: delete records after a specific time? In reply to
I think this work, every time, the whole programm is started, so every time, SOMEONE logs in, the autodelete thing happened. If I am wrong I will see this.

JPD,
thank you
Quote Reply
Re: delete records after a specific time? In reply to
I think this work, every time, the whole programm is started, so every time, SOMEONE logs in, the autodelete thing happened. If I am wrong I will see this.

JPD,
thank you
Quote Reply
Re: delete records after a specific time? In reply to
That's true, Mart. I didn't think about it that way. If you want it to do that, though, you'll really need to change some stuff in delete_records because you don't want everyone to get the "records deleted" message. Or you might copy the code from delete_records into the auto_delete subroutine.



------------------
JPD





Quote Reply
Re: delete records after a specific time? In reply to
Can you come back, after sub auto_delete in the sub html_home, and skip the delete_succes sub, so that no one notice, that something is deleted?

The normally deleting of own records by users must be intact.
Again, thanks.
Quote Reply
Re: delete records after a specific time? In reply to
Sure. You'll need to copy code from delete_records into the auto_delete subroutine.

Use the auto_delete subroutine I already gave above, except don't use the line
&delete_records

Instead, use the following:

Code:
my ($key, %delete_list, $rec_to_delete, @lines, $line, @data, $errstr, $succstr, $output, $restricted);
foreach $key (keys %in) {
if ($in{$key} eq "delete") {
$delete_list{$key} = 1;
}
}
open (DB, "<$db_file_name") or &cgierr("error in delete_records.
unable to open db file: $db_file_name.\nReason: $!");
if ($db_use_flock) { flock(DB, 1); }
@lines = <DB>;
close DB;
LINE: foreach $line (@lines) {
if ($line =~ /^$/) { next LINE; }
if ($line =~ /^#/) { $output .= $line; next LINE; }
chomp ($line);
@data = &split_decode($line);
$delete_list{$data[$db_key_pos]} ?
($delete_list{$data[$db_key_pos]} = 0) :
($output .= $line . "\n");
}
open (DB, ">$db_file_name") or &cgierr("error in delete_records. unable to open db file:
$db_file_name.\nReason: $!");
if ($db_use_flock) {
flock(DB, 2) or &cgierr("unable to get exclusive lock on $db_file_name.\nReason: $!");
}
print DB $output;
close DB;
&html_home;

------------------
JPD







[This message has been edited by JPDeni (edited March 24, 1999).]
Quote Reply
Re: delete records after a specific time? In reply to
I do not understand, what you mean with this part:
---Sure. You'll need to copy code from delete_records into the auto_delete subroutine.---
Maybe you coult explain??
Quote Reply
Re: delete records after a specific time? In reply to
The code I included in my last post is copied from the delete_records subroutine, although edited a bit. It needs to go into the auto_delete routine in order to keep the user from being taken to html_delete_success or html_delete_failure after the records are deleted.

I've been thinking about this and I see a potential problem with putting this into html_home because every time a user goes to the home page it's going to go through the routine. It could slow things down and will definitely be redundant. It might be better to run the subroutine whenever someone logs on, rather than every time they go to the home page. If you get more than one visitor a day, it will still be redundant, but not as bad.

Probably a good place to put it would be in auth.pl. Just after

open(AUTH, ">$auth_dir/$db_uid") or &cgierr("unable to open auth file: $auth_dir/$uid. Reason: $!\n");
print AUTH "$uid: $ENV{'REMOTE_HOST'}\n";
close AUTH;

add

&auto_delete;

And just so things are clear, I'll post the entire auto_delete subroutine. It can go anywhere in db.cgi -- you could just paste it onto the end of the script.

Code:
sub auto_delete {
$days = 30;
$time = time() - ($days * 86400);
my ($sec, $min, $hour, $day, $mon, $year, $dweek, $dyear, $daylight) = localtime($time);
my (@months) = qw!Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec!;
($day < 10) and ($day = "0$day");
$year = $year + 1900;
$in{'Date-lt'} = "$day-$months[$mon]-$year";
open(FILE, "< $db_file_name") or &cgierr("can't open $db_file_name: $!");
$count++ while <FILE>;
close FILE;
$in{'mh'} = $count;
my ($status,@hits) = &query("mod");
for (0 .. $db_total_hits - 1) {
%tmp = &array_to_hash($_, @hits);
$in{$tmp{$db_key}} = "delete";
}
my ($key, %delete_list, $rec_to_delete, @lines, $line, @data, $errstr, $succstr, $output, $restricted);
foreach $key (keys %in) {
if ($in{$key} eq "delete") {
$delete_list{$key} = 1;
}
}
open (DB, "<$db_file_name") or &cgierr("error in delete_records.
unable to open db file: $db_file_name.\nReason: $!");
if ($db_use_flock) { flock(DB, 1); }
@lines = <DB>;
close DB;
LINE: foreach $line (@lines) {
if ($line =~ /^$/) { next LINE; }
if ($line =~ /^#/) { $output .= $line; next LINE; }
chomp ($line);
@data = &split_decode($line);
$delete_list{$data[$db_key_pos]} ? ($delete_list{$data[$db_key_pos]} = 0) :
($output .= $line . "\n");
}
open (DB, ">$db_file_name") or &cgierr("error in delete_records. unable to open db file:
db_file_name.\nReason: $!");
if ($db_use_flock) {
flock(DB, 2) or &cgierr("unable to get exclusive lock on $db_file_name.\nReason: $!");
}
print DB $output;
close DB;
}

Notice that I took out the line
&html_home;

That's because, if you run the subroutine from auth.pl, it will take you to html_home anyway.

One thing to remember. I have not tested this, except that I'm pretty sure there aren't any syntax errors. Make up some data to test it on first, or at least be sure you back up your database to be sure it will do what you want it to do.


------------------
JPD





Quote Reply
Re: delete records after a specific time? In reply to
Thank you, but I think it will not work, I wonder, what the number of days is, a record is in the database, before it will be automaticly deleted, is it 30? I modify a record to 10-Jan-1999 and log-in, but the record was still there. What could be the reason?? I realy do not know...
Quote Reply
Re: delete records after a specific time? In reply to
How about:

Code:
sub auto_delete {
# ------------------------------------------
# Automatically removes entries older then $remove
# days old.
#
my $remove = 30; # Number of days old.
my $date_field = 2; # Position of date field.

my $today = &date_to_unix(&get_date);
my $removeby = $today - ($remove * 86400);

my (@lines, @values);

open (DB, $db_file_name) or &cgierr ("Can't open: $db_file_name. Reason: $!");
if ($db_use_flock) { flock (DB, 1); }
@lines = <DB>;
close DB;

open (DB, ">$db_file_name") or &cgierr ("Can't open: $db_file_name. Reason: $!");
if ($db_use_flock) { flock (DB, 2); }
foreach (@lines) {
next if /^#/;
next if /^\s*$/;
chomp;
@values = &split_decode ($_);
if ($removeby > &date_to_unix($values[$date_field])) {
next;
}
print DB $_, "\n";
}
close DB;
}

This is untested, but is a little quicker as it only goes through the database file once. Just save the subroutine at the end of db.cgi, then call &auto_delete whenever you want to prune the database. I would suggest not doing it every time, but maybe only once a user logs in (so add it in just before &html_home).

Hope that helps,

Alex
Quote Reply
Re: delete records after a specific time? In reply to
Cool! So much nicer when you really know what you're doing instead of my cut-and-paste stuff. Smile



------------------
JPD





Quote Reply
Re: delete records after a specific time? In reply to
I don't know. You might try it and let us know.


------------------
JPD





Quote Reply
Re: delete records after a specific time? In reply to
You could have a field that is "removeby" which contains the date the record should be removed. (This would be a lot easier to do than having the number of days that the record will stay.)

Using Alex's much better auto_delete, it would be
Code:
sub auto_delete {
# ------------------------------------------
# Automatically removes entries older than
# a predetermined date
#
my $date_field = 2; # Position of date-to-remove field.
my $today = &date_to_unix(&get_date);
my (@lines, @values);
open (DB, $db_file_name) or &cgierr ("Can't open: $db_file_name. Reason: $!");
if ($db_use_flock) { flock (DB, 1); }
@lines = <DB>;
close DB;
open (DB, ">$db_file_name") or &cgierr ("Can't open: $db_file_name. Reason: $!");
if ($db_use_flock) { flock (DB, 2); }
foreach (@lines) {
next if /^#/;
next if /^\s*$/;
chomp;
@values = &split_decode ($_);
if ($today > &date_to_unix($values[$date_field])) {
next;
}
print DB $_, "\n";
}
close DB;
}



------------------
JPD





Quote Reply
Re: delete records after a specific time? In reply to
The story goes on...

Do you think this will work??

code:
---------------------------------------------

sub auto_delete {
# ------------------------------------------# Automatically removes entries older then $remove# days old.#
my $remove_field = 3; #Position of the field with number of days old.
my $date_field = 2; #Position of date field.
my $today = &date_to_unix(&get_date);
my $removeby = $today - ($remove_field * 86400);
my (@lines, @values);
open (DB, $db_file_name) or &cgierr

--rest of the code...

[This message has been edited by mart (edited March 27, 1999).]

[This message has been edited by mart (edited March 27, 1999).]
Quote Reply
Re: delete records after a specific time? In reply to
Thank you, this works. But I will go one step further: Can a user, by adding a record, fill the $remove and save that in a field, so that the user can choose, how long his record stays in the db?, or am I going too far...

Anyway, with this part I am satisfied.

[This message has been edited by mart (edited March 27, 1999).]
Quote Reply
Re: delete records after a specific time? In reply to
JPD,
I've done the following, wait for 3 days, (the filled value), but when I startup and login, nothing was deleted. Could you see why???

sub auto_delete {
# ------------------------------------------
# Automatically removes entries older then $remove# days old.#

my $remove_field = 18; #Position of the field with number of days old.
my $date_field = 2; #Position of date field.
my $today = &date_to_unix(&get_date);
my $removeby = $today - ($remove_field * 86400);
my (@lines, @values);
open (DB, $db_file_name) or &cgierr

--rest of the code...



[This message has been edited by mart (edited March 30, 1999).]
Quote Reply
Re: delete records after a specific time? In reply to
I've got it work now, but it deletes all the records, no matter, what the value is of my number of days old variable. I wonder, does the var# removeby_field must have the same name as the field in the .cfg file? I mean, have I rename my field in the .cfg in -removeby- ? I think it is not nessesary, but I'm not sure.

Is it possible, that the script first delete all records and than, when there comes new records in, the autodelete fuction started. This is what happens now. It have deleted all mij old records, however the delete time was not come yet, and I add some new ones, they will stay in the DB, I think, until the deleting time comes.

If this make sence, I'll be glad, when you can tell me. Otherwise I'd like to see, what is wrong.

Thank you

[This message has been edited by mart (edited March 31, 1999).]
Quote Reply
Re: delete records after a specific time? In reply to
Code:
my $remove_field = 18; #Position of the field with number of days old.
my $date_field = 2; #Position of date field.
my $today = &date_to_unix(&get_date);
my $removeby = $today - ($remove_field * 86400);

This code makes it remove records that are 18 days old. If you look at the $removeby line, it is inserting the value of the variable $remove_field, not the value of the field in each record.

I'll try to edit the routine to get it to do what you want.

Code:
sub auto_delete {
# ------------------------------------------
# Automatically removes entries older than
# a predetermined date
#
my $removeby_field = 18;
my $today = &date_to_unix(&get_date);
my (@lines, @values);
open (DB, $db_file_name) or &cgierr ("Can't open: $db_file_name. Reason: $!");
if ($db_use_flock) { flock (DB, 1); }
@lines = <DB>;
close DB;
open (DB, ">$db_file_name") or &cgierr ("Can't open: $db_file_name. Reason: $!");
if ($db_use_flock) { flock (DB, 2); }
foreach (@lines) {
next if /^#/;
next if /^\s*$/;
chomp;
@values = &split_decode ($_);
if ($today > ($today -(86400 * $values[$removeby_field])) {
next;
}
print DB $_, "\n";
}
close DB;
}

Completely untested, but I think it should probably work.

------------------
JPD





Quote Reply
Re: delete records after a specific time? In reply to
Help,...
I did the mod in my db.cgi and the script gives an internal server error, but not only with this script, also with two or three other (same) reserve scripts. How is this possible??!!
Quote Reply
Re: delete records after a specific time? In reply to
I missed a typo. This might be the cause of your problem.

if ($today > ($today -(86400 * $values[$removeby_field])) {

should be

if ($today > ($today -(86400 * $values[$removeby_field]))) {



------------------
JPD





Quote Reply
Re: delete records after a specific time? In reply to
Debugging 101: Let's see what's actually happening with some use of print statments:

Code:
sub auto_delete {
# ------------------------------------------
# Automatically removes entries older than
# a predetermined date
#
my $removeby_field = 18;
my $today = &date_to_unix(&get_date);
my (@lines, @values);
print "Content-type: text/html\n\n <PRE>"; # Print headers.
open (DB, $db_file_name) or &cgierr ("Can't open: $db_file_name. Reason: $!");
if ($db_use_flock) { flock (DB, 1); }
@lines = <DB>;
close DB;

open (DB, ">$db_file_name") or &cgierr ("Can't open: $db_file_name. Reason: $!");
if ($db_use_flock) { flock (DB, 2); }
foreach (@lines) {
next if /^#/;
next if /^\s*$/;
chomp;
@values = &split_decode ($_);
print "Comparing: '$today' vs '$values[$removeby_field]' ... \n";
if ($today > ($today -(86400 * $values[$removeby_field])) {
print "Deleting\n";
next;
}
print DB $_, "\n";
}
close DB;
}

and run it. You should see it go through and see what is getting compared to what and which records DBMan decides to delete.

Hope this helps,

Alex
Quote Reply
Re: delete records after a specific time? In reply to
Thank you, I think this works fine now...
Quote Reply
Re: delete records after a specific time? In reply to
Just as I was about to go to sleep, this thread crossed my mind and I realized why Mart is having problems.

In order to have a field where you set the number of days before you delete a record, you also have to have a field where you set the date the record was added.

After

my $removeby_field = 18;

add

my $dateadded_field = 19; # change to the correct field number

And change

Code:
if ($today > ($today -(86400 * $values[$removeby_field]))) {

to

Code:
if ($today > (&date_to_unix($values[$dateadded_field]) + (86400 * $values[$removeby_field]))) {




------------------
JPD





> >