Gossamer Forum: Products: Links 2.0: Customization: Export only URL's in txt-file

Dec 21, 2007, 12:57 AM

uhrwerk

Novice (43 posts)

Dec 21, 2007, 12:57 AM

Post #1 of 16

Shortcut

Export only URL's in txt-file

Hello,

Is it possible to build in a some art of extension which will export only URL's from my database in txt-file. I want to use it for exter URL-verifier. I have a timeout restriction problem on my server and would like to verify links using some application on my desktop.

thanks in advance.

Dec 21, 2007, 2:53 AM

Andy

Veteran / Moderator (18441 posts)

Dec 21, 2007, 2:53 AM

Post #2 of 16

Shortcut

Re: [uhrwerk] Export only URL's in txt-file In reply to

Hi,

Should be as simple as:

Code:
#!/usr/bin/perl 

print "Content-Type: text/html \n\n"; 

my $data_folder = '/path/to/your/admin/data'; 

open(READIT,"<$data_folder/url.db") || die qq|Cant read: $data_folder/url.db .  Reason: $!|; 
open(WRITEIT,">$data_folder/urls_basic.txt") || die qq|Cant write: $data_folder/urls_basic.txt .  Reason: $!|; 
  while (<READIT>) { 
     chomp; 
     my @split = split /\|/, $_; 
     print WRITEIT $split[1] . "\n"; 
  } 
close(WRITEIT); 
close(READIT); 

print "DONE!";

Obviously change the path to $data_folder [ Smile

Then, when run -you should get a new file called urls_basic.txt , with all the URL's in.

Hope that helps.

Cheers

Andy (mod)
andy@ultranerds.co.uk

Dec 21, 2007, 8:18 AM

uhrwerk

Novice (43 posts)

Dec 21, 2007, 8:18 AM

Post #3 of 16

Shortcut

Re: [Andy] Export only URL's in txt-file In reply to

Hi.thank you for your reply!. It works! :))))

Dec 21, 2007, 8:34 AM

Andy

Veteran / Moderator (18441 posts)

Dec 21, 2007, 8:34 AM

Post #4 of 16

Shortcut

Re: [uhrwerk] Export only URL's in txt-file In reply to

Hi,

Glad to hear it Smile

Cheers

Andy (mod)
andy@ultranerds.co.uk

Dec 21, 2007, 8:42 AM

uhrwerk

Novice (43 posts)

Dec 21, 2007, 8:42 AM

Post #5 of 16

Shortcut

Re: [Andy] Export only URL's in txt-file In reply to

Can you help me to simply modify it?

I want that the" urls_basic.txt" file willbe saved in some other directory . not in cgi-bin directory. I have modified it to following:

Code:
#!/usr/bin/perl  

print "Content-Type: text/html \n\n";  

my $data_folder = '/var/www/vhosts/domain.com/cgi-bin/links3/admin/data'; 
my $download = '/var/www/vhosts/domain.com/httpdocs/fileadmin';  

open(READIT,"<$data_folder/url.db") || die qq|Cant read: $data_folder/url.db .  Reason: $!|;  
open(WRITEIT,">$download/urls_basic.txt") || die qq|Cant write: $download/urls_basic.txt .  Reason: $!|;  
  while (<READIT>) {  
     chomp;  
     my @split = split /\|/, $_;  
     print WRITEIT $split[1] . "\n";  
  }  
close(WRITEIT);  
close(READIT);  

print "DONE!";

But it will be not saved here. Waht did I wrong here?

thanks in advance

Dec 21, 2007, 8:46 AM

Andy

Veteran / Moderator (18441 posts)

Dec 21, 2007, 8:46 AM

Post #6 of 16

Shortcut

Re: [uhrwerk] Export only URL's in txt-file In reply to

Hi,

Are you getting an error message? Most likely, its because you need to CHMOD the folder to 777 :)

Hope that helps.

Cheers

Andy (mod)
andy@ultranerds.co.uk

Dec 21, 2007, 8:51 AM

uhrwerk

Novice (43 posts)

Dec 21, 2007, 8:51 AM

Post #7 of 16

Shortcut

Re: [Andy] Export only URL's in txt-file In reply to

I don't getting an error message. It seems to work. because I have onyl see "Done". The folder is already chmod with 777

Dec 21, 2007, 8:53 AM

uhrwerk

Novice (43 posts)

Dec 21, 2007, 8:53 AM

Post #8 of 16

Shortcut

Re: [uhrwerk] Export only URL's in txt-file In reply to

I want to make it because I can't open this file in my browser. Because it will be saved in cgi-bin directory an I receive the following message trying opening it in my browser

Quote:

Internal Server Error

The server encountered an internal error or misconfiguration and was unable to complete your request.

Please contact the server administrator, ave@domain.de and inform them of the time the error occurred, and anything you might have done that may have caused the error.

More information about this error may be available in the server error log

Dec 21, 2007, 8:58 AM

Andy

Veteran / Moderator (18441 posts)

Dec 21, 2007, 8:58 AM

Post #9 of 16

Shortcut

Re: [uhrwerk] Export only URL's in txt-file In reply to

Hi,

Try adding this:

Code:
    use CGI::Carp qw(fatalsToBrowser);

..right after:

Code:
#!/usr/bin/perl

..so it looks like:

Code:
#!/usr/bin/perl 

    use CGI::Carp qw(fatalsToBrowser);

Hopefully that will give you the error message then.

Cheers

Andy (mod)
andy@ultranerds.co.uk

Dec 21, 2007, 9:03 AM

uhrwerk

Novice (43 posts)

Dec 21, 2007, 9:03 AM

Post #10 of 16

Shortcut

Re: [Andy] Export only URL's in txt-file In reply to

I made it. I receive only the messeage "Done" . But The file will not be saved in the fileadmin dircetory!

Dec 21, 2007, 9:04 AM

Andy

Veteran / Moderator (18441 posts)

Dec 21, 2007, 9:04 AM

Post #11 of 16

Shortcut

Re: [uhrwerk] Export only URL's in txt-file In reply to

Hi,

Try this:

Code:
#!/usr/bin/perl  

print "Content-Type: text/html \n\n";  

my $data_folder = '/var/www/vhosts/domain.com/cgi-bin/links3/admin/data'; 
my $download = '/var/www/vhosts/domain.com/httpdocs/fileadmin';  

open(READIT,"<$data_folder/url.db") || die qq|Cant read: $data_folder/url.db .  Reason: $!|;  
open(WRITEIT,">$data_folder/urls_basic.txt") || die qq|Cant write: $data_folder/urls_basic.txt .  Reason: $!|;  
  while (<READIT>) {  
     chomp;  
     my @split = split /\|/, $_;  
     print WRITEIT $split[1] . "\n";  
  }  
close(WRITEIT);  
close(READIT);  

system("mv $data_folder/urls_basic.txt $download/urls_basic.txt") || die $!; 
print qq|Moving from: $data_folder/urls_basic.txt to $download/urls_basic.txt  \n<br />|; 

print "DONE!";

Cheers

Andy (mod)
andy@ultranerds.co.uk

Last edited by:

Andy: Dec 21, 2007, 9:06 AM

Dec 21, 2007, 9:10 AM

uhrwerk

Novice (43 posts)

Dec 21, 2007, 9:10 AM

Post #12 of 16

Shortcut

Re: [Andy] Export only URL's in txt-file In reply to

thank YOU!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Smile

it works!!

mery christmas :)

have a nice day!

Dec 21, 2007, 9:11 AM

Andy

Veteran / Moderator (18441 posts)

Dec 21, 2007, 9:11 AM

Post #13 of 16

Shortcut

Re: [uhrwerk] Export only URL's in txt-file In reply to

hehe - np =) Merry Xmas to you too.

Cheers

Andy (mod)
andy@ultranerds.co.uk

Dec 21, 2007, 10:16 AM

Bobsie

Veteran / Moderator (3111 posts)

Dec 21, 2007, 10:16 AM

Post #14 of 16

Shortcut

Re: [uhrwerk] Export only URL's in txt-file In reply to

Are you aware that there is already a file in your admin/data directory (url.db) that contains a list of URLs? It is in the format of nnn|URL where nnn is the ID number of the resource link and URL is the recorded database URL. That file is updated every time your rebuild the links.

Dec 22, 2007, 4:54 AM

Andy

Veteran / Moderator (18441 posts)

Dec 22, 2007, 4:54 AM

Post #15 of 16

Shortcut

Re: [Bobsie] Export only URL's in txt-file In reply to

Hi,

Wow, long time no seed Bobsie =)

I think he was, but he needed it in format of

url
url
url

..as the checker he is using only allows for one URL per line, and nothing else (thus the script I wrote to extract the URL's :))

Cheers

Andy (mod)
andy@ultranerds.co.uk

Dec 22, 2007, 8:05 AM

SandraR

User (424 posts)

Dec 22, 2007, 8:05 AM

Post #16 of 16

Shortcut

Re: [uhrwerk] Export only URL's in txt-file In reply to

I use: http://gsitecrawler.com/

It lets you select all types of formats and the software is FREE.
I like it because you can filter the results and let it work in the background.

Good luck

Sandra Roussel
Chonsa Group Design - Fresh Start Housing