I have recently discovered that my ISP's search engine (custom built) automatically searches Altavista and brings the results into their search engine and are displayed as if it was a totally seperate search engine.

I think it would be a good idea to be able to search in links then click a link where once clicked, links goes to Altavista then pulls the results without all of altavisa's formatting, banners, etc thus showing the results the way YOU want them to look and show YOUR banners.

Has this been done? If so please let me know how I can find it.

Thanks

------------------

This Is Doable, But The Legal Aspects & Writing The CGI To Strip Away AV Info Need To Be Looked Into Also.

DM

I wouldn't advise it.. Most of these search engines don't like people using their database and reformatting the output. It's easy enough to do though:

use LWP::Simple;
my $search_term = &url_encode($in{'query'});
my $av_results = get ("http://www.altavista.com/cgi-bin/query?q=$search_term");

Now you just have to strip out Alta Vista's html (note: &url_encode is a function taken from db.pl).

Cheers,

Alex

Hey,

Where must it writen in ?

C'ya
Martin Ackermann

------------------
http://www.schmetterling.nu

Yeah, I also want to know where exactly do we add that line Alex wrote to?

What other codes/lines must be included in order to strip out AltaVista's headers and footers?

Please let us know step-by-step on how to complete this excellent MOD feature.

Thanks,

John

[This message has been edited by jmurphy (edited December 06, 1998).]

As was mentioned uptopic, it's not a good idea to use someone else's content without their approval -- especially if you're taking out their formatting (ie. "brand name") and banners (ie. "income"). I don't know about AV, but InfoSeek and YAHOO have lots of disclaimers along those lines.

There IS one way to do it that apparantly the Search Engines don't have a problem with -- get one of those CGI scripts that do a world search on any or all of a number of search engines, and incorporate the input form into your Links search page (or vice versa).

I've seen this done on various non-Links sites, and the results come back to their site (keeping their site's style but leaving in the money-makers for whatever search engine is pirated.) If that's not legal, there are a lot of reputable sites out there breakin' de law<g>.

The question still remains... How do you get the results - searched from AltaVista for instance - to appear on your page instead of theirs?

helpnet, can you point us to such a program and how do we incorporate the results back to Links???

John

Anybody knows how to get this incorporated into Links???

Thanks!

John

I know some of you want to do this, but I don't see how it would work. I mean for instance when you search your directory you see the full describsion, but if you search Alta Vista you see this for an example.
Welcome to my page
my page contains all of the net into one...... I don't know about how you feel but if I was one of your vistors I would think somthing was up if I seen some had just a few words and then dots meaning that the describsion tag was a bit to long, and some had the full describsions no matter how long they was. Sorry if I made any of you mad by this, but I'm just saying what I think.
Thanks
Joey Bost
jbost@sheltonbbs.com

Alex, The code above, where can I insert it? I own two engines and one of them isn't links.

If you have to, email me.

Ed

As long as your goal is not to pirate a SEs content for your benefit w/o helping the SE, (using other programmers' skill for free to boot), here's another solution. Looksmart has a program to incorporate their results into your site.

------------------

They themselves - LooksSmart - is pulling search results from other search engines too. So are they also "pirating" other SE?

And what program is that may I ask?

John

Alex, let me know too by email.

Thanks!

John

I, Also, Would Like A Look At The Code -- If Possible.

Thanx

DM

This would be easy to do even w/o the LWP module....
I have done it with sockets before which is slower but just as good.
I would take alot of work to get the HTML the way you wanted it thought...

thanks,
Vivitron

It seems as if there is alot of confusion about what I am after. To have a look at my ISP's Altavista Searcher at http://www.samara.co.zw/cgi-bin/kangela.pl.

The overall idea that I have is that someone searches on my site for something. If it's not there or the user wants more results there is a link to "Search Altavista for Searchterm". The user is then presented with the altavista results but all within my design.

It sounds very simple to me when I look at the overall picture but from a programming point of view I am hopelessly useless.

I do hope that this will one day be possible. Thanks for your help though.

Regards
Mark Gilbert

------------------

i'm working on a script that will meta search alta vista, webcrawler, and hotbot.

john

[This message has been edited by jmurphy (edited December 12, 1998).]

Is it legal to search the other's database and print the results on your pages with your own style?

Thanks

I would think so.

Everybody is doing it and I don't see why the "little" guys can't do it.

John

Rock asked if it was legal to "brand" search engine results as your own. Each search engine licenses their product slightly differently but in general it's illegal to strip out all of the search engine's identifying characteristics and include the results as part of your own site.

jmurphy asked why the big guys can do it but the little guys can't. The answer is generally money. They bug guys spend large sums to obtain a site license for a particular search engine.

Okay, I am beginning to understand a little more about where this is leading. As far as searching Altavista and "Branding" I don't think this is the case. Although ISP has done this I would like to just chop of all the fancy stuff (banners, advertising, table outline, etc) and take the search results in one go and dump them on a page generated on the fly.

I would think this way I am not exactly "branding" their product as mine because all the results format is origional. I suppose it's a script that can strip formatting tags and dump the results.

The way I see it seems too complex for me to ever achieve but one day...
Here's the "formula" that I have...

1) User comes to site and searches
2) Gets some/no results but within the page will be a link saying "Can't find what you are looking for? Click Here to search Altavista".
3) Link points to program that grabs Altavista's page, strips all the banners up to the first result and after the last result on the page and creates a page on the fly (of which I have a certain amount of control over design) and dumps the results as they appear on Altavista (so the visitor does not feel lost).
4) User finds what he/she is after and goes to the site.
5) Once the script has outputed the results it scraps Altavista's page to save on space.

The biggest advantage that I think I have is that the user knows they are on my site but they have Altavista search engine to back them up. They will then continuously come back first looking on mine and then looking on Altavista.

Maybe it is illegal but I haven't read it anywhere concerning Altavista. I understand their banners are not being shown but my site doesn't get that many visitors on a daily basis for them to even worry about it. And to top it their formatting of the text remains the same.

If you think it is just too much to go into then I do understand. Any alternative suggestions out there? What about "search altavista" with a link forcing a new window. I don't have a clue how to do this because I know they have alot of routine call text in their search terms of which I wouldn't be able to understand.

Thanks again.

Mark Gilbert

Hi Mark,

I prefer your first idea too. Unfortunately I am no coder. Your last idea is easy add. Here is the link you need to add to site_html-pl:

<A HREF="http://www.altavista.digital.com/cgi-bin/query?q=$term " target="_blank">Altavista search</a>

Hope that helps.

Regards


Michael

P.S.: You may do a search (it HAS to be unsuccessful!) at my site. After not finding anything I offer 9 search engines to search at. My site is at

http://www.surfmaschine.de/pool

It is in german but I think you will manage to do an unsuccessful search nevertheless. The seach form is on the startpage. Just type in something like "hjbasfhjk", then you will find yourself at the correct page.



[This message has been edited by Zoul (edited December 12, 1998).]

*Sigh*. Fine. Taken Alex's code above and add &text=yes to the search string. That'll strip out everything you want removed.

bcurnow, can you be more specific?

Show us an example.

Where do we put all this to?

use LWP::Simple;
my $search_term = &url_encode($in{'query'});
my $av_results = get
("http://www.altavista.com/cgi-bin/query?q=$search_term");

&text=yes

John

hello, how would i put the search term like Macintosh+computers when the user types in Macinotsh computers?