i just used another few tables.. the main one just had the LinkID and the URL and first 500 characters... for searching..

then i addded code in search.cgi to search that..

then of course.. a perl spider that is a server process.. so it runs all the time..

problem is.. after about an hour.. you got a 200meg db..

Jerry Su