Actually, I just got done doing that a few weeks ago. It isn't that difficult. I found that the easiest way was to just download all of the files (through FTP) that I already had (since I knew they all worked). I went through all of the files and changed server specific information from my old ISP to my new ISP. The most notable things are: the path to Perl, the path to your files, and where your CGI files need to reside.
If you don't want to do that, you can grab your links.db and categories.db and that will save all of your actual links and categories. As you probably know, the site_html.pl and the links.cfg contain all of your customizations assuming you didn't do any custom hacks. If you did do custom hacks, your best bet is to keep all of your original code and alter it for a new server.
