You're supposed to put any of your 'private' directories in there to stop them being spidered, but then anyone can just bring the robots.txt file up in their browser and see what you're trying hide.
Would also like to hear if anyone has an answer that they use?
Ahh sorry, I thought you were talking generally about Robots.txt files. I didn't read it properly that you were having problems with the SPIDER plugin - I don't know the answer to that one.