I would like these tags to be merged under "web-crawler". This is how it now stands.
web-crawler (1,073 questions).
crawl (130 questions).
crawler (344 questions).
spider (208 questions).
Tag wiki for spider says "Synonym for web-crawler tag".
closed as off-topic by Sompuperoo, Aziz Shaikh, animuson, Martijn Pieters, rene Nov 24 at 11:01
This question appears to be off-topic. The users who voted to close gave this specific reason:
The author of this recently-suggested edit agrees that the 'crawl' tag covers the same ground. I rejected the edit to avoid duplicate tags, but a cleanup is definitely needed. I plan to start retagging these as time allows and would welcome additional help.