http://crawler.archive.org/

(Via Manageability – Open Source Web Crawlers Written in Java)