Java Web Crawler is a simple Web crawling utility written in Java. It supports the robots exclusion standard.
- Category: Crawlers
- License: Other
- HomePage: http://java.sun.com/developer/technicalArticles/ThirdParty/WebCrawler/
