A Web spidering library: handles robots.txt, scraping, finding more links, and doing it all over again.
Required Ruby Version
None
Authors
John Nagro
Versions
- 0.7.0 September 08, 2025 (10 KB)
- 0.6.0 September 08, 2025 (10 KB)
- 0.5.4 June 04, 2020 (15.5 KB)
- 0.5.3 April 23, 2018 (15 KB)
- 0.5.2 April 23, 2018 (15 KB)
- 0.4.2 July 06, 2008* (20.5 KB)