Wgit is a WWW indexer/scraper which crawls URL's and retrieves their page contents for later use. Also included in this package is a means to search indexed documents stored in a database. Therefore this library provides the main components of a WWW search engine. You can also use Wgit to copy entire website's HTML making it far more powerful than wget. The Wgit API is easily extendable allowing you to easily pull out the parts of a webpage that are important to you, the external links or keywords for example.
Required Ruby Version
~> 2.5
Authors
Michael Telford
Versions
- 0.12.1 August 18, 2025 (65.5 KB)
- 0.12.0 October 30, 2024 (65.5 KB)
- 0.11.0 January 19, 2024 (56.5 KB)
- 0.10.8 August 18, 2023 (51 KB)
- 0.10.7 December 01, 2022 (50.5 KB)
- 0.0.5 January 08, 2019 (22 KB)