crawlr 0.1.0

Crawlr is a modern Ruby web scraping framework built for respectful and efficient data extraction. Features async HTTP requests, robots.txt compliance, cookie management, proxy rotation, flexible CSS/XPath selectors, and comprehensive error handling. Designed for both simple scraping tasks and complex, large-scale data extraction projects.

Gemfile:
=

Installeer:
=

Versies:

0.2.3 October 03, 2025 (31,5 KB)
0.2.2 October 01, 2025 (31,5 KB)
0.2.1 September 30, 2025 (31 KB)
0.2.0 September 29, 2025 (31 KB)
0.1.0 September 29, 2025 (30,5 KB)

Runtime afhankelijkheden (5):

async ~> 2.32

async-http ~> 0.91.0

concurrent-ruby ~> 1.3

http-cookie ~> 1.0

nokogiri ~> 1.18

Development afhankelijkheden (5):

rake ~> 13.0

rspec ~> 3.0

rubocop ~> 1.60

ruby-prof ~> 1.7

webmock ~> 3.25

Show all transitive dependencies

Eigenaren:

Pushed by:

Authors:

Aristotelis Rapai

SHA 256 checksum:

Total downloads 1.045

Voor deze versie 181

Versie vrijgegeven:
September 29, 2025 12:09am

Licentie:
MIT

Required Ruby Version: >= 3.1.0

New versions require MFA: true

Version published with MFA: true

Links:

Startpagina Changelog Bug-tracker Download Review changes Badge RSS Report abuse Reverse dependencies