crawlr 0.2.0

Crawlr is a modern Ruby web scraping framework built for respectful and efficient data extraction. Features async HTTP requests, robots.txt compliance, cookie management, proxy rotation, flexible CSS/XPath selectors, and comprehensive error handling. Designed for both simple scraping tasks and complex, large-scale data extraction projects.

Gemfile:
=

installation:
=

Versions:

0.2.3 October 03, 2025 (31,5 ko)
0.2.2 October 01, 2025 (31,5 ko)
0.2.1 September 30, 2025 (31 ko)
0.2.0 September 29, 2025 (31 ko)
0.1.0 September 29, 2025 (30,5 ko)

Dépendances de Runtime (5):

async ~> 2.32

async-http ~> 0.91.0

concurrent-ruby ~> 1.3

http-cookie ~> 1.0

nokogiri ~> 1.18

Dépendances de Development (5):

rake ~> 13.0

rspec ~> 3.0

rubocop ~> 1.60

ruby-prof ~> 1.7

webmock ~> 3.25

Show all transitive dependencies

Propriétaires:

Pushed by:

Auteurs:

Aristotelis Rapai

Total de contrôle SHA 256:

Total de téléchargements 1 011

Pour cette version 233

Version publiée:
September 29, 2025 10:40pm

License:
MIT

Version de Ruby requise: >= 3.1.0

New versions require MFA: true

Version published with MFA: true

Liens:

Page d'accueil Changelog Suivi de bugs Télécharger Review changes Badge RSS Signaler un abus Reverse dependencies