RubyGems Navigation menu

cobweb 1.2.1

Cobweb is a web crawler that can use resque to cluster crawls to quickly crawl extremely large sites which is much more performant than multi-threaded crawlers. It is also a standalone crawler that has a sophisticated statistics monitoring interface to monitor the progress of the crawls.

Gemfile:
=

instalar:
=

Versões:

  1. 1.2.1 January 09, 2021 (9,28 MB)
  2. 1.2.0 February 25, 2019 (9,28 MB)
  3. 1.1.0 November 06, 2015 (9,28 MB)
  4. 1.0.29 November 05, 2015 (9,28 MB)
  5. 1.0.28 March 20, 2015 (9,28 MB)
Mostrar todas as versões (94)

Runtime Dependencies (9):

addressable >= 2.3.8
haml >= 4.0.7
json >= 1.8.3
nokogiri >= 1.6.0
rake >= 0
redis >= 3.2.1
redis-namespace >= 1.6.0
sinatra >= 1.4.6
slop >= 4.2.0

Development Dependencies (7):

bundle-audit >= 0
coveralls >= 0
mock_redis >= 0
rspec >= 0
rspec-core >= 0
sidekiq >= 0
thin >= 0
Mostrar todas as dependências

Donos:

Pushed by:

Autores:

  • Stewart McKee

SHA 256 checksum:

=
← Previous version

Total de downloads 332.461

Desta versão 2.885

Versão lançada:

Licença:

MIT

Versão Requerida do Ruby: >= 0

Links:

Homepage Documentação Download Review changes Badge Inscrever-se RSS Denunciar Abuso Dependências Reversas