A next-generation crawling and spidering framework.
- Updated
Oct 6, 2025 - Go
#
web-spider
Here are 8 public repositories matching this topic...
The unix-way web crawler
go cli golang crawler web-crawler web-scraping pentesting pentest golang-application web-spider pentest-tool unix-way
- Updated
Sep 12, 2025 - Go
🕷️ An easy-to-use spider written in Golang. (previous named GOPA.)
- Updated
May 19, 2021 - Go
Antch, a fast, powerful and extensible web crawling & scraping framework for Go
- Updated
May 31, 2020 - Go
Performs useful checks against a website, such as 404 errors reporting, structured data validation...
- Updated
Dec 15, 2019 - Go
An experimental web crawler to visualise & map the connections between domains
- Updated
Sep 7, 2024 - Go
Go application. Crawls the Steam store and collects Game records based on search criteria.
- Updated
Sep 13, 2019 - Go
The Concurrent Web Crawler is a Go-based application designed to crawl web pages efficiently using Go's powerful concurrency features.
java scraper spider multithreading async-python aiohttp asyncio html-parsing warc webcrawler graph-generation web-spider concurrent-web-crawler pluggable-modules
- Updated
Aug 26, 2025 - Go
Improve this page
Add a description, image, and links to the web-spider topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the web-spider topic, visit your repo's landing page and select "manage topics."