Web data collection

Updated 10:00 UTC.

The most used open source project in the Web data collection category is ripienaar/free-for-dev (142,883 stars), followed by NanmiCoder/MediaCrawler and KeygraphHQ/shannon. There is no 30 day star data yet, so growth is not ranked (data as of 2026-11-15).

Most used

  1. ripienaar/free-for-dev142,883 stars
  2. NanmiCoder/MediaCrawler71,136 stars
  3. KeygraphHQ/shannon51,911 stars

Quick comparison

  • ripienaar/free-for-dev

    A list of SaaS, PaaS and IaaS offerings that have free tiers of interest to devops and infradev

    License No dataMaintenance: Active

  • NanmiCoder/MediaCrawler

    小红书笔记 | 评论爬虫、抖音视频 | 评论爬虫、快手视频 | 评论爬虫、B 站视频 | 评论爬虫、微博帖子 | 评论爬虫、百度贴吧帖子 | 百度贴吧评论回复爬虫 | 知乎问答文章|评论爬虫

    License Custom license: Check the licenseMaintenance: Active

  • KeygraphHQ/shannon

    Shannon is an AI pentester for web applications and APIs. It analyzes your source code, identifies attack vectors, and executes real exploits to prove vulnerabilities before they reach production.

    License AGPL-3.0: Yes, with conditionsMaintenance: Active

  • NaiboWang/EasySpider

    A visual no-code/code-free web crawler/spider易采集:一个可视化浏览器自动化测试/数据采集/网页爬虫软件,可以无代码图形化的设计和执行爬虫任务。别名:ServiceWrapper面向Web应用的智能化服务封装系统。

    License AGPL-3.0: Yes, with conditionsMaintenance: Active

  • Usagi-org/ai-goofish-monitor

    基于 Playwright 和AI实现的闲鱼多任务实时/定时监控与智能分析系统,配备了功能完善的后台管理UI。帮助用户从闲鱼海量商品中,找到心仪产品。

    License MIT: YesMaintenance: Active

  • Not legal advice
Not legal advice
ProjectWhat it isLicenseMaintenance
ripienaar/free-for-devA list of SaaS, PaaS and IaaS offerings that have free tiers of interest to devops and infradevNo dataActive
NanmiCoder/MediaCrawler小红书笔记 | 评论爬虫、抖音视频 | 评论爬虫、快手视频 | 评论爬虫、B 站视频 | 评论爬虫、微博帖子 | 评论爬虫、百度贴吧帖子 | 百度贴吧评论回复爬虫 | 知乎问答文章|评论爬虫Custom license: Check the licenseActive
KeygraphHQ/shannonShannon is an AI pentester for web applications and APIs. It analyzes your source code, identifies attack vectors, and executes real exploits to prove vulnerabilities before they reach production.AGPL-3.0: Yes, with conditionsActive
NaiboWang/EasySpiderA visual no-code/code-free web crawler/spider易采集:一个可视化浏览器自动化测试/数据采集/网页爬虫软件,可以无代码图形化的设计和执行爬虫任务。别名:ServiceWrapper面向Web应用的智能化服务封装系统。AGPL-3.0: Yes, with conditionsActive
Usagi-org/ai-goofish-monitor基于 Playwright 和AI实现的闲鱼多任务实时/定时监控与智能分析系统,配备了功能完善的后台管理UI。帮助用户从闲鱼海量商品中,找到心仪产品。MIT: YesActive

All tools for this problem

5 projects, by 30 day growth, then stars.

  1. ripienaar/free-for-dev

    A list of SaaS, PaaS and IaaS offerings that have free tiers of interest to devops and infradev

    142,883 stars+660 in 7 daysMaintenance: Active

  2. NanmiCoder/MediaCrawler

    小红书笔记 | 评论爬虫、抖音视频 | 评论爬虫、快手视频 | 评论爬虫、B 站视频 | 评论爬虫、微博帖子 | 评论爬虫、百度贴吧帖子 | 百度贴吧评论回复爬虫 | 知乎问答文章|评论爬虫

    71,136 stars+502 in 7 daysMaintenance: Active

  3. KeygraphHQ/shannon

    Shannon is an AI pentester for web applications and APIs. It analyzes your source code, identifies attack vectors, and executes real exploits to prove vulnerabilities before they reach production.

    51,911 stars+482 in 7 daysMaintenance: Active

  4. NaiboWang/EasySpider

    A visual no-code/code-free web crawler/spider易采集:一个可视化浏览器自动化测试/数据采集/网页爬虫软件,可以无代码图形化的设计和执行爬虫任务。别名:ServiceWrapper面向Web应用的智能化服务封装系统。

    44,797 stars+27 in 7 daysMaintenance: Active

  5. Usagi-org/ai-goofish-monitor

    基于 Playwright 和AI实现的闲鱼多任务实时/定时监控与智能分析系统,配备了功能完善的后台管理UI。帮助用户从闲鱼海量商品中,找到心仪产品。

    19,963 stars+1,604 in 7 daysMaintenance: Active