- Python 3.6
- Scrapy 1.4
- json
- pymysql
- redis
git clone https://github.com/Dengqlbq/ZhiHuSpider.git
Rewrite the POST_DATA, QUESTION_COUNT, ANSWER_COUNT_PER_QUESTION, ANSWER_OFFSET and Mysql information in settings.py
cd zhihu/zhihu
scrapy crawl zhihu
Note: Before you run the project, make sure that you have created tables match the requirement