How to Scrape Millions of tweets with Python

HKN MZ
4 min readApr 11, 2021

Sometimes we need a users all tweets to csv file. But don’t have Twitter API or Twitter API doesn’t allow to scrape / download more than 3200 twees. That is why we will use snscrape library.

Snscrape is a scraper for social networking services (SNS). It scrapes things like user profiles, hashtags, or searches and returns the discovered items, e.g. the relevant posts. Here is the link. Not: snscrape requires Python 3.8 or higher.

Install snscrape package to your python enviroment . Just try to install snscrape with this command.

--

--

HKN MZ

I am writing about Sql Server, Elasticsearch and Python. İ am an Database Administrator on SQL Server and Elasticsearch more than 5+ years.