You can easily run the scraper in Gitpod, a browser-based development environment. Set it up in just 5 minutes by following these steps:
-
Visit this link and sign up using your GitHub account.
-
Once Signed Up, Open it in Gitpod.
-
In the terminal, run the following command to start scraping:
python main.py
-
Once the scraper has finished running, download the leads from the
outputfolder.
Just like with a local setup, you can configure the scraper in Gitpod by editing the main.py file and running the scraper using the python main.py command.
Also, it's important to regularly interact with the Gitpod environment, such as clicking within it every 30 minutes, to keep the machine active and prevent automatic shutdown.
If you don't want to click every 30 minutes, then we encourage to install Python on PC and run the scraper locally.
No, the scraper is very light on resources, so there won't be any noticeable difference whether you run it on an Intel Pentium processor with 4GB of RAM or an Intel i7 processor with 16GB of RAM.
Set the scrape_reviews argument to true.
queries = [
"web developers in bangalore"
]
Gmaps.places(queries, scrape_reviews=True)You can also configure the following options:
- reviews_max (Defaults: 20)
- reviews_sort (Defaults: Newest)
- Options are Most Relevant, Newest, Highest Rating, Lowest Rating
Here is an example that scrapes the Lowest Rated, 100 Reviews.
queries = [
"web developers in bangalore"
]
Gmaps.places(queries, scrape_reviews=True, reviews_max=100, reviews_sort=Gmaps.LOWEST_RATING)To scrape all reviews without any limit, you can set reviews_max to Gmaps.ALL_REVIEWS.
queries = [
"web developers in bangalore"
]
Gmaps.places(queries, scrape_reviews=True, reviews_max=Gmaps.ALL_REVIEWS)Please understand that some places may have thousands of reviews, so scraping all reviews may take a long time.
It's best to limit the number of reviews to scrape to a number like 100 or 1000.
Important: If you are a Data Scientist focused on scraping reviews for Data Analysis, we encourage you to use our Google Maps Reviews Scraper, as it is specially tailored for Data Scientists.
Seeing a lot of fields can be intimidating, so we have only kept the most important fields in the output.
However, you can select from upto 45+ fields.
To select all fields, use the following code:
queries = [
"web developers in bangalore"
]
Gmaps.places(queries, fields=Gmaps.ALL_FIELDS)To select specific fields only, use the following code:
queries = [
"web developers in bangalore"
]
fields = [
Gmaps.Fields.PLACE_ID,
Gmaps.Fields.NAME,
Gmaps.Fields.MAIN_CATEGORY,
Gmaps.Fields.RATING,
Gmaps.Fields.REVIEWS,
Gmaps.Fields.WEBSITE,
Gmaps.Fields.PHONE,
Gmaps.Fields.ADDRESS,
Gmaps.Fields.LINK,
]
Gmaps.places(queries, fields=fields)Please note that selecting more or fewer fields will not affect the scraping time; it will remain exactly the same.
So, don't fall into the trap of selecting fewer fields thinking it will decrease the scraping time, because it won't.
See the list of all supported fields here
- Scrape 100 Newest Google Maps Places Reviews of a Country and Store the result as JSON.
from botasaurus import bt
queries = Gmaps.Cities.India("your_target in")
scraped_places = Gmaps.places(queries, scrape_reviews=True, reviews_max=100)
bt.write_json(result, "scraped_places.json")- Read the Data for Panda Analysis
from botasaurus import bt
scraped_places = bt.read_json("scraped_places.json")
# Do whatever you want with scraped_placesPass the lang argument.
For example, if you want to scrape results in Spanish, you can do so by passing lang="Gmaps.Lang.Spanish".
queries = [
"web developers in bangalore"
]
Gmaps.places(queries, lang=Gmaps.Lang.Spanish)All Google Maps languages are supported. Some popular languages you may want to use are:
- Gmaps.Lang.Spanish
- Gmaps.Lang.English
- Gmaps.Lang.Japanese
- Gmaps.Lang.German
- Gmaps.Lang.Italian
See the list of all supported languages here
Use the links method to scrape information from specific Google Maps place links. Here's an example:
links = [
"https://www.google.com/maps/place/Zinavo-+Web+Development,+Web+Design+Company+in+Bangalore,+SEO+Services,+Digital+Marketing+Agency,+eCommerce+Web+Development/@13.01443,77.6480612,17z/data=!3m1!4b1!4m6!3m5!1s0x3bae172f3e7069f1:0xbcac5b2d393c2aa2!8m2!3d13.01443!4d77.6480612!16s%2Fg%2F11h0l3y9l?authuser=0&hl=en&entry=ttu",
"https://www.google.com/maps/place/SeekNEO+-+Web+Development,+Web+Design+Company+in+Bangalore,+eCommerce+Web+Development,+SEO+Services,+Digital+Marketing+Agency/@12.9863763,77.5473899,17z/data=!3m1!4b1!4m6!3m5!1s0x3bae13ac4bcc6641:0x1bf48a7dee3d5a51!8m2!3d12.9863763!4d77.5473899!16s%2Fg%2F11g2338zrl?authuser=0&hl=en&entry=ttu"
]
output_folder = "my-awesome-places"
Gmaps.links(links, output_folder)Provide the coordinates to the scraper as follows:
queries = [
"web developers in bangalore"
]
Gmaps.places(queries, geo_coordinates="12.900490, 77.571466")You can also adjust the zoom level, which will zoom in to the location before scraping:
- 1 (Whole Planet)
- 21 (House Level)
Zooming results in more places being scraped within a smaller area. Below is the same map at different zoom levels, yielding different numbers of results.
Image Credits: Apify
For example, to scrape places in Bangalore at zoom level 16:
queries = [
"web developers in bangalore"
]
Gmaps.places(queries, geo_coordinates="12.900490, 77.571466", zoom=16)We randomize cities for each user to increase the chances of sales.
If multiple users are targeting the same leads in the same cities, it reduces the opportunity for each.
Randomizing cities spreads them among our users, giving each user a better chance to make a sale.
No, you do not need proxies. You can scrape hundreds of thousands of leads without needing them.
Love It? Star It ⭐!
Become one of our amazing stargazers by giving us a star ⭐ on GitHub!
It's just one click, but it means the world to us.