Posted on Updated on
Online Reviews, Web Scraping

How to Scrape More than 100 Reviews from Amazon?

By Raunaq Singh

With the increasing importance of product reviews in influencing purchase decisions, the demand for scraping reviews from online marketplaces like Amazon has surged. However, Amazon recently limited the number of reviews one can view to just 100. But what if you need more data? In this article, we’ll explore a workaround using Unwrangle.com’s Amazon Product Reviews API and how you can potentially fetch up to 500 reviews.

Why the Limitation?

Amazon has implemented a limit, restricting users from viewing beyond the 10th page of product reviews, which equates to approximately 100 reviews. This has posed a challenge for researchers, marketers, and developers who rely on comprehensive review data for their projects.

Since mid July, 2023 Amazon displays 0 reviews for all products on its website beyond page 10.

The Workaround: Exploiting Filters

While it’s true that the direct view is limited, there’s a nifty trick you can employ. Amazon allows users to filter reviews based on different criteria, such as star ratings, helpfulness, or recency. Each filter can display up to 100 reviews, providing an avenue to extract a larger dataset.

Here’s how you can utilize Unwrangle.com’s API to leverage this:

1. Choose Your Product

Identify the Amazon product for which you need reviews and obtain its URL.

2. Initialize the API Call

Make a GET request to Unwrangle’s endpoint: /api/getter/?platform="amazon_reviews"

For instance, to fetch reviews for the latest iPhone, you’d use:

bashCopy codecurl -v -L 'https://data.unwrangle.com/api/getter/?platform=amazon_reviews&url=https%3A%2F%2Fwww.amazon.com%2FApple-iPhone-13-128GB-Blue%2Fdp%2FB09LNX6KQS%2F&api_key=API_KEY'

Note: Always ensure you add the -L option with curl to handle potential redirects.

3. Apply Filters

To the same product URL, apply Amazon’s review filters one by one and repeat the API call for each filter. This way, you can scrape reviews based on different star ratings, sorting them by helpfulness or recency.

Here are the query parameters you can utilize to apply filters:

  • filter_by_star: Specify the star rating (1-5) to filter reviews. E.g. all_stars, five_star, four_star, etc.
  • sort_by: Sort reviews by ‘recent’, ‘helpful’, etc.
  • page: Specify the page number to navigate through paginated results.

4. Compile the Data

Aggregate the results from each API call to have a comprehensive dataset of reviews.

In Conclusion

While Amazon’s new limitation might seem restrictive, with the right tools and methods, you can still access a rich dataset of reviews. Unwrangle.com’s Amazon Product Reviews API is a powerful ally in this quest, turning challenges into opportunities for richer data extraction.