Scrape Google Shopping Ads with Python in 2025

Understanding how your products and competitors’ offerings perform on Google Shopping Ads is essential for refining digital marketing strategies. Web scraping enables you to extract valuable data like product titles, prices, ratings, and ad positions directly from Google Shopping results. 

This tutorial provides a clear, step-by-step guide to scraping Google Shopping Ads using Python, incorporating proxy integration by OkeyProxy to handle anti-bot measures, and exporting data for analysis. 

Why Scrape Google Shopping Ads?

Scraping Google Shopping Ads (also called Product Listing Ads or PLAs) is a smart way for online sellers to learn more about the market and improve sales. It means using tools or code to collect information from Google Shopping search results such as product prices, sellers, and ad positions.

Monitor Competitor Actions

One of the main reasons for scraping these ads is to see what competitors are doing. For example, if a competitor suddenly lowers the price of a popular product or runs a limited-time discount, this can be spotted quickly. Knowing this helps decide how to price similar products or when to offer promotions.

Track Ad Performance

Scraping also shows how well ads are performing. For instance, tracking the position of a product in search results over time reveals whether ads are gaining more visibility or losing ground compared to others. This data supports improving ad content, like refining product titles or adjusting bids to increase exposure.

Discover Market Trends

Beyond this, scraping helps discover market trends by showing which products are appearing most frequently or gaining popularity during certain seasons. 

For example, an increase in ads for outdoor grills during spring can signal a seasonal trend, guiding inventory and marketing plans.

Protect Brand Integrity

For brand owners, scraping helps protect the brand. It can reveal unauthorized sellers offering discounted versions of products or breaking minimum advertised price policies. Detecting these violations early helps maintain pricing integrity and brand reputation.

Improve Product Listings

Scraping also provides ideas to improve product listings by learning from top sellers. Observing that the highest-ranking ads use clear, descriptive titles and high-quality images can inspire improvements in listing presentations to attract more customers.

Estimate Competitor Ad Spend

It helps estimate how much competitors might be spending on ads. For example, frequent appearances of certain products at the top of search results may indicate aggressive bidding strategies, suggesting the need to reconsider advertising budgets or keyword focus.

Save Time with Automation

Finally, scraping saves time by automating the process of checking ads. Instead of manually reviewing listings daily, automated scraping delivers real-time updates on price changes, stock availability, and new competitors entering the market.

Setup

Before starting, ensure you have:

 ● Python 3.x: Installed from Python.org.

 ● Libraries: Install requests, beautifulsoup4, pandas, and fake_useragent using: 

bash

pip install requests beautifulsoup4 pandas fake_useragent

 ● Proxy Service: Sign up for a proxy service like OkeyProxy and obtain your API key and proxy credentials.

 ● Text Editor/IDE: Use VS Code, PyCharm, or your preferred editor.

 ● Basic Python Knowledge: Familiarity with Python scripting and HTML parsing.

Step-by-Step Guide to Scraping Google Shopping Ads

Step 1: Set Up Your Environment 

1.  Create a Project Directory: Make a folder (e.g., google_shopping_scraper) for your script and data.

 a.  Navigate to it in your terminal or IDE.

 2.  Install Dependencies: Run the pip command above to install required libraries.

 a.  Confirm installations with pip list.

 3.  Configure Proxies: Log in to your OkeyProxy account (or your chosen proxy provider) and note your proxy server details (e.g., server:port, username, password).

 a.  Test proxy connectivity: 

bash

curl -x "http://username:password@server:port" https://httpbin.org/ip

Replace placeholders with your proxy credentials.

Step 2: Write the Scraping Script

Create a Python script (scraper.py) to fetch and parse Google Shopping Ads data. Below is a sample script using requests and BeautifulSoup with proxy integration.

python

import requests

from bs4 import BeautifulSoup

from fake_useragent import UserAgent

import pandas as pd

 

# Initialize User-Agent to mimic browser behavior

ua = UserAgent()

headers = {'User-Agent': ua.random}

 

# Proxy configuration

proxy = {

 'http': 'http://username:password@server:port',

 'https': 'http://username:password@server:port'

}

 

# Define the target URL for Google Shopping search

query = "wireless headphones"

url = f"https://www.google.com/search?tbm=shop&q={query.replace(' ', '+')}"

 

# Send HTTP request through proxy

response = requests.get(url, headers=headers, proxies=proxy)

response.raise_for_status() # Check for request errors

 

# Parse HTML content

soup = BeautifulSoup(response.text, 'html.parser')

 

# Extract product data

products = []

for item in soup.select('div.sh-dgr__content'):

 title = item.select_one('h3').text.strip() if item.select_one('h3') else 'N/A'

 price = item.select_one('span.a8Pemb').text.strip() if item.select_one('span.a8Pemb') else 'N/A'

 seller = item.select_one('div.aULzUe').text.strip() if item.select_one('div.aULzUe') else 'N/A'

 rating = item.select_one('span.Rsc7Yb').text.strip() if item.select_one('span.Rsc7Yb') else 'N/A'

 

 products.append({

 'Title': title,

 'Price': price,

 'Seller': seller,

 'Rating': rating

 })

 

# Print extracted data

for product in products:

 print(product)

Replace username, password, server, and port with your proxy credentials. Modify the query variable to target specific products.

Step 3: Extract Key Data Points

The script extracts:

 ● Product Title: Name of the product (e.g., “Sony WH-1000XM5 Headphones”).

 ● Price: Current price, including discounts.

 ● Seller: Retailer or platform offering the product.

 ● Rating: Customer ratings or reviews.

To find these elements:

 1.  Open Google Shopping in your browser and search for a product.

 2.  Use “Inspect” to view the HTML structure.

 3.  Identify CSS selectors (e.g., div.sh-dgr__content, span.a8Pemb) and update the script if Google’s HTML changes.

Step 4: Export Data for Analysis

Save the scraped data to a CSV file for analysis in tools like Excel or Tableau.

python

# Export to CSV

df = pd.DataFrame(products)

df.to_csv('google_shopping_data.csv', index=False)

print("Data exported to google_shopping_data.csv")

This creates a google_shopping_data.csv file with columns for Title, Price, Seller, and Rating. 

Use it to:

 ● Compare your product prices with competitors.

 ● Identify top sellers.

 ● Track customer sentiment via ratings.

Step 5: Integrate Proxies for Reliable Scraping

Google’s anti-bot systems may block repetitive requests from a single IP. Proxies help bypass these restrictions.

 1.  IP Rotation: Use a proxy service like OkeyProxy to rotate IPs automatically.

 a.  Update the proxy configuration: 

python

proxy = {

 'http': 'http://username:[email protected]:port',

 'https': 'http://username:[email protected]:port'

}

2.  Geo-Targeting:

 a.  Select proxies from specific regions to scrape localized results.

 b.  Configure via your proxy provider’s dashboard or API (e.g., country:US).

3.  Handle Rate Limits: Add delays to avoid triggering limits: 

python

import time

time.sleep(2) # 2-second delay

 a.  Throttling ensures compliance with Google’s terms. 

What is OkeyProxy?

OkeyProxy is a proxy service providing access to residential proxies for web scraping. It offers IPs across multiple countries, supports IP rotation, and ensures reliable data extraction. OkeyProxy is designed to bypass anti-bot systems, making it suitable for scraping tasks like Google Shopping Ads. 

You can explore its features and pricing at the proxy service page.

Step 6: Analyze and Optimize

Use the CSV data to:

 ● Monitor Competitors: Adjust pricing or bids based on competitors’ ad positions.

 ● Optimize Listings: Identify high-performing keywords or features.

 ● Track Trends: Analyze ratings to understand customer preferences.

Tools like Pandas or Power BI can visualize insights to refine your Google Ads strategy.

Comparison Table: Proxy Types for Scraping

Proxy Type	Advantages	Disadvantages	Best Use Case
Residential Proxies	High trust, low ban risk, mimics users	Slower, higher cost	Scraping Google Shopping, geo-targeted data
Datacenter Proxies	Fast, cost-effective, scalable	Higher ban risk, less trusted	Bulk data extraction, non-sensitive tasks
Mobile Proxies	High anonymity, mobile IP trust	Expensive, limited availability	Mobile-specific scraping, ad verification

Residential proxies are recommended for Google Shopping due to their reliability and low detection risk.

FAQs

1.  What are common technical challenges when scraping Google Shopping?

Common technical challenges include CAPTCHAs or IP bans from Google’s anti-bot systems.

2.  How do I integrate proxies with my scraper?

Configure the proxy dictionary with your provider’s server, username, and password. Test connectivity via curl or the provider’s dashboard.

3.  Why is configuring proxies difficult?

Incorrect credentials or server details may cause connection errors. To solve it, you need to verify proxy settings in your provider’s dashboard. Contact support for configuration help if necessary.

4.  How can businesses use scraped Google Shopping data?

Google Shopping Data can be use to monitor pricing to stay competitive, analyze ad positions to optimize bids, and track reviews to improve listings.

5.  How do I troubleshoot scraping failures?

Check CSS selectors for HTML changes. Test proxy connectivity. Use selenium for JavaScript-rendered pages. Log errors with try-except.

Conclusion

Scraping Google Shopping Ads with Python and proxies from OkeyProxy enables businesses to gain insights into product performance and competitor strategies. 

This tutorial provides a practical approach to set up a scraper, integrate proxies, and analyze data for actionable results. By leveraging tools like OkeyProxy for reliable data collection, you can optimize your digital marketing efforts and stay competitive in 2025. 

Visit OkeyProxy proxy solutions for your scraping needs.