Web Scraping for E-commerce: Building a Daraz Price Tracker
Web Scraping for E-commerce: Building a Daraz Price Tracker
Retailers across South Asia use Daraz price and stock data to drive their own pricing. Built well, a tracker runs on a small VPS and returns hours of manual work per day.
Legality First
Scrape only public product pages. Respect robots.txt where it applies to your use case, throttle politely, and never scrape private data or bypass paywalls. For commercial resale of the raw data, get legal advice in your jurisdiction.
Architecture
Fetching Pages
Use randomised delays and one concurrent request per proxy — anti-bot systems flag bursts, not volume. Rotate user agents and accept-language headers.
Parsing
Many e-commerce sites ship product data as JSON inside a tag. Parse the JSON directly instead of DOM scraping — it's faster and survives layout changes.
Storing History
One row per (product_id, checked_at). Do not overwrite the current price — history is the product.
Beating Anti-Bot
Alerts
A simple SQL query gives you a daily competitor drop alert: group by product, compare the min and max price over the window, and notify when the drop crosses a threshold.
Wrap-Up
Scraping is 20% code and 80% discipline: throttle, monitor, and store history. The winners are the teams that treat it as a data pipeline, not a hack.