Web Product Scraper

Web Product Scraper not working?
Fixes for the common problems.

Most problems have one of a handful of causes: the page had not finished loading, the shop lays its pages out unusually, a setting was left at its default, or the shop started refusing visits. Find the symptom below; each comes with what is happening and what to change.

Add to Chrome, it's free No account · no server · nothing leaves your browser
The Web Product Scraper side panel during a run: progress, the number of products done and a Stop button

Why is Web Product Scraper not finding products?

Usually because the shop builds its product list with scripts after the page opens. The panel re-reads such a page by itself a few seconds later; if it still finds nothing, press Re-generate, and if the shop is simply unusual, use Teach this site to point at one product card once.

When the page is read wrong

"No products" or an empty table

The panel says it is waiting for the products to appear and looks again after about 1.5, 3 and 5 seconds. If the table stays empty, scroll the page once and press Re-generate.

Wrong price, name or photo

Open Teach this site, press Pick next to the wrong field and click the right value on the page. The rule is saved for that shop.

No product list at all

In Teach this site, press Pick next to Product card and click any product: every product on the page then appears.

Only one product shown

You are on a product page, which is read as that one product. Open the category or search page to get the whole list.

Sizes and colours missing

Each product is one row with the price its page shows; options are not split into rows.

Recommended products left out

On purpose: "you may also like" blocks on a product page are never collected.

When a run does less than expected

Only the first page came through

Set the page range in step 2, or, for listings that load more as you scroll, the scroll rounds and "Load more" presses.

The run stopped by itself

It stops after three failed pages in a row, which usually means the shop is refusing visits. Raise the Pause, lower "At a time", or try again later.

Fewer products than the shop has

The run ends at Max products, 100 by default. Raise it, or run again: products already in the list are skipped.

A second run added nothing

Every product was already in your list. Press Clear first when you want a fresh copy, for example new prices.

Another shop's products in the list

The panel offers to clear them when you move to a new site. Choose Clear the list to keep each shop's data apart, or Keep them to combine.

When the files are not right

Only small photos

The listing shows thumbnails. Tick "Open every product page" so the full galleries are collected; Shopify stores give full-size photos even without it.

Some photos missing from the ZIP

The ZIP holds a _failed.txt naming every photo the shop would not hand over. Download those few by hand.

Odd characters in the CSV

The CSV is saved as UTF-8 with the marker Excel needs. If another program shows ₹ or € wrongly, import the file there as UTF-8.

Buttons missing in the PDF

Save the catalogue with Chrome's Save as PDF; other PDF printers can drop the links.

Still wrong? Send the page

Under the status line in the panel there is a "Site read wrong? Report it" link. It opens an email with the page address, the extension version and what the panel said, ready to send to the developer, so the reader can be fixed for that shop and every shop like it.

Frequently asked questions

Why does the panel say the site has not been allowed?

Each shop needs permission once. Press "Allow this site" and accept Chrome's prompt for that website.

Why is a price empty on some rows?

Those products show no price the reader could trust, often "price on request" or a price that appears only after choosing an option. The needs_review column lists such rows.

Can I make it go faster?

Raise "At a time" to open more product pages at once, up to 10. Some shops start refusing at five or more, so three is the safe default.

Does reinstalling help?

Rarely, and it empties your scraped list. Most problems are fixed by Re-generate, a setting, or Teach this site.

Add to Chrome, it's free