Quick list / feed scraping in the dev console

// function to create array of elements from an xpath let xeval = (s, each = e => e) => { const iter = document.evaluate(s, document, null, XPathResult.ANY_TYPE, null ); const elems = [] while(elem = iter.iterateNext()) { elems.push(each(elem)) } return elems } 

I was recently helping someone scrape app reviews and came across this tip. This will work to scrape any list / feed of elements.

The Problem

A lot of sites generate class names on page load, so you can't always use class names / id's to pull page data. The solution is to use XPATH. I was surprised by how easy this is!

• Scroll down the page a bit and choose an element you want to scrape

• Inspect the element and copy it's XPath

right click element in dev window -> copy -> XPath / full XPath

• it will look like:
  '.../div/div[10]/div/div[2]/div[1]/div[1]/span'

• by deleting '[10]' all like elements will be selected

• use the XPath to select the page elements

And that's it!

Here's the code