Regex To Scrape JavaScript

Question

I'm scraping a web page using casperjs, it works great with global JavaScript values, but now I'm stuck at getting out variables form a jQuery closure like this:

$(document).ready(function(){
var storeData = {
lon: "-5.91829",
lat: "54.65583",
name: "John Smith"
};
SOMEGLOBALVAR.storeFinder.addStore(storeData);
});

I can't find a way to to get my hands on lon, lat and name!

So I decided to select the script tag, then use regex to get out the code out of the script tag. Here's my ugly regex I built using an online editor:

/var storeData = {\n\t*id.*,\n\t.*(\n\t)*\n\t*.*\n\t*.*\n\t*[a-zA-Z: 0- 9,"]*\n\t*[a-zA-Z: 0-9,"]*\n\t*[a-zA-Z: 0-9,"]*\n\t*\};/g

So my question has two parts:

is there a way to access the lon, lat and name inside that function? (I'm scraping so I can't change the function itself)
How can I make my regex better?

Wiktor Stribiżew · Accepted Answer · 2015-04-13 17:05:29Z

3

Here you are, you can extract these values by using a regex with capturing groups (there are 3 of them):

lon: "([^"]*)"[\s\S]*?lat:\s+"([^"]*)"[\s\S]*?name:\s+"([^"]*)".*?

Note that I am using [\s\S] to match a newline as there is no Singleline option in JavaScript.

var re = /lon: "([^"]*)"[\s\S]*?lat:\s+"([^"]*)"[\s\S]*?name:\s+"([^"]*)".*?/; 
    var str = 'var storeData = {\nlon: "-5.91829",\nlat: "54.65583",\nname: "John Smith"\n};';
     
    if ((m = re.exec(str)) !== null) {
        document.getElementById("res").innerHTML = m[1] + "<br>" + m[2] + "<br>" + m[3];
    }

<div id="res"/>

answered Apr 13, 2015 at 17:05

Wiktor Stribiżew

631k41 gold badges502 silver badges632 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Regex To Scrape JavaScript

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related