node.js parsing html text to get a value to a javascript variable

Question

I' doing this successfully to get the help text of my page of interest.

router.get('/get', function (req, res) {
    var pg = 'https://j......com/f/resource'
    console.log('get', pg);
    requestify.get(pg).then(function (resp) {
        console.log(resp.body);
    });
});

Now that I have the page's text, I'm wanting to parse the text to get the value of a javascript variable which I know exists in the text.

<script> var x1 = {"p": {.......bla bla ...}};</script>

I know that sometimes the <script> tag will include the type attribute; but it will not always include the type attribute.

When I find the value of x1 I what to use that in my javascript's app's as a value in myVar variable.

If you do not have THE answer then your comment/tip as to what I should research is appreciated.

I was hoping I would find some module I can just drop the entire text into and have the module somehow just output all variables a values for me.

You may want to avoid the parsing aspect and try something "direct" like jsdom, where you can load (and execute) the page then grab the resulting value. Assuing it's as written, it should be easy to grab window.x1 since you're not competing with funky closures. — Brad Christie
– Brad Christie, Commented Jun 24, 2017 at 15:33
I'll have a look. Thanks for the tip. Sounds like the kind of thing I was hoping for. — user5903880
– user5903880, Commented Jun 24, 2017 at 15:42

Brad Christie · Accepted Answer · 2017-06-24 16:02:26Z

5

So you're not re-inventing the wheel, I feel like using JSDOM (and it's execution capabilities) would be the best best. To mock what you have:

const express   = require('express');
const jsdom     = require("jsdom");
const { JSDOM } = jsdom; // it exports a JSDOM class

// Mock a remote resource
const remote = express()
  .use('/', (req, res) => {
    res.send('<!DOCTYPE html><html lang="en-US"><head><title>Test document</title><script>var x1 = { "p": { "foo": "bar" } };</script></head><body></body></html>');
  })
  .listen(3001);

// Create "your" server
const local = express()
  .use('/', (req, res) => {
    // fetch the remote resource and load it into JSDOM. No need for
    // requestify, but you can use the JSDOM ctor and pass it a string
    // if you're doing something more complex than hitting an endpoint
    // (like passing auth, creds, etc.)
    JSDOM.fromURL('http://localhost:3001/', {
      runScripts: "dangerously" // allow <script> to run
    }).then((dom) => {
      // pass back the result of "x1" from the context of the
      // loaded dom page.
      res.send(dom.window.x1);
    });
  })
  .listen(3000);

I then receive back:

{"p":{"foo":"bar"}}

answered Jun 24, 2017 at 16:02

Brad Christie

102k16 gold badges160 silver badges200 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

user5903880 Over a year ago

Oh Wow! Worked AND So Simple. A rare combination to find = awesome.

Brad Christie Over a year ago

Indeed. Brief and concise are sometimes hard to come by, but luckily there are still great library authors out there with clean APIs.

Collectives™ on Stack Overflow

node.js parsing html text to get a value to a javascript variable

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related