making multiple api endpoint calls from array nodejs

Question

I'm having trouble making calls to API endpoints from an array. The array contains values that I need to feed into the endpoint in order to get data that I need. The trouble I am having is when I console log the data from the api call I receive JSON text. However, when I try to parse the text and get a specific value from the JSON text it is not giving me the results. Here is the code I am working on:

var http = require("http");
var areaCodes = ["'0071950')", "'0072850')", "'0073450')", "'0075700')", "'0076450')", "'0078700')", "'0900001')", "'0900000')"];
var results = [];
var encode = [];

    //source: http://stackoverflow.com/questions/17811827/get-a-json-via-http-request-in-nodej
for(var i = 0; i < areaCodes.length; i++){
    var realmStatus = "http://api.dol.gov/V1/Statistics/OES/OE_SERIES/?KEY=XXXXXXXXXXXXXXXXXXXXXXXXXX&$filter=(OCCUPATION_CODE eq '151131' ) and (AREA_CODE eq " + areaCodes[i];
     encode.push(encodeURI(realmStatus));

var options = {
        host: 'api.dol.gov',
        path: encode[i],
        type: 'GET',
        dataType: 'json',
        headers: {'accept' : 'application/json'}
};
console.log("Start");
var x = http.request(options,function(res){
    console.log("Connected");
     var str = '';
    res.on('data', function(chunk) {
        str += chunk;
    });
    res.on('data',function(data){
        if(res.statusCode == 200){
            //console.log("res status is good");
            try{
                 var dataA = JSON.parse(str);
                //run a for loop 
                for(var h = 0; h < dataA.d.results.length; h++){
                    var seriesNum = dataA.d.results[h].SERIES_ID; 
                    array.push(seriesNum);
                    console.log(seriesNum);
                }
                 //end for
            }catch(e){
                console.log('Error parsing JSON');
            }
        }
    });
});
x.end();
}//end for loop

I'd appreciate if someone would be willing to explain if there is anything I did incorrectly, or if there is something I am missing. Thanks!

samanime · Accepted Answer · 2017-04-18 21:03:47Z

0

It looks like your second res.on('data') (the one with the try...catch) should actually be a res.on('end'), which is fired once you've gotten the whole file.

Check out the example in the official Node.js documentation: https://nodejs.org/api/http.html#http_http_get_options_callback. Notice that they use res.on('data') once and res.on('end') once.

Beyond that, off-hand it looks like you should at least get something parsed.

answered Apr 18, 2017 at 21:03

samanime

26.8k17 gold badges99 silver badges155 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

learningToCode Over a year ago

hey thanks for the quick response, however, that does not work. It's odd because I used the same code for another endpoint and it works fine. The only difference is that in the other endpoint I did not use an array to store values to concatenate the endpoint.

learningToCode Over a year ago

The problem is that when I run the program I get "Error parsing JSON" meaning that the try block is not being executed. This is not a problem when I make a call for my other endpoint

samanime Over a year ago

I think the reason is because you're calling that second 'data' immediately after the first one. Depending on the size of the payload, you may get the entire JSON in just one data (which is probably why other endpoints work), but for this one, that payload requires multiple data calls. However, you're calling this after the first, so you probably only have like half of the JSON, which is why it throws an error.

learningToCode Over a year ago

the funny thing is that it would work sometimes, for instance a couple hours after the initial call it didn't work but then it gave me the results i wanted.. weird

samanime Over a year ago

That would lend more to what I said. If the payload got small enough (or fast enough) to come in on one data call, it'd work. If it's larger, the .on('data') would have to be called multiple times, but your code would try to parse JSON after the first.

|

Collectives™ on Stack Overflow

making multiple api endpoint calls from array nodejs

1 Answer 1

6 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

6 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related