Issue using tensorflow loadGraphModel from a signed URL on node.js

Question

I have uploaded the model.json file of my tensorflow graph to a private repository on an AWS S3 bucket, and am now trying to load the graph with the loadGraphModel (alongside with the binary files of the weight manifest values, group1-shard1of1). Here's my code, which I run with node (I've kept the bucket path and signature keys private)

TFJSConverter = require('@tensorflow/tfjs-converter')
const MODEL_URL = "https://[BucketName].s3.amazonaws.com/[PathToModel]/model.json?[credentials]&[securitykey]";
global.fetch = require('node-fetch')
TFJSConverter.loadGraphModel(MODEL_URL)

However the loadGraphModel function looks for a model url ending with '.json'. If not, it looks for the full model url and checks for a weight manifest file called weights_manifest.json, with no signature. An error request then follows:

UnhandledPromiseRejectionWarning: Error: Request to https://[BucketName].s3.amazonaws.com/[PathToModel]/model.json?[credentials]&[securitykey],https://[BucketName].s3.amazonaws.com/[PathToModel]/weights_manifest.json failed with status code 403. Please verify this URL points to the model JSON of the model to load.

I've checked that the signed url actually works, is there a solution for signed urls?

Installed versions: @tensorflow/[email protected] node v10.15.3

Many thanks!

Maybe download it to a temp directory and load it from there? — Sebastian Speitel
– Sebastian Speitel, Commented May 9, 2019 at 5:45

edkeveked · Accepted Answer · 2019-05-09 07:39:54Z

2

The correct library to use to load the model is tfjs and not tfjs-converter

let tf = require("@tensorflow/tfjs");
tf.loadGraphModel(MODEL_URL)

403 error is an authorization error response. Try to set the credentials in the request using requestInit of the object passed as parameter of loadGraphModel

answered May 9, 2019 at 7:39

edkeveked

18.4k10 gold badges60 silver badges95 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

tlc91 Over a year ago

I've added header with the signature: const mycred = '[mySignature]'; var myHeader = {'Authorization':mycred}; var myInit = {headers: myHeader}; const model = tf.loadGraphModel(MODEL_URL,{requestInit:myInit}); and I now get this error: Error: Request to s3.amazonaws.com/[PathToModel]/model.json failed with status code 400. Please verify this URL points to the model JSON of the model to load.

Ping Yu Over a year ago

Can you check the network tab in the developer tool of chrome, to see if the headers are set for the ajax requests to retrieve the model files?

lionbigcat Over a year ago

Did you manage to get this to work @tlc91? From my understanding, presigned S3 urls are limited to a single key (e.g. the model.json file), and tf.loadGraphModel would attempt to download the rest of the bin files in the defined prefix.

Daniel Hasegan · Accepted Answer · 2020-07-13 21:53:46Z

0

This worked for me:

const fetch = require('node-fetch')
global.fetch = fetch

but you can also try:

const fetch = require('node-fetch')
tf.loadGraphModel(MODEL_URL, { fetchFunc: fetch } )

as described in the documentation: https://js.tensorflow.org/api/latest/#loadGraphModel

answered Jul 13, 2020 at 21:53

Daniel Hasegan

7951 gold badge8 silver badges15 bronze badges

Collectives™ on Stack Overflow

Issue using tensorflow loadGraphModel from a signed URL on node.js

2 Answers 2

3 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related