Problem during prediction on tensorflow js prediction

Question

I have a problem on my tensorflow js model, I followed a course (link to the course) where I learned to create a tensorflow model and everything worked fine but the course doesn't show how to use the model so I developped myself this part but every time I try to predict a number I got the same result (2), I don't know why and I don't have the knowledge to fix that so I hope someone could help me fix that and provide an explenation.

The guest part of the code is here :

    function guessIt(){
  let inputTensor = tf.browser.fromPixels(document.getElementById('imageResult'), 1)// imageResult is an <img/> tag
    .reshape([1, 28, 28, 1])
    .cast('float32');
  let predictionResult =  modelJson.predict(inputTensor).dataSync();
  let recognizedDigit = predictionResult.indexOf(Math.max(...predictionResult));
  console.log(recognizedDigit);
  console.log(predictionResult);
}

var mousePressed = false;
var lastX, lastY;
var ctx;

//resize image with off-screen canvas
function imageToDataUri(img, width, height) {

    // create an off-screen canvas
    var canvas = document.createElement('canvas'),
        ctx = canvas.getContext('2d');

    // set its dimension to target size
    canvas.width = width;
    canvas.height = height;

    // draw source image into the off-screen canvas:
    ctx.drawImage(img, 0, 0, width, height);

    // encode image to data-uri with base64 version of compressed image
    return canvas.toDataURL("image/png");
}

function InitThis() {
    ctx = document.getElementById('sheet').getContext("2d");

    $('#sheet').mousedown(function (e) {
        mousePressed = true;
        Draw(e.pageX - $(this).offset().left, e.pageY - $(this).offset().top, false);
    });

    $('#sheet').mousemove(function (e) {
        if (mousePressed) {
            Draw(e.pageX - $(this).offset().left, e.pageY - $(this).offset().top, true);
        }
    });

    $('#sheet').mouseup(function (e) {
        mousePressed = false;
        let img = imageToDataUri(document.getElementById("sheet"),28,28)//resize it
        let imgElement = document.getElementById("imageResult").setAttribute("src",img);// display it
        guessIt();
    });
        $('#sheet').mouseleave(function (e) {
        mousePressed = false;
    });
}

function Draw(x, y, isDown) {
    if (isDown) {
        ctx.beginPath();
        ctx.strokeStyle = "000000";
        ctx.lineWidth = 9;
        ctx.lineJoin = "round";
        ctx.moveTo(lastX, lastY);
        ctx.lineTo(x, y);
        ctx.closePath();
        ctx.stroke();
    }
    lastX = x; lastY = y;
}

function clearArea() {
    // Use the identity matrix while clearing the canvas
    ctx.setTransform(1, 0, 0, 1, 0, 0);
    ctx.clearRect(0, 0, ctx.canvas.width, ctx.canvas.height);
    document.getElementById("imageResult").setAttribute("src","");
}

// init the cancas
document.addEventListener('DOMContentLoaded', InitThis);

The GitHub of the project is here : github

thanks in advance

It depends of many things. But most probably that you need to wait until the image has completed loading before predicting — edkeveked
– edkeveked, Commented Jan 6, 2020 at 15:14

edkeveked · Accepted Answer · 2020-01-08 13:03:11Z

1

The image should be predicted only when it has completed to load

const img = document.getElementById('imageResult')
 img.onload = function(){
 let inputTensor = tf.browser.fromPixels(document.getElementById('imageResult'), 1)// imageResult is an <img/> tag
    .reshape([1, 28, 28, 1])
    .cast('float32');
  let predictionResult =  modelJson.predict(inputTensor).dataSync();
  let recognizedDigit = predictionResult.indexOf(Math.max(...predictionResult));
  console.log(recognizedDigit);
  console.log(predictionResult);
}

Additionnaly, the initial canvas background is transparent because it has not been set. When converted to tensor the transparent background becomes black. The stroke style is also black. A black image on a black background... No matter what is drawn, it leads to the same tensor.

Either the strokestyle is changed, or the canvas background is filled or both (but both should not have the same color for the same reason explained above).

function imageToDataUri(img, width, height) {

    // create an off-screen canvas
    var canvas = document.createElement('canvas'),
        ctx = canvas.getContext('2d');


    // set its dimension to target size
    canvas.width = width;
    canvas.height = height;

    // canvas with white background
    ctx.fillStyle = 'white';
    ctx.fillRect(0, 0, canvas.width, canvas.height);


    // draw source image into the off-screen canvas:
    ctx.drawImage(img, 0, 0, width, height);

    // encode image to data-uri with base64 version of compressed image
    return canvas.toDataURL("image/png");
}

edited Jan 8, 2020 at 13:03

answered Jan 6, 2020 at 15:16

edkeveked

18.4k10 gold badges60 silver badges95 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

Brice Harismendy Over a year ago

I tried but still only get 2 as answer the file for the function is in js/load/model_using.js and is called in js/load/canvas_drawing.js line 40

edkeveked Over a year ago

I think that you will need to normalize the inputTensor. inputTensor.sub(127).div(127)

Brice Harismendy Over a year ago

I tried but I still get the same prediction for every image I try to predict but now with a bigger variation (0.07-0.14 instead of 0.09-0.10)

Brice Harismendy Over a year ago

ok I figured a lot of things out by your post and I've updated my github repository, I still struggle for some digit like 6 but it's good enought for a first try. a problem is that the trained image have a black background so for now I turned all canvas with fill to black and stroke to white. you really helped me on that one thanks ! :)

Brice Harismendy Over a year ago

I would have one last question, why to normalize the tensor you substract 127 and divide by 127 ? why this number ?

|

Collectives™ on Stack Overflow

Problem during prediction on tensorflow js prediction

1 Answer 1

6 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

6 Comments

Your Answer

Sign up or log in

Post as a guest

Related