How to deploy Pytorch in Python via a REST API with Flask?
I am working on AWS Sagemaker and my goal is to follow this tutorial from Pytorch's official documentation.

The original predict function from the tutorial above is the following: 

@app.route('/predict', methods=['POST'])
def predict():
    if request.method == 'POST':
        file = request.files['file']
        img_bytes = file.read()
        class_id, class_name = get_prediction(image_bytes=img_bytes)
        return jsonify({'class_id': class_id, 'class_name': class_name})


I was getting this error, so I added 'GET' as a method as mentioned in here. I also simplified my example to its minimal expression: 

from flask import Flask, jsonify, request

app = Flask(__name__)

@app.route('/predict', methods=['GET','POST'])
def predict():
    if request.method == 'POST':
        return jsonify({'class_name': 'cat'})
    return 'OK'

if __name__ == '__main__':
    app.run()


I perform requests with the following code: 

import requests

resp = requests.post("https://catdogclassifier.notebook.eu-west-1.sagemaker.aws/proxy/5000/predict",
                     files={"file": open('/home/ec2-user/SageMaker/cat.jpg', 'rb')})


resp is &lt;Response [200]&gt; but resp.json() returns JSONDecodeError: Expecting value: line 1 column 1 (char 0) Finally, resp.url points me to a page saying 'OK'. 

Moreover, this is the output of resp.content 

&lt;!DOCTYPE HTML&gt;

&lt;html&gt;
&lt;head&gt;
  &lt;style type="text/css"&gt;

#loadingImage {
    margin: 10em auto;
    width: 234px;
    height: 238px;
    background-repeat: no-repeat;
    background-image: url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAOoAAADuBAMAAADFK4ZrAAAAMFBMVEUAAAAaGhoXFxfo6Og3NzdPT0/4+PgmJibFxcWhoaF8fHzY2NiysrJmZmaNjY1DQ0OLk+cnAAAAAXRSTlMAQObYZgAABlpJREFUeNrtnGnITFEYx5+a3Ox1TjRFPpxbGmSZsmfLliKy73RKZImaSVmy7xFJtmSLLFlHdrIklH3fhSyJLOEDQpak4865951znmfGUuf34f3w1u13/+c+d+a555w74HA4HA6Hw+FwOBwOh+P/Z8DBfec3be3WbdO2U3P2SPgTeA1WXmuT9n2fse9/Eq1vvUhCoYntWzfqu1DBfdZ+Yz0oKA0WpblgWYhE6+UFzBub+8ZnYfBE7z1QIIY9ZoJFICYuLUxZDbkhWBEkboVqPdrJNGwuWJGIz0nQiD3pPYMibcFy0j+jHTZN8LEZglTktvpfstOWFozxd4SkJvAOWdpj7DvVJE46uJNgRvh9ZVbU78RxQxzbwIzpo0Vl/AFG6l0Q5tbE2qyo3+mIsR7wmQWVJwWiYq3FOzEb+Jikioq2ekuYHfyOioq2NkkxS8qdVFGR1lJvmS18rFRRcVZVv+bEr6ioKOuQtwxB5STAAoa3LmEY+AcoIfDWEYKhqDVpAUNbvTUMSVuBsKqoaPDWh+zPW1VF6Pzsibmff6u3INI56su9Vz+4Pyqdb2vpFAuDi693Z9Ub9POJZ9+8rkzk1XokXFpuyyAJikHzbvh5tMbehg5u26laf3MiLfJmHSiYTuLuINDwDjQXOayUTwie2CghjP09clgptZTYLKNa1875sdYPk0Ikh9P5sHqX9fH9LCESb246D9bSQm8RkkWe5sIU3bpLs1aekaOXvEy26gMcfwQ5aJoiWvUB5h8l5GKVIFqbiezu4KjBqXYmWp9mR+0JBuwWJGusud71GVA8RbIOFVlRr4ORlTbCe1FRYTLBqt83vI9ZVFoNl0plt7eYqMqKu6yfUVdVWVGXNT4fFVVZUXdrBYmKqqyou/WTWVSitUTWAE8yi0q0DlfFZD43Nplq3cECdDeLSrR6CxADPJlsDRZTDYmMqqz23xx8NCIqwlpSWH/dFGdka51gCc83qT+aVR+tchLRBCCswRKuKk2uCdnqBZ4geUtEb4ewxoIlvNqkEliRXLSeA4lnEI+6CuMZ+DIi0DFJ2xPVyVh/9lcBA2Kdi5JWk7ZZeRcwYaYW1rIeoVQLphBHzVqmzpFav1IGDPAOtVL0lWBE4/utIug9FYzwaiuk8Qp/7QgkOBwOh8PhcDgcDofJ00M9MMSzP6bB9lYRtD9jKL2gjmk9w2zS/5JgEfDKZg+WJfzfjqluuxqpMw5MqCuYolYSsZ6OWdiZos2n0KYE+XyTE+9sPZ9SVhRp7W496VoVDCjGiqSl9XJQFbqVvzOpx8ARXXBW6wm+BdoRRGs8aTuDz+fTrSa3fGntPIlWPt96LrOyRFmt74Jj5qOjrOTPiEuBQ9qRrVURCzPrMVb7cmymrd3SrFURu7HKSaKVz0fsmagGRGtVzG6sikQrn4/Y9MDXE61VMbtE45MMraSocJwFKC8NraSoxd+wAKMBZ1VREXuNeEuatSpqn378KMnKj1q1pOqy0rJOQq1hjQaa9TMYMF3bjEW0xhcbNBEtsg/KEK2sZtJ+R/cEoFr5HchBU6GtC5KtLHEyxythb7SrMoluZWMzlq+EVZN5sPKPSbtXwlTXT+pgbkaf/IG0XvZHKVZFIkrrzU6HVL2kWZX2rjTeKB2/AjSrQvSqBxoDzimponKGalX4+jzMvqs+C2E0kK0KXm7b7++oewOWaa8YaLVEtzIuJt7atwd+MGjAvpfvuQi/uyXNquMnWt/e9urZpmtd0z4LJ74WyFbd6/uM//gTRaUkwkqFd4e/YK0sCdar+Kh4a/WGb3HW8hJv5UfVy5lWxK8D3lodoDgq7IQkyqp64LopRNTFgLKqdj92Q1iX0jiJtKonm4Zp61LKANKqnmy8I7bWzYC2qoe4UhvsxvezJFirIt8yrjwJMFZ96+Ihi0tbbr29Us3A1wx0nebWm4CiVErfpRlbxTRytcyYd5lrQoDiZhVF+LWOZiJkQ+pgkw8LPn4GYIltEL5aolJpU7mlkwDPsCe9kiH/fc1zNDUqKQpPhp9MukhpgX50J3a+jR/pjN9MQmHwGl/1RcQTwSMJBSN2+n1Ku7yCj9ySgYJSe25X5ovgr0bd2imh0AxbueJ96tcvZI2aeO9FPfgjeI32nX2+qVu/TZuWHtw5CBwOh8PhcDgcDofD4XA4HP8i3wDmy/sFKv4WfAAAAABJRU5ErkJggg==);

    -webkit-animation:spin 4s linear infinite;
    -moz-animation:spin 4s linear infinite;
    animation:spin 4s linear infinite;
}


@-moz-keyframes spin { 100% { -moz-transform: rotate(360deg); } }
@-webkit-keyframes spin { 100% { -webkit-transform: rotate(360deg); } }
@keyframes spin { 100% { -webkit-transform: rotate(360deg); transform:rotate(360deg); } }

  &lt;/style&gt;
&lt;/head&gt;

&lt;body&gt;

  &lt;div id="loadingImage"&gt;&lt;/div&gt;

&lt;script type="text/javascript"&gt;



var RegionFinder = (function()
{
    function RegionFinder( location ) {
        this.location = location;
    }


    RegionFinder.prototype = {


        getURLWithRegion: function() {

            var isDynamicDefaultRegion = ifPathContains(this.location.pathname, "region/dynamic-default-region");

            var queryArgs = removeURLParameter(this.location.search, "region");

            var hashArgs = this.location.href.split("#")[1] || "";
            if (hashArgs) {
                hashArgs = "#" + hashArgs;
            }

            var region = this._getCurrentRegion();
            var newArgs = "region=" + region;
            if (_shouldAuth()) {
                newArgs = "needs_auth=true";
                region = "nil";
            }

            if (queryArgs &amp;&amp;
                queryArgs != "?") {
                queryArgs += "&amp;" + newArgs;
            } else {
                queryArgs = "?" + newArgs;
            }



            if (!region) {

                var contactUs = "https://portal.aws.amazon.com/gp/aws/html-forms-controller/contactus/aws-report-issue1";

                alert("How embarrassing! There is something wrong with this URL, please contact AWS at " + contactUs);
            }

            var pathname = isDynamicDefaultRegion ?  "/console/home" : this.location.pathname;

            return this.location.protocol + "//" + _getRedirectHostFromAttributes() +
                pathname + queryArgs + hashArgs;
        },


        _getCurrentRegion: function() {

            return _getRegionFromHash( this.location ) ||
                   _getRegionFromAttributes();
        }
    };



    function ifPathContains(url, parameter) {
        return (url.indexOf(parameter) != -1);
    }


    function removeURLParameter(url, parameter) {
        var urlparts= url.split('?');
        if (urlparts.length&gt;=2) {
            var prefix= encodeURIComponent(parameter);
            var pars= urlparts[1].split(/[&amp;;]/g);
            //reverse iteration as may be destructive
            for (var i= pars.length; i-- &gt; 0;) {
                if (pars[i].lastIndexOf(prefix, 0) !== -1) {
                    pars.splice(i, 1);
                }
            }
            url= urlparts[0]+'?'+pars.join('&amp;');
            return url;
        } else {
            return url;
        }
    }


    function _getRegionFromAttributes() {
        return "eu-west-1";
    };

    function _shouldAuth() {
        return "";
    };

    function _getRedirectHostFromAttributes() {
        return "eu-west-1.console.aws.amazon.com";
    }


    function _getRegionFromHash( location ) {

        var hashArgs = "#" + (location.href.split("#")[1] || "");


        var hashRegionArg = "";


        var match = hashArgs.match("region=([a-zA-Z0-9-]+)");
        if (match &amp;&amp; match.length &gt; 1 &amp;&amp; match[1]) {
            hashRegionArg = match[1];
        }
        return hashRegionArg;
    }

    return RegionFinder;
})();


var regionFinder = new RegionFinder( window.location );

window.location.href = regionFinder.getURLWithRegion();

&lt;/script&gt;


&lt;/body&gt;
&lt;/html&gt;


What am I missing? 

Looks like the content of your resp is HTML as opposed to JSON; this is likely a consequence of how the Jupyter server proxy endpoint you're attempting to POST to (https://catdogclassifier.notebook.eu-west-1.sagemaker.aws/proxy/5000/predict) is configured.

It looks like you're using a SageMaker notebook instance, so you might not have much control over this configuration. A workaround could be to instead deploy your Flask server as a SageMaker endpoint running outside JupyterLab, instead of directly on a notebook instance.

If you want to prototype using only a notebook instance, you can alternately just bypass the proxy entirely and simply call your Flask route relative to localhost from another notebook tab while the Flask server runs in your main notebook tab:

import requests

resp = requests.post("https://localhost:5000/predict",
                     files={"file": open('/home/ec2-user/SageMaker/cat.jpg', 'rb')})

