Get started with TensorFlow.js by building a digit recognizer from scratch in this quick start tutorial https://angularfirebase.com/lessons/tensorflow-js-quick-start/

TF.js docs - https://js.tensorflow.org
Machine Learning Crash Course - https://developers.google.com/machine-learning/crash-course/
Kaggle - https://www.kaggle.com/jeffd23
MNIST - https://ml4a.github.io/demos/confusion_mnist/
2024-02-04T07:39:54.226Z
nova-2
2-general-nova
2024-01-09.29447
30089e05-99d1-4376-b32e-c263170674af
dadf8479-5d81-4cba-9d9c-51fa63484546
126527c5199792eaf2a75eda535d06c95c423d545f3173b265a294c0f9ddcce2
67875a7f-c9c4-48a0-aa55-5bdb8a91c34a
deprecated
The most exciting thing to happen last week was the announcement of TensorFlow.
Js, a JavaScript library that allows us to build, train, and make predictions from machine learning models directly in the browser.
We'll use TensorFlow.
Js with Angular to take a hand drawn digit image and then predict its actual numeric value.
I'm super excited about this because it opens up a whole new world of possibilities to combine web development with machine learning.
Make sure to subscribe if you're new here, and you can grab the source code from Angular Firebase dotcom.
The first thing we need to answer is what is TensorFlow?
And the simple answer is that it's a library for performing mathematical computations.
But it's famous for building deep neural networks that power some of the most impressive artificial intelligence technology that we have in the world today.
The sad part of this answer is that if you don't have a background in machine learning, TensorFlow is not going to be very easy to use or very useful to you.
That being said, I started out with 0 machine learning knowledge a couple years ago, and I've worked my way up to an expert rake on Kaggle, which is the hub for data science competition around the world.
If you wanna get really good at machine learning, I recommend spending about half your time with traditional courses such as Coursera or Google's Machine Learning crash course, and spend the other half of your time applying what you learn in data science competitions on Kaggle.
But be careful because data science competitions can be very, very addicting.
If you're watching this video with 0 machine learning background, I still wanna give you a high level overview of how machine learning models work.
The typical narrative goes something like this.
We have a set of data and each item in that dataset has a label telling us what it is.
In this video, we're using a dataset called MNIST, which is a series of images of hand drawn digits.
Because you're most likely a human being, it's easy for you to recognize what these digits are.
But to a computer, each image just looks like a matrix of values and it has no way to discern the context or actual meaning of the image itself.
A typical image recognition convolutional neural network is going to look at many samples of these images and attempt to extract features out that will distinguish one digit from another.
The algorithm starts off completely random then slowly corrects itself after seeing more and more samples of these images.
So that's a very high level overview of how a TensorFlow model might work.
What I wanna show you now is how we build and train a model and then make predictions from it in the browser.
I'm going to generate a new Angular app with the Angular CLI, then we'll CDN the app and install TensorFlow.
Js.
For this first part, what I wanna do is show you how to build the most basic possible TensorFlow model which would just be a linear regression.
So we'll feed TensorFlow an array of values that should predict some other value.
Think of the relationship between a person's height and weight.
You should be able to use their height to roughly predict their weight.
We will train the machine learning model in Angular when this component is initialized.
And then when the user enters a new value into the form field, we'll run a new prediction with that TensorFlow model.
If you're more of a visual person, The actual dataset would look something like this with the blue dots being our samples and then the red line being the model that TensorFlow generates.
Let's go ahead and code this up by opening the app component in Angular, then we'll import TensorFlow under the namespace of TF.
Then I'm setting up a property called linear model that defines the actual TensorFlow model that will train and make predictions.
We're going to use the sequential API in TensorFlow.
Js which is a little bit higher level and easier to work with.
It's very similar to Keras if you have a Python background.
Then I'll also set up a property for the prediction value so we can show it in the front end UI.
In this demo, we're only training some randomly generated static data, So we'll do that as soon as the component is initialized.
TensorFlow.
Js works in promises, so it tends to be really easy to work with in JavaScript apps.
We'll set up an asynchronous function here called train new model.
Then we'll define an empty model by just calling TF sequential.
In this case, we're just performing a linear regression, and we can do that by adding a dense layer or a fully connected layer that will output a shape of 1 and also taking input with a shape 1.
In larger neural networks, you have multiple layers, and the dense layer keeps all of the neurons in each layer connected together.
But since we're only drawing a single straight line through the data, we don't really need to worry about that.
We just need to define the loss metric, which in this case will be mean squared error.
In other words, we're just trying to minimize the mean squared error of the predictions from the algorithm.
Then we also define an optimizer which is the function that determines how we reach this actual minimum error point.
SGD stands for stochastic gradient descent, And there's a number of different optimizers that you can choose from.
So that's actually our entire model right there.
Now we just need to train it by feeding it some data.
But we need to feed it that data in the form of a tensor.
But you might be asking yourself, what the hell is a tensor?
For our purposes, the easiest way to think about it is just as an array.
For example, a one dimensional tensor would look the same as an array of numbers.
The actual tensor terminology is not something you really need to get tripped up with especially when you're just learning TensorFlow.
As you can see here, we have one tensor named xs.
When you have a variable named x in machine learning, That means it's your actual training features for the dataset.
The y variable, on the other hand, is the label for that data.
It's the actual thing that we want our machine learning algorithm to predict.
Depending on the size of the dataset, training the model can take a long time.
But in this case, we just have a few values, so it's going to happen very quick.
And we make that happen by calling linear model fit with our x and y data.
The end result there is that we have a statistical model that can now predict values based on this dataset that we've provided.
To make a prediction, we'll call linear model predict then pass in the value in the form of a tensor.
That's going to return a prediction in the form of a tensor.
So now we just need to convert that tensor into something that we can use in Angular.
Everything in TensorFlow runs in the context of a session.
So to get out the data we actually want, we create an array and then call data sync on the tensor.
Now we'll switch over to our HTML and run this linear prediction method whenever the form value changes on this HTML form input.
If we pull up the app in the browser, we should get the results of our linear regression.
If you enter a smaller number, the prediction should decrease.
If you enter a larger number, it should increase.
Now that you know the basics, let's do something a little more complex.
Neural networks have really revolutionized image recognition problems.
In this example, we're going to take an image of a handwritten digit using an HTML canvas, And then we'll have our neural network predict what the actual value of that number is.
The training process for an algorithm like this is way too CPU and memory intensive to run-in the browser.
But one of the really cool things about TensorFlow.
Js is that we can import models that have been trained in Python.
So that means you can build your own machine learning model on your local system and then upload it to your web app so your users can use it on the web.
I included my model in the source code for this project, but I recommend going to Kaggle and participating in the digit recognizer competition using Keras to build your own from scratch.
Keras will allow you to export your trained model, And then TensorFlow has a tool that allows you to convert that model into a format that can be used specifically by TensorFlow.
Js.
To install that tool, you need to install TensorFlow.
Js with the Python package manager pip.
Then it gives you a command that will take the Keras file and not only convert it to TensorFlow.
Js, but it will also shard the file so you can load it efficiently on the web.
For right now, I'm just using the assets directory in Angular, but you'd wanna use a cloud storage bucket in the real world.
Now that our model has been converted to TensorFlow, I'm going to go back into the app component and add a load model method to n g on a net.
And a quick side note, I'm using this drawable directive to create the HTML canvas.
I'm not going to show you that code, but it's in the main repo if you wanna check it out.
You'll notice here that this time our model is typed to just a plain TensorFlow model.
That's the lower level API that's generally a little more difficult to use if you're a newcomer.
Then this time, our predictions will be an array of 10 different values where each value represents the probability that that index is the correct value for the drawing on the canvas.
So we can take the index with the highest probability and assume that that's the number that was drawn on the canvas.
Since our model is already pretrained, we can just load it with a simple one liner of TensorFlow load model.
But our predict method is going to be a little more tricky.
The first thing that makes it more tricky is that we need to think about memory management.
Neural networks can use a lot of memory, and we want to avoid memory leaks in our client side app.
To do that, we can run all of our code inside this TensorFlow tidy method.
That will ensure that all of our tensors are released from memory when the prediction is finished.
The actual image data that we want is coming from the HTML canvas that we're extracting seen as the image data type.
You can pass in a variety of different types here and it will convert the image into a tensor that can be used in the TensorFlow graph.
In our case, this model was trained with images that were 28 pixels by 28 pixels with just one color channel.
So we need to reshape our data into that shape.
The first value here is the batch size, which is going to be 1 because we just have one image, then 28 pixels by 28 pixels and one color channel.
The Python model that I trained for this demo is using an integer data type, But TensorFlow requires it to be a floating point, so we can change that by calling TensorFlow cast because all data types in a tensor need to be the same.
So now that we have our data in the correct format, we can just pass it into our model and call predict.
And just like last time, it returns the prediction as a tensor, So we'll convert that to an array and call data sync.
In this demo, I've set up my app to run this predict method every time a new image is drawn into the canvas.
It's not a perfect model, but it's about 90% accurate with my handwriting.
And you can clone this repo to try it out for yourself.
But if you're curious to visualize how the neural networks under the hood, I recommend going to the TensorFlow playground.
This was actually the original inspiration to to TensorFlow.
Js, and it shows you how neural networks build complex boundaries around datasets to make predictions.
I'm gonna go ahead and wrap things up there.
If you wanna see more TensorFlow content, please let me know in the comments.
And if you wanna learn how to build modern JavaScript apps for any form, consider becoming a pro member at angularfirebase.com.
You'll get access to a huge library of exclusive content as well as 1 on 1 project support.
Thanks for watching, and I'll see you soon.

Speaker 0: The most exciting thing to happen last week was the announcement of TensorFlow. Js, a JavaScript library that allows us to build, train, and make predictions from machine learning models directly in the browser. We'll use TensorFlow. Js with Angular to take a hand drawn digit image and then predict its actual numeric value. I'm super excited about this because it opens up a whole new world of possibilities to combine web development with machine learning.

Make sure to subscribe if you're new here, and you can grab the source code from Angular Firebase dotcom. The first thing we need to answer is what is TensorFlow? And the simple answer is that it's a library for performing mathematical computations. But it's famous for building deep neural networks that power some of the most impressive artificial intelligence technology that we have in the world today. The sad part of this answer is that if you don't have a background in machine learning, TensorFlow is not going to be very easy to use or very useful to you.

That being said, I started out with 0 machine learning knowledge a couple years ago, and I've worked my way up to an expert rake on Kaggle, which is the hub for data science competition around the world. If you wanna get really good at machine learning, I recommend spending about half your time with traditional courses such as Coursera or Google's Machine Learning crash course, and spend the other half of your time applying what you learn in data science competitions on Kaggle. But be careful because data science competitions can be very, very addicting. If you're watching this video with 0 machine learning background, I still wanna give you a high level overview of how machine learning models work. The typical narrative goes something like this.

We have a set of data and each item in that dataset has a label telling us what it is. In this video, we're using a dataset called MNIST, which is a series of images of hand drawn digits. Because you're most likely a human being, it's easy for you to recognize what these digits are. But to a computer, each image just looks like a matrix of values and it has no way to discern the context or actual meaning of the image itself. A typical image recognition convolutional neural network is going to look at many samples of these images and attempt to extract features out that will distinguish one digit from another.

The algorithm starts off completely random then slowly corrects itself after seeing more and more samples of these images. So that's a very high level overview of how a TensorFlow model might work. What I wanna show you now is how we build and train a model and then make predictions from it in the browser. I'm going to generate a new Angular app with the Angular CLI, then we'll CDN the app and install TensorFlow. Js.

For this first part, what I wanna do is show you how to build the most basic possible TensorFlow model which would just be a linear regression. So we'll feed TensorFlow an array of values that should predict some other value. Think of the relationship between a person's height and weight. You should be able to use their height to roughly predict their weight. We will train the machine learning model in Angular when this component is initialized.

And then when the user enters a new value into the form field, we'll run a new prediction with that TensorFlow model. If you're more of a visual person, The actual dataset would look something like this with the blue dots being our samples and then the red line being the model that TensorFlow generates. Let's go ahead and code this up by opening the app component in Angular, then we'll import TensorFlow under the namespace of TF. Then I'm setting up a property called linear model that defines the actual TensorFlow model that will train and make predictions. We're going to use the sequential API in TensorFlow.

Js which is a little bit higher level and easier to work with. It's very similar to Keras if you have a Python background. Then I'll also set up a property for the prediction value so we can show it in the front end UI. In this demo, we're only training some randomly generated static data, So we'll do that as soon as the component is initialized. TensorFlow.

Js works in promises, so it tends to be really easy to work with in JavaScript apps. We'll set up an asynchronous function here called train new model. Then we'll define an empty model by just calling TF sequential. In this case, we're just performing a linear regression, and we can do that by adding a dense layer or a fully connected layer that will output a shape of 1 and also taking input with a shape 1. In larger neural networks, you have multiple layers, and the dense layer keeps all of the neurons in each layer connected together.

But since we're only drawing a single straight line through the data, we don't really need to worry about that. We just need to define the loss metric, which in this case will be mean squared error. In other words, we're just trying to minimize the mean squared error of the predictions from the algorithm. Then we also define an optimizer which is the function that determines how we reach this actual minimum error point. SGD stands for stochastic gradient descent, And there's a number of different optimizers that you can choose from.

So that's actually our entire model right there. Now we just need to train it by feeding it some data. But we need to feed it that data in the form of a tensor. But you might be asking yourself, what the hell is a tensor? For our purposes, the easiest way to think about it is just as an array.

For example, a one dimensional tensor would look the same as an array of numbers. The actual tensor terminology is not something you really need to get tripped up with especially when you're just learning TensorFlow. As you can see here, we have one tensor named xs. When you have a variable named x in machine learning, That means it's your actual training features for the dataset. The y variable, on the other hand, is the label for that data.

It's the actual thing that we want our machine learning algorithm to predict. Depending on the size of the dataset, training the model can take a long time. But in this case, we just have a few values, so it's going to happen very quick. And we make that happen by calling linear model fit with our x and y data. The end result there is that we have a statistical model that can now predict values based on this dataset that we've provided.

To make a prediction, we'll call linear model predict then pass in the value in the form of a tensor. That's going to return a prediction in the form of a tensor. So now we just need to convert that tensor into something that we can use in Angular. Everything in TensorFlow runs in the context of a session. So to get out the data we actually want, we create an array and then call data sync on the tensor.

Now we'll switch over to our HTML and run this linear prediction method whenever the form value changes on this HTML form input. If we pull up the app in the browser, we should get the results of our linear regression. If you enter a smaller number, the prediction should decrease. If you enter a larger number, it should increase. Now that you know the basics, let's do something a little more complex.

Neural networks have really revolutionized image recognition problems. In this example, we're going to take an image of a handwritten digit using an HTML canvas, And then we'll have our neural network predict what the actual value of that number is. The training process for an algorithm like this is way too CPU and memory intensive to run-in the browser. But one of the really cool things about TensorFlow. Js is that we can import models that have been trained in Python.

So that means you can build your own machine learning model on your local system and then upload it to your web app so your users can use it on the web. I included my model in the source code for this project, but I recommend going to Kaggle and participating in the digit recognizer competition using Keras to build your own from scratch. Keras will allow you to export your trained model, And then TensorFlow has a tool that allows you to convert that model into a format that can be used specifically by TensorFlow. Js. To install that tool, you need to install TensorFlow.

Js with the Python package manager pip. Then it gives you a command that will take the Keras file and not only convert it to TensorFlow. Js, but it will also shard the file so you can load it efficiently on the web. For right now, I'm just using the assets directory in Angular, but you'd wanna use a cloud storage bucket in the real world. Now that our model has been converted to TensorFlow, I'm going to go back into the app component and add a load model method to n g on a net.

And a quick side note, I'm using this drawable directive to create the HTML canvas. I'm not going to show you that code, but it's in the main repo if you wanna check it out. You'll notice here that this time our model is typed to just a plain TensorFlow model. That's the lower level API that's generally a little more difficult to use if you're a newcomer. Then this time, our predictions will be an array of 10 different values where each value represents the probability that that index is the correct value for the drawing on the canvas.

So we can take the index with the highest probability and assume that that's the number that was drawn on the canvas. Since our model is already pretrained, we can just load it with a simple one liner of TensorFlow load model. But our predict method is going to be a little more tricky. The first thing that makes it more tricky is that we need to think about memory management. Neural networks can use a lot of memory, and we want to avoid memory leaks in our client side app.

To do that, we can run all of our code inside this TensorFlow tidy method. That will ensure that all of our tensors are released from memory when the prediction is finished. The actual image data that we want is coming from the HTML canvas that we're extracting seen as the image data type. You can pass in a variety of different types here and it will convert the image into a tensor that can be used in the TensorFlow graph. In our case, this model was trained with images that were 28 pixels by 28 pixels with just one color channel.

So we need to reshape our data into that shape. The first value here is the batch size, which is going to be 1 because we just have one image, then 28 pixels by 28 pixels and one color channel. The Python model that I trained for this demo is using an integer data type, But TensorFlow requires it to be a floating point, so we can change that by calling TensorFlow cast because all data types in a tensor need to be the same. So now that we have our data in the correct format, we can just pass it into our model and call predict. And just like last time, it returns the prediction as a tensor, So we'll convert that to an array and call data sync.

In this demo, I've set up my app to run this predict method every time a new image is drawn into the canvas. It's not a perfect model, but it's about 90% accurate with my handwriting. And you can clone this repo to try it out for yourself. But if you're curious to visualize how the neural networks under the hood, I recommend going to the TensorFlow playground. This was actually the original inspiration to to TensorFlow.

Js, and it shows you how neural networks build complex boundaries around datasets to make predictions. I'm gonna go ahead and wrap things up there. If you wanna see more TensorFlow content, please let me know in the comments. And if you wanna learn how to build modern JavaScript apps for any form, consider becoming a pro member at angularfirebase.com. You'll get access to a huge library of exclusive content as well as 1 on 1 project support.

Thanks for watching, and I'll see you soon.
The most exciting thing to happen last week was the announcement of TensorFlow. Js, a JavaScript library that allows us to build, train, and make predictions from machine learning models directly in the browser. We'll use TensorFlow. Js with Angular to take a hand drawn digit image and then predict its actual numeric value.
machine learning
I'm super excited about this because it opens up a whole new world of possibilities to combine web development with machine learning.
Make sure to subscribe if you're new here, and you can grab the source code from Angular Firebase dotcom. The first thing we need to answer is what is TensorFlow? And the simple answer is that it's a library for performing mathematical computations. But it's famous for building deep neural networks that power some of the most impressive artificial intelligence technology that we have in the world today. The sad part of this answer is that if you don't have a background in machine learning, TensorFlow is not going to be very easy to use or very useful to you. That being said, I started out with 0 machine learning knowledge a couple years ago, and I've worked my way up to an expert rake on Kaggle, which is the hub for data science competition around the world. If you wanna get really good at machine learning, I recommend spending about half your time with traditional courses such as Coursera or Google's Machine Learning crash course, and spend the other half of your time applying what you learn in data science competitions on Kaggle. But be careful because data science competitions can be very, very addicting. If you're watching this video with 0 machine learning background, I still wanna give you a high level overview of how machine learning models work. The typical narrative goes something like this. We have a set of data and each item in that dataset has a label telling us what it is. In this video, we're using a dataset called MNIST, which is a series of images of hand drawn digits. Because you're most likely a human being, it's easy for you to recognize what these digits are.
machine learning
neural networks
deep learning
artificial intelligence
deepgram
data
data analysis
But to a computer, each image just looks like a matrix of values and it has no way to discern the context or actual meaning of the image itself. A typical image recognition convolutional neural network is going to look at many samples of these images and attempt to extract features out that will distinguish one digit from another. The algorithm starts off completely random then slowly corrects itself after seeing more and more samples of these images. So that's a very high level overview of how a TensorFlow model might work. What I wanna show you now is how we build and train a model and then make predictions from it in the browser. I'm going to generate a new Angular app with the Angular CLI, then we'll CDN the app and install TensorFlow. Js. For this first part, what I wanna do is show you how to build the most basic possible TensorFlow model which would just be a linear regression. So we'll feed TensorFlow an array of values that should predict some other value. Think of the relationship between a person's height and weight. You should be able to use their height to roughly predict their weight. We will train the machine learning model in Angular when this component is initialized. And then when the user enters a new value into the form field, we'll run a new prediction with that TensorFlow model. If you're more of a visual person, The actual dataset would look something like this with the blue dots being our samples and then the red line being the model that TensorFlow generates. Let's go ahead and code this up by opening the app component in Angular, then we'll import TensorFlow under the namespace of TF.
neural networks
machine learning
visualization
artificial intelligence
modeling
photography
deep learning
Then I'm setting up a property called linear model that defines the actual TensorFlow model that will train and make predictions.
We're going to use the sequential API in TensorFlow. Js which is a little bit higher level and easier to work with. It's very similar to Keras if you have a Python background. Then I'll also set up a property for the prediction value so we can show it in the front end UI. In this demo, we're only training some randomly generated static data, So we'll do that as soon as the component is initialized. TensorFlow. Js works in promises, so it tends to be really easy to work with in JavaScript apps. We'll set up an asynchronous function here called train new model. Then we'll define an empty model by just calling TF sequential. In this case, we're just performing a linear regression, and we can do that by adding a dense layer or a fully connected layer that will output a shape of 1 and also taking input with a shape 1. In larger neural networks, you have multiple layers, and the dense layer keeps all of the neurons in each layer connected together. But since we're only drawing a single straight line through the data, we don't really need to worry about that. We just need to define the loss metric, which in this case will be mean squared error.
neural networks
machine learning
modeling
deep learning
In other words, we're just trying to minimize the mean squared error of the predictions from the algorithm.
Then we also define an optimizer which is the function that determines how we reach this actual minimum error point. SGD stands for stochastic gradient descent, And there's a number of different optimizers that you can choose from. So that's actually our entire model right there. Now we just need to train it by feeding it some data. But we need to feed it that data in the form of a tensor. But you might be asking yourself, what the hell is a tensor? For our purposes, the easiest way to think about it is just as an array.
neural networks
machine learning
artificial intelligence
For example, a one dimensional tensor would look the same as an array of numbers. The actual tensor terminology is not something you really need to get tripped up with especially when you're just learning TensorFlow. As you can see here, we have one tensor named xs. When you have a variable named x in machine learning, That means it's your actual training features for the dataset. The y variable, on the other hand, is the label for that data. It's the actual thing that we want our machine learning algorithm to predict. Depending on the size of the dataset, training the model can take a long time. But in this case, we just have a few values, so it's going to happen very quick. And we make that happen by calling linear model fit with our x and y data. The end result there is that we have a statistical model that can now predict values based on this dataset that we've provided. To make a prediction, we'll call linear model predict then pass in the value in the form of a tensor. That's going to return a prediction in the form of a tensor. So now we just need to convert that tensor into something that we can use in Angular. Everything in TensorFlow runs in the context of a session. So to get out the data we actually want, we create an array and then call data sync on the tensor. Now we'll switch over to our HTML and run this linear prediction method whenever the form value changes on this HTML form input. If we pull up the app in the browser, we should get the results of our linear regression.
machine learning
data
neural networks
artificial intelligence
modeling
If you enter a smaller number, the prediction should decrease.
If you enter a larger number, it should increase. Now that you know the basics, let's do something a little more complex. Neural networks have really revolutionized image recognition problems. In this example, we're going to take an image of a handwritten digit using an HTML canvas, And then we'll have our neural network predict what the actual value of that number is. The training process for an algorithm like this is way too CPU and memory intensive to run-in the browser. But one of the really cool things about TensorFlow. Js is that we can import models that have been trained in Python. So that means you can build your own machine learning model on your local system and then upload it to your web app so your users can use it on the web. I included my model in the source code for this project, but I recommend going to Kaggle and participating in the digit recognizer competition using Keras to build your own from scratch. Keras will allow you to export your trained model, And then TensorFlow has a tool that allows you to convert that model into a format that can be used specifically by TensorFlow.
neural networks
machine learning
programming
deep learning
artificial intelligence
Js. To install that tool, you need to install TensorFlow. Js with the Python package manager pip. Then it gives you a command that will take the Keras file and not only convert it to TensorFlow. Js, but it will also shard the file so you can load it efficiently on the web. For right now, I'm just using the assets directory in Angular, but you'd wanna use a cloud storage bucket in the real world.
Now that our model has been converted to TensorFlow, I'm going to go back into the app component and add a load model method to n g on a net. And a quick side note, I'm using this drawable directive to create the HTML canvas. I'm not going to show you that code, but it's in the main repo if you wanna check it out. You'll notice here that this time our model is typed to just a plain TensorFlow model. That's the lower level API that's generally a little more difficult to use if you're a newcomer. Then this time, our predictions will be an array of 10 different values where each value represents the probability that that index is the correct value for the drawing on the canvas. So we can take the index with the highest probability and assume that that's the number that was drawn on the canvas. Since our model is already pretrained, we can just load it with a simple one liner of TensorFlow load model. But our predict method is going to be a little more tricky. The first thing that makes it more tricky is that we need to think about memory management. Neural networks can use a lot of memory, and we want to avoid memory leaks in our client side app. To do that, we can run all of our code inside this TensorFlow tidy method. That will ensure that all of our tensors are released from memory when the prediction is finished. The actual image data that we want is coming from the HTML canvas that we're extracting seen as the image data type. You can pass in a variety of different types here and it will convert the image into a tensor that can be used in the TensorFlow graph. In our case, this model was trained with images that were 28 pixels by 28 pixels with just one color channel. So we need to reshape our data into that shape. The first value here is the batch size, which is going to be 1 because we just have one image, then 28 pixels by 28 pixels and one color channel. The Python model that I trained for this demo is using an integer data type, But TensorFlow requires it to be a floating point, so we can change that by calling TensorFlow cast because all data types in a tensor need to be the same. So now that we have our data in the correct format, we can just pass it into our model and call predict. And just like last time, it returns the prediction as a tensor, So we'll convert that to an array and call data sync. In this demo, I've set up my app to run this predict method every time a new image is drawn into the canvas. It's not a perfect model, but it's about 90% accurate with my handwriting. And you can clone this repo to try it out for yourself. But if you're curious to visualize how the neural networks under the hood, I recommend going to the TensorFlow playground. This was actually the original inspiration to to TensorFlow. Js, and it shows you how neural networks build complex boundaries around datasets to make predictions. I'm gonna go ahead and wrap things up there. If you wanna see more TensorFlow content, please let me know in the comments. And if you wanna learn how to build modern JavaScript apps for any form, consider becoming a pro member at angularfirebase.com.
probability
neural networks
machine learning
data
geometry
modeling
visualization
artificial intelligence
deep learning
You'll get access to a huge library of exclusive content as well as 1 on 1 project support. Thanks for watching, and I'll see you soon.
websites
multimedia
streaming
The most exciting thing to happen last week was the announcement of TensorFlow. Js, a JavaScript library that allows us to build, train, and make predictions from machine learning models directly in the browser. We'll use TensorFlow. Js with Angular to take a hand drawn digit image and then predict its actual numeric value. I'm super excited about this because it opens up a whole new world of possibilities to combine web development with machine learning. Make sure to subscribe if you're new here, and you can grab the source code from Angular Firebase dotcom. The first thing we need to answer is what is TensorFlow? And the simple answer is that it's a library for performing mathematical computations. But it's famous for building deep neural networks that power some of the most impressive artificial intelligence technology that we have in the world today. The sad part of this answer is that if you don't have a background in machine learning, TensorFlow is not going to be very easy to use or very useful to you. That being said, I started out with 0 machine learning knowledge a couple years ago, and I've worked my way up to an expert rake on Kaggle, which is the hub for data science competition around the world. If you wanna get really good at machine learning, I recommend spending about half your time with traditional courses such as Coursera or Google's Machine Learning crash course, and spend the other half of your time applying what you learn in data science competitions on Kaggle. But be careful because data science competitions can be very, very addicting. If you're watching this video with 0 machine learning background, I still wanna give you a high level overview of how machine learning models work. The typical narrative goes something like this. We have a set of data and each item in that dataset has a label telling us what it is. In this video, we're using a dataset called MNIST, which is a series of images of hand drawn digits. Because you're most likely a human being, it's easy for you to recognize what these digits are. But to a computer, each image just looks like a matrix of values and it has no way to discern the context or actual meaning of the image itself. A typical image recognition convolutional neural network is going to look at many samples of these images and attempt to extract features out that will distinguish one digit from another. The algorithm starts off completely random then slowly corrects itself after seeing more and more samples of these images. So that's a very high level overview of how a TensorFlow model might work. What I wanna show you now is how we build and train a model and then make predictions from it in the browser. I'm going to generate a new Angular app with the Angular CLI, then we'll CDN the app and install TensorFlow. Js. For this first part, what I wanna do is show you how to build the most basic possible TensorFlow model which would just be a linear regression. So we'll feed TensorFlow an array of values that should predict some other value. Think of the relationship between a person's height and weight. You should be able to use their height to roughly predict their weight. We will train the machine learning model in Angular when this component is initialized. And then when the user enters a new value into the form field, we'll run a new prediction with that TensorFlow model. If you're more of a visual person, The actual dataset would look something like this with the blue dots being our samples and then the red line being the model that TensorFlow generates. Let's go ahead and code this up by opening the app component in Angular, then we'll import TensorFlow under the namespace of TF. Then I'm setting up a property called linear model that defines the actual TensorFlow model that will train and make predictions. We're going to use the sequential API in TensorFlow. Js which is a little bit higher level and easier to work with. It's very similar to Keras if you have a Python background. Then I'll also set up a property for the prediction value so we can show it in the front end UI. In this demo, we're only training some randomly generated static data, So we'll do that as soon as the component is initialized. TensorFlow. Js works in promises, so it tends to be really easy to work with in JavaScript apps. We'll set up an asynchronous function here called train new model. Then we'll define an empty model by just calling TF sequential. In this case, we're just performing a linear regression, and we can do that by adding a dense layer or a fully connected layer that will output a shape of 1 and also taking input with a shape 1. In larger neural networks, you have multiple layers, and the dense layer keeps all of the neurons in each layer connected together. But since we're only drawing a single straight line through the data, we don't really need to worry about that. We just need to define the loss metric, which in this case will be mean squared error. In other words, we're just trying to minimize the mean squared error of the predictions from the algorithm. Then we also define an optimizer which is the function that determines how we reach this actual minimum error point. SGD stands for stochastic gradient descent, And there's a number of different optimizers that you can choose from. So that's actually our entire model right there. Now we just need to train it by feeding it some data. But we need to feed it that data in the form of a tensor. But you might be asking yourself, what the hell is a tensor? For our purposes, the easiest way to think about it is just as an array. For example, a one dimensional tensor would look the same as an array of numbers. The actual tensor terminology is not something you really need to get tripped up with especially when you're just learning TensorFlow. As you can see here, we have one tensor named xs. When you have a variable named x in machine learning, That means it's your actual training features for the dataset. The y variable, on the other hand, is the label for that data. It's the actual thing that we want our machine learning algorithm to predict. Depending on the size of the dataset, training the model can take a long time. But in this case, we just have a few values, so it's going to happen very quick. And we make that happen by calling linear model fit with our x and y data. The end result there is that we have a statistical model that can now predict values based on this dataset that we've provided. To make a prediction, we'll call linear model predict then pass in the value in the form of a tensor. That's going to return a prediction in the form of a tensor. So now we just need to convert that tensor into something that we can use in Angular. Everything in TensorFlow runs in the context of a session. So to get out the data we actually want, we create an array and then call data sync on the tensor. Now we'll switch over to our HTML and run this linear prediction method whenever the form value changes on this HTML form input. If we pull up the app in the browser, we should get the results of our linear regression. If you enter a smaller number, the prediction should decrease. If you enter a larger number, it should increase. Now that you know the basics, let's do something a little more complex. Neural networks have really revolutionized image recognition problems. In this example, we're going to take an image of a handwritten digit using an HTML canvas, And then we'll have our neural network predict what the actual value of that number is. The training process for an algorithm like this is way too CPU and memory intensive to run-in the browser. But one of the really cool things about TensorFlow. Js is that we can import models that have been trained in Python. So that means you can build your own machine learning model on your local system and then upload it to your web app so your users can use it on the web. I included my model in the source code for this project, but I recommend going to Kaggle and participating in the digit recognizer competition using Keras to build your own from scratch. Keras will allow you to export your trained model, And then TensorFlow has a tool that allows you to convert that model into a format that can be used specifically by TensorFlow. Js. To install that tool, you need to install TensorFlow. Js with the Python package manager pip. Then it gives you a command that will take the Keras file and not only convert it to TensorFlow. Js, but it will also shard the file so you can load it efficiently on the web. For right now, I'm just using the assets directory in Angular, but you'd wanna use a cloud storage bucket in the real world. Now that our model has been converted to TensorFlow, I'm going to go back into the app component and add a load model method to n g on a net. And a quick side note, I'm using this drawable directive to create the HTML canvas. I'm not going to show you that code, but it's in the main repo if you wanna check it out. You'll notice here that this time our model is typed to just a plain TensorFlow model. That's the lower level API that's generally a little more difficult to use if you're a newcomer. Then this time, our predictions will be an array of 10 different values where each value represents the probability that that index is the correct value for the drawing on the canvas. So we can take the index with the highest probability and assume that that's the number that was drawn on the canvas. Since our model is already pretrained, we can just load it with a simple one liner of TensorFlow load model. But our predict method is going to be a little more tricky. The first thing that makes it more tricky is that we need to think about memory management. Neural networks can use a lot of memory, and we want to avoid memory leaks in our client side app. To do that, we can run all of our code inside this TensorFlow tidy method. That will ensure that all of our tensors are released from memory when the prediction is finished. The actual image data that we want is coming from the HTML canvas that we're extracting seen as the image data type. You can pass in a variety of different types here and it will convert the image into a tensor that can be used in the TensorFlow graph. In our case, this model was trained with images that were 28 pixels by 28 pixels with just one color channel. So we need to reshape our data into that shape. The first value here is the batch size, which is going to be 1 because we just have one image, then 28 pixels by 28 pixels and one color channel. The Python model that I trained for this demo is using an integer data type, But TensorFlow requires it to be a floating point, so we can change that by calling TensorFlow cast because all data types in a tensor need to be the same. So now that we have our data in the correct format, we can just pass it into our model and call predict. And just like last time, it returns the prediction as a tensor, So we'll convert that to an array and call data sync. In this demo, I've set up my app to run this predict method every time a new image is drawn into the canvas. It's not a perfect model, but it's about 90% accurate with my handwriting. And you can clone this repo to try it out for yourself. But if you're curious to visualize how the neural networks under the hood, I recommend going to the TensorFlow playground. This was actually the original inspiration to to TensorFlow. Js, and it shows you how neural networks build complex boundaries around datasets to make predictions. I'm gonna go ahead and wrap things up there. If you wanna see more TensorFlow content, please let me know in the comments. And if you wanna learn how to build modern JavaScript apps for any form, consider becoming a pro member at angularfirebase.com. You'll get access to a huge library of exclusive content as well as 1 on 1 project support. Thanks for watching, and I'll see you soon.
The
the
most
most
exciting
exciting
thing
thing
to
to
happen
happen
last
last
week
week
was
was
the
the
announcement
announcement
of
of
TensorFlow.
tensorflow
Js,
js
a
a
JavaScript
javascript
library
library
that
that
allows
allows
us
us
to
to
build,
build
train,
train
and
and
make
make
predictions
predictions
from
from
machine
machine
learning
learning
models
models
directly
directly
in
in
the
the
browser.
browser
We'll
we'll
use
use
TensorFlow.
tensorflow
Js
js
with
with
Angular
angular
to
to
take
take
a
a
hand
hand
drawn
drawn
digit
digit
image
image
and
and
then
then
predict
predict
its
its
actual
actual
numeric
numeric
value.
value
I'm
i'm
super
super
excited
excited
about
about
this
this
because
because
it
it
opens
opens
up
up
a
a
whole
whole
new
new
world
world
of
of
possibilities
possibilities
to
to
combine
combine
web
web
development
development
with
with
machine
machine
learning.
learning
Make
make
sure
sure
to
to
subscribe
subscribe
if
if
you're
you're
new
new
here,
here
and
and
you
you
can
can
grab
grab
the
the
source
source
code
code
from
from
Angular
angular
Firebase
firebase
dotcom.
dotcom
The
the
first
first
thing
thing
we
we
need
need
to
to
answer
answer
is
is
what
what
is
is
TensorFlow?
tensorflow
And
and
the
the
simple
simple
answer
answer
is
is
that
that
it's
it's
a
a
library
library
for
for
performing
performing
mathematical
mathematical
computations.
computations
But
but
it's
it's
famous
famous
for
for
building
building
deep
deep
neural
neural
networks
networks
that
that
power
power
some
some
of
of
the
the
most
most
impressive
impressive
artificial
artificial
intelligence
intelligence
technology
technology
that
that
we
we
have
have
in
in
the
the
world
world
today.
today
The
the
sad
sad
part
part
of
of
this
this
answer
answer
is
is
that
that
if
if
you
you
don't
don't
have
have
a
a
background
background
in
in
machine
machine
learning,
learning
TensorFlow
tensorflow
is
is
not
not
going
going
to
to
be
be
very
very
easy
easy
to
to
use
use
or
or
very
very
useful
useful
to
to
you.
you
That
that
being
being
said,
said
I
i
started
started
out
out
with
with
0
0
machine
machine
learning
learning
knowledge
knowledge
a
a
couple
couple
years
years
ago,
ago
and
and
I've
i've
worked
worked
my
my
way
way
up
up
to
to
an
an
expert
expert
rake
rake
on
on
Kaggle,
kaggle
which
which
is
is
the
the
hub
hub
for
for
data
data
science
science
competition
competition
around
around
the
the
world.
world
If
if
you
you
wanna
wanna
get
get
really
really
good
good
at
at
machine
machine
learning,
learning
I
i
recommend
recommend
spending
spending
about
about
half
half
your
your
time
time
with
with
traditional
traditional
courses
courses
such
such
as
as
Coursera
coursera
or
or
Google's
google's
Machine
machine
Learning
learning
crash
crash
course,
course
and
and
spend
spend
the
the
other
other
half
half
of
of
your
your
time
time
applying
applying
what
what
you
you
learn
learn
in
in
data
data
science
science
competitions
competitions
on
on
Kaggle.
kaggle
But
but
be
be
careful
careful
because
because
data
data
science
science
competitions
competitions
can
can
be
be
very,
very
very
very
addicting.
addicting
If
if
you're
you're
watching
watching
this
this
video
video
with
with
0
0
machine
machine
learning
learning
background,
background
I
i
still
still
wanna
wanna
give
give
you
you
a
a
high
high
level
level
overview
overview
of
of
how
how
machine
machine
learning
learning
models
models
work.
work
The
the
typical
typical
narrative
narrative
goes
goes
something
something
like
like
this.
this
We
we
have
have
a
a
set
set
of
of
data
data
and
and
each
each
item
item
in
in
that
that
dataset
dataset
has
has
a
a
label
label
telling
telling
us
us
what
what
it
it
is.
is
In
in
this
this
video,
video
we're
we're
using
using
a
a
dataset
dataset
called
called
MNIST,
mnist
which
which
is
is
a
a
series
series
of
of
images
images
of
of
hand
hand
drawn
drawn
digits.
digits
Because
because
you're
you're
most
most
likely
likely
a
a
human
human
being,
being
it's
it's
easy
easy
for
for
you
you
to
to
recognize
recognize
what
what
these
these
digits
digits
are.
are
But
but
to
to
a
a
computer,
computer
each
each
image
image
just
just
looks
looks
like
like
a
a
matrix
matrix
of
of
values
values
and
and
it
it
has
has
no
no
way
way
to
to
discern
discern
the
the
context
context
or
or
actual
actual
meaning
meaning
of
of
the
the
image
image
itself.
itself
A
a
typical
typical
image
image
recognition
recognition
convolutional
convolutional
neural
neural
network
network
is
is
going
going
to
to
look
look
at
at
many
many
samples
samples
of
of
these
these
images
images
and
and
attempt
attempt
to
to
extract
extract
features
features
out
out
that
that
will
will
distinguish
distinguish
one
one
digit
digit
from
from
another.
another
The
the
algorithm
algorithm
starts
starts
off
off
completely
completely
random
random
then
then
slowly
slowly
corrects
corrects
itself
itself
after
after
seeing
seeing
more
more
and
and
more
more
samples
samples
of
of
these
these
images.
images
So
so
that's
that's
a
a
very
very
high
high
level
level
overview
overview
of
of
how
how
a
a
TensorFlow
tensorflow
model
model
might
might
work.
work
What
what
I
i
wanna
wanna
show
show
you
you
now
now
is
is
how
how
we
we
build
build
and
and
train
train
a
a
model
model
and
and
then
then
make
make
predictions
predictions
from
from
it
it
in
in
the
the
browser.
browser
I'm
i'm
going
going
to
to
generate
generate
a
a
new
new
Angular
angular
app
app
with
with
the
the
Angular
angular
CLI,
cli
then
then
we'll
we'll
CDN
cdn
the
the
app
app
and
and
install
install
TensorFlow.
tensorflow
Js.
js
For
for
this
this
first
first
part,
part
what
what
I
i
wanna
wanna
do
do
is
is
show
show
you
you
how
how
to
to
build
build
the
the
most
most
basic
basic
possible
possible
TensorFlow
tensorflow
model
model
which
which
would
would
just
just
be
be
a
a
linear
linear
regression.
regression
So
so
we'll
we'll
feed
feed
TensorFlow
tensorflow
an
an
array
array
of
of
values
values
that
that
should
should
predict
predict
some
some
other
other
value.
value
Think
think
of
of
the
the
relationship
relationship
between
between
a
a
person's
person's
height
height
and
and
weight.
weight
You
you
should
should
be
be
able
able
to
to
use
use
their
their
height
height
to
to
roughly
roughly
predict
predict
their
their
weight.
weight
We
we
will
will
train
train
the
the
machine
machine
learning
learning
model
model
in
in
Angular
angular
when
when
this
this
component
component
is
is
initialized.
initialized
And
and
then
then
when
when
the
the
user
user
enters
enters
a
a
new
new
value
value
into
into
the
the
form
form
field,
field
we'll
we'll
run
run
a
a
new
new
prediction
prediction
with
with
that
that
TensorFlow
tensorflow
model.
model
If
if
you're
you're
more
more
of
of
a
a
visual
visual
person,
person
The
the
actual
actual
dataset
dataset
would
would
look
look
something
something
like
like
this
this
with
with
the
the
blue
blue
dots
dots
being
being
our
our
samples
samples
and
and
then
then
the
the
red
red
line
line
being
being
the
the
model
model
that
that
TensorFlow
tensorflow
generates.
generates
Let's
let's
go
go
ahead
ahead
and
and
code
code
this
this
up
up
by
by
opening
opening
the
the
app
app
component
component
in
in
Angular,
angular
then
then
we'll
we'll
import
import
TensorFlow
tensorflow
under
under
the
the
namespace
namespace
of
of
TF.
tf
Then
then
I'm
i'm
setting
setting
up
up
a
a
property
property
called
called
linear
linear
model
model
that
that
defines
defines
the
the
actual
actual
TensorFlow
tensorflow
model
model
that
that
will
will
train
train
and
and
make
make
predictions.
predictions
We're
we're
going
going
to
to
use
use
the
the
sequential
sequential
API
api
in
in
TensorFlow.
tensorflow
Js
js
which
which
is
is
a
a
little
little
bit
bit
higher
higher
level
level
and
and
easier
easier
to
to
work
work
with.
with
It's
it's
very
very
similar
similar
to
to
Keras
keras
if
if
you
you
have
have
a
a
Python
python
background.
background
Then
then
I'll
i'll
also
also
set
set
up
up
a
a
property
property
for
for
the
the
prediction
prediction
value
value
so
so
we
we
can
can
show
show
it
it
in
in
the
the
front
front
end
end
UI.
ui
In
in
this
this
demo,
demo
we're
we're
only
only
training
training
some
some
randomly
randomly
generated
generated
static
static
data,
data
So
so
we'll
we'll
do
do
that
that
as
as
soon
soon
as
as
the
the
component
component
is
is
initialized.
initialized
TensorFlow.
tensorflow
Js
js
works
works
in
in
promises,
promises
so
so
it
it
tends
tends
to
to
be
be
really
really
easy
easy
to
to
work
work
with
with
in
in
JavaScript
javascript
apps.
apps
We'll
we'll
set
set
up
up
an
an
asynchronous
asynchronous
function
function
here
here
called
called
train
train
new
new
model.
model
Then
then
we'll
we'll
define
define
an
an
empty
empty
model
model
by
by
just
just
calling
calling
TF
tf
sequential.
sequential
In
in
this
this
case,
case
we're
we're
just
just
performing
performing
a
a
linear
linear
regression,
regression
and
and
we
we
can
can
do
do
that
that
by
by
adding
adding
a
a
dense
dense
layer
layer
or
or
a
a
fully
fully
connected
connected
layer
layer
that
that
will
will
output
output
a
a
shape
shape
of
of
1
1
and
and
also
also
taking
taking
input
input
with
with
a
a
shape
shape
1.
1
In
in
larger
larger
neural
neural
networks,
networks
you
you
have
have
multiple
multiple
layers,
layers
and
and
the
the
dense
dense
layer
layer
keeps
keeps
all
all
of
of
the
the
neurons
neurons
in
in
each
each
layer
layer
connected
connected
together.
together
But
but
since
since
we're
we're
only
only
drawing
drawing
a
a
single
single
straight
straight
line
line
through
through
the
the
data,
data
we
we
don't
don't
really
really
need
need
to
to
worry
worry
about
about
that.
that
We
we
just
just
need
need
to
to
define
define
the
the
loss
loss
metric,
metric
which
which
in
in
this
this
case
case
will
will
be
be
mean
mean
squared
squared
error.
error
In
in
other
other
words,
words
we're
we're
just
just
trying
trying
to
to
minimize
minimize
the
the
mean
mean
squared
squared
error
error
of
of
the
the
predictions
predictions
from
from
the
the
algorithm.
algorithm
Then
then
we
we
also
also
define
define
an
an
optimizer
optimizer
which
which
is
is
the
the
function
function
that
that
determines
determines
how
how
we
we
reach
reach
this
this
actual
actual
minimum
minimum
error
error
point.
point
SGD
sgd
stands
stands
for
for
stochastic
stochastic
gradient
gradient
descent,
descent
And
and
there's
there's
a
a
number
number
of
of
different
different
optimizers
optimizers
that
that
you
you
can
can
choose
choose
from.
from
So
so
that's
that's
actually
actually
our
our
entire
entire
model
model
right
right
there.
there
Now
now
we
we
just
just
need
need
to
to
train
train
it
it
by
by
feeding
feeding
it
it
some
some
data.
data
But
but
we
we
need
need
to
to
feed
feed
it
it
that
that
data
data
in
in
the
the
form
form
of
of
a
a
tensor.
tensor
But
but
you
you
might
might
be
be
asking
asking
yourself,
yourself
what
what
the
the
hell
hell
is
is
a
a
tensor?
tensor
For
for
our
our
purposes,
purposes
the
the
easiest
easiest
way
way
to
to
think
think
about
about
it
it
is
is
just
just
as
as
an
an
array.
array
For
for
example,
example
a
a
one
one
dimensional
dimensional
tensor
tensor
would
would
look
look
the
the
same
same
as
as
an
an
array
array
of
of
numbers.
numbers
The
the
actual
actual
tensor
tensor
terminology
terminology
is
is
not
not
something
something
you
you
really
really
need
need
to
to
get
get
tripped
tripped
up
up
with
with
especially
especially
when
when
you're
you're
just
just
learning
learning
TensorFlow.
tensorflow
As
as
you
you
can
can
see
see
here,
here
we
we
have
have
one
one
tensor
tensor
named
named
xs.
xs
When
when
you
you
have
have
a
a
variable
variable
named
named
x
x
in
in
machine
machine
learning,
learning
That
that
means
means
it's
it's
your
your
actual
actual
training
training
features
features
for
for
the
the
dataset.
dataset
The
the
y
y
variable,
variable
on
on
the
the
other
other
hand,
hand
is
is
the
the
label
label
for
for
that
that
data.
data
It's
it's
the
the
actual
actual
thing
thing
that
that
we
we
want
want
our
our
machine
machine
learning
learning
algorithm
algorithm
to
to
predict.
predict
Depending
depending
on
on
the
the
size
size
of
of
the
the
dataset,
dataset
training
training
the
the
model
model
can
can
take
take
a
a
long
long
time.
time
But
but
in
in
this
this
case,
case
we
we
just
just
have
have
a
a
few
few
values,
values
so
so
it's
it's
going
going
to
to
happen
happen
very
very
quick.
quick
And
and
we
we
make
make
that
that
happen
happen
by
by
calling
calling
linear
linear
model
model
fit
fit
with
with
our
our
x
x
and
and
y
y
data.
data
The
the
end
end
result
result
there
there
is
is
that
that
we
we
have
have
a
a
statistical
statistical
model
model
that
that
can
can
now
now
predict
predict
values
values
based
based
on
on
this
this
dataset
dataset
that
that
we've
we've
provided.
provided
To
to
make
make
a
a
prediction,
prediction
we'll
we'll
call
call
linear
linear
model
model
predict
predict
then
then
pass
pass
in
in
the
the
value
value
in
in
the
the
form
form
of
of
a
a
tensor.
tensor
That's
that's
going
going
to
to
return
return
a
a
prediction
prediction
in
in
the
the
form
form
of
of
a
a
tensor.
tensor
So
so
now
now
we
we
just
just
need
need
to
to
convert
convert
that
that
tensor
tensor
into
into
something
something
that
that
we
we
can
can
use
use
in
in
Angular.
angular
Everything
everything
in
in
TensorFlow
tensorflow
runs
runs
in
in
the
the
context
context
of
of
a
a
session.
session
So
so
to
to
get
get
out
out
the
the
data
data
we
we
actually
actually
want,
want
we
we
create
create
an
an
array
array
and
and
then
then
call
call
data
data
sync
sync
on
on
the
the
tensor.
tensor
Now
now
we'll
we'll
switch
switch
over
over
to
to
our
our
HTML
html
and
and
run
run
this
this
linear
linear
prediction
prediction
method
method
whenever
whenever
the
the
form
form
value
value
changes
changes
on
on
this
this
HTML
html
form
form
input.
input
If
if
we
we
pull
pull
up
up
the
the
app
app
in
in
the
the
browser,
browser
we
we
should
should
get
get
the
the
results
results
of
of
our
our
linear
linear
regression.
regression
If
if
you
you
enter
enter
a
a
smaller
smaller
number,
number
the
the
prediction
prediction
should
should
decrease.
decrease
If
if
you
you
enter
enter
a
a
larger
larger
number,
number
it
it
should
should
increase.
increase
Now
now
that
that
you
you
know
know
the
the
basics,
basics
let's
let's
do
do
something
something
a
a
little
little
more
more
complex.
complex
Neural
neural
networks
networks
have
have
really
really
revolutionized
revolutionized
image
image
recognition
recognition
problems.
problems
In
in
this
this
example,
example
we're
we're
going
going
to
to
take
take
an
an
image
image
of
of
a
a
handwritten
handwritten
digit
digit
using
using
an
an
HTML
html
canvas,
canvas
And
and
then
then
we'll
we'll
have
have
our
our
neural
neural
network
network
predict
predict
what
what
the
the
actual
actual
value
value
of
of
that
that
number
number
is.
is
The
the
training
training
process
process
for
for
an
an
algorithm
algorithm
like
like
this
this
is
is
way
way
too
too
CPU
cpu
and
and
memory
memory
intensive
intensive
to
to
run-in
run-in
the
the
browser.
browser
But
but
one
one
of
of
the
the
really
really
cool
cool
things
things
about
about
TensorFlow.
tensorflow
Js
js
is
is
that
that
we
we
can
can
import
import
models
models
that
that
have
have
been
been
trained
trained
in
in
Python.
python
So
so
that
that
means
means
you
you
can
can
build
build
your
your
own
own
machine
machine
learning
learning
model
model
on
on
your
your
local
local
system
system
and
and
then
then
upload
upload
it
it
to
to
your
your
web
web
app
app
so
so
your
your
users
users
can
can
use
use
it
it
on
on
the
the
web.
web
I
i
included
included
my
my
model
model
in
in
the
the
source
source
code
code
for
for
this
this
project,
project
but
but
I
i
recommend
recommend
going
going
to
to
Kaggle
kaggle
and
and
participating
participating
in
in
the
the
digit
digit
recognizer
recognizer
competition
competition
using
using
Keras
keras
to
to
build
build
your
your
own
own
from
from
scratch.
scratch
Keras
keras
will
will
allow
allow
you
you
to
to
export
export
your
your
trained
trained
model,
model
And
and
then
then
TensorFlow
tensorflow
has
has
a
a
tool
tool
that
that
allows
allows
you
you
to
to
convert
convert
that
that
model
model
into
into
a
a
format
format
that
that
can
can
be
be
used
used
specifically
specifically
by
by
TensorFlow.
tensorflow
Js.
js
To
to
install
install
that
that
tool,
tool
you
you
need
need
to
to
install
install
TensorFlow.
tensorflow
Js
js
with
with
the
the
Python
python
package
package
manager
manager
pip.
pip
Then
then
it
it
gives
gives
you
you
a
a
command
command
that
that
will
will
take
take
the
the
Keras
keras
file
file
and
and
not
not
only
only
convert
convert
it
it
to
to
TensorFlow.
tensorflow
Js,
js
but
but
it
it
will
will
also
also
shard
shard
the
the
file
file
so
so
you
you
can
can
load
load
it
it
efficiently
efficiently
on
on
the
the
web.
web
For
for
right
right
now,
now
I'm
i'm
just
just
using
using
the
the
assets
assets
directory
directory
in
in
Angular,
angular
but
but
you'd
you'd
wanna
wanna
use
use
a
a
cloud
cloud
storage
storage
bucket
bucket
in
in
the
the
real
real
world.
world
Now
now
that
that
our
our
model
model
has
has
been
been
converted
converted
to
to
TensorFlow,
tensorflow
I'm
i'm
going
going
to
to
go
go
back
back
into
into
the
the
app
app
component
component
and
and
add
add
a
a
load
load
model
model
method
method
to
to
n
n
g
g
on
on
a
a
net.
net
And
and
a
a
quick
quick
side
side
note,
note
I'm
i'm
using
using
this
this
drawable
drawable
directive
directive
to
to
create
create
the
the
HTML
html
canvas.
canvas
I'm
i'm
not
not
going
going
to
to
show
show
you
you
that
that
code,
code
but
but
it's
it's
in
in
the
the
main
main
repo
repo
if
if
you
you
wanna
wanna
check
check
it
it
out.
out
You'll
you'll
notice
notice
here
here
that
that
this
this
time
time
our
our
model
model
is
is
typed
typed
to
to
just
just
a
a
plain
plain
TensorFlow
tensorflow
model.
model
That's
that's
the
the
lower
lower
level
level
API
api
that's
that's
generally
generally
a
a
little
little
more
more
difficult
difficult
to
to
use
use
if
if
you're
you're
a
a
newcomer.
newcomer
Then
then
this
this
time,
time
our
our
predictions
predictions
will
will
be
be
an
an
array
array
of
of
10
10
different
different
values
values
where
where
each
each
value
value
represents
represents
the
the
probability
probability
that
that
that
that
index
index
is
is
the
the
correct
correct
value
value
for
for
the
the
drawing
drawing
on
on
the
the
canvas.
canvas
So
so
we
we
can
can
take
take
the
the
index
index
with
with
the
the
highest
highest
probability
probability
and
and
assume
assume
that
that
that's
that's
the
the
number
number
that
that
was
was
drawn
drawn
on
on
the
the
canvas.
canvas
Since
since
our
our
model
model
is
is
already
already
pretrained,
pretrained
we
we
can
can
just
just
load
load
it
it
with
with
a
a
simple
simple
one
one
liner
liner
of
of
TensorFlow
tensorflow
load
load
model.
model
But
but
our
our
predict
predict
method
method
is
is
going
going
to
to
be
be
a
a
little
little
more
more
tricky.
tricky
The
the
first
first
thing
thing
that
that
makes
makes
it
it
more
more
tricky
tricky
is
is
that
that
we
we
need
need
to
to
think
think
about
about
memory
memory
management.
management
Neural
neural
networks
networks
can
can
use
use
a
a
lot
lot
of
of
memory,
memory
and
and
we
we
want
want
to
to
avoid
avoid
memory
memory
leaks
leaks
in
in
our
our
client
client
side
side
app.
app
To
to
do
do
that,
that
we
we
can
can
run
run
all
all
of
of
our
our
code
code
inside
inside
this
this
TensorFlow
tensorflow
tidy
tidy
method.
method
That
that
will
will
ensure
ensure
that
that
all
all
of
of
our
our
tensors
tensors
are
are
released
released
from
from
memory
memory
when
when
the
the
prediction
prediction
is
is
finished.
finished
The
the
actual
actual
image
image
data
data
that
that
we
we
want
want
is
is
coming
coming
from
from
the
the
HTML
html
canvas
canvas
that
that
we're
we're
extracting
extracting
seen
seen
as
as
the
the
image
image
data
data
type.
type
You
you
can
can
pass
pass
in
in
a
a
variety
variety
of
of
different
different
types
types
here
here
and
and
it
it
will
will
convert
convert
the
the
image
image
into
into
a
a
tensor
tensor
that
that
can
can
be
be
used
used
in
in
the
the
TensorFlow
tensorflow
graph.
graph
In
in
our
our
case,
case
this
this
model
model
was
was
trained
trained
with
with
images
images
that
that
were
were
28
28
pixels
pixels
by
by
28
28
pixels
pixels
with
with
just
just
one
one
color
color
channel.
channel
So
so
we
we
need
need
to
to
reshape
reshape
our
our
data
data
into
into
that
that
shape.
shape
The
the
first
first
value
value
here
here
is
is
the
the
batch
batch
size,
size
which
which
is
is
going
going
to
to
be
be
1
1
because
because
we
we
just
just
have
have
one
one
image,
image
then
then
28
28
pixels
pixels
by
by
28
28
pixels
pixels
and
and
one
one
color
color
channel.
channel
The
the
Python
python
model
model
that
that
I
i
trained
trained
for
for
this
this
demo
demo
is
is
using
using
an
an
integer
integer
data
data
type,
type
But
but
TensorFlow
tensorflow
requires
requires
it
it
to
to
be
be
a
a
floating
floating
point,
point
so
so
we
we
can
can
change
change
that
that
by
by
calling
calling
TensorFlow
tensorflow
cast
cast
because
because
all
all
data
data
types
types
in
in
a
a
tensor
tensor
need
need
to
to
be
be
the
the
same.
same
So
so
now
now
that
that
we
we
have
have
our
our
data
data
in
in
the
the
correct
correct
format,
format
we
we
can
can
just
just
pass
pass
it
it
into
into
our
our
model
model
and
and
call
call
predict.
predict
And
and
just
just
like
like
last
last
time,
time
it
it
returns
returns
the
the
prediction
prediction
as
as
a
a
tensor,
tensor
So
so
we'll
we'll
convert
convert
that
that
to
to
an
an
array
array
and
and
call
call
data
data
sync.
sync
In
in
this
this
demo,
demo
I've
i've
set
set
up
up
my
my
app
app
to
to
run
run
this
this
predict
predict
method
method
every
every
time
time
a
a
new
new
image
image
is
is
drawn
drawn
into
into
the
the
canvas.
canvas
It's
it's
not
not
a
a
perfect
perfect
model,
model
but
but
it's
it's
about
about
90%
90%
accurate
accurate
with
with
my
my
handwriting.
handwriting
And
and
you
you
can
can
clone
clone
this
this
repo
repo
to
to
try
try
it
it
out
out
for
for
yourself.
yourself
But
but
if
if
you're
you're
curious
curious
to
to
visualize
visualize
how
how
the
the
neural
neural
networks
networks
under
under
the
the
hood,
hood
I
i
recommend
recommend
going
going
to
to
the
the
TensorFlow
tensorflow
playground.
playground
This
this
was
was
actually
actually
the
the
original
original
inspiration
inspiration
to
to
to
to
TensorFlow.
tensorflow
Js,
js
and
and
it
it
shows
shows
you
you
how
how
neural
neural
networks
networks
build
build
complex
complex
boundaries
boundaries
around
around
datasets
datasets
to
to
make
make
predictions.
predictions
I'm
i'm
gonna
gonna
go
go
ahead
ahead
and
and
wrap
wrap
things
things
up
up
there.
there
If
if
you
you
wanna
wanna
see
see
more
more
TensorFlow
tensorflow
content,
content
please
please
let
let
me
me
know
know
in
in
the
the
comments.
comments
And
and
if
if
you
you
wanna
wanna
learn
learn
how
how
to
to
build
build
modern
modern
JavaScript
javascript
apps
apps
for
for
any
any
form,
form
consider
consider
becoming
becoming
a
a
pro
pro
member
member
at
at
angularfirebase.com.
angularfirebase.com
You'll
you'll
get
get
access
access
to
to
a
a
huge
huge
library
library
of
of
exclusive
exclusive
content
content
as
as
well
well
as
as
1
1
on
on
1
1
project
project
support.
support
Thanks
thanks
for
for
watching,
watching
and
and
I'll
i'll
see
see
you
you
soon.
soon
success
The machine learning model in discussing TensorFlow is easy to use and can predict numeric values in high volume machine learning models. The use of a dataset called MNIST and a sequential API to train a model is recommended for creating a prediction model. A tool called Kaggle is recommended for building models and a digit recognizable competition is suggested. The importance of memory management and reshaping data into a certain format is emphasized. The speaker promotes a community and a community page for more TensorFlow content.
TensorFlow.js Quick Start
The most exciting thing to happen last week was the announcement of TensorFlow. Js, a JavaScript library that allows us to build, train, and make predictions from machine learning models directly in the browser. We'll use TensorFlow. Js with Angular to take a hand drawn digit image and then predict its actual numeric value. I'm super excited about this because it opens up a whole new world of possibilities to combine web development with machine learning. Make sure to subscribe if you're new here, and you can grab the source code from Angular Firebase dotcom. The first thing we need to answer is what is TensorFlow? And the simple answer is that it's a library for performing mathematical computations. But it's famous for building deep neural networks that power some of the most impressive artificial intelligence technology that we have in the world today. The sad part of this answer is that if you don't have a background in machine learning, TensorFlow is not going to be very easy to use or very useful to you. That being said, I started out with 0 machine learning knowledge a couple years ago, and I've worked my way up to an expert rake on Kaggle, which is the hub for data science competition around the world. If you wanna get really good at machine learning, I recommend spending about half your time with traditional courses such as Coursera or Google's Machine Learning crash course, and spend the other half of your time applying what you learn in data science competitions on Kaggle. But be careful because data science competitions can be very, very addicting. If you're watching this video with 0 machine learning background, I still wanna give you a high level overview of how machine learning models work. The typical narrative goes something like this. We have a set of data and each item in that dataset has a label telling us what it is. In this video, we're using a dataset called MNIST, which is a series of images of hand drawn digits. Because you're most likely a human being, it's easy for you to recognize what these digits are. But to a computer, each image just looks like a matrix of values and it has no way to discern the context or actual meaning of the image itself. A typical image recognition convolutional neural network is going to look at many samples of these images and attempt to extract features out that will distinguish one digit from another. The algorithm starts off completely random then slowly corrects itself after seeing more and more samples of these images. So that's a very high level overview of how a TensorFlow model might work. What I wanna show you now is how we build and train a model and then make predictions from it in the browser. I'm going to generate a new Angular app with the Angular CLI, then we'll CDN the app and install TensorFlow. Js. For this first part, what I wanna do is show you how to build the most basic possible TensorFlow model which would just be a linear regression. So we'll feed TensorFlow an array of values that should predict some other value. Think of the relationship between a person's height and weight. You should be able to use their height to roughly predict their weight. We will train the machine learning model in Angular when this component is initialized. And then when the user enters a new value into the form field, we'll run a new prediction with that TensorFlow model. If you're more of a visual person, The actual dataset would look something like this with the blue dots being our samples and then the red line being the model that TensorFlow generates. Let's go ahead and code this up by opening the app component in Angular, then we'll import TensorFlow under the namespace of TF. Then I'm setting up a property called linear model that defines the actual TensorFlow model that will train and make predictions. We're going to use the sequential API in TensorFlow. Js which is a little bit higher level and easier to work with. It's very similar to Keras if you have a Python background. Then I'll also set up a property for the prediction value so we can show it in the front end UI. In this demo, we're only training some randomly generated static data, So we'll do that as soon as the component is initialized. TensorFlow. Js works in promises, so it tends to be really easy to work with in JavaScript apps. We'll set up an asynchronous function here called train new model. Then we'll define an empty model by just calling TF sequential. In this case, we're just performing a linear regression, and we can do that by adding a dense layer or a fully connected layer that will output a shape of 1 and also taking input with a shape 1. In larger neural networks, you have multiple layers, and the dense layer keeps all of the neurons in each layer connected together. But since we're only drawing a single straight line through the data, we don't really need to worry about that. We just need to define the loss metric, which in this case will be mean squared error. In other words, we're just trying to minimize the mean squared error of the predictions from the algorithm. Then we also define an optimizer which is the function that determines how we reach this actual minimum error point. SGD stands for stochastic gradient descent, And there's a number of different optimizers that you can choose from. So that's actually our entire model right there. Now we just need to train it by feeding it some data. But we need to feed it that data in the form of a tensor. But you might be asking yourself, what the hell is a tensor? For our purposes, the easiest way to think about it is just as an array. For example, a one dimensional tensor would look the same as an array of numbers. The actual tensor terminology is not something you really need to get tripped up with especially when you're just learning TensorFlow. As you can see here, we have one tensor named xs. When you have a variable named x in machine learning, That means it's your actual training features for the dataset. The y variable, on the other hand, is the label for that data. It's the actual thing that we want our machine learning algorithm to predict. Depending on the size of the dataset, training the model can take a long time. But in this case, we just have a few values, so it's going to happen very quick. And we make that happen by calling linear model fit with our x and y data. The end result there is that we have a statistical model that can now predict values based on this dataset that we've provided. To make a prediction, we'll call linear model predict then pass in the value in the form of a tensor. That's going to return a prediction in the form of a tensor. So now we just need to convert that tensor into something that we can use in Angular. Everything in TensorFlow runs in the context of a session. So to get out the data we actually want, we create an array and then call data sync on the tensor. Now we'll switch over to our HTML and run this linear prediction method whenever the form value changes on this HTML form input. If we pull up the app in the browser, we should get the results of our linear regression. If you enter a smaller number, the prediction should decrease. If you enter a larger number, it should increase. Now that you know the basics, let's do something a little more complex. Neural networks have really revolutionized image recognition problems. In this example, we're going to take an image of a handwritten digit using an HTML canvas, And then we'll have our neural network predict what the actual value of that number is. The training process for an algorithm like this is way too CPU and memory intensive to run-in the browser. But one of the really cool things about TensorFlow. Js is that we can import models that have been trained in Python. So that means you can build your own machine learning model on your local system and then upload it to your web app so your users can use it on the web. I included my model in the source code for this project, but I recommend going to Kaggle and participating in the digit recognizer competition using Keras to build your own from scratch. Keras will allow you to export your trained model, And then TensorFlow has a tool that allows you to convert that model into a format that can be used specifically by TensorFlow. Js. To install that tool, you need to install TensorFlow. Js with the Python package manager pip. Then it gives you a command that will take the Keras file and not only convert it to TensorFlow. Js, but it will also shard the file so you can load it efficiently on the web. For right now, I'm just using the assets directory in Angular, but you'd wanna use a cloud storage bucket in the real world. Now that our model has been converted to TensorFlow, I'm going to go back into the app component and add a load model method to n g on a net. And a quick side note, I'm using this drawable directive to create the HTML canvas. I'm not going to show you that code, but it's in the main repo if you wanna check it out. You'll notice here that this time our model is typed to just a plain TensorFlow model. That's the lower level API that's generally a little more difficult to use if you're a newcomer. Then this time, our predictions will be an array of 10 different values where each value represents the probability that that index is the correct value for the drawing on the canvas. So we can take the index with the highest probability and assume that that's the number that was drawn on the canvas. Since our model is already pretrained, we can just load it with a simple one liner of TensorFlow load model. But our predict method is going to be a little more tricky. The first thing that makes it more tricky is that we need to think about memory management. Neural networks can use a lot of memory, and we want to avoid memory leaks in our client side app. To do that, we can run all of our code inside this TensorFlow tidy method. That will ensure that all of our tensors are released from memory when the prediction is finished. The actual image data that we want is coming from the HTML canvas that we're extracting seen as the image data type. You can pass in a variety of different types here and it will convert the image into a tensor that can be used in the TensorFlow graph. In our case, this model was trained with images that were 28 pixels by 28 pixels with just one color channel. So we need to reshape our data into that shape. The first value here is the batch size, which is going to be 1 because we just have one image, then 28 pixels by 28 pixels and one color channel. The Python model that I trained for this demo is using an integer data type, But TensorFlow requires it to be a floating point, so we can change that by calling TensorFlow cast because all data types in a tensor need to be the same. So now that we have our data in the correct format, we can just pass it into our model and call predict. And just like last time, it returns the prediction as a tensor, So we'll convert that to an array and call data sync. In this demo, I've set up my app to run this predict method every time a new image is drawn into the canvas. It's not a perfect model, but it's about 90% accurate with my handwriting. And you can clone this repo to try it out for yourself. But if you're curious to visualize how the neural networks under the hood, I recommend going to the TensorFlow playground. This was actually the original inspiration to to TensorFlow. Js, and it shows you how neural networks build complex boundaries around datasets to make predictions. I'm gonna go ahead and wrap things up there. If you wanna see more TensorFlow content, please let me know in the comments. And if you wanna learn how to build modern JavaScript apps for any form, consider becoming a pro member at angularfirebase.com. You'll get access to a huge library of exclusive content as well as 1 on 1 project support. Thanks for watching, and I'll see you soon.
The machine learning model in discussing TensorFlow is easy to use and can predict numeric values in high volume machine learning models. The use of a dataset called MNIST and a sequential API to train a model is recommended for creating a prediction model. A tool called Kaggle is recommended for building models and a digit recognizable competition is suggested. The importance of memory management and reshaping data into a certain format is emphasized. The speaker promotes a community and a community page for more TensorFlow content.