# Apache MXNet - Crash Course Part 2

## Basic Neural Network

This is a follow-along from the MXNet crash course: Create NN

• We create a basic Neural Network in this post.
• Follow along from previous post
• Have fun!
``````from mxnet import nd
from mxnet.gluon import nn
``````

## Create your Neural Network’s first layer

``````layer = nn.Dense(2)
layer
``````
``````Dense(None -> 2, linear)
``````
• Then initialize its weights with the default initialization method, which draws random values unformly from [-0.7, 0.7]
``````layer.initialize()
``````
• Then do a forward pass with random data. We create (3,4) shape random input x and feed into the layer to compute the output.
``````x = nd.random_uniform(-1, 1, (3, 4))
layer(x)
``````
``````[[ 0.01587485  0.03087313]
[ 0.02257253 -0.02103142]
[ 0.06961896  0.01239835]]
<NDArray 3x2 @cpu(0)>
``````
``````layer.weight.data()
``````
``````[[-0.00873779 -0.02834515  0.05484822 -0.06206018]
[ 0.06491279 -0.03182812 -0.01631819 -0.00312688]]
<NDArray 2x4 @cpu(0)>
``````

## Chain layers into a neural network

• First, a simple case that a neural network is a chain of layers. During the forward pass, we run layers sequentially one-by-one. The following code implements a famous network called LeNet through nn.Sequential
``````net = nn.Sequential()

# Add a sequence of layers.
net.add(# Similar to Dense, it is not necessary to specify the input channels
# by the argument 'in_channels', which will be automatically inferred
# in the first forward pass. Also, we apply a rely activation on the output.
# In addition, we can use a tuple to specify a non-square kerne size, such as
# 'kernel_size=(2,4)'
nn.Conv2D(channels=6, kernel_size=5, activation='relu'),
# One can also use a tuple to specify non-symmetric pool and stride sizes
nn.MaxPool2D(pool_size=2, strides=2),
nn.Conv2D(channels=16, kernel_size=3, activation='relu'),
nn.MaxPool2D(pool_size=2, strides=2),
# The dense layer will automatically reshape the 4-D output of last
# max pooling layer into the 2-D shape: (x.shape, x.size/x.shape)
nn.Dense(120, activation='relu'),
nn.Dense(84, activation='relu'),
nn.Dense(10)
)
net
``````
``````Sequential(
(0): Conv2D(None -> 6, kernel_size=(5, 5), stride=(1, 1))
(1): MaxPool2D(size=(2, 2), stride=(2, 2), padding=(0, 0), ceil_mode=False)
(2): Conv2D(None -> 16, kernel_size=(3, 3), stride=(1, 1))
(3): MaxPool2D(size=(2, 2), stride=(2, 2), padding=(0, 0), ceil_mode=False)
(4): Dense(None -> 120, Activation(relu))
(5): Dense(None -> 84, Activation(relu))
(6): Dense(None -> 10, linear)
)
``````
• The usage of nn.Sequential() is similar to nn.Dense. They are both subclasses of nn.Block.
• Now initialize the weights and run the forward pass.
``````net.initialize()

# Input shape is (batch_size, color_channels, height, width)
x = nd.random_uniform(shape=(4, 1, 28, 28))
y = net(x)
y.shape
``````
``````(4, 10)
``````
• We can use [] to index a particular layer. For example, the following accesses the 1st layer’s weight and 6th layer’s bias.
``````(net.weight.data().shape, net.bias.data().shape)
``````
``````((6, 1, 5, 5), (84,))
``````

## Create a Neural Network flexibly

• In nn.Sequential, MXNet will automatically construct the forward function that sequentially executes added layers.
• Introducing another way to contruct a network with a flexible forward function.
• Create a subclass of nn.Block and implement two methods:
• __init__ create the layers
• forward define the forward function
``````
class MixMLP(nn.Block):
def __init__(self, **kwargs):
# Run `nn.Block`'s init method
super(MixMLP, self).__init__(**kwargs)
self.blk = nn.Sequential()
nn.Dense(4, activation='relu'))
self.dense = nn.Dense(5)
def forward(self, x):
y = nd.relu(self.blk(x))
print(y)
return self.dense(y)

net = MixMLP()
net

``````
``````MixMLP(
(blk): Sequential(
(0): Dense(None -> 3, Activation(relu))
(1): Dense(None -> 4, Activation(relu))
)
(dense): Dense(None -> 5, linear)
)
``````
• Usage of net remains the same as before
``````net.initialize()
x = nd.random_uniform(shape=(2,2))
net(x)
``````
``````[[0.         0.00072302 0.00043636 0.00045482]
[0.         0.00081594 0.00049244 0.00051327]]
<NDArray 2x4 @cpu(0)>

[[ 3.9329490e-05  1.1599804e-05  4.5617679e-05 -1.8504743e-05
2.4952336e-05]
[ 4.4383840e-05  1.3090528e-05  5.1480143e-05 -2.0882841e-05
2.8159035e-05]]
<NDArray 2x5 @cpu(0)>
``````
• Accessing a particular layer’s weights can also be done similarly:
``````net.blk.weight.data()
``````
``````[[ 0.0521711  -0.02633957 -0.03170411]
[-0.01043678  0.04172656  0.05394727]
[-0.04401097  0.02518312  0.06339083]
[-0.00614183  0.02624836 -0.00232279]]
<NDArray 4x3 @cpu(0)>
``````
``````
``````
{"ft_lang_label":"__label__en","ft_lang_prob":0.69510275,"math_prob":0.9840188,"size":5308,"snap":"2020-45-2020-50","text_gpt3_token_len":1604,"char_repetition_ratio":0.09973605,"word_repetition_ratio":0.024128687,"special_character_ratio":0.36605126,"punctuation_ratio":0.20123565,"nsfw_num_words":0,"has_unicode_error":false,"math_prob_llama3":0.99748003,"pos_list":[0,1,2],"im_url_duplicate_count":[null,null,null],"WARC_HEADER":"{\"WARC-Type\":\"response\",\"WARC-Date\":\"2020-10-22T21:14:12Z\",\"WARC-Record-ID\":\"<urn:uuid:e59f2580-1038-4cd6-97eb-b75e880de644>\",\"Content-Length\":\"29354\",\"Content-Type\":\"application/http; msgtype=response\",\"WARC-Warcinfo-ID\":\"<urn:uuid:7292cee0-7c6b-46ed-898e-ad6ac09baea1>\",\"WARC-Concurrent-To\":\"<urn:uuid:5f84841b-b25c-4eda-9a1d-efee25a913ef>\",\"WARC-IP-Address\":\"185.199.108.153\",\"WARC-Target-URI\":\"http://www.mycpu.org/MXNet_CrashCourse-Part2/\",\"WARC-Payload-Digest\":\"sha1:ZVTDVRINT2UTQOIRLPFBYJJRCKZEZXDV\",\"WARC-Block-Digest\":\"sha1:ZU7WGHTCYDUYIO7ECXY2HPJOTSKOTOUT\",\"WARC-Identified-Payload-Type\":\"text/html\",\"warc_filename\":\"/cc_download/warc_2020/CC-MAIN-2020-45/CC-MAIN-2020-45_segments_1603107880038.27_warc_CC-MAIN-20201022195658-20201022225658-00317.warc.gz\"}"}