you do the same for additionx and are numpy tensors def naive_add(xy)assert len( shape= assert shape = shape copy(for in range( shape[ ])for in range( shape[ ]) [ij+ [ijreturn avoid overwriting the input tensor on the same principleyou can do element-wise multiplicationsubtractionand so on in practicewhen dealing with numpy arraysthese operations are available as welloptimized built-in numpy functionswhich themselves delegate the heavy lifting to basic linear algebra subprograms (blasimplementation if you have one installed (which you shouldblas are low-levelhighly parallelefficient tensor-manipulation routines that are typically implemented in fortran or soin numpyyou can do the following element-wise operationand it will be blazing fastimport numpy as np np maximum( element-wise addition element-wise relu broadcasting our earlier naive implementation of naive_add only supports the addition of tensors with identical shapes but in the dense layer introduced earlierwe added tensor with vector what happens with addition when the shapes of the two tensors being added differwhen possibleand if there' no ambiguitythe smaller tensor will be broadcasted to match the shape of the larger tensor broadcasting consists of two steps axes (called broadcast axesare added to the smaller tensor to match the ndim of the larger tensor the smaller tensor is repeated alongside these new axes to match the full shape of the larger tensor let' look at concrete example consider with shape ( and with shape ( ,firstwe add an empty first axis to ywhose shape becomes ( thenwe repeat times alongside this new axisso that we end up with tensor with shape ( )where [ := for in range( at this pointwe can proceed to add and ybecause they have the same shape in terms of implementationno new tensor is createdbecause that would be terribly inefficient the repetition operation is entirely virtualit happens at the algorithmic level rather than at the memory level but thinking of the vector being licensed to