advanced deep-learning best practices (continuedyou can grasp this concept with signal-processing analogyif you have an audioprocessing pipeline that consists of series of operationseach of which takes as input the output of the previous operationthen if one operation crops your signal to low-frequency range (for example - khz)the operations downstream will never be able to recover the dropped frequencies any loss of information is permanent residual connectionsby reinjecting earlier information downstreampartially solve this issue for deep-learning models vanishing gradients in deep learning backpropagationthe master algorithm used to train deep neural networksworks by propagating feedback signal from the output loss down to earlier layers if this feedback signal has to be propagated through deep stack of layersthe signal may become tenuous or even be lost entirelyrendering the network untrainable this issue is known as vanishing gradients this problem occurs both with deep networks and with recurrent networks over very long sequences--in both casesa feedback signal must be propagated through long series of operations you're already familiar with the solution that the lstm layer uses to address this problem in recurrent networksit introduces carry track that propagates information parallel to the main processing track residual connections work in similar way in feedforward deep networksbut they're even simplerthey introduce purely linear information carry track parallel to the main layer stackthus helping to propagate gradients through arbitrarily deep stacks of layers layer weight sharing one more important feature of the functional api is the ability to reuse layer instance several times when you call layer instance twiceinstead of instantiating new layer for each callyou reuse the same weights with every call this allows you to build models that have shared branches--several branches that all share the same knowledge and perform the same operations that isthey share the same representations and learn these representations simultaneously for different sets of inputs for exampleconsider model that attempts to assess the semantic similarity between two sentences the model has two inputs (the two sentences to compareand outputs score between and where means unrelated sentences and means sentences that are either identical or reformulations of each other such model could be useful in many applicationsincluding deduplicating natural-language queries in dialog system in this setupthe two input sentences are interchangeablebecause semantic similarity is symmetrical relationshipthe similarity of to is identical to the similarity of to for this reasonit wouldn' make sense to learn two independent models for licensed to