A neural network is an
interconnected group of nodes, akin
to the vast network of neurons in the
human brain.
Learning algorithms for neural networks use local search to
choose the weights that will get the right output for each input
during training. The most common training technique is the
backpropagation algorithm.[105] Neural networks learn to model
complex relationships between inputs and outputs and find
patterns in data. In theory, a neural network can learn any
function.[106]
In feedforward neural networks the signal passes in only one
direction.[107] Recurrent neural networks feed the output signal
back into the input, which allows short-term memories of previous
input events. Long short term memory is the most successful
network architecture for recurrent networks.[108] Perceptrons[109]
use only a single layer of neurons; deep learning[110] uses multiple
layers. Convolutional neural networks strengthen the connection
between neurons that are "close" to each other—this is especially important in image processing, where a
local set of neurons must identify an "edge" before the network can identify an object.[111]
Deep learning[110] uses several layers of neurons between the network's
inputs and outputs. The multiple layers can progressively extract higher-
level features from the raw input. For example, in image processing, lower
layers may identify edges, while higher layers may identify the concepts
relevant to a human such as digits, letters, or faces.[112]
Deep learning has profoundly improved the performance of programs in
many important subfields of artificial intelligence, including computer
vision, speech recognition, natural language processing, image
classification,[113] and others. The reason that deep learning performs so
well in so many applications is not known as of 2023.[114] The sudden success of deep learning in 2012–
2015 did not occur because of some new discovery or theoretical breakthrough (deep neural networks and
backpropagation had been described by many people, as far back as the 1950s)[i] but because of two
factors: the incredible increase in computer power (including the hundred-fold increase in speed by
switching to GPUs) and the availability of vast amounts of training data, especially the giant curated
datasets used for benchmark testing, such as ImageNet.[j]
Generative pre-trained transformers (GPT) are large language models (LLMs) that generate text based on
the semantic relationships between words in sentences. Text-based GPT models are pretrained on a large
corpus of text that can be from the Internet. The pretraining consists of predicting the next token (a token
being usually a word, subword, or punctuation). Throughout this pretraining, GPT models accumulate
knowledge about the world and can then generate human-like text by repeatedly predicting the next
token. Typically, a subsequent training phase makes the model more truthful, useful, and harmless,
usually with a technique called reinforcement learning from human feedback (RLHF). Current GPT
Deep learning
GPT