Donkey Small

DonkeySmall

AI & ML interests

None yet

Recent Activity

reacted to schuler's post with โค๏ธ 3 days ago
๐Ÿ”ฎ GPT-3 implemented in pure Free Pascal! https://github.com/joaopauloschuler/gpt-3-for-pascal This implementation follows the GPT-3 Small architecture from the landmark paper "Language Models are Few-Shot Learners": ``` โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ Input Layer โ”‚ โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค โ”‚ Token & Positional โ”‚ โ”‚ Embedding โ”‚ โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค โ”‚ 12x Transformer โ”‚ โ”‚ Blocks โ”‚ โ”‚ - 12 heads โ”‚ โ”‚ - 768 hidden dims โ”‚ โ”‚ - 3072 intermediate โ”‚ โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค โ”‚ Output Layer โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ ``` Clean Pascal Implementation ``` for CntLayer := 1 to {Layers=}12 do begin Result.AddTransformerBlockCAI( {Heads=}12, {intermediate dimensions=}4*768, {NoForward=}true, {HasNorm=}true, false ); end; ```
liked a Space 3 months ago
ZhengPeng7/BiRefNet_demo
View all activity

Organizations

OCR-Datasets's profile picture