Suvradeep Das

Suvradeep Das

he/him

Language models & social good

Posts by Tags

Datascience

Demystifying Tokenization: The First Step in Building LLMs

less than 1 minute read

Published:

Ever wondered how large language models (LLMs) like GPT-4 understand text? Well, they don’t—at least, not like humans do! Instead, they break text down into tokens, the fundamental building blocks of all AI-generated language.

Deepseek

Dissecting the Attention Mechanism and Transformers

less than 1 minute read

Published:

Transformers have revolutionized NLP, powering models like GPT-4o, Claude, and DeepSeek. But what makes them so effective? The answer lies in their attention mechanism, which enables models to focus on relevant information rather than processing everything equally.

LLM

Dissecting the Attention Mechanism and Transformers

less than 1 minute read

Published:

Transformers have revolutionized NLP, powering models like GPT-4o, Claude, and DeepSeek. But what makes them so effective? The answer lies in their attention mechanism, which enables models to focus on relevant information rather than processing everything equally.

Demystifying Tokenization: The First Step in Building LLMs

less than 1 minute read

Published:

Ever wondered how large language models (LLMs) like GPT-4 understand text? Well, they don’t—at least, not like humans do! Instead, they break text down into tokens, the fundamental building blocks of all AI-generated language.

Naturallanguageprocessing

Demystifying Tokenization: The First Step in Building LLMs

less than 1 minute read

Published:

Ever wondered how large language models (LLMs) like GPT-4 understand text? Well, they don’t—at least, not like humans do! Instead, they break text down into tokens, the fundamental building blocks of all AI-generated language.

Seq2seq

Dissecting the Attention Mechanism and Transformers

less than 1 minute read

Published:

Transformers have revolutionized NLP, powering models like GPT-4o, Claude, and DeepSeek. But what makes them so effective? The answer lies in their attention mechanism, which enables models to focus on relevant information rather than processing everything equally.

Tokenization

Demystifying Tokenization: The First Step in Building LLMs

less than 1 minute read

Published:

Ever wondered how large language models (LLMs) like GPT-4 understand text? Well, they don’t—at least, not like humans do! Instead, they break text down into tokens, the fundamental building blocks of all AI-generated language.

Transformers

Dissecting the Attention Mechanism and Transformers

less than 1 minute read

Published:

Transformers have revolutionized NLP, powering models like GPT-4o, Claude, and DeepSeek. But what makes them so effective? The answer lies in their attention mechanism, which enables models to focus on relevant information rather than processing everything equally.