NLP Task Taxonomy

Every major NLP task predates Transformers, and every one of them is now dominated by Transformer-based approaches — but the classical approach to each is worth knowing, both for historical grounding and because classical methods are sometimes still the right, cheaper tool for a well-defined production task.

Intuition: Constrained vs. Open-Ended Output

The tasks on this page split cleanly along one axis: does the model select from a fixed, small set of outputs (sequence labeling, classification, extractive QA), or does it generate genuinely new text (translation, abstractive summarization, generative QA)? Constrained tasks are structurally safer — a classifier can't hallucinate a label that doesn't exist, extractive QA can't hallucinate an answer not present in the source — at the cost of being unable to produce anything the fixed output space doesn't already contain. Every classical/modern contrast below is really a version of this same trade-off.

Sequence Labeling

The general framing behind POS tagging and NER: assign a label to every token in a sequence, where the labels of nearby tokens are dependent on each other (not independent per-token classification). Classical: HMMs/CRFs, modeling the label sequence's transition structure explicitly (see the real Viterbi decode on Classical NLP). Modern: a BiLSTM or Transformer encoder with a classification head per token, letting the model learn label dependencies implicitly through its own representations rather than an explicit transition matrix.

Text Classification & Sentiment Analysis

Assigning a label (or set of labels) to an entire document/sentence rather than per-token. Classical: bag-of-words or TF-IDF features feeding a Logistic Regression, Naive Bayes, or SVM classifier. Real bag-of-words scoring, live — including exactly where it structurally breaks:

Still a completely reasonable, fast, interpretable baseline for well-scoped classification tasks with limited labeled data. Modern: fine-tuning a BERT-style encoder (see Attention & Transformers) or prompting an LLM directly (see Prompt Engineering) — better accuracy, especially on nuanced/context-dependent sentiment (exactly the negation-scope case above), at higher compute cost per prediction.

Machine Translation

Translating text from one language to another — the task that originally motivated the encoder-decoder Transformer architecture (Attention & Transformers — The T5 Lineage):

• Statistical Machine Translation (SMT): pre-neural approaches built from explicit translation probability tables (phrase-to-phrase translation likelihoods learned from parallel corpora) combined with a language model scoring fluency — a pipeline of separately-optimized components, not end-to-end trained.
• Neural Machine Translation (NMT): end-to-end sequence-to-sequence models (originally RNN encoder-decoder with attention, see Sequence Models — Seq2Seq with Attention, now Transformer encoder-decoder) trained directly on parallel text — a single model learns the entire mapping, substantially outperforming SMT's pipeline once enough training data and compute were available.
• Evaluation: BLEU (n-gram precision overlap against reference translations) remains the standard automatic metric despite well-known limitations. Real BLEU-style precision, computed live, on exactly the case that exposes the limitation:

See LLM Evaluation & RAGOps — Traditional Metrics for how LLM-as-judge approaches are increasingly used alongside or instead of BLEU for more nuanced translation quality assessment.

Summarization

• Extractive summarization: select and concatenate existing sentences/phrases directly from the source document — guaranteed factually grounded, but can read disjointedly since it's a selection, not a composition.
• Abstractive summarization: generate genuinely new sentences that convey the source's meaning — reads more naturally, but introduces real risk of hallucination. Toggle between the two on the same source below — one of them contains a deliberately planted factual error, exactly the faithfulness/groundedness concern covered in LLM Evaluation & RAGOps:

Modern LLMs perform abstractive summarization by default when prompted to summarize — the extractive/abstractive distinction remains relevant for understanding why a summary might be unfaithful to its source (an abstractive method structurally can be, an extractive one structurally cannot).

Question Answering

• Extractive QA: given a question and a passage, identify the exact span of the passage that answers it — framed as predicting a start and end token position, a structurally different (and simpler, more constrained) task than free-form generation. Real softmax, real argmax, real predicted span:

• Generative/open-domain QA: generate a free-form answer, potentially synthesizing information across multiple sources or requiring knowledge not explicitly present in any single retrieved passage — the task framing behind RAG, which combines retrieval (find relevant passages) with generative QA (compose an answer from them) rather than either extractive matching or ungrounded generation alone.

Code: A Real Extractive-QA Span Predictor

The exact computation the diagram above runs, in the form a real fine-tuned model's head actually takes:

import torch
import torch.nn.functional as F

def predict_span(start_logits: torch.Tensor, end_logits: torch.Tensor, tokens: list[str]) -> str:
    start_probs = F.softmax(start_logits, dim=-1)
    end_probs = F.softmax(end_logits, dim=-1)
    start_idx = torch.argmax(start_probs).item()
    end_idx = torch.argmax(end_probs).item()
    return " ".join(tokens[start_idx:end_idx + 1])  # can only ever point INTO the passage

NLP section complete. Next: LLMs & GenAI — where this entire task taxonomy gets absorbed into general-purpose, prompted Transformer models rather than one specialized architecture per task.