Neural Mastery
← Practice
Scaled Dot-Product Attention Scoreeasy

Scaled Dot-Product Attention Score

easy Inside an LLM 10–15 min

NumPy allowed · plain Python earns 10 bonus points

Attention step 1/4

Tokens attend to tokens

Q · Kᵀ / √d → softmax → weighted values, computed in the page.
Open lesson →
01tokens
02Q K V
03scores
04attention
modelslearnrelationships
models32%
learn38%
relationships30%

Q · Kᵀ / √d scores normalized with softmax

Implement the core scaled dot-product similarity score that powers transformer attention.

Task

Implement attention_score(q, k) that computes dot(q, k) / sqrt(d_k). This is the fundamental score in every transformer attention layer.

Function Signature

attention_score(q: list[float], k: list[float]) -> float

Examples

Example 1: Basic 4-dim vectors
Input: q = [1, 0, 1, 0] k = [1, 1, 0, 0]
Output: 0.5
Example 2: Two-dim vectors
Input: q = [1, 2] k = [3, 4]
Output: 7.7781745930520225
Example 3: Zero query vector
Input: q = [0, 0] k = [1, 1]
Output: 0

Constraints

  • q and k must have equal length, else raise ValueError.
  • Use math.sqrt for the scaling factor.
  • A plain-Python solution earns 10 bonus points.
Python3Saved ✓
def attention_score(q: list[float], k: list[float]) -> float:
"""Compute the scaled dot-product attention score between query q and key k.
Score = dot(q, k) / sqrt(d_k) where d_k = len(q).
Args:
q: Query vector (list of floats).
k: Key vector of same length as q (list of floats).
Returns:
float: The scaled attention score.
Raises:
ValueError: If q and k have different lengths.
"""
# Your implementation here
pass

Case 1: Basic 4-dim vectors
Input: q = [1, 0, 1, 0] k = [1, 1, 0, 0]
Expected: 0.5
Case 2: Two-dim vectors
Input: q = [1, 2] k = [3, 4]
Expected: 7.7781745930520225
Case 3: Zero query vector
Input: q = [0, 0] k = [1, 1]
Expected: 0