STRATA
Skip to study
XLVII

SOURCE RECORD / 2017

Attention without recurrence

DocumentedR01 · 2026-11-18
Scope of this record

KNOWN refers to the architecture and task results reported in the original paper, not a universal claim about intelligence or current model quality.

Statement

The Transformer paper introduces a sequence architecture based on attention rather than recurrent or convolutional sequence-processing layers and evaluates it on translation tasks. [source]

Equation

Unrecorded

The connection is scoped to this source. No broader cross-domain equality is established by this entry.

Observation

The abstract motivates an architecture that is more parallelizable than the recurrent approaches used as its comparison. Its empirical claims are tied to specified machine-translation tasks. [source]

Progressive questions

What does attention mix, and what information about order must be supplied? Which baseline has comparable data and compute? Does an attention weight explain a model decision, or is that a separate question?

Source response

The source reports translation results and training-efficiency comparisons. Those historical results should not be transported into a present-day benchmark without matching the experimental setup. [source]

Interpretation

XLVII proposal: expose the actual matrices and token ordering in a small demonstration. A highlighted attention edge is a numerical weight, not automatically a causal explanation or evidence of human-like understanding.

Conclusion

This entry anchors a particular architectural transition. Wider cognitive interpretations remain unestablished here. [source]

Next experiment

Proposed: build a small fixed-input attention calculation with independently checked tensor shapes and a reference implementation. Compare outputs before and after controlled token changes. Only add training or latency claims after measuring a specified model, dataset and device.

Work through a related example.

IV.10 / Transformers and attention

Calculate a tiny attention mixture.

Use two value vectors, (1,0) and (0,1), with attention weights 0.8 and 0.2. Their weighted output is (0.8,0.2). Change the weights while keeping the values fixed, then change the values while keeping the weights fixed. This separates what is selected from what is carried by the selected representations.

IV.12 / Large models and agents

Evaluate a public-document lookup task.

Give a system a fixed set of documents and ask it to find a date with its supporting passage. Record the chosen tool call, returned evidence, final answer, and whether the citation actually supports the date. Introduce a missing document and inspect whether the system correctly preserves the unresolved state rather than inventing a result.

X.06 / Optimization, control, and decision theory

Solve a small allocation problem.

Let x and y be nonnegative activity levels with x + y ≤ 4, x ≤ 2, and y ≤ 3. Maximize the illustrative value 3x + 2y. The feasible choice x = 2, y = 2 gives value 10; compare the other corner points to verify the optimum. Then change one coefficient and inspect whether the preferred allocation changes.

Follow earlier and later work.

Before

  • Proposed connection · UNKNOWNPrediction and residual error

    Editorial research connection to investigate; not a verified correspondence, causal lineage, or mathematical equivalence.

After

  • Proposed connection · UNKNOWNCompute has a budget

    Editorial research connection to investigate; not a verified correspondence, causal lineage, or mathematical equivalence.

Follow the chain
  1. Prediction and residual error · 1 link away · UNKNOWN
  2. Compute has a budget · 1 link away · UNKNOWN
  3. Memory as a collective state · 2 links away · UNKNOWN
  4. The chemical compass · 2 links away · UNKNOWN
  5. What survives a replication? · 2 links away · UNKNOWN
  6. Information is not meaning · 3 links away · UNKNOWN
  7. Compression, reopened · 3 links away · UNKNOWN
  8. Compression → classification · 4 links away · UNKNOWN
  9. Erasure has conditions · 4 links away · UNKNOWN
  10. Dynamic Markov modelling · 5 links away · KNOWN
  11. Reversibility and the record · 5 links away · UNKNOWN

Placement.

Technology

This entry records the cited source and a proposed next investigation. It makes no claim of historical abandonment, suppression, or independent reproduction.

Sources.

  1. Attention Is All You Need

    Ashish Vaswani et al. / author manuscript · arXiv:1706.03762; original 12 June 2017; abstract.

    Abstract and manuscript metadata inspected; equations, training logs and checkpoints not independently verified. Checked 2026-11-18.

Record history
  1. 2026-11-18 / R1 / SOURCE_SEEDED

    Source-limited editorial seed with explicit inspection scope. Research questions and proposed experiments are not executed results.