Notes on AI and machine learning.

Posts about papers and technical topics I find useful or interesting, with derivations, examples, and references where they help.

01 / Post index

Post tags are separate from the task/method/goal taxonomy used for Research.

2 posts

Series

Topics

Approach

Series · Part 2

25 Aug 2026Core path about 15 minutes · optional section about 7 minutes

Rereading the Transformer Upgrade Path

Rereading the Transformer Upgrade Path (2): How RoPE Writes Relative Position into Attention

Starting from a single identity for two-dimensional rotations, this article explains how RoPE uses the same sin/cos signals to rotate Queries and Keys so that position enters the attention score only through relative displacement.

Series · Transformer upgrade pathTopic · Position encodingTopic · TransformerApproach · Mathematical derivation

Read post

Series · Part 1

24 Aug 2026About 15 minutes

Rereading the Transformer Upgrade Path

Rereading the Transformer Upgrade Path (1): Tracing Sinusoidal Positional Encoding to Its Source

Starting from the idea that equal relative distances should induce equal positional interactions, this article uses clocks, the unit circle, and multiple frequencies to explain sinusoidal positional encoding, long-range decorrelation, and its relationship to RoPE.

Series · Transformer upgrade pathTopic · Position encodingTopic · TransformerApproach · Mathematical derivation

Read post