Notes on AI and machine learning.

Posts about papers and technical topics I find useful or interesting, with derivations, examples, and references where they help.

01 / Post index

Post tags are separate from the task/method/goal taxonomy used for Research.

1 post

Series

Topics

Approach

Series · Part 1

24 Aug 202620 min read

Rereading the Transformer Upgrade Path

Rereading the Transformer Upgrade Path (1): Tracing Sinusoidal Position Encoding to Its Source

A derivation of how sinusoidal position encoding exposes relative offsets through rotations, with careful limits on frequency schedules, decay, Taylor expansions, and the bridge to RoPE.

Series · Transformer upgrade pathTopic · Position encodingTopic · TransformerApproach · Mathematical derivation

Read post