https://www.youngju.dev/blog 천천히 올바르게. AI Researcher & DevOps Engineer Youngju's tech blog. GPU/CUDA, LLM, MLOps, Kubernetes AI workloads, distributed training, and data engineering. ko fjvbn2003@gmail.com (Youngju Kim) fjvbn2003@gmail.com (Youngju Kim) Fri, 26 Jun 2026 00:00:00 GMT https://www.youngju.dev/blog/llm/2026-06-26-positional-encoding-rope-explained.en https://www.youngju.dev/blog/llm/2026-06-26-positional-encoding-rope-explained.en Starting from why Transformers need positional information, this article explains sinusoidal, learned, and relative positional encodings step by step, then RoPE and ALiBi. It connects length extrapolation and context extension (NTK, YaRN) through to multimodal M-RoPE and summarizes common pitfalls. Fri, 26 Jun 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) llmpositional-encodingropealibilong-contexttransformer https://www.youngju.dev/blog/llm/2026-06-26-positional-encoding-rope-explained.ja https://www.youngju.dev/blog/llm/2026-06-26-positional-encoding-rope-explained.ja なぜTransformerに位置情報が必要かから始め、sinusoidal、learned、相対位置エンコーディング、そしてRoPEとALiBiを段階的に説明します。長さの外挿とコンテキスト拡張(NTK, YaRN)、マルチモーダルのM-RoPEまでつなげ、よくある落とし穴を整理します。 Fri, 26 Jun 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) llmpositional-encodingropealibilong-contexttransformer https://www.youngju.dev/blog/llm/2026-06-26-positional-encoding-rope-explained https://www.youngju.dev/blog/llm/2026-06-26-positional-encoding-rope-explained 왜 Transformer에 위치 정보가 필요한지부터 시작해 sinusoidal, learned, 상대 위치 인코딩, 그리고 RoPE와 ALiBi를 단계적으로 설명합니다. 길이 외삽과 컨텍스트 확장(NTK, YaRN), 멀티모달의 M-RoPE까지 연결하고 흔한 함정을 정리합니다. Fri, 26 Jun 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) llmpositional-encodingropealibilong-contexttransformer