https://www.youngju.dev/blog 천천히 올바르게. AI Researcher & DevOps Engineer Youngju's blog. GPU/CUDA, LLM, MLOps, Kubernetes AI workloads, and data engineering — plus mindset essays on confidence, routines, health, and sport psychology. ko fjvbn2003@gmail.com (Youngju Kim) fjvbn2003@gmail.com (Youngju Kim) Sat, 22 Aug 2026 00:00:00 GMT https://www.youngju.dev/blog/ai/tiny-models-04-image-to-text.en https://www.youngju.dev/blog/ai/tiny-models-04-image-to-text.en We joined a CNN encoder to a transformer decoder and captioned Fashion-MNIST images. With 1.37 million parameters and ten minutes of training, the label hit rate reached 91%. This part looks at what cross-attention does in an encoder-decoder setup, and why the EOS bug that dropped Part 3 accuracy to 7.5% never appeared in this code — with the two functions placed side by side. Sat, 22 Aug 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) aicaptioningmultimodaltransformerpytorchtiny-models https://www.youngju.dev/blog/ai/tiny-models-04-image-to-text.ja https://www.youngju.dev/blog/ai/tiny-models-04-image-to-text.ja CNN エンコーダとトランスフォーマーデコーダをつないで、Fashion-MNIST の画像にキャプションをつけるモデルを作りました。137万パラメータ、10分の学習でラベル的中率91%です。エンコーダ・デコーダ構造で cross-attention が何をするのか、そして第3回で正解率を7.5%に落とした EOS バグがなぜこのコードになかったのかを、2つの関数を並べて確認します。 Sat, 22 Aug 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) aicaptioningmultimodaltransformerpytorchtiny-models https://www.youngju.dev/blog/ai/tiny-models-04-image-to-text https://www.youngju.dev/blog/ai/tiny-models-04-image-to-text CNN 인코더와 트랜스포머 디코더를 이어 붙여 Fashion-MNIST 이미지에 캡션을 다는 모델을 만들었습니다. 137만 파라미터, 10분 학습으로 라벨 적중률 91%가 나왔습니다. 인코더-디코더 구조에서 cross-attention이 무엇을 하는지, 그리고 3편에서 정확도를 7.5%로 떨어뜨렸던 EOS 버그가 왜 이 코드에는 없었는지를 두 함수를 나란히 놓고 확인합니다. Sat, 22 Aug 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) aicaptioningmultimodaltransformerpytorchtiny-models https://www.youngju.dev/blog/ai/tiny-models-04-image-to-text.zh https://www.youngju.dev/blog/ai/tiny-models-04-image-to-text.zh 把 CNN 编码器接上 Transformer 解码器，给 Fashion-MNIST 图像配字幕。137万参数、十分钟训练，标签命中率达到 91%。本文讲清编码器-解码器结构中 cross-attention 在做什么，并把两个函数并排放在一起，说明让第3篇正确率跌到 7.5% 的 EOS bug 为何没有出现在这份代码里。 Sat, 22 Aug 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) aicaptioningmultimodaltransformerpytorchtiny-models