Choosing Code Models: Completion vs Chat, FIM, and Licenses

• Code Models Are Two Different Products
• Completion Needs Fill-in-the-Middle
• Do Not Chat with a Base Model
• Conversational Code Models
• Agentic Coding Is Yet Another Requirement
• Why Context Length Matters Especially for Code
• The Category Where Licensing Needs the Most Care
• Code Example
• The Order to Decide In
• Try It Yourself
• Series Navigation
• References

Model details were read directly from the Hugging Face pages on 2026-08-12. Model cards and licenses change, so check the original again before you use anything.

Code Models Are Two Different Products

The first fork in choosing a code model is not size but purpose. Completing at the cursor inside an editor and producing an explained answer to a question demand different abilities.

Completion is all latency. The result has to arrive in the pause after the user stops typing, so it usually means a small model, and it has to see the code after the cursor as well as before it. Conversation can afford a few seconds but demands explanation and reasoning. Try to solve both with one model and you usually end up mediocre at each.

Completion Needs Fill-in-the-Middle

Editor completion is hard because there is code after the cursor. Continue from the prefix alone and you collide with a function already defined below. That is why you need training that supplies both prefix and suffix and asks the model to fill the middle.

The google/codegemma-7b card goes as far as naming the tokens to use: <|fim_prefix|>, <|fim_suffix|>, <|fim_middle|>, and <|file_separator|>. If you are building an editor extension yourself you have to match that format exactly, and since tokens differ by model, swapping models also means changing the prompt assembly code.

Be careful: instruct variants do not necessarily carry a FIM statement. The Qwen/Qwen2.5-Coder-7B-Instruct page does not state fill-in-the-middle. For completion, checking the base model in the same family is the right move.

Do Not Chat with a Base Model

The bigcode/starcoder2-7b card states outright that it is not an instruction model and that commands like asking it to write a function that computes the square root do not work well. The Qwen/Qwen2.5-Coder-1.5B card likewise states that using base language models for conversations is not recommended.

Miss those sentences, throw a question at a base model, and the model starts continuing your question. Then the team concludes the model is bad. In fact they simply used an object built for a different purpose. Start by checking whether Instruct appears at the end of the repository name.

The bigcode/starcoder2-7b context statement is also worth a look. It says 16,384, but with a note about a 4,096 sliding window attention. The maximum length on the label and the range actually attended to in one pass can differ. The training data is stated as The Stack v2 across 17 programming languages.

Conversational Code Models

The training composition the deepseek-ai/deepseek-coder-6.7b-instruct card gives is unusually concrete. A ratio of 87 percent code to 13 percent English and Chinese natural language tells you something about which languages it will explain code in. The card states that DeepSeek Coder supports commercial use and points to the LICENSE-MODEL document for details.

Agentic Coding Is Yet Another Requirement

Letting a model read files and run commands to drive work forward requires a function-calling format.

Qwen/Qwen3-Coder-30B-A3B-Instruct is apache-2.0 with 30.5B total and 3.3B activated, and mentions tool-calling capability along with a specially designed function call format for agentic coding. It states native support for 262,144 tokens of context, extendable up to 1M tokens using Yarn. The card also states that this model supports only non-thinking mode and does not generate <think></think> blocks, and advises reducing the context to a shorter value such as 32,768 if out-of-memory issues appear.

Because it is an MoE architecture, the arithmetic from the earlier post applies directly. Speed follows the 3.3B activated, but all 30.5B of the weights still have to be in memory.

Why Context Length Matters Especially for Code

In code, context is not conversation length but reference scope. Fixing one function means bringing in its call sites, its type definitions, and its test file, and those three usually live in different files.

So the 16K of deepseek-ai/deepseek-coder-6.7b-instruct and the 262,144 of Qwen/Qwen3-Coder-30B-A3B-Instruct are not a quality gap but a difference in the size of task each can hold. For a single-file edit, 16K is plenty; for work that sweeps a whole repository, you are in a different band from the start.

The Qwen/Qwen2.5-Coder-7B-Instruct card warns that vLLM only supports static YaRN, which means the scaling factor stays constant regardless of input length and can affect performance on shorter texts. For a service dominated by short requests, like completion, leaving the extension setting permanently on can cost you.

The Category Where Licensing Needs the Most Care

Code model output goes straight into your product, which puts license verification one notch above the other categories.

bigcode/starcoder2-7b is marked bigcode-openrail-m, and the page names it BigCode OpenRAIL-M v1. That family publishes weights widely while attaching clauses restricting particular uses, so assuming it works like Apache or MIT is a mistake. google/codegemma-7b is gemma, and deepseek-ai/deepseek-coder-6.7b-instruct uses its own deepseek identifier pointing at a LICENSE-MODEL document. The Qwen code models are marked apache-2.0.

Read the quality limitations on the same cards alongside. bigcode/starcoder2-7b states that generated code is not guaranteed to work as intended, can be inefficient, and may contain bugs or exploits. That sentence directly contradicts any expectation that a code review step can be dropped.

Read the full license text yourself and put commercial use through legal review. This post only relays what the cards say and is not legal advice.

Code Example

# Example: ask a conversational code model for a refactor
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "Qwen/Qwen2.5-Coder-7B-Instruct"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype="auto", device_map="auto")

messages = [
    {"role": "system", "content": "You are a careful code reviewer."},
    {"role": "user", "content": "Harden the error handling here.\n\ndef load(p):\n    return open(p).read()"},
]
prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

inputs = tok(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=512)
print(tok.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

The Order to Decide In

1. Split first between completion, conversation, and agentic use.
2. For completion, keep only repositories that state fill-in-the-middle, and confirm the token format.
3. Check whether Instruct is in the repository name, and do not chat with base models.
4. Count how many files one task must reference and set your context requirement from that.
5. Record the license field verbatim and open the linked full text to find use restrictions.
6. Move the code quality limitations the card states into your code review requirements.

Try It Yourself

• Prompt Engineering Guide — practice shaping code request prompts.
• GPU VRAM Calculator for LLMs — check whether a small completion model fits your dev machine.
• LLM API Cost Calculator — compare the cost structure of self-hosting against API calls.

Series Navigation

• Previous: Vision and Multimodal: Image Understanding, OCR, and VLMs
• Next: Licenses and Deployment: Reading the Types and Verifying Quantized Builds

References

• Every value in the tables was read directly from that model page on Hugging Face on 2026-08-12. Anything absent from the page is written as not stated.
• On google/codegemma-7b the prose describing a 7 billion pretrained variant and the 9B params in the spec box disagree. Both are reported as written.
• Read the full license text yourself and put commercial use through legal review.