fix(llm-server): Phi-4-mini-instruct incompatible with transformers 5.x (tied_weights_keys API change) #11

New issue

Closed

opened 2026-04-02 17:00:43 -07:00 by pyr0ball · 1 comment

pyr0ball commented

2026-04-02 17:00:43 -07:00

Owner

Problem

Phi-4-mini-instruct ships a custom modeling_phi3.py. Two issues are partially fixed; one remains:

SlidingWindowCache removed from transformers.cache_utils — fixed with try/except stub
LossKwargs removed from transformers.utils — fixed with TypedDict stub
get_expanded_tied_weights_keys() in transformers 5.x expects tied_mapping as a dict; Phi-3 returns a list — NOT YET FIXED, causes:

AttributeError: 'list' object has no attribute 'keys'
  File transformers/modeling_utils.py, line 2471, in get_expanded_tied_weights_keys
    if all(regex.match(k) for k in tied_mapping.keys() | tied_mapping.values())

Impact

Phi-4-mini-instruct cannot load. Qwen2.5-3B-Instruct is sole production model.

Preferred fix

Patch the local modeling_phi3.py to override get_tied_weights_keys() returning a dict (transformers 5.x format):

def get_tied_weights_keys(self):
    return {"lm_head.weight": "model.embed_tokens.weight"}

Files

/Library/Assets/LLM/vllm/models/Phi-4-mini-instruct/modeling_phi3.py — local model code
circuitforge_core/resources/profiles/public/single-gpu-*.yaml — Phi-4-mini removed from candidates for now

## Problem Phi-4-mini-instruct ships a custom `modeling_phi3.py`. Two issues are partially fixed; one remains: 1. `SlidingWindowCache` removed from `transformers.cache_utils` — **fixed** with try/except stub 2. `LossKwargs` removed from `transformers.utils` — **fixed** with TypedDict stub 3. `get_expanded_tied_weights_keys()` in transformers 5.x expects `tied_mapping` as a dict; Phi-3 returns a list — **NOT YET FIXED**, causes: ``` AttributeError: 'list' object has no attribute 'keys' File transformers/modeling_utils.py, line 2471, in get_expanded_tied_weights_keys if all(regex.match(k) for k in tied_mapping.keys() | tied_mapping.values()) ``` ## Impact Phi-4-mini-instruct cannot load. Qwen2.5-3B-Instruct is sole production model. ## Preferred fix Patch the local `modeling_phi3.py` to override `get_tied_weights_keys()` returning a dict (transformers 5.x format): ```python def get_tied_weights_keys(self): return {"lm_head.weight": "model.embed_tokens.weight"} ``` ## Files - `/Library/Assets/LLM/vllm/models/Phi-4-mini-instruct/modeling_phi3.py` — local model code - `circuitforge_core/resources/profiles/public/single-gpu-*.yaml` — Phi-4-mini removed from candidates for now

pyr0ball commented

2026-04-02 17:22:51 -07:00

Author

Owner

Root cause (updated): transformers 5.x changed _tied_weights_keys from list[str] to dict[str, str]. get_expanded_tied_weights_keys() reads the attribute directly (self._tied_weights_keys) — not via a method — so a method override does nothing.

Fix applied to /Library/Assets/LLM/vllm/models/Phi-4-mini-instruct/modeling_phi3.py:

# was:
_tied_weights_keys = ["lm_head.weight"]
# now:
_tied_weights_keys = {"lm_head.weight": "model.embed_tokens.weight"}

HF module cache cleared after patching. Model loads and get_expanded_tied_weights_keys() returns the correct dict. Verified with transformers 5.x. Closes #11.

**Root cause (updated):** transformers 5.x changed `_tied_weights_keys` from `list[str]` to `dict[str, str]`. `get_expanded_tied_weights_keys()` reads the attribute directly (`self._tied_weights_keys`) — not via a method — so a method override does nothing. **Fix applied to `/Library/Assets/LLM/vllm/models/Phi-4-mini-instruct/modeling_phi3.py`:** ```python # was: _tied_weights_keys = ["lm_head.weight"] # now: _tied_weights_keys = {"lm_head.weight": "model.embed_tokens.weight"} ``` HF module cache cleared after patching. Model loads and `get_expanded_tied_weights_keys()` returns the correct dict. Verified with `transformers` 5.x. Closes #11.