[NVBug: 6445613] handle nested vocab_size for multimodal configs (Gemma4)

[juhi10071998]

Summary

• Gemma4Config stores vocab_size under text_config.vocab_size rather than at the top level, causing AttributeError during TRT-LLM checkpoint export
• Added hasattr fallback so multimodal configs with nested vocab_size work correctly while all flat-config models (Llama, Mistral, etc.) are unaffected

Root cause

torch_to_tensorrt_llm_checkpoint in model_config_export.py assumed model.config.vocab_size always exists. For multimodal HF models (Gemma4 and similar), vocab_size is nested under text_config.

Test plan

• Verified fix logic inside epic_banach container with AutoConfig.for_model("gemma4") — correctly reads text_config.vocab_size = 262144
• Verified flat-config models (Llama) still read vocab_size directly — unchanged behavior
• Full end-to-end test with google/gemma-4-E2B-it + INT4 AWQ quantization pending model weights

References

• NVBug: https://nvbugspro.nvidia.com/bug/6445613
• GitHub issue: [Bug]: Gemma4 INT4 AWQ quantization fails during export: Gemma4Config has no attribute vocab_size TensorRT-LLM#16171

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: c6ab5b2c-775a-44ea-95ae-d544be34dc0b

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

PR Preview Action v1.8.1
🚀 View preview at https://NVIDIA.github.io/Model-Optimizer/pr-preview/pr-2040/
Built to branch gh-pages at 2026-07-31 22:12 UTC. Preview will be ready when the GitHub Pages deployment is complete.

[codecov]

Codecov Report

❌ Patch coverage is 66.66667% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 66.80%. Comparing base (a23390d) to head (6f8ef86).

Files with missing lines	Patch %	Lines
modelopt/torch/export/model_config_export.py	66.66%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2040      +/-   ##
==========================================
- Coverage   69.97%   66.80%   -3.18%     
==========================================
  Files         519      519              
  Lines       59399    59401       +2     
==========================================
- Hits        41565    39681    -1884     
- Misses      17834    19720    +1886     

Flag	Coverage Δ
examples	43.11% <66.66%> (-0.20%)	⬇️
gpu	20.94% <0.00%> (-11.77%)	⬇️
regression	15.00% <0.00%> (+0.07%)	⬆️
unit	55.15% <0.00%> (-0.01%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.