<h1 id="qwen3-vl-4.3b-semantic">Qwen3-VL-4.3B-semantic</h1>An expanded-vocabulary variant of <code>Qwen/Qwen3-VL-4B-Instruct</code>, designed for use as a text encoder in Krea 2.The original 36 decoder layers and vision tower are retained. The main architectural modification is an expanded <code>embed_tokens</code> / tied <code>lm_head</code> vocabulary containing additional whole-word English tokens.<a target="_blank" rel="ugc" href="https://huggingface.co/Sorenvza/Qwen3-VL-4_3B-semantic">https://huggingface.co/Sorenvza/Qwen3-VL-4_3B-semantic</a><h2 id="what-changed">What changed</h2>| | Base | This model || ----------------- | -------: | ---------------------------------------: || Tokenizer length | 151,936 | 268,590 || Added word tokens | — | 116,654 || New parameters | — | 298,634,240 || Hidden size | 2,560 | 2,560 || Decoder layers | 36 | 36 || Vision tower | Original | Original || Main modification | — | Expanded <code>embed_tokens</code> / tied <code>lm_head</code> |Semantic initializationThe added embeddings were constructed in several stages:<ol><li>English lexical candidates were selected from the external lexical resources described below.</li><li>Initial semantic representations were obtained from ConceptNet Numberbatch.</li><li>WordNet-derived hypernym relationships were used to impose additional lexical structure.</li><li>The semantic representations were aligned to the original Qwen embedding space using a linear least-squares projection based on the mean embeddings of the corresponding original subword representations.</li><li>The resulting vectors were used to initialize the added Qwen vocabulary rows.</li></ol>This procedure should be understood as semantic embedding expansion and initialization, not as full pretraining or continued pretraining of the Qwen3-VL transformer.<h2 id="data-and-external-resources">Data and external resources</h2>This model incorporates information derived from several external lexical and semantic resources.<h3 id="conceptnet-numberbatch">ConceptNet Numberbatch</h3>ConceptNet Numberbatch 19.08 was used as the initial semantic representation for the added vocabulary.Source:<ul><li>ConceptNet</li><li>ConceptNet Numberbatch</li><li>Version: 19.08</li></ul>The applicable ConceptNet / Numberbatch license and attribution requirements apply to the corresponding derived data.<h3 id="wordnet-open-multilingual-wordnet">WordNet / Open Multilingual WordNet</h3>WordNet-derived lexical relations, including hypernym information, were used during semantic embedding construction.The local lexical environment used the NLTK <code>omw-1.4</code> resource where applicable.The applicable WordNet / OMW licensing and attribution requirements apply to the corresponding derived data.<h3 id="wiktionary-wiktextract">Wiktionary / Wiktextract</h3>Wiktionary-derived lexical information was obtained through Wiktextract / Kaikki where applicable.The exact dump or snapshot used to construct the released lexical data should be recorded for reproducibility.Wiktionary-derived data is subject to the applicable Wiktionary licensing terms, including CC BY-SA and GFDL where applicable.<h2></h2>

Files with SHA256: 8cd07476b53275f3774d0bbc0b2cf975dd6374c2bda4f41665f79e1e7ccd51c3