4 Commits

Author	SHA1	Message	Date
Michael Yang	440a3823a6	fix(tokenizer): add special tokens to empty inputs (#13091)	2025-11-18 11:16:56 -08:00
Michael Yang	5994e8e8fd	embedding gemma model (#12181) ... * ollama: add embeddings	2025-09-04 09:09:07 -07:00
Jeffrey Morgan	9e125d884c	model: treat 'user defined' tokens as special tokens (#11077)	2025-06-16 16:03:16 -07:00
Michael Yang	333e360422	model: handle multiple eos tokens (#10577) ... * get eos_token_id from generation_config.json * refactor * include both ids and strings in trace * comments * remove special case for gemma3 special vocab (#10743)	2025-05-16 13:40:23 -07:00