[WIP]feat: Add initial support for SeedOSS model.

[XiChen0415]

Motivation

Add initial support for Seed-OSS model which is released on 8.21. The code relies on a forked version of transformers(https://github.com/Fazziekey/transformers/tree/seed-oss).
Though the code is already tested on H20, it should not be merged before the transformers update, thus this PR is marked as WIP for now.

Modifications

Accuracy Tests

Benchmarking and Profiling

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.

[gemini-code-assist]

Summary of Changes

Hello @XiChen0415, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces initial support for the SeedOSS large language model. It integrates the model's architecture and components, including its specific reasoning format, into the system. This enables the platform to utilize and process outputs from the newly released SeedOSS model, relying on a forked version of the Hugging Face transformers library.

Highlights

• SeedOSS Model Implementation: Implemented the core architecture for the SeedOSS model, including its multi-layer perceptron, attention mechanism, decoder layers, and the overall model structure for causal language modeling.
• Reasoning Format Integration: Added a dedicated SeedOssDetector to the reasoning parser, allowing the system to correctly identify and process reasoning content delimited by seed:think and </seed:think> tokens from SeedOSS models.
• Infrastructure Integration: Integrated the new model components into the existing parallel processing and weight loading infrastructure.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces support for the SeedOSS model, largely adapted from the existing Qwen2 implementation. The changes to reasoning_parser.py to add a SeedOssDetector are correct and follow the existing pattern. However, I've identified two critical issues in the new seed_oss.py model file. One is a potential NameError in load_kv_cache_scales due to incorrect logic flow, and the other is a bug in load_weights related to iterator consumption that could lead to incorrect weight loading. I've provided detailed comments and a suggested fix for the first issue. Please address these critical bugs.

[gemini-code-assist]

Using next(filter(..., weights)) here will consume the weights iterator. If weights is a generator, this will cause the outer loop to skip subsequent weights or fail to find model.embed_tokens.weight if it has already been passed, potentially leading to incorrect model loading or a StopIteration error.

A better approach is to find and cache embed_token_weights once before the main loop. This might require converting weights to a list if it's a one-time iterable.

[adarshxs]

I believe this also needs a custom tool parser?

[XiChen0415]

I believe this also needs a custom tool parser?

Sure thing. Just pushed function_call_parser for SeedOSS model. Haven't got time for testing this parser, feel free to give it a try and leave a comment :)