[NVIDIA] adding DSR1 deployment guide on B200

[kushanam]

Motivation

We’re adding a deployment guide for running DSR1 on B200. Since the set of parameters varies greatly depending on workload and datatype, this guide is meant to help streamline efficient deployment. Please note that the list of features is evolving daily, so it will be important to keep the guide up to date.

Modifications

A new folder will store all upcoming recipes along with the corresponding guide

Accuracy Tests

N/A

Benchmarking and Profiling

N/A

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.

[gemini-code-assist]

Summary of Changes

Hello @kushanam, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a new deployment guide for running the DeepSeek R1 large language model on NVIDIA B200 GPUs using SGLang. The guide aims to streamline efficient deployment by providing comprehensive instructions for setting up the environment, including Docker image creation, model weight acquisition, and configuring SGLang for optimal performance with FP8 and NVFP4 quantized models. It also covers benchmarking and performance tuning for various workload requirements.

Highlights

• New Deployment Guide: A new Markdown file, docs/recipes/dsr1_b200_deployment_guide.md, has been added to provide a comprehensive guide for deploying the DeepSeek R1 model on NVIDIA B200 GPUs.
• DeepSeek R1 Model Deployment: The guide provides step-by-step instructions for running DeepSeek R1 with SGLang, including support for FP8 and NVFP4 quantization, and covers the installation of SGLang and FlashInfer components.
• Docker Image Building and Running: Instructions are included for building a Docker image with SGLang and its dependencies, along with commands for running the container with appropriate GPU access and shared memory settings.
• Model Weight Management: The guide details how to download both FP4 and FP8 quantized versions of the DeepSeek R1 model weights from Hugging Face directly within the Docker environment.
• SGLang Server and Client Configuration: Detailed examples and flag descriptions are provided for launching the SGLang server and client, with specific configurations tailored for high-throughput and low-latency scenarios for both FP8 and FP4 models.
• Performance Benchmarking: Commands and explanations are provided for benchmarking the accuracy and performance of the deployed DeepSeek R1 model using sglang.bench_serving.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[kushanam]

@merrymercy and @zhyncs I suggest adding deployment guides for each model and keeping this document continuously updated (by the authors). Let me know your thoughts.

[gemini-code-assist]

Warning

Gemini encountered an error creating the review. You can try again by commenting /gemini review.

[zhyncs]

hold on