vlm: tensor hash kernel

[mickqian]

Motivation

Previously, for gpu-tensor, the hash process required by multimodal model will first:

1. move the tensor D2H
2. then hash the tensor with sha256

With some simple profiling, hashing a normal image feature (e.g., from sgl-logo image, shape=[3312,1176], dtype=float32 in qwen-vl cases) would cost ~80ms.

Modifications

1. Add a triton hash kernel, which does:
   1. hash the blocks parallelly on GPU
   2. reduce the block results sequentially on CPU

Profiling

Hash performance

	time(1000 times)
original hash	11s
triton hash	~0.6s

MMMU

	accuracy(before)	time(before)	accuracy(after)	time(after)
Gemma-3-4b-it	0.384	226.2	0.384	218.3
Qwen2.5-VL-7B-Instruct	0.467	352.7	0.467	338.2
Minicpmv	0.436	232.5	0.436	230.7

Correctness

1. Consistency: The kernel first hash tensor with blocks, then performs a reduce on block results sequentially.
2. Collision: The collision probability is zero, on 10000 randomly-generated tensors with the same shape as real data, with the following script:

def test_hash_collision(hasher, name, num_tensors=10000, tensor_shape=(128,)):
    hashes = set()
    collision_count = 0

    for i in range(num_tensors):
        tensor = torch.rand(
         size=tensor_shape, dtype=torch.float32, device="cuda"
        ) * 2 - 1
        h = hasher(tensor)
        if h in hashes:
            collision_count += 1
        else:
            hashes.add(h)

    print(f"hasher: {name}")
    print(f"Tested {num_tensors} random tensors of shape {tensor_shape}")
    print(f"Collisions found: {collision_count}")
    print(f"Collision rate: {collision_count / num_tensors:.6f}")

Future Work

1. Should we just implement MurmurHash/xxHash directly?

Checklist

• Format your code according to the Code Formatting with Pre-Commit.
• Add unit tests as outlined in the Running Unit Tests.
• Update documentation / docstrings / example tutorials as needed, according to Writing Documentation.
• Provide throughput / latency benchmark results and accuracy evaluation results as needed, according to Benchmark and Profiling and Accuracy Results.
• For reviewers: If you haven't made any contributions to this PR and are only assisting with merging the main branch, please remove yourself as a co-author when merging the PR.
• Please feel free to join our Slack channel at https://slack.sglang.ai to discuss your PR.

[JustinTong0323]

a typo is found...

[merrymercy]

Why will a tensor be on GPU?

[mickqian]

if the fast version of the processor is enabled, the returned tensor will be on GPU be default, seehere

[merrymercy]

sum is not a good combinator for hash function

[merrymercy]

This is a very bad hash function! @yizhang2077 @zhyncs @mickqian

[merrymercy]

related links:
NVIDIA/TensorRT-LLM#4145
pytorch/pytorch#2569