You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, PLY files serve as the de facto standard through their sheer simplicity and ubiquity. This extension aims to bring structure and conformity to the Gaussian Splat space while utilizing glTF to its fullest extent. Gaussian Splats are essentially a superset of a traditional point cloud, so we approached the extension with this mindset. The **position**, **rotation**, **scale**, and **diffuse color** properties are stored as standard attributes on a point primitive. If the point primitive contains the extension the renderer can know to render the point primitive as Gaussian Splats instead of a points.
37
+
Currently, PLY files serve as the de facto standard through their sheer simplicity and ubiquity. This extension aims to bring structure and conformity to the Gaussian Splat space while utilizing glTF to its fullest extent. Gaussian splats are essentially a superset of a traditional point cloud, so we approached the extension with this mindset. If the point primitive contains the extension the renderer can know to render the point primitive as Gaussian splats instead of a points. Gaussian splats are defined by a position, color, rotation, and scale. We store these values as attributes on primitives.
38
+
39
+
This approach allows for an easy fallback in the event the glTF is loaded within a renderer that doesn't support Gaussian splats. In this scenario, the glTF file will render as a sparse point cloud to the user. It also allows for easy integration with existing compression like meshopt.
25
40
26
-
This approach allows for an easy fallback in the event the glTF is loaded within a renderer that doesn't support Gaussian Splats. In this scenario, the glTF file will render as a sparse point cloud to the user. It also allows for easy integration with existing compression like meshopt.
41
+
## Adding Gaussian Splats to Primitives
42
+
43
+
As Gaussian splats are defined by a position color, rotation and scale, we both map to existing attributes and define new ones. These are attached to a primitive by defining the `extensions.KHR_gaussian_splatting` property on that mesh.
To define a Gaussian splat, it must contain the following attributes:
48
+
49
+
| Splat Data | glTF Attribute | Accessor Type | Component Type |
50
+
| --- | --- | --- | --- |
51
+
| Position | POSITION | VEC3 | float |
52
+
| Color (SH0 (Diffuse) and alpha) | COLOR_0 | VEC4 | unsigned byte normalized |
53
+
| Rotation |_ROTATION | VEC4 | float |
54
+
| Scale |_SCALE | VEC3 | float |
55
+
56
+
Standard PLY splat formats have opacity and color separate, however they are combined into the COLOR_0 attribute here.
37
57
38
-
Spherical Harmonic channels 1 through 15, which map the splat specular, are currently unused by the extension.
58
+
This implementation only contains the zeroth spherical harmonic for diffuse color. Spherical Harmonic channels 1 through 15, which map the splat specular, are currently unused by the extension.
39
59
40
60
### Extension attributes
41
61
42
-
| Attributes | Type | Description | Required
62
+
`extensions.KHR_gaussian_splatting` may contain the following values:
63
+
64
+
| Attributes | Type | Description | Required |
43
65
| --- | --- | --- | --- |
44
66
| quantizedPositionScale | number | Scale value for dequantizing POSITION attribute values | No, default: `1.0`|
45
67
68
+
```json
69
+
{
70
+
"extensions": {
71
+
"KHR_gaussian_splatting": {
72
+
"quantizedPositionScale": 13.228187255859375
73
+
}
74
+
}
75
+
}
76
+
```
77
+
78
+
### Transforming Gaussian Splat Data for glTF
79
+
80
+
The data output from the Gaussian splat training process that is stored in the typical PLY must be transformed before storing in glTF.
81
+
82
+
#### Diffuse Color
83
+
84
+
Each color channel `red`, `green`, and `blue` must each be multiplied by the zeroth-order spherical harmonic constant `0.28209479177387814`
46
85
47
-
## Extending glTF node
86
+
#### Alpha
87
+
88
+
Alpha must be activated through the logistic function $\sigma(a) = \frac{1}{1 + e^{-a}}$ which constrains it to the range `[0 - 1)`
89
+
90
+
It can then be converted to an `unsigned byte` color channel.
91
+
92
+
#### Scale
93
+
94
+
'Activated' via exponentiation similar to `alpha`. $\exp(x)$
Before storing values into attributes, the data will have been preprocessed from the Gaussian Splat training process.
98
-
99
-
| Attribute | Process |
100
-
| --- | --- |
101
-
| Alpha | Activated through logistic function `sigmoid(a) = 1 / (1 + e^(-a))`|
102
-
| Scale | Natural exponentiation `exp(scale)`|
103
-
| Rotation | Normalized Quaternion |
104
-
| Diffuse Color | Multiplied by zero-order spherical harmonic constant `0.28209479177387814`|
105
-
106
-
107
150
## Implementation
108
151
109
152
_This section is non-normative_
110
153
111
154
In this example, we follow a reference implementation for rendering Gaussian Splats.
112
155
113
-
In the vertex shader, we first must compute covariance in 3D and then 2D space. In optimizing implementations, 3D covariance can be computed ahead of time.
156
+
In the vertex shader, we first must compute covariance in 3D and then 2D space. In optimizing implementations, 3D covariance can be computed ahead of time.
This is currently implemented within [3D Tiles and CesiumJS as an experimental feature](https://github.com/CesiumGS/cesium/tree/splat-shader).
0 commit comments