Quality is low in comparison to other viewers

[hybridherbst]

Thanks for starting Forge!

I made some quality comparisons and I can't get the Forge viewer to look nearly as good as the others, both for low and for high splat counts.
The results are much blurrier (in both cases) and downright bad (for the high-splatcount version).

Is this known? Are you planning to improve quality?

Screenshots

Viewer	10k splats	1200k splats
Forge
SuperSplat
mkkellogg

Reproduction files

I attached the 10k splat file here: AnisoLamp2.ply.zip

The 1200k splat file can be directly viewed:

• on Forge: https://sparkjs.dev/viewer/?url=https%3A%2F%2Fd28zzqy0iyovbz.cloudfront.net%2F1b0b385d%2Fscene.compressed.ply
• on SuperSplat: https://superspl.at/view?id=1b0b385d

Additional Info

In case it's related, I'm on a high-dpi screen – not sure if maybe DPI isn't correctly taken into account in the viewer?

[dmarcos]

Thanks so much for taking the time to compare them all. Looking into it.

[asundqui]

Hey there @hybridherbst , thank you for bringing this to our attention and doing all the work to show comparisons and get us the repro. We've looked into it and I think we have a fix!

Before fix:

After fix:

Indeed it was terrible before :), and with the fix it looks much better. I won't claim that it's going to be identical to the other viewers, because there are so many factors involved in rendering splats, for example internal reduced-bit representations, sorting precision (Forge lowers sorting precision for faster sort updates), sorting metric (by default we use a non-standard "radial distance" sort instead of Z-depth, but you can flip a boolean to change this).

We also made a live example you can click on (use WASD keys + mouse to move around like an FPS) to examine: https://snowy-phrygian-arrow.glitch.me

Long story short: In src/shaders/splatDefines.glsl we had a setting MIN_ALPHA = 0.01, which culled any splats with < 1% opacity during rendering because we had believed it wouldn't make a difference in appearance while maybe slightly improving performance. Your .PLY scan is however mostly transparent! In fact 98.7% of the splats are < 5% opacity. With MIN_ALPHA=0.01 we were discarding 37% of your splats before rendering.

We've merged a fix that sets MIN_ALPHA = 0.5/255, which is the lowest conceivable (rounded) value that could be encoded in a Uint8 (1/255, and 0.5 of that if the hardware rounds up). This completely changed the rendering of your splats and to me looks the same quality as the other two renderers.

If you're running this locally, we would love to hear if updating to the latest main commit fixes this for you? Please note that we have NOT yet updated our website forge.dev/examples with this fix, so you can only test this locally right now! (the glitch.me link above includes an updated build)

A few more things because you mentioned them:

1. Forge does support high-DPI, but it's not turned on by default in the viewer. If you're running locally you can check out examples/editor, which has a checkbox setting in the GUI to toggle high-DPI on and off. Your scan looks almost the same at high/lo-DPI to me, but it's slightly crispier at high-DPI (at the expense of rendering a bit slower of course).

2. Forge defaults to "radial distance" sort instead of "Z-depth" sort, which is different than most renderers + reconstruction software, but we do it because it's more stable under rotation. This is easy to change: forge.defaultView.sortRadial = false. We've also updated examples/editor so there's a toggle that can switch that.

3. Forge sorts splats with float16 precision to improve performance. We want to add a high-precision sort in the future for use cases where this matters more. However you may notice a bit more flickering/Z-fighting between splats in your scans because of this.

4. Our goal is for any scan / use case to work well in Forge. However, I do think your 1200K scan isn't the best for 3DGS... curious what software you're using for reconstruction? 98.7% splats are < 5% opacity, which means the lamp is actually almost completely transparent :) Layering 1.2M splats on top of each other makes it look opaque in the end, but it's very inefficient to render so many super-transparent splats on top of each other. Not sure if you can change the reconstruction settings, but for an object that's solid like yours, you'd ideally like most splats to be near 100% opacity. It's possible the highly anisotropic nature of the brushed metal makes it difficult for the reconstruction to tell transparent from solid. But I do think if you want this scan to render well + efficiently, some tuning may be beneficial here.

Please let us know if this fix works for you! We'd love for Forge to work for you. Also please let me know if any of the above is unclear and I can expand on it!

[dmarcos]

@hybridherbst following up. let us know when you can if the issue is gone or if there’s additional work we need to do. Thank you!

[zzh-tech]

Hi @dmarcos , thanks for this great work! I found the quality of Spark is still lower than other viewers. Are there any parameters/options that I can tweak to improve the quality if I have enough computing resources?

[hybridherbst]

I agree that the quality in Spark.js is still a lot lower and results are pretty blurry.
When viewing up close the differences are less, but still noticeable.

Here's a current comparison between

• https://sparkjs.dev/viewer/?url=https%3A%2F%2Fd28zzqy0iyovbz.cloudfront.net%2F1b0b385d%2Fscene.compressed.ply
• https://superspl.at/view?id=1b0b385d

Spark	SuperSplat

[dmarcos]

Hi @dmarcos , thanks for this great work! I found the quality of Spark is still lower than other viewers. Are there any parameters/options that I can tweak to improve the quality if I have enough computing resources?

Thanks. Can you provide a file we can test where you see lower quality? thanks

[asundqui]

@hybridherbst do you want to try it in our Editor instead? There's a checkbox on the right menu "High DPI" that will use the full devicePixelRatio on your computer:

https://sparkjs.dev/examples/editor/?url=https%3A%2F%2Fd28zzqy0iyovbz.cloudfront.net%2F1b0b385d%2Fscene.compressed.ply

Check out the image below, I'm not sure if I'm seeing it blurrier or different than SuperSplat, maybe I'm not looking close enough?

[zzh-tech]

Thanks. Can you provide a file we can test where you see lower quality? thanks

Sure, for example, see the commonly used dataset playroom, and I found the same "blurry" issue in my private dataset. (ps. I ran both locally and correctly set the devicePixelRatio)

The screenshot from mkkellogg:

And the screenshot from Spark:

But I like Spark since its code logic is clearer and more flexible. Hopefully this quality issue will be resolved soon.

[asundqui]

But I like Spark since its code logic is clearer and more flexible. Hopefully this quality issue will be resolved soon.

I have an idea: Do you know how these scan reconstructions were created? There are two options in SparkRenderer that may be the cause: preBlurAmount=0.0 and blurAmount=0.3. Their default values assume the scene was optimized with the "3DGS anti-aliasing" fix enabled, while many scenes including the standard reconstruction scenes were trained with preBlurAmount=0.3 and blurAmount=0.0.

Swapping these values when looking at the standard bicycle.ply reconstruction scene for example changes how the bicycle spokes look, for example. But these are parameters that have to be set to match how the reconstruction was made.

Could you please try fiddling with those two values in the SparkRenderer in your local examples/editor/index.html? You could add them to the gui and get sliders for both preBlurAmount and blurAmount.

[zzh-tech]

Thanks, I tried but did not work.

[asundqui]

Thanks, I tried but did not work.

@zzh-tech I tried and it fixed the difference, I'm pretty sure this is what's going on for you since you are using the "playroom" standard scene, which was created before the anti-alias fix.

MKellogg's renderer also has the same feature you can adjust, you can read about it here: mkkellogg/GaussianSplats3D#142

Basically, before this issue was discovered (see Mip-NeRF if you really want to learn the details), everyone was adding 0.3 to the projected 2DGS covariance diagonal, effectively blurring all splats by 0.5 pixels. The 3DGS optimizers were using this as a rendering target, including the standard reconstruction scenes like "playroom" that you're using.

Unfortunately, if you think about it, this makes no sense physically. The blurring of the splat by incrementing the covariance matrix is a great trick - it's essentially convolving two gaussians together. But as things "blur", they also should "fade out". And this was missing in the original 3DGS paper calculation. The correct way to do this (and achieve correct anti-aliasing) is to do the 0.3 convolution via the covariance diagonal, but also to "blur" the Gaussian by the ratio of the before/after Gaussian area. This effectively "spreads out" the integral so things are blurred correctly.

I believe most reconstruction tools (Postshot, etc) today using the default settings Spark uses (preBlurAmount (no AA) = 0.0 and blurAmount (AA) = 0.3).

In order for your playroom test to render correctly, you need to adjust it to the pre-AA settings: preBlurAmount (no AA) = 0.3 and blurAmount (AA) = 0.0).

Here are some images to show you the difference. Open these in separate browser tabs and swap between them. First is your screenshot from MKellogg:

Spark screenshot (High DPI enabled, adjusted settings Blur (no AA) = 0.3, Blur (AA) = 0.0):

Spark screenshot (High DPI enabled, default settings Blur (no AA) = 0.0, Blur (AA) = 0.3):

If you compare the text of the top row of books between the screenshots I think you'd see that it makes most of the difference. With the default settings the text indeeds renders differently, but that's expected.

To make it simpler for you to test, we just merged a PR that adds these settings in the Editor's GUI: #54

It's been deployed here: https://sparkjs.dev/examples/editor/

Open the "Settings" menu on the right. Enable "High DPI". Drag playroom.ply into the editor to load it. Play with the two sliders "Blur Amount (no AA)" and (AA).

[zzh-tech]

Thanks, I tried but did not work.

I mean, even if I adjust these two parameters, there is still a gap.

The screen shot of the Editor's GUI, default setting Blur (no AA) = 0.0, Blur (AA) = 0.3):

The screen shot of the Editor's GUI Blur, adjusted settings (no AA) = 0.3, Blur (AA) = 0.0):

MKellogg:

[zzh-tech]

I cannot share private data. Let me use "playroom" for the same comparison.

The screen shot of the Editor's GUI, default setting Blur (no AA) = 0.0, Blur (AA) = 0.3):

The screen shot of the Editor's GUI Blur, adjusted settings (no AA) = 0.3, Blur (AA) = 0.0):

MKellogg:

[asundqui]

Thanks, I tried but did not work.

I mean, even if I adjust these two parameters, there is still a gap.

Is this with "High DPI" checked on? Have you tried toggling "Radial sort" and see if that could be the difference?

[dmarcos]

Thanks, I tried but did not work.

I mean, even if I adjust these two parameters, there is still a gap.

The screen shot of the Editor's GUI, default setting Blur (no AA) = 0.0, Blur (AA) = 0.3):

The screen shot of the Editor's GUI Blur, adjusted settings (no AA) = 0.3, Blur (AA) = 0.0):

MKellogg:

thanks for putting the time to compare. Is ply for this available?