GS/HW: Support for integer depth.

[TJnotJT]

Status: this is a WIP for early testing. There are several performance and code cleanup issues to be addressed. Committed on top of ROV, but has not been fully tested.

Description of Changes

Adds support for integer depth format to DX12/VK/GL/DX11.

Procedure for interpolating 32 bit Z values was given by @TellowKrinkle:

1. In the vertex shader send the minimum Z value of each triangle as a flat uint attribute, along with the offset as an interpolated float attribute.
2. In the fragment shader add the flat minimum value to the interpolated float value.

This should work as long as each triangle's Z range fits in 24 bits (which appears to be the case in all GS dumps examined so far).

Rationale behind Changes

Makes depth accuracy about the same as SW when the full 32 bit range is used non-trivially (i.e., upper bits of depth take multiple values). I don't know of many cases where this is actually needed, so this is more of a "nice to have" feature. It comes at a significant performance cost, so a less expensive fix is preferable (Z floor and/or limit 24 bit setting) unless absolutely required.

Draws with integer depth will usually require full barriers since we need to read/write the buffer for depth testing. This might be impractical in many cases, though it could be more usable when combined with ROVs.

Suggested Testing Steps

Use the following setting in the INI:

[EmuCore/GS]
HWZIntegerMode = 1

The values are:

• HWZIntegerMode = 0: Disabled.
• HWZIntegerMode = 1: Enabled. Use integer depth when the Z values go outside the 24 bit range and switch back to regular depth if the range goes back.
• HWZIntegerMode = 2: Always. Always use integer depth. This is more more testing than for practical use.
  Then use VK and/or DX12.

Example

The only case I know where 32 bit Z is used non-trivially (thanks @refractionpcsx2 ):

Shox_SLUS-20533_20260111064920.gs.xz (see the signs in the distance and sides of the road)

Master VK

Master SW

PR VK

Did you use AI to help find, test, or implement this issue or feature?

Yes, to get help with the API usage/shader programming and info about extensions. Also, for reviewing the 64 bit arithmetic code.

[MarioSonic2987]

Fixes z-fighting in the following games without using the Limit Depth to 24 bits hack:

Harry Potter and the Goblet of Fire

Before:

After:

It no longer breaks the graphics in this cutscene (see #13805 (comment)):

Worms 3D

Before:

After:

Worms Forts: Under Siege

Before:

After:

Worms 4 Mayhem

Before:

After:

[bigol83]

Does this include the recent zfighting fixes that are in master? If it does this breaks those, with a couple of games i tested it went back to big zfighting just like it was before zfighting recent fixes were merged.

gsdump
https://www.swisstransfer.com/d/b80a8856-b700-4d3b-a656-182c69e1eef6

[TJnotJT]

Kingdom Hearts II - It needs HWZIntegerMode set to 2. If not, z-fighting strikes back.

In theory there should be no difference between mode 1 and 2 so this might be a bug. Would it be possible to have a dump showing this?

Does this include the recent zfighting fixes that are in master? If it does this breaks those, with a couple of games i tested it went back to big zfighting just like it was before zfighting recent fixes were merged.

Yes, there was a rebase error with Z floor in VK that is hopefully fixed on the last push.

[MarioSonic2987]

There's no z-fighting anymore in KH2 with HWZIntegerMode = 1.

GS Dump

[bigol83]

Kingdom Hearts II - It needs HWZIntegerMode set to 2. If not, z-fighting strikes back.

In theory there should be no difference between mode 1 and 2 so this might be a bug. Would it be possible to have a dump showing this?

Does this include the recent zfighting fixes that are in master? If it does this breaks those, with a couple of games i tested it went back to big zfighting just like it was before zfighting recent fixes were merged.

Yes, there was a rebase error with Z floor in VK that is hopefully fixed on the last push.

Fixed in last push 👍

[TJnotJT]

Last push incorporates a simpler and more performant process for interpolating 32 bit integer values on the GPU, given by @TellowKrinkle.

It also adds an initial port to DX11 and GL.

Changes have not been thoroughly tested and may have bugs.