Question [Crash] Valorant — Black Screen with DX11 External Overlay on Windows 11 DWM

[byte_corvus]

Anyone else running into black screen issues with external overlays on Win11 specifically for Valorant?

I have been banging my head against the wall with this one. My external overlay works perfectly on Win10 and handles other titles like PUBG on Win11 without a hitch. However, as soon as I hook the overlay in Valorant on Win11, the game buffer goes pitch black—audio and input are still active, but the visual output is dead. Closing the process brings it back instantly.

Tech breakdown of the current implementation:

1. Overlay Subsystem: Standard Win32 layered window, WS_EX_TRANSPARENT | WS_EX_LAYERED, using DwmExtendFrameIntoClientArea for transparency.
2. Pipeline: Legacy DX11, SwapEffect = DXGI_SWAP_EFFECT_DISCARD, BufferDesc = DXGI_FORMAT_R8G8B8A8_UNORM.

I have already tried hijacking existing trusted overlays to mask the window presence, but Vanguard still nukes the buffer. It seems like a specific interaction between Vanguard and the DWM compositor on modern Windows builds.

The Questions:

1. DWM vs. Vanguard: Is Vanguard actively blocking the DwmExtendFrameIntoClientArea call to prevent legacy overlay injection?
2. Flip Model: Have any of you successfully resolved this by switching to DirectComposition or moving to the

   DXGI_SWAP_EFFECT_FLIP_DISCARD

   model to match the OS-native flip mode optimizations?
3. Alternatives: Is it time to ditch the Win32 layered window approach entirely for a different hooking method to remain invisible to the kernel monitor?

I know most of you are moving toward DMA for this game to bypass VGK entirely, but for those still working on Ring 3 visuals, how are you handling the compositor? If anyone has tips on getting a stable overlay to play nice with the Windows 11 desktop composition while Valorant is active, drop your thoughts.

Interested to see if anyone has a cleaner way to handle the overlay swap chain without triggering the blackout. Thanks.