Question OpenGL — wglSwapBuffers Hook Return Logic & Recursion Issues

[byte_corvus]

Anyone currently digging into OpenGL internals has probably hit this wall. You’re trying to hijack the render loop via wglSwapBuffers to draw your menu or ESP, but the moment you try to call the original function, the game either hangs or CTDs.

The logic in the snippet provided is a textbook example of why manual detours require more than just a GetProcAddress. If you’re overwriting the first 5 bytes of the function with a JMP, calling that same address from within your hook just sends you right back to the start of your own hook. Infinite recursion — game over.

The Technical Breakdown
Look at the provided base:

typedef bool(__thiscall* wglSwapBuffers)(HDC hdc);
wglSwapBuffers swapBuffers = (wglSwapBuffers)OpenGL::wglSwapBuffers;

bool hwglSwapBuffers(HDC hdc) {
  // This just jumps back to the start of the hook if you used a 5-byte JMP
  return swapBuffers(hdc);
}

Critical Errors in This Implementation:

1. Calling Convention: wglSwapBuffers is WINAPI (__stdcall), not __thiscall. Using the wrong stack cleanup logic is a one-way ticket to a stack corruption crash.
2. The Trampoline Problem: Your "hook" function is likely a simple detour. Once you've patched the prologue of the original function in opengl32.dll, that original address is "poisoned." You can't just call it and expect it to work unless you've preserved the "stolen bytes" and jumped back to original_address + 5.
3. Memory Protection: Ensure you're using VirtualProtect correctly when applying the patch, but that's secondary to the logic loop here.

Spoiler: Structure Details

namespace OpenGL {
  static HMODULE OpenGL{ NULL };
  static DWORD wglSwapBuffers{ NULL };
}

void upd() {
  OpenGL::OpenGL = { GetModuleHandle("OPENGL32.dll") };
  if (OpenGL::OpenGL)
    OpenGL::wglSwapBuffers = { reinterpret_cast<DWORD>(GetProcAddress(OpenGL::OpenGL, "wglSwapBuffers")) };
}

Preventive Troubleshooting
If you aren't using a library like MinHook or Detours, you need to implement a gateway. A gateway is a small buffer of memory that executes the bytes you replaced, then jumps to the rest of the original function. Without this, your

swapBuffers(hdc)

call is literally just calling your

hwglSwapBuffers

again if the JMP is still there.

Also, check your DEP (Data Execution Prevention) settings. If your gateway isn't marked as PAGE_EXECUTE_READWRITE, the CPU will trip as soon as it hits your trampoline.

Are you guys still rolling your own hooking engines for OpenGL, or is everyone just moving to MinHook to avoid the boilerplate?