Bypassing DEP via Oddly Specific ROP Gadgets

02 June 2026 ยท 14 min

Stack-based Overflows and Defence Mechanisms

Traditional stack-based buffer overflow exploits offered a straightforward path to arbitrary code execution. The recipe starts with some processing of user input that has improper length validation, such as uses of the strcpy or gets C functions. Supplying specific strings to these functions causes the program to write beyond the specified input buffer, overflowing into other sections of the stack frame. This provides attackers with two abilities. Firstly, it would be possible to overwrite the contents of the return pointer which resides on the stack frame, allowing redirection of execution flow. Secondly, it allows arbitrary content to be written into the stack itself. Arbitrary code execution can be achieved through a combination of both abilities, where attackers write instructions into the stack, then trigger its execution by overwriting the return pointer to execute an instruction such as JMP ESP or CALL ESP. This allows for execution of an injected payload in the context of the application.

Modern operating systems neutralized this approach to arbitrary code execution with the implementation of executable-space protection. This involves marking memory segments as non-executable, such that attempts to execute instructions residing in certain segments will trigger exceptions. Windows implements this as Data Execution Prevention (DEP), which enforces page-level memory permissions, primarily designating memory segments handling dynamic data such as the stack and heap as readable and writable but non-executable. This protection renders traditional approaches of injecting code directly onto the stack obsolete, as arbitrary code written into the stack can no longer be executed by the program with just a simple JMP ESP call.

Return Oriented Programming

Return-Oriented Programming (ROP) emerged as a technique for bypassing executable-space protection mechanisms such as DEP. Rather than injecting executable code directly onto the stack or heap, ROP leverages existing instructions that are already present within the target application’s executable. These are typically instructions located in the .text segment of the application, or imported dynamic link libraries (common ones being kernel32.dll and ntdll.dll on Windows systems).

The building blocks of ROP are known as gadgets, which are short, pre-existing instruction sequences that end with a RET instruction. These gadgets can individually achieve effects such as transferring data between registers or arithmetic on data within registers. Attackers can chain multiple gadgets to perform more complex actions by loading the memory addresses of subsequent gadgets into the stack. This is possible as all gadgets end with a RET instruction; once a gadget completes execution and returns, the next address is popped off the stack and into the instruction pointer (EIP or RIP), allowing for a chain of gadgets to be executed.

The elegance of ROP is that it renders DEP ineffective, as every individual instruction executed as part of a chain of ROP gadgets lies within memory pages that are marked as executable, never triggering access violations. While attackers still need to find some manner of loading addresses of gadgets onto the stack, DEP is not violated as the values only represent addresses, and are never treated and executed as instructions by the system.

Windows APIs to Disable or Bypass DEP

Individual ROP gadgets only contain small numbers of instructions preceding the RET instruction, allowing for only relatively simple actions such as adding two registers together or moving contents between two registers, well short of what can be considered arbitrary code execution. Despite this, ROP chains can still be combined with certain APIs exposed by the OS to attain code execution while respecting DEP.

One such example is the VirtualAlloc function found on Windows, which can be used to change the state of a region of memory pages within the virtual address space of the process in question.

LPVOID VirtualAlloc(
  [in, optional] LPVOID lpAddress,
  [in]           SIZE_T dwSize,
  [in]           DWORD  flAllocationType,
  [in]           DWORD  flProtect
);

Thus, an approach to leverage existing stack-based buffer overflow vulnerabilities to gain arbitrary code execution on environments with DEP lies in the flProtect argument. The API supports supplying the value 0x40 (PAGE_EXECUTE_READWRITE) which marks a specified memory region as readable, writable and executable. On top of this, it is possible to specify arbitrary memory regions for this change through the lpAddress argument. Effectively, it would be possible to roll back the executable-space protections on the stack introduced by DEP by calling the VirtualAlloc API, specifying all read, write and execute permissions on the region of memory allocated for the stack frames for a given executable. Once complete, it would be possible to copy shellcode onto the stack and have it executed, gaining code execution without violating DEP.

Another useful API is the WriteProcessMemory API which writes data to an area of memory in a specified process.

BOOL WriteProcessMemory(
  [in]  HANDLE  hProcess,
  [in]  LPVOID  lpBaseAddress,
  [in]  LPCVOID lpBuffer,
  [in]  SIZE_T  nSize,
  [out] SIZE_T  *lpNumberOfBytesWritten
);

Now, instead of making the stack executable, storing the shellcode on the stack and triggering it, WriteProcessMemory copies the shellcode into another region of memory in the process which is already marked as executable, before returning execution flow to that region to gain code execution, likewise without violating DEP. Note that even if the target section is not initially writable, WriteProcessMemory enables writing to the specified area of memory, performs the write, then disables write access once again.

WriteProcessMemory Approach

This section documents the considerations and ROP gadgets that can be handy in using WriteProcessMemory to “patch” shellcode into a process’ memory, but the approach and methodology works similarly for techniques using VirtualAlloc. Before dealing with the requirements for calling WriteProcessMemory, the following prerequisites need to be performed first:

  1. Verify that a buffer overflow vulnerability exists in the target application, and that you are able to overwrite the EIP value stored on the stack.
  2. Identify the offset required to reach and overwrite EIP on the stack (using something like msf-pattern_create and msf-pattern_offset).
  3. Iterate through and identify the bad characters - bytes that are dropped, filtered or altered by the application when it is copied onto the stack. Usual culprits include the NULL byte (0x00, treated as a string terminator by functions such as strcpy and sprintf which will terminate any copy process), line feed and carriage return bytes (0x0A and 0x0D, treated as end-of-line by functions like fgets which will also terminate any copy operation).
  4. WriteProcessMemory requires the lpBaseAddress as a destination for the shellcode. As the shellcode contains instructions to be executed, it needs to be located somewhere in memory that already has execute permissions, one suitable candidate is the .text section. However, overwriting the .text section (which contains the program’s original execution instructions) may cause it to crash. To prevent the shellcode from overwriting the original instructions and crashing the program, a section of unused memory inside the .text section must be located. This can be identified by searching for spaces that contain contiguous 0x00 bytes within the .text section with a debugger. Ideally, this area of memory should also be aligned to a memory page. This is known as a code cave.

Stack Setup

To leverage the stack-based buffer overflow vulnerability to call WriteProcessMemory, the calling convention for functions must be adhered to. This means setting up the stack in a certain way to replicate a legitimate function call, along with supplying the required functions.

+0x00 address of WriteProcessMemory (dummy value) 
+0x04 return address (address within code cave) 
+0x08 hprocess (-1) 
+0x0c lpBaseAddress (address within code cave) 
+0x10 lpBuffer (dummy) 1
+0x14 nSize (dummy) 
+0x18 lpNumberOfBytesWritten (dummy)

The following arguments are required to be passed to WriteProcessMemory

On the surface, one might opt to directly include the required arguments into the buffer overflow string, such that it will be loaded into the stack. However, there will be certain challenges with setting up the stack in this manner:

Thus, we are limited to only loading dummy values that do not contain bad characters (such as 0x45454545) into the stack via the buffer overflow vulnerability, before finding ROP gadgets to also “patch” in the correct values for the calling stack. The planned layout of the stack will look something like this.

Stack layout

Interesting Gadgets

In constructing ROP chains to complete and patch the arguments required for the call to WriteProcessMemory , we are usually restricted to the gadgets that are already present within the executable. It is often the case that simple gadgets that achieve specific actions are not always available, meaning that we may have to rely on more complex or longer ROP gadgets to perform simple actions such as adding two registers. These gadgets often come with side-effects that have to be handled, or require a more convoluted way to perform the action. Finding these gadgets requires imagination and intuition, but here are some non-exhaustive approaches to perform certain tasks.

Saving the Stack Address (or somewhere nearby)

One significant obstacle in patching the required arguments for WriteProcessMemory into the skeleton is handling dynamic addresses, primarily the lpBuffer (where the shellcode to be executed is located within the stack), as the precise address on the stack may change between runs. Thus, it cannot be hardcoded and needs to be dynamically obtained with ROP gadgets. Additionally, the address of the stack is also needed to dynamically patch the values in the calling stack skeleton with ROP gadgets. Hence, it is important to store this address in a less-volatile general-purpose register (such as EAX, EBX, ECX, EDX) so that it can be referenced throughout the exploit ROP chain.

The simplest option would be to find a gadget that copies the value of ESP to another register. Note that we do not need the exact address of the location within the stack, somewhere near the region of memory will do. However, it should not be taken for granted that simple, straightforward gadgets like these are always present.

MOV EAX, ESP; RET;

Often, the desired action (in this case, getting a copy of the memory address somewhere near the stack), exists alongside another instruction that introduced unwanted side-effects.

PUSH ESP; POP ECX; RET 0x08;

In the example above, the stack address (ESP) is effectively “copied” into the ECX register, but it is important to also notice the impacts of the RET 0x08 instruction which adds 8 to the value of ESP once the gadget returns execution. To neutralize the side-effects of this gadget, a padding of 0x08 bytes needs to be added to the resulting ROP chain to ensure that no gadgets will be skipped over once the previous returns. The exploit chain will thus look something like this:

rop += pack("<L", (0x37563f96)) # address of PUSH ESP; POP EAX; RET 0x08; 
rop += pack("<L", (0x55555555)) # 4 bytes junk padding
rop += pack("<L", (0x55555555)) # 4 bytes junk padding
rop += pack("<L", (0x4163a052)) # address of XOR EAX, EAX; RET

In the absence of a singular instruction which achieves a desired effect, it may be useful to consider how multiple gadgets can be chained together to complete the action.

MOV EBP, ESP; POP EAX; RET;
DEC EBX; LEA EAX, dword [EBP-0x0A]; RET;

The two gadgets above still effectively copy the address of somewhere near the stack into the EAX register. In the first gadget, the stack address is copied into the EBP register. While this might suffice, working with the EBP register if often discouraged as it can be significantly changed as the calling stack progresses through the ROP chain. However, this can be combined with the second gadget, importantly the LEA EAX, dword [EBP-0x0A] instruction. Since ESP is now stored in EBP, this will load the effective address of a specified variable and store the result into EAX. When dword [EBP-0x0A] is supplied as the argument, it is important to note that it does not actually dereference the memory at EBP-0x0A, the actual effect is that the address of the variable located at EBP-0x0A will be calculated. Since EBP holds the value of ESP, we can obtain the address of ESP-0x0A in EAX.

Address Arithmetic

Another category of useful gadgets is one that enables obtaining values that contain NULL bytes or bad characters that would otherwise not be able to be introduced into the stack via the buffer overflow vulnerability. For instance, the lpNumberOfBytesWritten argument for WriteProcessMemory needs a value of 0x0, but this cannot be directly loaded into the stack to form the function calling frame. ROP gadgets would be needed to “patch” the correct value.

As certain values cannot be worked on directly, gadgets such as incrementing and decrementing a value already existing in a register can be useful to perform small scale addition and subtraction operations. These can be repeated to achieve larger values to be added or subtracted.

INC EAX; RET;
DEC EAX; RET;

These are particularly useful when traversing the calling stack for WriteProcessMemory. For instance, if EAX contains the address pointing to hProcess, something like the code snippet below can be used to move to the next argument, lpBaseAddress

for i in range(8):
    rop += rop += pack("<L", (0x493458b0)) # address of INC EAX; RET;

However, the process of performing addition or subtraction can become very tedious once the value gets into the hundreds. For instance, shellcode to obtain remote code execution often ends up being beyond 300 bytes in size, meaning that such value needs to be loaded into the calling frame skeleton somehow, as the nSize argument to WriteProcessMemory. Loading directly into the stack will fail due to the leading null bytes, but relying on INC instructions exclusively will cause the overall exploit string to be excessively long. To address this, a combination of the two gadgets below can be used to perform addition of large values.

# load -360 to stack
POP EAX; RET;
NEG EAX; RET;

Instead of working with the hexadecimal representation of 360 (0x00000168), we first negate the value before loading it into the stack. Note that the correct negated value will be represented as -(x+1), meaning that the correct value in this case will be (0xfffffe97 or -361). This will then be popped off the stack into EAX, before it is negated to obtain the value of 360 (0x168) in EAX. The resulting ROP chain will look something like this:

rop += pack("<L", (0x43bbe930)) # address of POP EAX; RET; 
rop += pack("<L", (0xfffffe97)) # negation of 0x168 into EAX 
rop += pack("<L", (0x42a90254)) # address of NEG EAX; RET;

Some neat tricks can be done with certain operators to obtain the NULL (0x00) value into the register, which may be required for some arguments. Examples include subtracting a register from itself, or performing an exclusive or on a register with itself - there can be many combinations so pick a gadget that is present and is relatively short.

XOR EAX; EAX; RET; 
SUB EBX; EBX; RET;

Stack Pivoting to Trigger Function Call

Once the WriteProcessMemory calling frame skeleton has been patched with the correct values as arguments, the last step in the ROP chain would be to populate the ESP register with the address of a region in the stack which contains the address of WriteProcessMemory, its return address and the required arguments. The desired effect is for the ESP register to point to first DWORD (address of WriteProcessMemory), such that when the last gadget executes the RET instruction, that DWORD will be popped into the EIP register, causing execution to be redirected successfully. From here, the standard calling convention is adhered to, with the top of the stack now containing a DWORD that is the return address after WriteProcessMemory completes execution, with subsequent DWORDs containing the arguments supplied to the API call.

An example of a gadget which performs stack pivoting is the following. Assuming that the stack address is stored in EAX, it will be pushed onto the stack, then popped back into the ESP register. Once the final RET instruction is executed, WriteProcessMemory will be called with the patched arguments.

PUSH EAX; RET;
POP ESP; RET;

Another simpler gadget is the one below, which makes use of only one XCHG instruction. Assuming that the stack address is stored in EAX, it swaps it into ESP, achieving the same effect.

XCHG EAX; ESP; RET;

Another interesting gadget which allows for stack pivoting is one that contains a LEAVE instruction.

LEAVE; RET;

The LEAVE instruction performs the following actions:

MOV ESP, EBP; 
POP EBP;

Thus it provides another avenue towards controlling the contents of the ESP register - if you have control over the EBP register, a stack pivot can be successfully performed via the LEAVE instruction, as it will copy the contents of EBP to ESP.

Stepping through the call to WriteProcessMemory will cause the shellcode that is loaded into the stack to be written into a code cave within the process memory that possesses execute permissions. Once WriteProcessMemory returns, the next DWORD, which is the address of the code cave, will be popped into the EIP register, causing execution to be redirected to the code cave, allowing for execution of arbitrary code, completing the exploit process.