Security in Context: The BeyondTrust Blog

Welcome to Security in Context

Bringing you news and commentary on solutions and strategies for protecting critical IT infrastructure in the context of your business.

DEP Down Part 2: Why is DEP failing?

Posted October 21, 2010    The eEye Research Team

In the first part of the series “DEP Down”, we discussed how DEP (Data Execution Prevention) is not always enabled on the application targeted by attackers. When it is enabled, it can be defeated in a number of ways:

  1. Return-to-libc attacks
    These attacks, while normally limited to simple system commands, will always evade DEP as code will never execute from non-executable memory.
  2. Resetting the NX bit on the protected page to allow execution
    Exploit code can accomplish this by calling the VirtualProtect API passing in the address of malicious code and specifying PAGE_READ_WRITE_EXECUTE.
  3. Disabling DEP for current process
    An attacker could call SetProcessDEPPolicy (supported on Windows XP SP3, Vista SP1 and Windows 2008) to disable DEP for the current process with the caveat that it can be accomplished only if the process has not called this function itself.
    Another way to accomplish this is by calling the NtSetInformationProcess API to disable DEP on the current process. A good article explaining this bypass can be found here.
  4. Allocating new writable and executable memory then copying the shellcode payload to it and jumping to it
  5. Copying the shellcode to an already existing writable memory area and jumping to it

But how would an attacker be able to do any of the above if DEP will not let it execute in the first place? The answer lies in using a technique known as ROP (Return Oriented Programming). In a nutshell, ROP is a very clever way of using pieces of already existing code (called snippets or gadgets) from the current process and chaining them together in order to disable DEP (or perform other nefarious actions).

Without going into the details of ROP (although we do recommend reading the document linked above as well as Dino Dai Zovi’s excellent presentation), we should point out that there is one additional requirement for ROP to succeed: module(s) from which code will be “borrowed” must be loaded at predictable and consistent locations. And that is exactly what used to happen until ASLR (Address Space Layout Randomization) came around.

ASLR will throw a wrench in the wheel by randomizing the location where modules load and by doing so, it makes it impossible to reliably find where to jump in a ROP chain. The problem with ASLR is that it is an opt-in feature by default (it can be forced set to always on via registry) and all it takes is one module to be compiled by the vendor without the /DYNAMICBASE linker flag to provide enough ROPe to hang yourself (pun intended).

A classic example of such an exploit, and maybe the first, was the LibTiff Adobe Reader exploit. The exploit makes use of one Adobe DLL which did not use ASLR (BIB.dll) from which it managed to “borrow” enough code to allocate executable memory, copy the payload to it, and then jump to it. Pretty impressive, right?

The weak point in this case was the memory where the code was copied was left as PAGE_READ_WRITE_EXECUTE (it needed the WRITE access to be able to copy to it in the first place). That is enough information for a security product capable of tracking where code is being executed from to detect foul play. Knowing that Adobe Reader is not supposed to execute from writable memory (even if marked as executable) is a dead giveaway.

To recap, DEP is a great security measure offered by Windows to defend against attacks exploiting buffer overflow vulnerabilities – especially when used with ASLR – however it is not invulnerable and additional security is still needed.

Blog by:
Laurentiu Nicula
Founding Software Engineer


Leave a Reply

Additional articles


Answering the age-old question, ‘What’s plugged into my network?’

Posted October 9, 2015    Alejandro DaCosta

“What’s plugged into my network?” is a question I hear frequently from security administrators. And, really, it’s no surprise why. No longer do we have to account just for the physical servers in our datacenters, workstations and a few network devices. Now we need to keep track of roaming laptops, dynamic virtual systems, off-site cloud deployments and BYOD.


Closing the Vulnerability Gap

Posted October 7, 2015    Brian Chappell

Managing vulnerabilities is a significant challenge for many organizations. The main difficulties with managing this manifest in two key areas. The first is that the list isn’t static. The second is priority.


Scottrade Breach: Identified by Federal Officials

Posted October 5, 2015    Morey Haber

Late afternoon on October 2nd, news leaked out of another large security breach, now at Scottrade. The identity count of records, in the millions again (4.6 million is the latest). This breach comes on the second day of national CyberSecurity month, the first being Experian/T-Mobile breach.