I have written an antimalware program that uses an NT driver as part of its solution. Although the driver works correctly as is, I would like to pack/encrypt it in some fashion to avoid simple disassembly and analysis by malware writers seeking to evade my program. A colleague and I have been working on this and have run into a problem.
We have encrypted the .reloc section of the driver image. However, when it is decrypted at runtime, as soon as the first byte of the .reloc section is modified in memory, the system (VirtualPC) reboots. There is no BSOD, just an immediate reboot.
We are fairly confident that there is nothing wrong with the assembly code itself, for two reasons.
(1) If the driver headers are modified so that the same code runs in ring 3, the relocs are decrypted and applied successfully.
(2) If a small test driver is encrypted instead of the actual driver used in my program, the runtime decryption again is successful and the driver loads.
We suspect that the problem may have to do with paging because of reason (2) above. Is this possible? If not, what else might cause this behavior?
The small test driver is less than 2*PAGE_SIZE (8K) large, whereas the real driver is much larger.
I have not specified any #pragma alloc_text directives in any of the code, which I thought meant every driver function was by default non-pageable. Is this true? Is DriverEntry put in the INIT section by default if no pragma is specified? In which section are the other driver functions put?
I do allocate dynamically from paged pool – but I don’t think that will change the pageability of the driver image itself.
Any guidance would be greatly appreciated. Thank you very much!