I have developed a minifilter driver which encrypt and decrypt a txt file on fly when user tries to write and read it. It is intercepting IRP_MJ_READ callback postread for decrypting buffer and IRP_MJ_WRITE callback prewrite for encrypting buffer to write. It used bcrypt.h AES 128 algorithm. It is working fine for small txt file but for big txt file (Even medium one) it is not working.
I think that issue is due to multiple buffer created for reading and writing file in case of big file while in case file is small it is making single buffer for both read and write.
First of all, read about standard and isolation minifilters. File-level encryption is very hard to implement, and in the case of a large file the problems are most likely due to the use of mapped I/O.
The problem with this architecture, in addition to mentioned multiple read / write buffers, is a fundamental one defined by the Windows file system architecture.
Windows file system driver is free to decide what to do with IO initiated by user application. For example, consider a write IO. These requests can be passed through directly to an underlying volume device, thus skipping the cache, or they can be processed through the file system cache. In the former case the written data needs to be encrypted, in the latter case no encryption is required, as the pages in the file system cache is shared with memory mapped files, so user applications want to see unecrypted data for memory mapped files.
The above is true even for user initiated non-cached IO. A file system driver can convert it to a cached IO and process through the cache, for example to preserve data integrity, so the modified data is visible to application using cached IO or memory mapped files.
That means you cannot convert all user IO to cached or non-cached to make this behaviour deterministic, as this can be done only for a particular file system driver, if can be done at all.
A file system driver will try to preserve data coherency between volume data and cache, thus defeating many attempts to bypass the filesystem cache.
You cannot convert all user IO to paging IO as well to make IO target deterministic (a volume in this case), as this makes data for mapped files / cache incoherent with data on volume. You can try to purge all pages for memory mapped files / cache before doing paging IO, but this might not succeed and slows down the system and can even deadlock it if done carelessly.