Windows System Software -- Consulting, Training, Development -- Unique Expertise, Guaranteed Results

Before Posting...
Please check out the Community Guidelines in the Announcements and Administration Category.

End of file encryption

Sergey_PisarevSergey_Pisarev Member - All Emails Posts: 198
Hello!
I am writing encryption minifilter based on swapbuffers sample. For now I plan to use aes algorithm that doesn’t increase output buffer size. So for now I don’t need isolation filter.

However I am thinking about writes and reads at the end of the file if file size is not multiple of sector size. In that case part of encrypted buffer won’t get to disk and only partial buffer will be returned. Therefore file tail won’t be encrypted correctly since I use block cipher.

For now I think about just obfuscating end of file xoring it with some key.

Is there some better way to solve this problem?

I also thought about writing and reading directly to/from volume, but this won’t work- requirement is for user to be able to copy file on any number of storages on machines without encryption filter. He shouldn’t be able to read this file on external machine, but file should be read correctly on work machine after any number of copies.

Thank you !

Comments

  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 7,676

    For now I plan to use aes algorithm that doesn’t increase output buffer size. So for now I don’t need isolation filter.

    Increasing the buffer size isn't the only thing that would require isolation, right? If you need to support simultaneous clear and raw views of the file... you need an isolation filter.

    Just sayin'

    I am thinking about writes and reads at the end of the file if file size is not multiple of sector size

    I let somebody who actually knows what they're talking about answer this (like @Scott_Noone_(OSR) or @rod_widdowson) but don't you zero fill to the block size, and then handle the valid end of data by setting Valid Data Length appropriately??

    Peter

    Peter Viscarola
    OSR
    @OSRDrivers

  • Sergey_PisarevSergey_Pisarev Member - All Emails Posts: 198
    Thank you Peter !
    I definitely hope fs experts will chime in.

    Could you please explain what is clear and raw views.
  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 7,676

    Could you please explain what is clear and raw views.

    So, I' assuming... since you're doing this as a Minifilter... that your encryption works (a) transparently, (b) on a per-file basis, and (c) provides automatic decryption of a file only to SOME (properly authorized) users. This means that users who have proper access to some file will see DECRYPTED (clear text) data and other users (those who do not have proper access to the file) will see ENCRYPTED (raw text) data.

    So...
    "Clear" = "Decrypted"
    "Raw" = "Data that's encrypted"

    The primary motivation for an isolation filter is to provide multiple, simultaneous, views of the same file. So, an authorized user opens a particular word document... they see nice, normal, decrypted data. While some backup program running in the background simultaneously opens this same file... and it sees the encrypted "raw" version of the file.

    Peter

    Peter Viscarola
    OSR
    @OSRDrivers

  • Sergey_PisarevSergey_Pisarev Member - All Emails Posts: 198
    Thank you very much Peter !

    We assuming that anyone who can login is ok user. So fortunately I don’t need to implement different views. The goal is to encrypt files on usb sticks to prevent data leak from organization.

    I would love to write isolation filter. Unfortunately I’m afraid I won’t be given 1.5 -2 years to do it :(
  • Martin_DrábMartin_Dráb Member - All Emails Posts: 80

    For now I think about just obfuscating end of file xoring it with some key.

    What scheme/algorithm are you using to encrypt file contents?

    Martin Dráb

  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 7,676

    In his original post, he said he’s using AES and given he’s concerned about sector size, I’d guess CBC mode?

    Peter

    Peter Viscarola
    OSR
    @OSRDrivers

  • Sergey_PisarevSergey_Pisarev Member - All Emails Posts: 198

    Hello guys !
    My plan is to use aes in ctr mode

  • Martin_DrábMartin_Dráb Member - All Emails Posts: 80

    My plan is to use aes in ctr mode

    well, CTR mode does not require any padding (e.g. size of the ciphertext matech the plaintext, except the initialization vector). It just turns the block cipher into a stream one. So, if you know where to store the initialization vector (there is no need to keep it secret, the whole world can know it), you need not to change size of the encrypted file.

    But maybe, I understood your problem the wrong way.

    Martin Dráb

  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 7,676

    My plan is to use aes in ctr mode

    Interesting choice.

    You do realize that CTR mode isn't supported by CNG?

    Peter

    Peter Viscarola
    OSR
    @OSRDrivers

  • Sergey_PisarevSergey_Pisarev Member - All Emails Posts: 198

    @Martin_Dráb said:

    My plan is to use aes in ctr mode

    well, CTR mode does not require any padding (e.g. size of the ciphertext matech the plaintext, except the initialization vector). It just turns the block cipher into a stream one. So, if you know where to store the initialization vector (there is no need to keep it secret, the whole world can know it), you need not to change size of the encrypted file.

    But maybe, I understood your problem the wrong way.

    Thank you Martin. Are you sure ? I think I still need to cipher in 16 bytes blocks. So if file size is 17 bytes I can’t cipher all content by aes. I mean aes is still block cipher, right ? I saw ctr implementation on github and I think it uses 16 byte blocks

    @Peter_Viscarola_(OSR) said:

    My plan is to use aes in ctr mode

    Interesting choice.

    You do realize that CTR mode isn't supported by CNG?

    Peter

    Thank you Peter. Yes I know that, I knew I will need to implement algorithm myself.

  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 7,676

    I think I still need to cipher in 16 bytes blocks.

    Yes, you need to cipher in 16 byte blocks. That's a basic requirement of AES, regardless of the mode used.

    So if file size is 17 bytes I can’t cipher all content by aes

    Well, you cipher 32 bytes and then "adjust", right? 1 byte of data and 16 bytes of zeros (which become encrypted).

    Peter

    Peter Viscarola
    OSR
    @OSRDrivers

  • Sergey_PisarevSergey_Pisarev Member - All Emails Posts: 198

    @Peter_Viscarola_(OSR) said:

    I think I still need to cipher in 16 bytes blocks.

    Yes, you need to cipher in 16 byte blocks. That's a basic requirement of AES, regardless of the mode used.

    Thank you for confirmation Peter.

    So if file size is 17 bytes I can’t cipher all content by aes

    Well, you cipher 32 bytes and then "adjust", right? 1 byte of data and 16 bytes of zeros (which become encrypted).

    Peter

    I can encrypt extended buffer but i can’t write this buffer, since that would increased file size. And I can’t decipher from partial buffer.

  • Dejan_MaksimovicDejan_Maksimovic Member - All Emails Posts: 260
    via Email
    Not in CTR/XTS/CTS mode.

    >> I think I still need to cipher in 16 bytes blocks.
    >
    > Yes, you need to cipher in 16 byte blocks. That's a basic requirement of
    > AES, regardless of the mode used.
  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 7,676

    Well, you DO just write the extra data... it you need to maintain your own metadata to know where the EOF really is.

    Did you read this article, about file sizes?

    Peter

    Peter Viscarola
    OSR
    @OSRDrivers

  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 7,676

    Not in CTR/XTS/CTS mode.

    I stand humbly corrected.

    As I searched for an authoritative source to quote, I found this very nice explanation courtesy of the IETF.

    Peter

    Peter Viscarola
    OSR
    @OSRDrivers

  • Martin_DrábMartin_Dráb Member - All Emails Posts: 80

    Yes, you need to cipher in 16 byte blocks. That's a basic requirement of AES, regardless of the mode used.

    Yes, you need to encrypt in 16 byte blocks, that is correct. The trick here is that CTR uses the block cipher "only" as a key stream generator. The key stream is then just XORed on the plaintext. So, for N bytes of plaintext, you generate
    [code]
    (N + 15) / 16 * 16
    [/code]
    bytes of key stream (16 is the block size). Then, you use only N bytes of the key stream to do the actual encryption (XOR) step. Decryption works along the same lines (since CTR actually does not require the "block cipher primitive" to be able to decrypt...).

    So, no extra bytes needed (expect for the IV).

    You do realize that CTR mode isn't supported by CNG?

    Well, implementing CTR is not that difficult (unlike GCM and other more exotic stuff), so I would not consider this being a big deal.

    Martin Dráb

  • rod_widdowsonrod_widdowson Member - All Emails Posts: 1,112

    Sorry, late to the game here...

    Interesting conversations about different encryption modes, thanks all for broadening my knowledge..

    Beyond that

    • Isolation has nothing to do with encryption. Shadow file objects are just a way of allowing a filter to separate the file object that the application sees from the file object that the filesystem uses. Throw support for caching into the works and you have isolation. It's a tool. If you want to present two different views of a file then Isolation is super useful and the canonical solution is encrypted/plain text. But (plucking ridiculous solutions out of the air) you could use it to present an XML view or a JSON view of the same data, or the ANSI or "unicode" view of a text file. There is even a case for isolation when all you want to do is wrest control of the cache from the FSD.
    • It is quite feasible to implement per file encryption without isolation. You just only get to chose once per reboot whether you see the file encrypted or decrypted (remember you don't own the cache so you don't get to decide what gets caches or when, you can get close but not enough to make guarantees). If you really wanted to own implementing the application full stack there are FSCTLs out there call things like "Read Encrypted" to help get access
    • Size changing a file also has nothing to with encryption, its another tool. The primary reason to size change a file is to store metadata in it in a manner which doesn't depend on ADS. In the encryption case it's handy for things like the key. After all if you are not associating a unique key with the file why are you not doing the encryption at the device level (which is so much easier)
  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 7,676

    Gosh I love Mr. Widdowson’s answers...

    Peter

    Peter Viscarola
    OSR
    @OSRDrivers

  • Sergey_PisarevSergey_Pisarev Member - All Emails Posts: 198
    edited February 27
    Thank you very much Rod !
    I understand that isolation is not the only option, but I always thought that this is the “right” way to do encryption.

    Also I would like to thank all participants of this thread. Thank you guys very much! This discussion was very insightful. My thanks are little late because I wasn’t able to post for some reason, but here they are.

    Looks like I have got my ntfsd posting ability back. Thank you our osr masters :)
  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 7,676
    edited February 27

    @Sergey_Pisarev ... We(the mods) didn’t do anything to change, disable, or enable, or in any way alter your posting ability. So... mystery!

    Edited to add: Your posts were in the spam queue for some reason. I’ll mark you as a “trusted” member so that your posts won’t be subject to such evaluation again.

    Anytime you post and it doesn’t appear, toss a message in the A&A Section to alert us. We don’t get notified of the spam queue filling up, and it can sometimes be weeks before we check it unless somebody prompts us.

    And, yes... it does indeed catch a LOT of annoying spam.

    Peter

    Peter Viscarola
    OSR
    @OSRDrivers

Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Upcoming OSR Seminars
OSR has suspended in-person seminars due to the Covid-19 outbreak. But, don't miss your training! Attend via the internet instead!
Kernel Debugging 30 Mar 2020 OSR Seminar Space
Developing Minifilters 20 Apr 2020 LIVE ONLINE
Writing WDF Drivers 11 May 2020 LIVE ONLINE
Internals & Software Drivers 28 Sept 2020 Dulles, VA