Encryption filter driver?

NTFSD
1

Hi all,
I’m writing a file system encryption filter driver (an amazingly
new concept, I know) and have a few questions that I was hoping some of
the battle-tested filter driver veterans could enlighten me on, if you
are not sick of doing it yet. Also, I know this is a large development
effort, will take a very long time, and will be fraught with nasty
issues and will almost be impossible, so you can try to discourage me,
but I’m not giving up :wink: Here are my questions:

  1. Do you consider it theoretically possible to build, or has anyone
    successfully completed, a filter driver that adds a header/trailer to
    files for metadata and adjusts file offsets/sizes without the
    header/trailer being page aligned? I’ve seen traffic in the archives
    that discusses this sort of thing, but I was curious to know if I should
    even attempt such an architecture. It seems there are numerous issues
    with this when interactions with the VMM and CM are taken into account.

  2. Is it generally a “bad idea” to post to a worker thread in the
    completion routine of a paging i/o read or write? I’m assuming it is.

  3. Can I roll my own IRPs and used these to perform reads in a paging
    i/o read or write dispatch routine without problems?

  4. Can I block in the dispatch routines for paging i/o reads/writes?

Thanks for your help.

Matt

2

(1) Yes there are such examples. Complex beasts.

(2) No. Using work routines from completion routines (paging I/O or not) is
perfectly acceptable, provided you understand the synchronization issues
involved.

(3) Paging I/O operations are called at APC_LEVEL, so you need to understand
the ramifications of this. But NTFS and FAT roll their own IRPs all the
time in the context of paging I/O operations…

(4) Block for what? Block for locks? No, that won’t work. Block for
external events, like disk I/O to complete? Sure, that works fine. Keep in
mind the locking hierarchy here - file system locks, then cache manager,
then memory manager.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: Matthew White [mailto:xxxxx@fsnnet.net]
Sent: Friday, March 07, 2003 3:15 PM
To: File Systems Developers
Subject: [ntfsd] Encryption filter driver?

Hi all,
I’m writing a file system encryption filter driver (an amazingly
new concept, I know) and have a few questions that I was hoping some of
the battle-tested filter driver veterans could enlighten me on, if you
are not sick of doing it yet. Also, I know this is a large development
effort, will take a very long time, and will be fraught with nasty
issues and will almost be impossible, so you can try to discourage me,
but I’m not giving up :wink: Here are my questions:

  1. Do you consider it theoretically possible to build, or has anyone
    successfully completed, a filter driver that adds a header/trailer to
    files for metadata and adjusts file offsets/sizes without the
    header/trailer being page aligned? I’ve seen traffic in the archives
    that discusses this sort of thing, but I was curious to know if I should
    even attempt such an architecture. It seems there are numerous issues
    with this when interactions with the VMM and CM are taken into account.

  2. Is it generally a “bad idea” to post to a worker thread in the
    completion routine of a paging i/o read or write? I’m assuming it is.

  3. Can I roll my own IRPs and used these to perform reads in a paging
    i/o read or write dispatch routine without problems?

  4. Can I block in the dispatch routines for paging i/o reads/writes?

Thanks for your help.

Matt

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

3

Thanks for the response, Tony. The help is much appreciated. In
relation to item (1) below (the complex beasts), I was wondering if you,
or anyone else who has done such work, could explain one issue I have
been wondering about. In an earlier post you said this:

“The difficult case for any filter that attempts to manipulate data
sizes in
the file systems is memory mapped files - the size information comes
from
the VM system, not the file system. While the VM system gets the
initial
size from the file system (via an IRP_MJ_QUERY_INFORMATION) the file
system
reports any subsequent file size changes to the VM system using a
programmatic interface (CcSetFileSize as I recall.)”

And this:

“Of course, many applications only use the read/write interface.
Notepad,
however, uses the memory mapped API and is thus a great test application
for
a filter driver of this type.”

Are there any general strategies that would be useful to work
around/with the issue of the underlying file system calling
CcSetFileSizes() to change the file size(s) in the cache manager for a
filter driver that would like to mess with file sizes? There must be
some way to do this (since these types of filters are possible), but I
don’t see right away how this issue can be resolved at all, since this
call by the file system into the cache manager is independent of a
filter driver. I have seen a couple posts in the archives describing
such things as duplicated file objects being used. Just want to know if
there is a “right way” to do this type of thing, or if it’s kind of a
“you’re on your own” thing. If anyone has any comments on this, many
thanks.

Matt

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Tony Mason
Sent: Friday, March 07, 2003 3:33 PM
To: File Systems Developers
Subject: [ntfsd] RE: Encryption filter driver?

(1) Yes there are such examples. Complex beasts.

(2) No. Using work routines from completion routines (paging I/O or
not) is perfectly acceptable, provided you understand the
synchronization issues involved.

(3) Paging I/O operations are called at APC_LEVEL, so you need to
understand the ramifications of this. But NTFS and FAT roll their own
IRPs all the time in the context of paging I/O operations…

(4) Block for what? Block for locks? No, that won’t work. Block for
external events, like disk I/O to complete? Sure, that works fine.
Keep in mind the locking hierarchy here - file system locks, then cache
manager, then memory manager.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: Matthew White [mailto:xxxxx@fsnnet.net]
Sent: Friday, March 07, 2003 3:15 PM
To: File Systems Developers
Subject: [ntfsd] Encryption filter driver?

Hi all,
I’m writing a file system encryption filter driver (an amazingly
new concept, I know) and have a few questions that I was hoping some of
the battle-tested filter driver veterans could enlighten me on, if you
are not sick of doing it yet. Also, I know this is a large development
effort, will take a very long time, and will be fraught with nasty
issues and will almost be impossible, so you can try to discourage me,
but I’m not giving up :wink: Here are my questions:

  1. Do you consider it theoretically possible to build, or has anyone
    successfully completed, a filter driver that adds a header/trailer to
    files for metadata and adjusts file offsets/sizes without the
    header/trailer being page aligned? I’ve seen traffic in the archives
    that discusses this sort of thing, but I was curious to know if I should
    even attempt such an architecture. It seems there are numerous issues
    with this when interactions with the VMM and CM are taken into account.

  2. Is it generally a “bad idea” to post to a worker thread in the
    completion routine of a paging i/o read or write? I’m assuming it is.

  3. Can I roll my own IRPs and used these to perform reads in a paging
    i/o read or write dispatch routine without problems?

  4. Can I block in the dispatch routines for paging i/o reads/writes?

Thanks for your help.

Matt

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

You are currently subscribed to ntfsd as: xxxxx@fsnnet.net
To unsubscribe send a blank email to xxxxx@lists.osr.com

4

Notepad doesn’t keep the file open while working with it. For that
reason it is not as good test application as one might think.

“Of course, many applications only use the read/write interface.
Notepad, however, uses the memory mapped API and is thus a great test
application for a filter driver of this type.”

–
Kind regards, Dejan M. MVP for DDK
http://www.alfasp.com E-mail: xxxxx@alfasp.com
Alfa Transparent File Encryptor - Transparent file encryption services.
Alfa File Protector - File protection and hiding library for Win32
developers.
Alfa File Monitor - File monitoring library for Win32 developers.

5

The approaches I know:

  • Don’t change the size of the data within the file; if you have extra
    information, store it in a different location (e.g., stream or separate
    database or file.)
  • Create a secondary file object (IoCreateStreamFileObjectLite will do the
    trick) and pass that secondary file object to NTFS. The primary file object
    (the one that arrived at your driver) then points to a device object
    controlled by your filter. The downside to this approach is that *you* are
    now responsible for managing the VM state. This increases complexity by at
    least an order of magnitude (essentially, you are implementing your own file
    system layered on top of FAT or NTFS.)

There may be other approaches that people have found to work for them and if
that is the case I’d love to hear their suggestions so we can augment the
FAQ.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: Matthew White [mailto:xxxxx@fsnnet.net]
Sent: Friday, March 14, 2003 8:50 PM
To: File Systems Developers
Subject: [ntfsd] RE: Encryption filter driver?

Thanks for the response, Tony. The help is much appreciated. In
relation to item (1) below (the complex beasts), I was wondering if you,
or anyone else who has done such work, could explain one issue I have
been wondering about. In an earlier post you said this:

“The difficult case for any filter that attempts to manipulate data
sizes in
the file systems is memory mapped files - the size information comes
from
the VM system, not the file system. While the VM system gets the
initial
size from the file system (via an IRP_MJ_QUERY_INFORMATION) the file
system
reports any subsequent file size changes to the VM system using a
programmatic interface (CcSetFileSize as I recall.)”

And this:

“Of course, many applications only use the read/write interface.
Notepad,
however, uses the memory mapped API and is thus a great test application
for
a filter driver of this type.”

Are there any general strategies that would be useful to work
around/with the issue of the underlying file system calling
CcSetFileSizes() to change the file size(s) in the cache manager for a
filter driver that would like to mess with file sizes? There must be
some way to do this (since these types of filters are possible), but I
don’t see right away how this issue can be resolved at all, since this
call by the file system into the cache manager is independent of a
filter driver. I have seen a couple posts in the archives describing
such things as duplicated file objects being used. Just want to know if
there is a “right way” to do this type of thing, or if it’s kind of a
“you’re on your own” thing. If anyone has any comments on this, many
thanks.

Matt

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Tony Mason
Sent: Friday, March 07, 2003 3:33 PM
To: File Systems Developers
Subject: [ntfsd] RE: Encryption filter driver?

(1) Yes there are such examples. Complex beasts.

(2) No. Using work routines from completion routines (paging I/O or
not) is perfectly acceptable, provided you understand the
synchronization issues involved.

(3) Paging I/O operations are called at APC_LEVEL, so you need to
understand the ramifications of this. But NTFS and FAT roll their own
IRPs all the time in the context of paging I/O operations…

(4) Block for what? Block for locks? No, that won’t work. Block for
external events, like disk I/O to complete? Sure, that works fine.
Keep in mind the locking hierarchy here - file system locks, then cache
manager, then memory manager.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: Matthew White [mailto:xxxxx@fsnnet.net]
Sent: Friday, March 07, 2003 3:15 PM
To: File Systems Developers
Subject: [ntfsd] Encryption filter driver?

Hi all,
I’m writing a file system encryption filter driver (an amazingly
new concept, I know) and have a few questions that I was hoping some of
the battle-tested filter driver veterans could enlighten me on, if you
are not sick of doing it yet. Also, I know this is a large development
effort, will take a very long time, and will be fraught with nasty
issues and will almost be impossible, so you can try to discourage me,
but I’m not giving up :wink: Here are my questions:

  1. Do you consider it theoretically possible to build, or has anyone
    successfully completed, a filter driver that adds a header/trailer to
    files for metadata and adjusts file offsets/sizes without the
    header/trailer being page aligned? I’ve seen traffic in the archives
    that discusses this sort of thing, but I was curious to know if I should
    even attempt such an architecture. It seems there are numerous issues
    with this when interactions with the VMM and CM are taken into account.

  2. Is it generally a “bad idea” to post to a worker thread in the
    completion routine of a paging i/o read or write? I’m assuming it is.

  3. Can I roll my own IRPs and used these to perform reads in a paging
    i/o read or write dispatch routine without problems?

  4. Can I block in the dispatch routines for paging i/o reads/writes?

Thanks for your help.

Matt

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

You are currently subscribed to ntfsd as: xxxxx@fsnnet.net
To unsubscribe send a blank email to xxxxx@lists.osr.com

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

6

The first solution is unpractical and the second is very ugly. I tried also
to put a header in front of the encrypted file but I had a lot of problems
emulating the file objects and FCBs so I gave up. The solution I found, and
I am working on it now, is to put the header at the end of file. That way
the size problem is not anymore so important for the cache and the mapping
between the memory pages and the HDD pages remains natural. The header is
written at the end of file in a separate page to optimize a bit the writing
process and of course it is rewritten every time the file usefull data
exceeds the allocated pages. I don’t know yet what problems will arrise when
the cache is trying to read/write the header page, but I am confident with
this solution. Anyway, I think it is the only solution.
It is just an ideea, hope it is usefull.

Daniel

----- Original Message -----
From: “Tony Mason”
To: “File Systems Developers”
Sent: Sunday, March 16, 2003 1:11 PM
Subject: [ntfsd] RE: Encryption filter driver?

> The approaches I know:
>
> - Don’t change the size of the data within the file; if you have extra
> information, store it in a different location (e.g., stream or separate
> database or file.)
> - Create a secondary file object (IoCreateStreamFileObjectLite will do the
> trick) and pass that secondary file object to NTFS. The primary file
object
> (the one that arrived at your driver) then points to a device object
> controlled by your filter. The downside to this approach is that you
are
> now responsible for managing the VM state. This increases complexity by
at
> least an order of magnitude (essentially, you are implementing your own
file
> system layered on top of FAT or NTFS.)
>
> There may be other approaches that people have found to work for them and
if
> that is the case I’d love to hear their suggestions so we can augment the
> FAQ.
>
> Regards,
>
> Tony
>
> Tony Mason
> Consulting Partner
> OSR Open Systems Resources, Inc.
> http://www.osr.com
>
>
> -----Original Message-----
> From: Matthew White [mailto:xxxxx@fsnnet.net]
> Sent: Friday, March 14, 2003 8:50 PM
> To: File Systems Developers
> Subject: [ntfsd] RE: Encryption filter driver?
>
> Thanks for the response, Tony. The help is much appreciated. In
> relation to item (1) below (the complex beasts), I was wondering if you,
> or anyone else who has done such work, could explain one issue I have
> been wondering about. In an earlier post you said this:
>
> “The difficult case for any filter that attempts to manipulate data
> sizes in
> the file systems is memory mapped files - the size information comes
> from
> the VM system, not the file system. While the VM system gets the
> initial
> size from the file system (via an IRP_MJ_QUERY_INFORMATION) the file
> system
> reports any subsequent file size changes to the VM system using a
> programmatic interface (CcSetFileSize as I recall.)”
>
> And this:
>
> “Of course, many applications only use the read/write interface.
> Notepad,
> however, uses the memory mapped API and is thus a great test application
> for
> a filter driver of this type.”
>
> Are there any general strategies that would be useful to work
> around/with the issue of the underlying file system calling
> CcSetFileSizes() to change the file size(s) in the cache manager for a
> filter driver that would like to mess with file sizes? There must be
> some way to do this (since these types of filters are possible), but I
> don’t see right away how this issue can be resolved at all, since this
> call by the file system into the cache manager is independent of a
> filter driver. I have seen a couple posts in the archives describing
> such things as duplicated file objects being used. Just want to know if
> there is a “right way” to do this type of thing, or if it’s kind of a
> “you’re on your own” thing. If anyone has any comments on this, many
> thanks.
>
> Matt
>
> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com] On Behalf Of Tony Mason
> Sent: Friday, March 07, 2003 3:33 PM
> To: File Systems Developers
> Subject: [ntfsd] RE: Encryption filter driver?
>
>
> (1) Yes there are such examples. Complex beasts.
>
> (2) No. Using work routines from completion routines (paging I/O or
> not) is perfectly acceptable, provided you understand the
> synchronization issues involved.
>
> (3) Paging I/O operations are called at APC_LEVEL, so you need to
> understand the ramifications of this. But NTFS and FAT roll their own
> IRPs all the time in the context of paging I/O operations…
>
> (4) Block for what? Block for locks? No, that won’t work. Block for
> external events, like disk I/O to complete? Sure, that works fine.
> Keep in mind the locking hierarchy here - file system locks, then cache
> manager, then memory manager.
>
> Regards,
>
> Tony
>
> Tony Mason
> Consulting Partner
> OSR Open Systems Resources, Inc.
> http://www.osr.com
>
>
> -----Original Message-----
> From: Matthew White [mailto:xxxxx@fsnnet.net]
> Sent: Friday, March 07, 2003 3:15 PM
> To: File Systems Developers
> Subject: [ntfsd] Encryption filter driver?
>
> Hi all,
> I’m writing a file system encryption filter driver (an amazingly
> new concept, I know) and have a few questions that I was hoping some of
> the battle-tested filter driver veterans could enlighten me on, if you
> are not sick of doing it yet. Also, I know this is a large development
> effort, will take a very long time, and will be fraught with nasty
> issues and will almost be impossible, so you can try to discourage me,
> but I’m not giving up :wink: Here are my questions:
>
> 1. Do you consider it theoretically possible to build, or has anyone
> successfully completed, a filter driver that adds a header/trailer to
> files for metadata and adjusts file offsets/sizes without the
> header/trailer being page aligned? I’ve seen traffic in the archives
> that discusses this sort of thing, but I was curious to know if I should
> even attempt such an architecture. It seems there are numerous issues
> with this when interactions with the VMM and CM are taken into account.
>
> 2. Is it generally a “bad idea” to post to a worker thread in the
> completion routine of a paging i/o read or write? I’m assuming it is.
>
> 3. Can I roll my own IRPs and used these to perform reads in a paging
> i/o read or write dispatch routine without problems?
>
> 4. Can I block in the dispatch routines for paging i/o reads/writes?
>
> Thanks for your help.
>
> Matt
>
>
>
>
> —
> You are currently subscribed to ntfsd as: xxxxx@osr.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
> —
> You are currently subscribed to ntfsd as: xxxxx@fsnnet.net
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
>
> —
> You are currently subscribed to ntfsd as: xxxxx@osr.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
> —
> You are currently subscribed to ntfsd as: xxxxx@ipdevel.ro
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
>
>

7

It certainly isn’t the only solution - I’ve seen examples of all three.

The problem that I’ve always seen with the header at the end of the file is
that it is now visible to applications. While that probably does not matter
for all applications, it will interfere with some applications.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: Daniel Turcanu [mailto:xxxxx@ipdevel.ro]
Sent: Sunday, March 16, 2003 6:22 AM
To: File Systems Developers
Subject: [ntfsd] RE: Encryption filter driver?

The first solution is unpractical and the second is very ugly. I tried also
to put a header in front of the encrypted file but I had a lot of problems
emulating the file objects and FCBs so I gave up. The solution I found, and
I am working on it now, is to put the header at the end of file. That way
the size problem is not anymore so important for the cache and the mapping
between the memory pages and the HDD pages remains natural. The header is
written at the end of file in a separate page to optimize a bit the writing
process and of course it is rewritten every time the file usefull data
exceeds the allocated pages. I don’t know yet what problems will arrise when
the cache is trying to read/write the header page, but I am confident with
this solution. Anyway, I think it is the only solution.
It is just an ideea, hope it is usefull.

Daniel

----- Original Message -----
From: “Tony Mason”
To: “File Systems Developers”
Sent: Sunday, March 16, 2003 1:11 PM
Subject: [ntfsd] RE: Encryption filter driver?

> The approaches I know:
>
> - Don’t change the size of the data within the file; if you have extra
> information, store it in a different location (e.g., stream or separate
> database or file.)
> - Create a secondary file object (IoCreateStreamFileObjectLite will do the
> trick) and pass that secondary file object to NTFS. The primary file
object
> (the one that arrived at your driver) then points to a device object
> controlled by your filter. The downside to this approach is that you
are
> now responsible for managing the VM state. This increases complexity by
at
> least an order of magnitude (essentially, you are implementing your own
file
> system layered on top of FAT or NTFS.)
>
> There may be other approaches that people have found to work for them and
if
> that is the case I’d love to hear their suggestions so we can augment the
> FAQ.
>
> Regards,
>
> Tony
>
> Tony Mason
> Consulting Partner
> OSR Open Systems Resources, Inc.
> http://www.osr.com
>
>
> -----Original Message-----
> From: Matthew White [mailto:xxxxx@fsnnet.net]
> Sent: Friday, March 14, 2003 8:50 PM
> To: File Systems Developers
> Subject: [ntfsd] RE: Encryption filter driver?
>
> Thanks for the response, Tony. The help is much appreciated. In
> relation to item (1) below (the complex beasts), I was wondering if you,
> or anyone else who has done such work, could explain one issue I have
> been wondering about. In an earlier post you said this:
>
> “The difficult case for any filter that attempts to manipulate data
> sizes in
> the file systems is memory mapped files - the size information comes
> from
> the VM system, not the file system. While the VM system gets the
> initial
> size from the file system (via an IRP_MJ_QUERY_INFORMATION) the file
> system
> reports any subsequent file size changes to the VM system using a
> programmatic interface (CcSetFileSize as I recall.)”
>
> And this:
>
> “Of course, many applications only use the read/write interface.
> Notepad,
> however, uses the memory mapped API and is thus a great test application
> for
> a filter driver of this type.”
>
> Are there any general strategies that would be useful to work
> around/with the issue of the underlying file system calling
> CcSetFileSizes() to change the file size(s) in the cache manager for a
> filter driver that would like to mess with file sizes? There must be
> some way to do this (since these types of filters are possible), but I
> don’t see right away how this issue can be resolved at all, since this
> call by the file system into the cache manager is independent of a
> filter driver. I have seen a couple posts in the archives describing
> such things as duplicated file objects being used. Just want to know if
> there is a “right way” to do this type of thing, or if it’s kind of a
> “you’re on your own” thing. If anyone has any comments on this, many
> thanks.
>
> Matt
>
> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com] On Behalf Of Tony Mason
> Sent: Friday, March 07, 2003 3:33 PM
> To: File Systems Developers
> Subject: [ntfsd] RE: Encryption filter driver?
>
>
> (1) Yes there are such examples. Complex beasts.
>
> (2) No. Using work routines from completion routines (paging I/O or
> not) is perfectly acceptable, provided you understand the
> synchronization issues involved.
>
> (3) Paging I/O operations are called at APC_LEVEL, so you need to
> understand the ramifications of this. But NTFS and FAT roll their own
> IRPs all the time in the context of paging I/O operations…
>
> (4) Block for what? Block for locks? No, that won’t work. Block for
> external events, like disk I/O to complete? Sure, that works fine.
> Keep in mind the locking hierarchy here - file system locks, then cache
> manager, then memory manager.
>
> Regards,
>
> Tony
>
> Tony Mason
> Consulting Partner
> OSR Open Systems Resources, Inc.
> http://www.osr.com
>
>
> -----Original Message-----
> From: Matthew White [mailto:xxxxx@fsnnet.net]
> Sent: Friday, March 07, 2003 3:15 PM
> To: File Systems Developers
> Subject: [ntfsd] Encryption filter driver?
>
> Hi all,
> I’m writing a file system encryption filter driver (an amazingly
> new concept, I know) and have a few questions that I was hoping some of
> the battle-tested filter driver veterans could enlighten me on, if you
> are not sick of doing it yet. Also, I know this is a large development
> effort, will take a very long time, and will be fraught with nasty
> issues and will almost be impossible, so you can try to discourage me,
> but I’m not giving up :wink: Here are my questions:
>
> 1. Do you consider it theoretically possible to build, or has anyone
> successfully completed, a filter driver that adds a header/trailer to
> files for metadata and adjusts file offsets/sizes without the
> header/trailer being page aligned? I’ve seen traffic in the archives
> that discusses this sort of thing, but I was curious to know if I should
> even attempt such an architecture. It seems there are numerous issues
> with this when interactions with the VMM and CM are taken into account.
>
> 2. Is it generally a “bad idea” to post to a worker thread in the
> completion routine of a paging i/o read or write? I’m assuming it is.
>
> 3. Can I roll my own IRPs and used these to perform reads in a paging
> i/o read or write dispatch routine without problems?
>
> 4. Can I block in the dispatch routines for paging i/o reads/writes?
>
> Thanks for your help.
>
> Matt
>
>
>
>
> —
> You are currently subscribed to ntfsd as: xxxxx@osr.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
> —
> You are currently subscribed to ntfsd as: xxxxx@fsnnet.net
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
>
> —
> You are currently subscribed to ntfsd as: xxxxx@osr.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
> —
> You are currently subscribed to ntfsd as: xxxxx@ipdevel.ro
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
>
>

—
You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

8

My filter implements the “don’t change file sizes, user must tell you the
files which are encrypted” approach. I.e. the user tells which files are
encrypted (or a program), and the driver just encrypts/decrypts OTF.
The drawback with changing file sizes is that either the listing of files
will find incorrect sizes, or you would have unacceptable performance
degradation - sometimes you cannot correctly work this out for security context
and other security reasons.

Tony Mason wrote:

The approaches I know:

  • Don’t change the size of the data within the file; if you have extra
    information, store it in a different location (e.g., stream or separate
    database or file.)
  • Create a secondary file object (IoCreateStreamFileObjectLite will do the
    trick) and pass that secondary file object to NTFS. The primary file object
    (the one that arrived at your driver) then points to a device object
    controlled by your filter. The downside to this approach is that *you* are
    now responsible for managing the VM state. This increases complexity by at
    least an order of magnitude (essentially, you are implementing your own file
    system layered on top of FAT or NTFS.)

–
Kind regards, Dejan M. MVP for DDK
http://www.alfasp.com E-mail: xxxxx@alfasp.com
Alfa Transparent File Encryptor - Transparent file encryption services.
Alfa File Protector - File protection and hiding library for Win32 developers.
Alfa File Monitor - File monitoring library for Win32 developers.

9

In one product I was involved in developing, the “header” was at the end of
the file. For the file to be opened with decryption enabled two conditions
were required: 1) Correct program opening the file with a supported file
extension. 2) A user logged in with credentials to open that file.

For the programs (Microsoft Office 2000 & Notepad) that were permitted to
see the plaintext, the file size was modified to eliminate the header size
from the real file size. These programs didn’t seem to have a problem with
the techniques used. All other programs could see the file size which
included the header. This permitted the user to send the encrypted file to
other people and have it remain encrypted unless they also possessed the
correct credentials.

Part of the problem with a general purpose encryption is where do you
maintain the required information. With NTFS you can use an alternate
stream and let NTFS worry about the details, but I wonder if it can maintain
the security when a program such as WinWord creates temporary work files -
will the alternate stream be copied? Another solution is to use a by
directory methodology where the info about the directory is stored in some
other location. This was used by a MS-DOS/Win 95 security product,
Watchdog, that I worked on for several years. It is no longer on the
market.

For anyone who wants an idea about good, strong, and secure encryption, I am
fond of using the GemPlus GPK16000, which can create a private/public RSA
key pair. The private key never leaves the card, so possession of the card
is all that is required, though a password/pin number for the card can be
used to require two access items. Even with this, you still have the
problem of identifying which files are encrypted and what session/symmetric
key was used. This all would be a lot easier if NTFS was documented as
FASTFAT has been since the first IFS Kit was released.

----- Original Message -----
From: “Tony Mason”
To: “File Systems Developers”
Sent: Sunday, March 16, 2003 6:45 AM
Subject: [ntfsd] RE: Encryption filter driver?

> It certainly isn’t the only solution - I’ve seen examples of all three.
>
> The problem that I’ve always seen with the header at the end of the file
is
> that it is now visible to applications. While that probably does not
matter
> for all applications, it will interfere with some applications.
>
> Regards,
>
> Tony
>
> Tony Mason
> Consulting Partner
> OSR Open Systems Resources, Inc.
> http://www.osr.com
>
>
> -----Original Message-----
> From: Daniel Turcanu [mailto:xxxxx@ipdevel.ro]
> Sent: Sunday, March 16, 2003 6:22 AM
> To: File Systems Developers
> Subject: [ntfsd] RE: Encryption filter driver?
>
> The first solution is unpractical and the second is very ugly. I tried
also
> to put a header in front of the encrypted file but I had a lot of problems
> emulating the file objects and FCBs so I gave up. The solution I found,
and
> I am working on it now, is to put the header at the end of file. That way
> the size problem is not anymore so important for the cache and the mapping
> between the memory pages and the HDD pages remains natural. The header is
> written at the end of file in a separate page to optimize a bit the
writing
> process and of course it is rewritten every time the file usefull data
> exceeds the allocated pages. I don’t know yet what problems will arrise
when
> the cache is trying to read/write the header page, but I am confident with
> this solution. Anyway, I think it is the only solution.
> It is just an ideea, hope it is usefull.
>
> Daniel
>
> ----- Original Message -----
> From: “Tony Mason”
> To: “File Systems Developers”
> Sent: Sunday, March 16, 2003 1:11 PM
> Subject: [ntfsd] RE: Encryption filter driver?
>
>
> > The approaches I know:
> >
> > - Don’t change the size of the data within the file; if you have extra
> > information, store it in a different location (e.g., stream or separate
> > database or file.)
> > - Create a secondary file object (IoCreateStreamFileObjectLite will do
the
> > trick) and pass that secondary file object to NTFS. The primary file
> object
> > (the one that arrived at your driver) then points to a device object
> > controlled by your filter. The downside to this approach is that you
> are
> > now responsible for managing the VM state. This increases complexity by
> at
> > least an order of magnitude (essentially, you are implementing your own
> file
> > system layered on top of FAT or NTFS.)
> >
> > There may be other approaches that people have found to work for them
and
> if
> > that is the case I’d love to hear their suggestions so we can augment
the
> > FAQ.
> >
> > Regards,
> >
> > Tony
> >
> > Tony Mason
> > Consulting Partner
> > OSR Open Systems Resources, Inc.
> > http://www.osr.com
> >
> >
> > -----Original Message-----
> > From: Matthew White [mailto:xxxxx@fsnnet.net]
> > Sent: Friday, March 14, 2003 8:50 PM
> > To: File Systems Developers
> > Subject: [ntfsd] RE: Encryption filter driver?
> >
> > Thanks for the response, Tony. The help is much appreciated. In
> > relation to item (1) below (the complex beasts), I was wondering if you,
> > or anyone else who has done such work, could explain one issue I have
> > been wondering about. In an earlier post you said this:
> >
> > “The difficult case for any filter that attempts to manipulate data
> > sizes in
> > the file systems is memory mapped files - the size information comes
> > from
> > the VM system, not the file system. While the VM system gets the
> > initial
> > size from the file system (via an IRP_MJ_QUERY_INFORMATION) the file
> > system
> > reports any subsequent file size changes to the VM system using a
> > programmatic interface (CcSetFileSize as I recall.)”
> >
> > And this:
> >
> > “Of course, many applications only use the read/write interface.
> > Notepad,
> > however, uses the memory mapped API and is thus a great test application
> > for
> > a filter driver of this type.”
> >
> > Are there any general strategies that would be useful to work
> > around/with the issue of the underlying file system calling
> > CcSetFileSizes() to change the file size(s) in the cache manager for a
> > filter driver that would like to mess with file sizes? There must be
> > some way to do this (since these types of filters are possible), but I
> > don’t see right away how this issue can be resolved at all, since this
> > call by the file system into the cache manager is independent of a
> > filter driver. I have seen a couple posts in the archives describing
> > such things as duplicated file objects being used. Just want to know if
> > there is a “right way” to do this type of thing, or if it’s kind of a
> > “you’re on your own” thing. If anyone has any comments on this, many
> > thanks.
> >
> > Matt
> >
> > -----Original Message-----
> > From: xxxxx@lists.osr.com
> > [mailto:xxxxx@lists.osr.com] On Behalf Of Tony Mason
> > Sent: Friday, March 07, 2003 3:33 PM
> > To: File Systems Developers
> > Subject: [ntfsd] RE: Encryption filter driver?
> >
> >
> > (1) Yes there are such examples. Complex beasts.
> >
> > (2) No. Using work routines from completion routines (paging I/O or
> > not) is perfectly acceptable, provided you understand the
> > synchronization issues involved.
> >
> > (3) Paging I/O operations are called at APC_LEVEL, so you need to
> > understand the ramifications of this. But NTFS and FAT roll their own
> > IRPs all the time in the context of paging I/O operations…
> >
> > (4) Block for what? Block for locks? No, that won’t work. Block for
> > external events, like disk I/O to complete? Sure, that works fine.
> > Keep in mind the locking hierarchy here - file system locks, then cache
> > manager, then memory manager.
> >
> > Regards,
> >
> > Tony
> >
> > Tony Mason
> > Consulting Partner
> > OSR Open Systems Resources, Inc.
> > http://www.osr.com
> >
> >
> > -----Original Message-----
> > From: Matthew White [mailto:xxxxx@fsnnet.net]
> > Sent: Friday, March 07, 2003 3:15 PM
> > To: File Systems Developers
> > Subject: [ntfsd] Encryption filter driver?
> >
> > Hi all,
> > I’m writing a file system encryption filter driver (an amazingly
> > new concept, I know) and have a few questions that I was hoping some of
> > the battle-tested filter driver veterans could enlighten me on, if you
> > are not sick of doing it yet. Also, I know this is a large development
> > effort, will take a very long time, and will be fraught with nasty
> > issues and will almost be impossible, so you can try to discourage me,
> > but I’m not giving up :wink: Here are my questions:
> >
> > 1. Do you consider it theoretically possible to build, or has anyone
> > successfully completed, a filter driver that adds a header/trailer to
> > files for metadata and adjusts file offsets/sizes without the
> > header/trailer being page aligned? I’ve seen traffic in the archives
> > that discusses this sort of thing, but I was curious to know if I should
> > even attempt such an architecture. It seems there are numerous issues
> > with this when interactions with the VMM and CM are taken into account.
> >
> > 2. Is it generally a “bad idea” to post to a worker thread in the
> > completion routine of a paging i/o read or write? I’m assuming it is.
> >
> > 3. Can I roll my own IRPs and used these to perform reads in a paging
> > i/o read or write dispatch routine without problems?
> >
> > 4. Can I block in the dispatch routines for paging i/o reads/writes?
> >
> > Thanks for your help.
> >
> > Matt
> >
> >
> >
> >
> > —
> > You are currently subscribed to ntfsd as: xxxxx@osr.com
> > To unsubscribe send a blank email to xxxxx@lists.osr.com
> >
> > —
> > You are currently subscribed to ntfsd as: xxxxx@fsnnet.net
> > To unsubscribe send a blank email to xxxxx@lists.osr.com
> >
> >
> >
> > —
> > You are currently subscribed to ntfsd as: xxxxx@osr.com
> > To unsubscribe send a blank email to xxxxx@lists.osr.com
> >
> > —
> > You are currently subscribed to ntfsd as: xxxxx@ipdevel.ro
> > To unsubscribe send a blank email to xxxxx@lists.osr.com
> >
> >
> >
> >
>
>
>
>
>
> —
> You are currently subscribed to ntfsd as: xxxxx@osr.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
> —
> You are currently subscribed to ntfsd as: xxxxx@yoshimuni.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com