Encryption filter question

NTFSD
21

Alas! The code I made yesterday works on NTFS, and makes no bug checks - but it doesn’t on FAT:-(
Real fun:-)

“Fuller, Rob” wrote:

While FASTFAT only uses stream files for METADATA, NTFS uses stream file objects for NORMAL file caching under a number of common scenarios. Thus, he does need to know the name of some of those stream file objects. However, he cannot get it by querying the stream file object, so he must get it during the IRP_MJ_CREATE of the user’s file object, and associate it through the FsContext pointer with the NTFS stream file object (both file object’s point to the same FCB.) Note that this is pretty tricky, hence error prone. Good luck, Dejan! This is what makes NT file system filtering “fun.”

–
Kind regards, Dejan M. www.alfasp.com
E-mail: xxxxx@alfasp.com ICQ#: 56570367
Alfa File Monitor - File monitoring library for Win32 developers.
Alfa File Protector - File protection and hiding library for Win32 developers.
Alfa Registry Monitor - Registry monitoring library for Win32 developers.
Alfa Registry Protector - Registry protection library for Win32 developers.

22

> You are right that he may see a file-object he wouldn’t have seen in the

create - however the FsContext points to the same FCB.
Filters should be keying off the FsContext and not the file-object
itself. I still haven’t gotten a convincing answer why this is needed.

I’ve answered that about 3 times in other posts (because I don’t use
headers, but want to use file names sent via user mode process)
Hmm, I just found out why directory and meta data FO’s FsContext was
not in my list - I was ignoring these during create…
Now to see if that’s all there is to it…

–
Kind regards, Dejan M. www.alfasp.com
E-mail: xxxxx@alfasp.com ICQ#: 56570367
Alfa File Monitor - File monitoring library for Win32 developers.
Alfa File Protector - File protection and hiding library for Win32
developers.
Alfa Registry Monitor - Registry monitoring library for Win32 developers.
Alfa Registry Protector - Registry protection library for Win32 developers.

23

I remember crossing that bridge… I don’t recall whether I did it more than once or not :slight_smile:

-----Original Message-----
From: Dejan Maksimovic [mailto:xxxxx@alfasp.com]
Sent: Monday, July 01, 2002 4:49 PM
To: File Systems Developers
Subject: [ntfsd] Re: Encryption filter question

Alas! The code I made yesterday works on NTFS, and makes no bug checks - but it doesn’t on FAT:-(
Real fun:-)

“Fuller, Rob” wrote:

While FASTFAT only uses stream files for METADATA, NTFS uses stream file objects for NORMAL file caching under a number of common scenarios. Thus, he does need to know the name of some of those stream file objects. However, he cannot get it by querying the stream file object, so he must get it during the IRP_MJ_CREATE of the user’s file object, and associate it through the FsContext pointer with the NTFS stream file object (both file object’s point to the same FCB.) Note that this is pretty tricky, hence error prone. Good luck, Dejan! This is what makes NT file system filtering “fun.”

–
Kind regards, Dejan M. www.alfasp.com
E-mail: xxxxx@alfasp.com ICQ#: 56570367
Alfa File Monitor - File monitoring library for Win32 developers.
Alfa File Protector - File protection and hiding library for Win32 developers.
Alfa Registry Monitor - Registry monitoring library for Win32 developers.
Alfa Registry Protector - Registry protection library for Win32 developers.

You are currently subscribed to ntfsd as: xxxxx@inin.com
To unsubscribe send a blank email to %%email.unsub%%

24

That’s definitely not the most efficient way to write an encryption
filter, nor is it reliable.
Unless you don’t intend this to filter to circulate/work outside your
test machine, I am afraid this wouldn’t work.

Having user-mode process tell you which file to encrypt/decrypt is fine

  • however you should do this up-front, i.e. at create. You do not need
    to - and should not do name queries on every i/o.

Ravi

This posting is provided “AS IS” with no warranties, and confers no
rights.

-----Original Message-----
From: Dejan Maksimovic [mailto:xxxxx@alfasp.com]
Sent: Monday, July 01, 2002 2:48 PM
To: File Systems Developers
Subject: [ntfsd] Re: Encryption filter question

But, I’m not doing this the “usual” route, i.e. I am not using ANY
headers. A user mode process is to provide the name of the files that
are to be encrypted, so that is why I must know the name during each
Read/Write.

Ravisankar Pudipeddi wrote:

Dejan -

If you haven’t seen the FsContext before (and you remember it in
post-IRP_MJ_CREATE), it’s one of 2 things:
. Your driver loaded after the file was opened (so you missed
the create). You shouldn’t be encrypting/decrypting this file
. The file system is using a stream file object for meta-data
access - again you wouldn’t need to.

This is the reason I believe you shouldn’t be really needing to know
the name of the filter during paging i/o - and for encryption filters
you shouldn’t be really caring about the name past create.

–
Kind regards, Dejan M. www.alfasp.com
E-mail: xxxxx@alfasp.com ICQ#: 56570367
Alfa File Monitor - File monitoring library for Win32 developers. Alfa
File Protector - File protection and hiding library for Win32
developers. Alfa Registry Monitor - Registry monitoring library for
Win32 developers. Alfa Registry Protector - Registry protection library
for Win32 developers.

You are currently subscribed to ntfsd as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to %%email.unsub%%

25

To make things “worse” people are still using NT:-)

“Fuller, Rob” wrote:

That was pretty subtle.

There is a scenario where your suggested tracking scheme breaks. The user file is opened, NTFS initiates the caching with a stream file object, then the user file is closed. The filter cleans up its tracking structures on the close of the user file. The first paging IO write is received by the filter. Note that the filter already cleaned up its tracking structure before the delayed write took place when the filter would have seen the stream file object for the first time.

Of course this is easily addressed in XP or later with the PerStreamContext routines. However, people are still using 2K.

–
Kind regards, Dejan M. www.alfasp.com
E-mail: xxxxx@alfasp.com ICQ#: 56570367
Alfa File Monitor - File monitoring library for Win32 developers.
Alfa File Protector - File protection and hiding library for Win32 developers.
Alfa Registry Monitor - Registry monitoring library for Win32 developers.
Alfa Registry Protector - Registry protection library for Win32 developers.

26

For the last time, I DON’T query during I/O, UNLESS I HAVE TO. (Sorry
to shout:-)
I only query if the FsContext is unknown, which does happen.
There are already 3 filters out there that do it this way, and
successfully. They were in the market for over 5 years.

Dejan.

Ravisankar Pudipeddi wrote:

That’s definitely not the most efficient way to write an encryption
filter, nor is it reliable.
Unless you don’t intend this to filter to circulate/work outside your
test machine, I am afraid this wouldn’t work.

Having user-mode process tell you which file to encrypt/decrypt is fine

  • however you should do this up-front, i.e. at create. You do not need
    to - and should not do name queries on every i/o.

–
Kind regards, Dejan M. www.alfasp.com
E-mail: xxxxx@alfasp.com ICQ#: 56570367
Alfa File Monitor - File monitoring library for Win32 developers.
Alfa File Protector - File protection and hiding library for Win32
developers.
Alfa Registry Monitor - Registry monitoring library for Win32 developers.
Alfa Registry Protector - Registry protection library for Win32 developers.

27

At least in NT 4 you will see an IRP_MJ_CLEANUP for a file object with FO_STREAM_FILE set when NTFS creates a stream file object. You can use that in NT 4 to associate the NTFS stream file object with its user file object. Enter W2K where there is a new API called IoCreateStreamFileObjectLite that does NOT send the IRP_MJ_CLEANUP. Now that’s “fun!”

-----Original Message-----
From: Dejan Maksimovic [mailto:xxxxx@alfasp.com]
Sent: Monday, July 01, 2002 5:13 PM
To: File Systems Developers
Subject: [ntfsd] Re: Encryption filter question

To make things “worse” people are still using NT:-)

“Fuller, Rob” wrote:

That was pretty subtle.

There is a scenario where your suggested tracking scheme breaks. The user file is opened, NTFS initiates the caching with a stream file object, then the user file is closed. The filter cleans up its tracking structures on the close of the user file. The first paging IO write is received by the filter. Note that the filter already cleaned up its tracking structure before the delayed write took place when the filter would have seen the stream file object for the first time.

Of course this is easily addressed in XP or later with the PerStreamContext routines. However, people are still using 2K.

–
Kind regards, Dejan M. www.alfasp.com
E-mail: xxxxx@alfasp.com ICQ#: 56570367
Alfa File Monitor - File monitoring library for Win32 developers.
Alfa File Protector - File protection and hiding library for Win32 developers.
Alfa Registry Monitor - Registry monitoring library for Win32 developers.
Alfa Registry Protector - Registry protection library for Win32 developers.

You are currently subscribed to ntfsd as: xxxxx@inin.com
To unsubscribe send a blank email to %%email.unsub%%

28

Well I’m seeing contradictory responses. Maybe I understood your mail
below:
“A user mode process is to provide the name of the files that
are to be encrypted, so that is why I must know the name during each
Read/Write”

As I am stated in my first response, if you see an ‘unknown’ FsContext -
you should be ignoring the file anyway, since it would be default
uninteresting due to the 2 specific conditions under which the unknown
FsContext would appear to you. ( it’s i/o on metat-data, or it’s for a
file which was created before your filter loaded).

If you see an exception to the above: i.e. you see an FsContext you
haven’t seen during post-create processing - but the i/o is for a user
data stream for any MS filesystems, please do let me know.

Ravi

This posting is provided “AS IS” with no warranties, and confers no
rights
-----Original Message-----
From: Dejan Maksimovic [mailto:xxxxx@alfasp.com]
Sent: Monday, July 01, 2002 3:15 PM
To: File Systems Developers
Subject: [ntfsd] Re: Encryption filter question

For the last time, I DON’T query during I/O, UNLESS I HAVE TO.
(Sorry to shout:-)
I only query if the FsContext is unknown, which does happen.
There are already 3 filters out there that do it this way, and
successfully. They were in the market for over 5 years.

Dejan.

Ravisankar Pudipeddi wrote:

That’s definitely not the most efficient way to write an encryption
filter, nor is it reliable. Unless you don’t intend this to filter to
circulate/work outside your test machine, I am afraid this wouldn’t
work.

Having user-mode process tell you which file to encrypt/decrypt is
fine

  • however you should do this up-front, i.e. at create. You do not need
    to - and should not do name queries on every i/o.

–
Kind regards, Dejan M. www.alfasp.com
E-mail: xxxxx@alfasp.com ICQ#: 56570367
Alfa File Monitor - File monitoring library for Win32 developers. Alfa
File Protector - File protection and hiding library for Win32
developers. Alfa Registry Monitor - Registry monitoring library for
Win32 developers. Alfa Registry Protector - Registry protection library
for Win32 developers.

You are currently subscribed to ntfsd as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to %%email.unsub%%

29

That’s what I’m saying!

Ravisankar Pudipeddi wrote:

Well I’m seeing contradictory responses. Maybe I understood your mail
below:
“A user mode process is to provide the name of the files that
are to be encrypted, so that is why I must know the name during each
Read/Write”

As I am stated in my first response, if you see an ‘unknown’ FsContext -
you should be ignoring the file anyway, since it would be default
uninteresting due to the 2 specific conditions under which the unknown
FsContext would appear to you. ( it’s i/o on metat-data, or it’s for a
file which was created before your filter loaded).

If you see an exception to the above: i.e. you see an FsContext you
haven’t seen during post-create processing - but the i/o is for a user
data stream for any MS filesystems, please do let me know.

–
Kind regards, Dejan M. www.alfasp.com
E-mail: xxxxx@alfasp.com ICQ#: 56570367
Alfa File Monitor - File monitoring library for Win32 developers.
Alfa File Protector - File protection and hiding library for Win32
developers.
Alfa Registry Monitor - Registry monitoring library for Win32 developers.
Alfa Registry Protector - Registry protection library for Win32 developers.

30

i don’t know how exactly u will be encrypting/decrypting the file or how u
will be determining which files to encrypt/decrypt, but maybe what u can do
is to have the application open the file and send the handle down to the
driver through an IOCTL. When the driver receives the IOCTL, it will be able
to get the file object (corresponding to the handle) in the irp, and derive
the fscontext from there. however, this sugestion is more applicable to when
the application is the one that initiates the encryption/decryption.

Ho Mun Chuen
@@ “Not everything that counts can be counted;
<” )~ and not everything that can be counted counts"
//\ … Albert Einstein
----- Original Message -----
From: “Dejan Maksimovic”
To: “File Systems Developers”
Sent: Tuesday, July 02, 2002 5:48 AM
Subject: [ntfsd] Re: Encryption filter question

But, I’m not doing this the “usual” route, i.e. I am not using ANY
headers. A user mode process is to provide the name of the files that are to
be encrypted, so that is why I must know the name during each Read/Write.

Ravisankar Pudipeddi wrote:

> Dejan -
>
> If you haven’t seen the FsContext before (and you remember it in
> post-IRP_MJ_CREATE), it’s one of 2 things:
> . Your driver loaded after the file was opened (so you missed
> the create). You shouldn’t be encrypting/decrypting this file
> . The file system is using a stream file object for meta-data
> access - again you wouldn’t need to.
>
> This is the reason I believe you shouldn’t be really needing to know the
> name of the filter during paging i/o - and for encryption filters you
> shouldn’t be really caring about the name past create.

–
Kind regards, Dejan M. www.alfasp.com
E-mail: xxxxx@alfasp.com ICQ#: 56570367
Alfa File Monitor - File monitoring library for Win32 developers.
Alfa File Protector - File protection and hiding library for Win32
developers.
Alfa Registry Monitor - Registry monitoring library for Win32 developers.
Alfa Registry Protector - Registry protection library for Win32 developers.

—
You are currently subscribed to ntfsd as: xxxxx@pmail.ntu.edu.sg
To unsubscribe send a blank email to %%email.unsub%%

31

> Dejan,

Why do you want to use a map to match encryption data with file object
names ? I think it would be more practical, altough maybe more difficult to
code, to add extra bytes to a file like a “metadata” regarding encryption
attributtes. I think that keeping a database which maps file names to
encryption attributtes and so is unpractical , if for no more than copy
deployment, read only media deployment, and so on

Regards, Dan

Hi,

I also work on similar problem (maybe more complicated than your).
It is true that using some kind of map or hash-table for keep a trace on
file names is not a good idea - it consumes both memory and processor
time, which must be avoided in filter drivers.
What I did with this problem. FILE_OBJECT.Flags has unused bits in high
byte.
It’s risky of course, but I do not met any driver which use them.
So you can simply “mark” your file object during IRP processing with new
flags values to indicat if this file is for decryption, as well as file is
alrady decrypted.
On my case it works.

Good luck!

32

I’m pretty sure dozens of subscribers to this list who have released commercial filters can attest to what a stupid idea this is. Any mechanism in a filter that doesn’t interoperate with a similar mechanism in another filter will be a support nightmare.

-----Original Message-----
From: Toni Garabedyan [mailto:xxxxx@hotmail.com]
Sent: Friday, July 05, 2002 8:02 AM
To: File Systems Developers
Subject: [ntfsd] Re: Encryption filter question

Dejan,

Why do you want to use a map to match encryption data with file object
names ? I think it would be more practical, altough maybe more difficult to
code, to add extra bytes to a file like a “metadata” regarding encryption
attributtes. I think that keeping a database which maps file names to
encryption attributtes and so is unpractical , if for no more than copy
deployment, read only media deployment, and so on

Regards, Dan

Hi,

I also work on similar problem (maybe more complicated than your).
It is true that using some kind of map or hash-table for keep a trace on
file names is not a good idea - it consumes both memory and processor
time, which must be avoided in filter drivers.
What I did with this problem. FILE_OBJECT.Flags has unused bits in high
byte.
It’s risky of course, but I do not met any driver which use them.
So you can simply “mark” your file object during IRP processing with new
flags values to indicat if this file is for decryption, as well as file is
alrady decrypted.
On my case it works.

Good luck!

You are currently subscribed to ntfsd as: xxxxx@inin.com
To unsubscribe send a blank email to %%email.unsub%%

33

It’s true for a big risk if there are more than one filter using this
method (especialy from diferent vendors). But in particular case, when the
filter is implemented with the clear idea to work ONLY for specific
application service in well defined system environment, I think its a very
light decision (not stupid!). Could you enumerate some real-world exams of
systems with more than one FS filter drivers (not Microsoft developped)??
I dunno.

34

> light decision (not stupid!). Could you enumerate some real-world
exams of

systems with more than one FS filter drivers (not Microsoft
developped)??

Usually this discovers issues. Testing filters together with NAI or
such often causes interoperability issues.

Max

35

Most customers insist on running anti-virus products that use file system filters. Thus, if you’re producing a commercial filter that is NOT an anti-virus products, your customer will likely have multiple filters loaded.

-----Original Message-----
From: Toni Garabedyan [mailto:xxxxx@hotmail.com]
Sent: Thursday, July 18, 2002 4:11 AM
To: File Systems Developers
Subject: [ntfsd] Re: Encryption filter question

It’s true for a big risk if there are more than one filter using this
method (especialy from diferent vendors). But in particular case, when the
filter is implemented with the clear idea to work ONLY for specific
application service in well defined system environment, I think its a very
light decision (not stupid!). Could you enumerate some real-world exams of
systems with more than one FS filter drivers (not Microsoft developped)??
I dunno.

You are currently subscribed to ntfsd as: xxxxx@inin.com
To unsubscribe send a blank email to %%email.unsub%%