adding modification information

NTDEV
1

Hello all,

If we modify certain documents like MSWORD odc or EXCEL doc, then the
software stores this information in the file itself and later on some one
can check who modified it last.

I want to incorporate the same functionality for all user files, no matter
whether it is text or zip or xyz.

Here is what I am thinking od doing…

  1. add a special header/footer to to these files using an FS filter driver.
  2. When the file is requested for read, remove this dat structure of mine
    and supply the original file to the caller, when it is saved or written back
    to the disk I append his login id to the list and append the list to the
    file and write it.
  3. I need to track all users who modified/opened teh file, so the list size
    will grow.

Is this strategy feasable using a FS filter driver?

For those who wonder how I will read this extra information, well I might
think of implementing a custom viewer or extend the file properties tab
sheets.

Please comment and suggest the pros and cons of this approach.

IS there a better way to achieve the same?

bedanto

2

If your dealing with NTFS, why not just tuck this extra info into the ADS?

“add a special header/footer to to these files using an FS filter
driver.” - The FsF driver could do this, but monkeying with the file
itself would just added extra work and complications; creating a master
file (xml style) that a fsf could manipulate and track changes seems
like it would be easier…

Bedanto wrote:

Hello all,

If we modify certain documents like MSWORD odc or EXCEL doc, then the
software stores this information in the file itself and later on some
one can check who modified it last.

I want to incorporate the same functionality for all user files, no
matter whether it is text or zip or xyz.

Here is what I am thinking od doing…

  1. add a special header/footer to to these files using an FS filter
    driver.
  2. When the file is requested for read, remove this dat structure of
    mine and supply the original file to the caller, when it is saved or
    written back to the disk I append his login id to the list and append
    the list to the file and write it.
  3. I need to track all users who modified/opened teh file, so the list
    size will grow.

Is this strategy feasable using a FS filter driver?

For those who wonder how I will read this extra information, well I
might think of implementing a custom viewer or extend the file
properties tab sheets.

Please comment and suggest the pros and cons of this approach.

IS there a better way to achieve the same?

bedanto
— Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256 You are currently
subscribed to ntdev as: unknown lmsubst tag argument: ‘’ To
unsubscribe send a blank email to xxxxx@lists.osr.com

3

>If your dealing with NTFS, why not just tuck this extra info into the ADS?

No, it should be file system independent.

The FsF driver could do this, but monkeying with the file
itself would just added extra work and complications; creating a master
file (xml style) that a fsf could manipulate and track changes seems
like it would be easier…

Matt, in my case I don’t think that this is going to work. What if the user
emails the file to some one else i nthe organisation, and I need to append
his name also and all his changes also, how do I sync the xml file? How do I
convince the mail client to email that file also?

Bedanto

4

Well, you busted me on the email thing, but using your approach, what
happens the first time someone in your org tries to email one of these
modified files to someout outside of your company? What happens when you
try to upload a modified zip or email a word doc to a client; it ain’t
going to open…

A file server/CVS type approach would fix that with a change log file…
sticky situation…

Bedanto wrote:

>If your dealing with NTFS, why not just tuck this extra info into the
ADS?

No, it should be file system independent.

>The FsF driver could do this, but monkeying with the file
>itself would just added extra work and complications; creating a master
>file (xml style) that a fsf could manipulate and track changes seems
>like it would be easier…

Matt, in my case I don’t think that this is going to work. What if the
user emails the file to some one else i nthe organisation, and I need
to append his name also and all his changes also, how do I sync the
xml file? How do I convince the mail client to email that file also?

Bedanto

— Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256 You are currently
subscribed to ntdev as: unknown lmsubst tag argument: ‘’ To
unsubscribe send a blank email to xxxxx@lists.osr.com

5

Hi Matt, Bedanto,

We are also trying to do something similar here. But We use two different
files, one that contains the information we want, and ofcourse the file that
needs to be tracked. Going by this approach also I don’t find any stable way
to solve the problem :frowning:

Bedanto, I would be interested to know what approach solves the issues
discused by Matt.

6

>Well, you busted me on the email thing, but using your approach, what

happens the first time someone in your org tries to email one of these
modified files to someout outside of your company?

Well, in the original file, we append that the file was emailed to
xxxxx@yyy.zzz and the story has to end. I know there are huge loop holes in
this, and we are yetto decide on a safe appraoch. That is wy I am discussing
these issues with experts here. May be they can give me vision, if not
solutions, they can point out more problems in this approach, which would
help me evaluate this driver approach better.

What happens when you
try to upload a modified zip or email a word doc to a client; it ain’t
going to open…

What we are planning is, when the file is opened, we are stripping it off
it’s context info and storing it to a seperate data structure, so when
zipped the context is appened to the original file saying it is was zipped.
I don’t think we can doanything more than this. And the XML meta file
approach also doesnt help much, does it?

Please suggest.

Bedanto.

PS:- Amitr0, can you explain what all you are planning to do in your project

A file server/CVS type approach would fix that with a change log file…
sticky situation…

7

As matt says, a very sticky situation.

Well Bedanto, we are trying to track documents for certain operations. If
the user does those operations (the theory behind trapping those operations
is huge and messy) a log is generated and also stored ina file.

The zip problem is bugging us in this case, as when the file is zipped, we
need to zip the log file also, thus we need to patch winzip, which
unfortunately is not he only means to zip up !!! :frowning:

Putting the log in the file itself seems to solve this problem very quickly,
as it makes the doc independent of the utility used to zip. But as Matt
suggests what if he emails the zip to an external party, he can never open
it :frowning:

We either have to track the zip file also, else do some unholy thing like
block attachments :slight_smile:

Matt any suggestions?

8

Since most email clients and web browsers use Common Dialog to select
file attachments, you could use two files as Amitr0 suggested. Using
standard api,
you could associate two different files with the same name but different
extensions (meaning if one was selected, the companion would be also,
thus both would be sent). If the recipient didn’t have the appropriate
software, the second file with the extended info would be harmless. If
the user did have the right software to read it, it would detect it…???

amitr0 wrote:

Hi Matt, Bedanto,

We are also trying to do something similar here. But We use two
different files, one that contains the information we want, and
ofcourse the file that needs to be tracked. Going by this approach
also I don’t find any stable way to solve the problem :frowning:

Bedanto, I would be interested to know what approach solves the issues
discused by Matt.
— Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256 You are currently
subscribed to ntdev as: unknown lmsubst tag argument: ‘’ To
unsubscribe send a blank email to xxxxx@lists.osr.com

9

Bedanto,

what you and I are trying to do, has been termed impossible by many. As I
delve deeper into the project, I have started agreeing with the gurus out
here.

Since most email clients and web browsers use Common Dialog to select
file attachments, you could use two files as Amitr0 suggested. Using
standard api,
you could associate two different files with the same name but different
extensions (meaning if one was selected, the companion would be also,
thus both would be sent).

I do not understand this part, can you elaborate, which API you are
addressing, also IIRC common dialogs come with extensions predefined and
selected when we call them, so how can the second file be attached
automatically?

10

The common dialog window contains a listview window. Once the common
dialog window is created, you can grab a handle too it, and enum it’s
child windows to get a handle to the listview. When an item is selected
in that listview window (for example somefile.doc), you can then search
and select a second file (somefile.ext) programatically.

Email and webbrowsers open the common dialog with all files accessible.
Even if you open a common dialog with only *.exe selected, it isn’t hard
to make it search for a secondary file extension and highlight. You can
also make it so a highlighted extension can not he un-highlighted…
Therefor, when you approve the file selection, both somefile.doc and
somefile.ext are both attached…

Search the web, there are samples out there and it doesn’t require evil
hooking or undocumented functions…

amitr0 wrote:

Bedanto,

what you and I are trying to do, has been termed impossible by many.
As I delve deeper into the project, I have started agreeing with the
gurus out here.

>Since most email clients and web browsers use Common Dialog to select
>file attachments, you could use two files as Amitr0 suggested. Using
>standard api,
>you could associate two different files with the same name but different
>extensions (meaning if one was selected, the companion would be also,
>thus both would be sent).

I do not understand this part, can you elaborate, which API you are
addressing, also IIRC common dialogs come with extensions predefined
and selected when we call them, so how can the second file be attached
automatically?

— Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256 You are currently
subscribed to ntdev as: unknown lmsubst tag argument: ‘’ To
unsubscribe send a blank email to xxxxx@lists.osr.com

11

Bedanto wrote:

>If your dealing with NTFS, why not just tuck this extra info into the
ADS?

No, it should be file system independent.

>The FsF driver could do this, but monkeying with the file
>itself would just added extra work and complications; creating a master
>file (xml style) that a fsf could manipulate and track changes seems
>like it would be easier…

Matt, in my case I don’t think that this is going to work. What if the
user emails the file to some one else i nthe organisation, and I need
to append his name also and all his changes also, how do I sync the
xml file? How do I convince the mail client to email that file also?

That problem is not solveable. Your filter will not know what
application is reading the file, and for what purpose. Thus, you would
have to strip out the tracking information when ANY application reads
the file, including e-mail programs. Thus, the file that the e-mail
client ships will not have the tracking information anyway.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

12

>Your filter will not know what

application is reading the file, and for what purpose.

Tim, why wont the filter know which app requested for it, after all, caller
context is retained in the case of FS filters, isn’t it?

13

amitr0 wrote:

>Your filter will not know what
>application is reading the file, and for what purpose.

Tim, why wont the filter know which app requested for it, after all,
caller context is retained in the case of FS filters, isn’t it?

Is it? And even if it is, what will that tell him? I just don’t think
there is enough context.

Let’s say he is able to determine that Outlook is reading the file. How
will he know whether it is destined for an address outside the company,
in which case he will want to eliminate the extra cruft? If the reader
is zip.exe, or 7zip.exe, how will he know whether to keep the extra
info? The problem is unsolveable.

And let’s say someone does e-mail an encrufted document to someone else
in the company, who saves it to disk. He is going to have to recognize
that the document contains the encruftion, so he can use that, instead
of creating it anew. Can he ensure that it won’t be faked?

I just think he is on the wrong track with this. I think it would be
entirely practical to create and maintain some kind of permanent journal
of all accesses and modifications to all files, and his “carry along”
technique would not generate any information that a sophisticated log
analysis utility couldn’t also generate.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

14

>Is it? And even if it is, what will that tell him? I just don’t think

there is enough context.

Well atleast the file name of the caller could be extracted.

Let’s say he is able to determine that Outlook is reading the file. How
will he know whether it is destined for an address outside the company,
in which case he will want to eliminate the extra cruft?

Correct, I never said just doing this is going to solve it. OUTLOOK and
others are cases that need to be handled seperately. Infact, in the
prototype we developed, we had to use MSAA to track OUTLOOK and other office
apps, and a lot of API hooking (oops sorry, but I am trying to get rid of
all ofthem and find better approaches). Also the two file approach doesnt
help much, we are currently using it, and know where it fails.

If the reader
is zip.exe, or 7zip.exe, how will he know whether to keep the extra
info? The problem is unsolveable.

We will not, we will simply take it as a transformation operation and remove
the header from the doc and append it to the zip. Now lets take tht the zip
was emailed, that is where our taps into OUTLOOK can tell us whether the
mail is going within the org or outside it. I know, I know, it is not a good
approach, and I am trying to solve the “unsolveable problem”, and honestly
tim, I agree that it cannot be solved, unless the number of apps supported
by the program are the only ones that can ever be installed and run on the
machine.

And let’s say someone does e-mail an encrufted document to someone else
in the company, who saves it to disk. He is going to have to recognize
that the document contains the encruftion, so he can use that, instead
of creating it anew.

If the doc reaches the user properly, I dont think that would be herculian.
Infact maintaining the context and sending it across is the difficult part.

Can he ensure that it won’t be faked?

No I cannot. Atleast not today!!!

I just think he is on the wrong track with this.

Me too, just that I am sharing with you all the observations I have already
made on this issue.

I think it would be entirely practical to create and maintain some kind of
permanent journal
of all accesses and modifications to all files, and his “carry along”
technique would not generate any information that a sophisticated log
analysis utility couldn’t also generate.

And if he does that, that is the logs are maintained in a XML type of a file
seperate from the actual doc, then if the file is emailed, how does he send
the XML file also, or how does the zip operation get handled?

amitr0

15

Why Windows audit log and ACLs not enough for such?

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

----- Original Message -----
From: “Bedanto”
To: “Windows System Software Devs Interest List”
Sent: Friday, December 16, 2005 9:27 AM
Subject: [ntdev] adding modification information

Hello all,

If we modify certain documents like MSWORD odc or EXCEL doc, then the
software stores this information in the file itself and later on some one
can check who modified it last.

I want to incorporate the same functionality for all user files, no matter
whether it is text or zip or xyz.

Here is what I am thinking od doing…

1. add a special header/footer to to these files using an FS filter driver.
2. When the file is requested for read, remove this dat structure of mine
and supply the original file to the caller, when it is saved or written back
to the disk I append his login id to the list and append the list to the
file and write it.
3. I need to track all users who modified/opened teh file, so the list size
will grow.

Is this strategy feasable using a FS filter driver?

For those who wonder how I will read this extra information, well I might
think of implementing a custom viewer or extend the file properties tab
sheets.

Please comment and suggest the pros and cons of this approach.

IS there a better way to achieve the same?

bedanto

—
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

16

amitr0 wrote:

> I think it would be entirely practical to create and maintain some
kind of permanent journal
>of all accesses and modifications to all files, and his “carry along”
>technique would not generate any information that a sophisticated log
>analysis utility couldn’t also generate.

And if he does that, that is the logs are maintained in a XML type of
a file seperate from the actual doc, then if the file is emailed, how
does he send the XML file also, or how does the zip operation get handled?

You don’t. You recognize the fact that a single file has migrated to
some other machine when you do your log analysis. If you need to track
this information for a single file within your enterprise, you just
collect the XML logs from all the machines in your enterprise every day
and collate them. File name and file size are probably sufficient for
identifying a single migratory file.

Seriously, I do not think there is a single piece of information that
the live tracking could supply that a good log analyzer could not also
supply.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

17

Tim, amitr0 and others,

Thank you for such a lovely discussion on the pros and cons of the different
approaches.

Currently, my architecture is…

All ‘tracked’ files are stored in a central server, the user has to download
these files to the local machine using a special protocol. Once downloaded,
certain information (like who downloaded, time etc) are createrd and stored
in a seperate file in the same directory as the ‘tracked file’. This secret
file is updated as and when the file is tampered with.

However this approach fails miserably when we try to zip the file up. As the
secret file doesn’t get zipped and all context is lost after that.

So I was wondering whether a single file approach, where all the infor is
also appened to the same file to make things simpler.

From what amitr0 said, I got some hope that this is gonna work, but Tim
seems to disagree completely.

Ofcourse, apart from storing the secret file information we also have a log
generated in the central server that the admin can view to find out what
happened to the secret files after they were downloaded.

Both the single file (orig file + secret info) and two files (where the logs
are saved in a diff file) fails at some point. Is there a better way to
tackle this problem?

Regards,

Bedanto

18

“This secret file is updated as and when the file is tampered with.” -
What method are you using to track the changes locally?

“However this approach fails miserably when we try to zip the file up.
As the secret file doesn’t get zipped and all context is lost after
that.” - if you are using a fsfd to monitor for changes, adding the
ability to a fsfd to detect the creation of a .zip file would be rather
simple. Upon an event being signaled (a new .zip created), a user app or
service could insert the secret file into the newly created zip. Files
can easily be added to an existing archive.

Bedanto wrote:

Tim, amitr0 and others,

Thank you for such a lovely discussion on the pros and cons of the
different approaches.

Currently, my architecture is…

All ‘tracked’ files are stored in a central server, the user has to
download these files to the local machine using a special protocol.
Once downloaded, certain information (like who downloaded, time etc)
are createrd and stored in a seperate file in the same directory as
the ‘tracked file’. This secret file is updated as and when the file
is tampered with.

However this approach fails miserably when we try to zip the file up.
As the secret file doesn’t get zipped and all context is lost after that.

So I was wondering whether a single file approach, where all the infor
is also appened to the same file to make things simpler.

From what amitr0 said, I got some hope that this is gonna work, but
Tim seems to disagree completely.

Ofcourse, apart from storing the secret file information we also have
a log generated in the central server that the admin can view to find
out what happened to the secret files after they were downloaded.

Both the single file (orig file + secret info) and two files (where
the logs are saved in a diff file) fails at some point. Is there a
better way to tackle this problem?

Regards,

Bedanto
— Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256 You are currently
subscribed to ntdev as: unknown lmsubst tag argument: ‘’ To
unsubscribe send a blank email to xxxxx@lists.osr.com

19

>“This secret file is updated as and when the file is tampered with.” -

What method are you using to track the changes locally?

That depends on the type of file. For word, we use MSAA to track the
commands like copy, paste, save as etc. For Notepad/MSPAINT we hook into the
app and track using different techniques. There is no filter driver involved
at this stage. I was trying to introduce one to bypass some of the problems
in the current approach.

if you are using a fsfd to monitor for changes, adding the
ability to a fsfd to detect the creation of a .zip file would be rather
simple. Upon an event being signaled (a new .zip created), a user app or
service could insert the secret file into the newly created zip.
How do we know that the zip contains my secret file, can a kernel driver
know of hte contents that goes into an archieve?

?Files
?can easily be added to an existing archive.

WEll this I think would depend on the zip utility being used. Winzip exports
it’s apis to be programmed, so if winzip is installed, I an add the secret
file into the archieve, but if he uses his own “super-duper-zip” then what?

So our program becomes dependent on a third party utility to function, and
breaks down miserably if someone uninstalls Winzip!

Also, for some reason, the inherent funda of using two seperate files to
keep track of related information, looks bad to me. Hence the thought of a
single file appraoch.

Bedanto

20

“How do we know that the zip contains my secret file, can a kernel
driver know of hte contents that goes into an archieve?” - check in
usermode, use the zlib library. You can add files and remove them and
search the archive to see if your secret file is present… Using
zlib, even if someone uninstalled winzip, so what, it would still function.

Zip files created with Winzip use the Lempel-Ziv algorithm and can be
uncompressed with most unzip utilities, not solely Winzip. An
application using zlib would be fine.

Bedanto wrote:

>“This secret file is updated as and when the file is tampered with.” -
>What method are you using to track the changes locally?

That depends on the type of file. For word, we use MSAA to track the
commands like copy, paste, save as etc. For Notepad/MSPAINT we hook
into the app and track using different techniques. There is no filter
driver involved at this stage. I was trying to introduce one to bypass
some of the problems in the current approach.

>if you are using a fsfd to monitor for changes, adding the
>ability to a fsfd to detect the creation of a .zip file would be rather
>simple. Upon an event being signaled (a new .zip created), a user app or
>service could insert the secret file into the newly created zip.
How do we know that the zip contains my secret file, can a kernel
driver know of hte contents that goes into an archieve?

?Files
?can easily be added to an existing archive.

WEll this I think would depend on the zip utility being used. Winzip
exports it’s apis to be programmed, so if winzip is installed, I an
add the secret file into the archieve, but if he uses his own
“super-duper-zip” then what?

So our program becomes dependent on a third party utility to function,
and breaks down miserably if someone uninstalls Winzip!

Also, for some reason, the inherent funda of using two seperate files
to keep track of related information, looks bad to me. Hence the
thought of a single file appraoch.

Bedanto

— Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256 You are currently
subscribed to ntdev as: unknown lmsubst tag argument: ‘’ To
unsubscribe send a blank email to xxxxx@lists.osr.com