Well, based on your description, it seems to me that the problem comes from
the fact that some open modes (and operations) apply to the file as a whole
instead of just one stream (delete, FILE_SUPERSEDE, FILE_OVERWRITE
etc…) so the streams are not symmetrical (the default one can impact the
managed one but not the other way around).
Have you considered making “unmanaged” applications also use a stream? For
example, for foo.txt, you’d have foo.txt:managed and foo.txt:unmanaged and
just reparse any unmanaged open to foo.txt:unmanaged. This way the two
streams are symmetrical and deleting and overwriting one won’t affect the
other ? Of course, this would mean that your filter must always be running
and the distinction between managed and unmanaged disappears (or at least
becomes very blurry).
Thanks,
Alex.
On Wed, Jun 4, 2014 at 2:13 AM, wrote:
> Hi Marion,
>
> Yes, other then the file location, there’s no correlation between the
> contents of the primary/non-managed and managed streams.
>
>
> Okay, so based on the advise, I’m going to go with removing the
> FILE_OVERWRITE(_IF) flags (on the primary stream) and implementing this
> functionality in my post-create.
> I’ll do some testing on NTFS to work out exactly what it does for
> overwrites, but it’ll most likely be similar to the FastFat copy, in that
> I’ll need to purge the file (and possibly the section cache), update the
> file attributes and report a notification change.
>
> Thanks to Tony, Alex and Marion
> Ged.
>
>
>
> From: Marion Bond
> Sent: Wednesday, 4 June 2014 01:07
>
> To: Windows File Systems Devs Interest List
>
> Are you saying that there is no correlation between the contents of the
> managed view and the unmanaged view? If that is the case, then you should
> certainly intercept the overwrite and perform your own behaviour. This
> would not be a hack, but merely the correct way to maintain the
> independence between the data in multiple streams
>
> If, on the other hand, the data in these streams has some correlation, and
> I find it hard to believe that there is any useful scenario where they do
> not, then you need to define the protocol by which they interact. I have
> the suspicion that you are trying to optimize the case where an application
> writes almost the same data, and the question is whether you can detect
> that case any more quickly than you can recomputed the result.
>
> Sent from Surface Pro
>
> From: Ged Murphy
> Sent: Tuesday, June 03, 2014 7:19 PM
> To: Windows File Systems Devs Interest List
>
> Sorry guys, it seems I didn’t make it very clear as to the goal of the
> design.
>
> Essentially I have two ways of running all applications on the box,. An
> app can either run in a managed mode (whereby it’s managed by our software)
> or an unmanaged mode.
> When running in unmanaged mode, apps interact with files in a normal way.
> When running in managed mode, we reparse file creates on primary streams to
> managed streams and all data is written and read to that managed stream.
> Additional attributes can be added to managed streams, such as encryption
> or compression. As far as managed apps are concerned, they still think
> they’re accessing the primary stream.
>
> Therefore, notepad++ running in normal mode will see one view of a file,
> but notepad++ running in managed mode will see an entirely different view
> of the file.
> The problem arises when a non-managed app opens a primary stream with
> FILE_OVERWRITE. This doesn’t get reparsed to the managed stream, so the
> overwrite is done on the main stream which removes all the ADS’ and
> essentially deletes the view of the file which managed apps see.
>
> This was one of those ‘oh sh*t’ moments when you realise your design has a
> hole and any possible fix dirties what was a previously clean solution….
>
> Ged.
>
>
>
> On 04/06/2014 00:00, “Alex Carp” wrote:
>
> Yeah, like I said before, I’ve mostly used ADS as a mechanism to store
> metadata about file contents, so when the contents went away the metadata
> was no longer accurate (and necessary). However, perhaps Ged is
> implementing something like file versions, where all the old contents of
> the file are preserved in ADS or some such.
>
> However, I wonder if the file name comes into play at all. For example,
> for a file version system, one should probably take into account the file
> name for a specific version as well, so maybe ADS isn’t a good choice for
> things that need to be tracked across renames… Anyway, I’m really just
> guessing here, Ged didn’t say anything more specific…
>
> Thanks,
> Alex.
>
>
> On Tue, Jun 3, 2014 at 3:20 PM, Marion Bond
> wrote:
>
>> Interesting. You made it sound like the file you are working with are
>> some kind of specially organized databases or something like that rather
>> than any arbitrary file. I am curious what kind of meta data you might want
>> to persist across a complete replacement of the contents that wasn’t
>> triggered in some controlled way you control. In your example, nothing
>> would stop me from pasting in an entirely different document in Wordpad and
>> then saving. Replication information might be useful after something like
>> that, but it shouldn’t occupy GB. Backup and restore data might occupy GB,
>> but using alternate data streams seems like a poor way to implement it -
>> especially as VSS and even recycle bin functionality already exists in the
>> base OS
>>
>> Sent from Surface Pro
>>
>> From: Ged Murphy
>> Sent: Tuesday, June 03, 2014 6:07 PM
>>
>> To: Windows File Systems Devs Interest List
>>
>> The IO can come from any application on the box, so it could come at any
>> time.
>> Wordpad is an example of an app which uses this flag when opening files
>> for writing
>>
>>
>> On 03/06/2014 22:55, “Marion Bond” wrote:
>>
>> How often do you expect this operation in real life? If the file IO is
>> from your application, I would expect it not to do this without a good
>> reason. And if the file IO is from another application (ie a restore by a
>> backup application), then maybe regenerating multiple GB of data isn’t such
>> a bad thing
>>
>> Sent from Surface Pro
>>
>>
>> —
>> NTFSD is sponsored by OSR
>>
>> OSR is hiring!! Info at http://www.osr.com/careers
>>
>> For our schedule of debugging and file system seminars visit:
>> http://www.osr.com/seminars
>>
>> To unsubscribe, visit the List Server section of OSR Online at
>> http://www.osronline.com/page.cfm?name=ListServer
>>
>> —
>> NTFSD is sponsored by OSR
>>
>> OSR is hiring!! Info at http://www.osr.com/careers
>>
>> For our schedule of debugging and file system seminars visit:
>> http://www.osr.com/seminars
>>
>> To unsubscribe, visit the List Server section of OSR Online at
>> http://www.osronline.com/page.cfm?name=ListServer
>>
>
> — NTFSD is sponsored by OSR OSR is hiring!! Info at
> http://www.osr.com/careers For our schedule of debugging and file system
> seminars visit: http://www.osr.com/seminars To unsubscribe, visit the
> List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
>
> —
> NTFSD is sponsored by OSR
>
> OSR is hiring!! Info at http://www.osr.com/careers
>
> For our schedule of debugging and file system seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
> —
> NTFSD is sponsored by OSR
>
> OSR is hiring!! Info at http://www.osr.com/careers
>
> For our schedule of debugging and file system seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
> —
> NTFSD is sponsored by OSR
>
> OSR is hiring!! Info at http://www.osr.com/careers
>
> For our schedule of debugging and file system seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>