Cache Manager Pinning Interface.

OSR_Community_User · December 11, 2006, 12:35pm

Hi All,

I’ll start with a question and than follow with a more specific description of what I am trying to achieve.

Q. Is it possible to assynchroneously process the lazy-writer threads, and notify the CM that the specified page is not dirty any more only after it is committed to disk?

I am writting a netwrok file system that has both client and server side caching. Even after the client side cache is written to the server I want to keep it around till I receive a confirmation from the server that data was committed to disk. In case of a server side error I want to re-send the client data.

Here is what I do:

On a client side I utilize the Cc pinning interface to cache the user writes. I map and pin the write buffers processing the user write requests using the CcMapData and CcPinMappedData Apis. I mark the data as dirty using CcSetDirtyPinnedData, and I add the resulting BCB to the FCB link list.

Some time later I force the flush of these BCBs using the CcRepinBcb/CcUnpinRepinnedBcb, which writes out all data that was previously was marked as dirty to the server. Next, if I receive notification that the server encountered an error committing the data to disk, I want to mark the data as dirty again via CcSetDirtyPinnedData, but it does not seem to work.

Here is amore simplified description of above using a sequence of API calls:

In FsdWrite thread:

CcMapData
CcPinMappedData
CcSetDirtyPinnedData
Add BCB to the link list of FCB and Return

In a worker thread per each BCB in the link list of FCB:

CcRepinBcb
CcUnpinRepinnedBcb - this triggers the write of dirty data
…
Send message to the server to commit the data to the disk - in case of an error do the following.
CcSetDirtyPinnedData, and than repeat steps 1-3. However, the problem is that I do not see any writes going through when I call CcRepinBcb/CcUnpinRepinnedBcb Apis next time, which is at the core of my problem.

Thanks in advance for all your responses.

-Ilya.

OSR_Community_User · December 12, 2006, 1:02pm

I do not think I violate any contracts, at least I am trying not to.
I am trying to extend an existing functionality where cached data is lazy written some time after user write request was completed. What I am trying to do is to keep the data in memory after the lazy write went through - this way all future IOs will be satisfied with in-memory pages. Once I receive a confirmation that data was committed, I will tell CM to unlock the specified pages.
It seems that what I should do is to keep track of the dirty pages and lazy write them to the disk myself. I should lock the pages via CcMapData and CcPinMappedData APIs, and keep them locked till I receive a confirmation from the server that the data was committed. I will use the returned buffer from CcMapData to modify the data in-memory; and asynchronously write it to the server using my worker (lazy-writer) thread.
Any comments on the above approach will be greatly appreciated.
BTW., I realize that this method will not work with memory-mapped files, which is an accepted limitation.
Thanks,
Ilya.

From: Tracy Camp

> I suspect you are not going to be successful with this because it seems
> you might be volating the basic contract of a file-system, namely that
> once you have completed a write that the file system guarantees that it
> will remain involitile.
>
> If there are conditions where remote ACK is needed to guaranteed the
> write, it is best to not complete the MM’s write IRP until you have that
> ACK.
>
> NFS has such an issue and various flavors of ACK exist in the NFS
> protocol, “ACK I got your packet” and in some situations a “ACK I
> completed the operation”
>
> t.
>
>
> On Mon, 11 Dec 2006, xxxxx@comcast.net wrote:
>
> > Hi All,
> >
> > I’ll start with a question and than follow with a more specific description of
> what I am trying to achieve.
> >
> > Q. Is it possible to assynchroneously process the lazy-writer threads, and
> notify the CM that the specified page is not dirty any more only after it is
> committed to disk?
> >
> > I am writting a netwrok file system that has both client and server side
> caching. Even after the client side cache is written to the server I want to
> keep it around till I receive a confirmation from the server that data was
> committed to disk. In case of a server side error I want to re-send the client
> data.
> >
> > Here is what I do:
> >
> > On a client side I utilize the Cc pinning interface to cache the user
> > writes. I map and pin the write buffers processing the user write
> > requests using the CcMapData and CcPinMappedData Apis. I mark the data
> > as dirty using CcSetDirtyPinnedData, and I add the resulting BCB to the
> > FCB link list.
> >
> > Some time later I force the flush of these BCBs using the
> > CcRepinBcb/CcUnpinRepinnedBcb, which writes out all data that was
> > previously was marked as dirty to the server. Next, if I receive
> > notification that the server encountered an error committing the data to
> > disk, I want to mark the data as dirty again via CcSetDirtyPinnedData,
> > but it does not seem to work.
> >
> > Here is amore simplified description of above using a sequence of API
> > calls:
> >
> > In FsdWrite thread:
> > 1. CcMapData
> > 2. CcPinMappedData
> > 3. CcSetDirtyPinnedData
> > 4. Add BCB to the link list of FCB and Return
> >
> > In a worker thread per each BCB in the link list of FCB:
> > 1. CcRepinBcb
> > 2. CcUnpinRepinnedBcb - this triggers the write of dirty data
> > …
> > 3. Send message to the server to commit the data to the disk - in case of an
> error do the following.
> > 4. CcSetDirtyPinnedData, and than repeat steps 1-3. However, the problem is
> that I do not see any writes going through when I call
> CcRepinBcb/CcUnpinRepinnedBcb Apis next time, which is at the core of my
> problem.
> >
> > Thanks in advance for all your responses.
> >
> > -Ilya.
> > —
> > Questions? First check the IFS FAQ at
> https://www.osronline.com/article.cfm?id=17
> >
> > You are currently subscribed to ntfsd as: xxxxx@openmars.com
> > To unsubscribe send a blank email to xxxxx@lists.osr.com

Slava_Imameyev · December 12, 2006, 5:21pm

> Q. Is it possible to assynchroneously process the lazy-writer threads, and notify the CM that the specified page is not dirty any more only after it is committed to disk?

I think you want to avoid the popup message “write behind failed”.
The page frames are marked as not dirty before write and if the write failed the page frames are marked again as dirty. If the lazy writer fails to write pages the system creates the popup msg “write behind failed”, but there is one exception - if STATUS_FILE_LOCK_CONFLICT is returned the lazy writer thread requeues the request and will repeat an attempt to write pages from the shared cache map. So simply return STATUS_FILE_LOCK_CONFLICT.

Some time later I force the flush of these BCBs using the CcRepinBcb/CcUnpinRepinnedBcb

If you set WriteThrough to TRUE, then you mark the page frames as dirty and calls MmFlushSection.
And again, if STATUS_FILE_LOCK_CONFLICT is returned the system will repeat an attempt to write data from the lazy writer thread.
Also, CcSetDirtyPinnedData doesn’t mark the page frames as dirty - it marks the BCB as dirty.

I should lock the pages via CcMapData and CcPinMappedData

This APIs don’t lock page frames associated with a file mapped in the cache!

wrote in message news:xxxxx@ntfsd…
Hi All,

I’ll start with a question and than follow with a more specific description of what I am trying to achieve.

Q. Is it possible to assynchroneously process the lazy-writer threads, and notify the CM that the specified page is not dirty any more only after it is committed to disk?

I am writting a netwrok file system that has both client and server side caching. Even after the client side cache is written to the server I want to keep it around till I receive a confirmation from the server that data was committed to disk. In case of a server side error I want to re-send the client data.

Here is what I do:

On a client side I utilize the Cc pinning interface to cache the user writes. I map and pin the write buffers processing the user write requests using the CcMapData and CcPinMappedData Apis. I mark the data as dirty using CcSetDirtyPinnedData, and I add the resulting BCB to the FCB link list.

Some time later I force the flush of these BCBs using the CcRepinBcb/CcUnpinRepinnedBcb, which writes out all data that was previously was marked as dirty to the server. Next, if I receive notification that the server encountered an error committing the data to disk, I want to mark the data as dirty again via CcSetDirtyPinnedData, but it does not seem to work.

Here is amore simplified description of above using a sequence of API calls:

In FsdWrite thread:
1. CcMapData
2. CcPinMappedData
3. CcSetDirtyPinnedData
4. Add BCB to the link list of FCB and Return

In a worker thread per each BCB in the link list of FCB:
1. CcRepinBcb
2. CcUnpinRepinnedBcb - this triggers the write of dirty data
…
3. Send message to the server to commit the data to the disk - in case of an error do the following.
4. CcSetDirtyPinnedData, and than repeat steps 1-3. However, the problem is that I do not see any writes going through when I call CcRepinBcb/CcUnpinRepinnedBcb Apis next time, which is at the core of my problem.

Thanks in advance for all your responses.

-Ilya.

OSR_Community_User · December 12, 2006, 6:24pm

Slava,

thanks for you reply. While returning the STATUS_FILE_LOCK_CONFLICT processing the lazy-writer threads is an interesting option, it is not really applicable in my case. My goal here is to keep data in shared memory even after a successful lazy-writer request is complete, and if neccessary, mark this shared data as dirty again and force the CM to requeue the lazy-writer thread.

A call to CcRepinBcb/CcUnpinRepinnedBcb APIs will trigger a lazy-writer thread to flush the data described by BCB to disk. Obviously the data had to be marked as dirty previously via CcSetDirtyPinnedData API. Note that even if lazy-writer threads complete the operation successfully, let’s assume they always do, it is still possible that the data will not get committed on the server side - this is similar to client side “write behind failed” error/message. If I find out that the server side write behind failed, can I mark the relevant BCBs as dirty again, and than flush the data again using CcRepinBcb/CcUnpinRepinnedBcb APIs??? I assume that BCB is valid and locked till CcUnpinData is called.

I think another way to do it, which I described in my previous e-mail, is to keep track of the dirty pages and lazy write them to the disk myself. I should lock the pages via CcMapData and CcPinMappedData APIs, and keep them locked till I receive a confirmation from the server that the data was committed. I will use the returned buffer from CcMapData to modify the data in-memory; and asynchronously write it to the server using my worker (lazy-writer) thread. The downside with this method is that I may loose all optimizations that CM already has built in. e.g. small file writes.

Any thoughts?

Thanks,

Ilya.

-------------- Original message --------------
From: “Slava Imameyev”

> Q. Is it possible to assynchroneously process the lazy-writer threads, and notify the CM that the specified page is not dirty any more only after it is committed to disk?

I think you want to avoid the popup message “write behind failed”.
The page frames are marked as not dirty before write and if the write failed the page frames are marked again as dirty. If the lazy writer fails to write pages the system creates the popup msg “write behind failed”, but there is one exception - if STATUS_FILE_LOCK_CONFLICT is returned the lazy writer thread requeues the request and will repeat an attempt to write pages from the shared cache map. So simply return STATUS_FILE_LOCK_CONFLICT.

>Some time later I force the flush of these BCBs using the CcRepinBcb/CcUnpinRepinnedBcb

If you set WriteThrough to TRUE, then you mark the page frames as dirty and calls MmFlushSection.
And again, if STATUS_FILE_LOCK_CONFLICT is returned the system will repeat an attempt to write data from the lazy writer thread.
Also, CcSetDirtyPinnedData doesn’t mark the page frames as dirty - it marks the BCB as dirty.

>I should lock the pages via CcMapData and CcPinMappedData

This APIs don’t lock page frames associated with a file mapped in the cache!

wrote in message news:xxxxx@ntfsd…
Hi All,

I’ll start with a question and than follow with a more specific description of what I am trying to achieve.

Q. Is it possible to assynchroneously process the lazy-writer threads, and notify the CM that the specified page is not dirty any more only after it is committed to disk?

I am writting a netwrok file system that has both client and server side caching. Even after the client side cache is written to the server I want to keep it around till I receive a confirmation from the server that data was committed to disk. In case of a server side error I want to re-send the client data.

Here is what I do:

On a client side I utilize the Cc pinning interface to cache the user writes. I map and pin the write buffers processing the user write requests using the CcMapData and CcPinMappedData Apis. I mark the data as dirty using CcSetDirtyPinnedData, and I add the resulting BCB to the FCB link list.

Some time later I force the flush of these BCBs using the CcRepinBcb/CcUnpinRepinnedBcb, which writes out all data that was previously was marked as dirty to the server. Next, if I receive notification that the server encountered an error committing the data to disk, I want to mark the data as dirty again via CcSetDirtyPinnedData, but it does not seem to work.

Here is amore simplified description of above using a sequence of API calls:

In FsdWrite thread:
1. CcMapData
2. CcPinMappedData
3. CcSetDirtyPinnedData
4. Add BCB to the link list of FCB and Return

In a worker thread per each BCB in the link list of FCB:
1. CcRepinBcb
2. CcUnpinRepinnedBcb - this triggers the write of dirty data
…
3. Send message to the server to commit the data to the disk - in case of an error do the following.
4. CcSetDirtyPinnedData, and than repeat steps 1-3. However, the problem is that I do not see any writes going through when I call CcRepinBcb/CcUnpinRepinnedBcb Apis next time, which is at the core of my problem.

Thanks in advance for all your responses.

-Ilya.

—
Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

Slava_Imameyev · December 12, 2006, 6:53pm

> If I find out that the server side write behind failed, can I mark the relevant BCBs as dirty again, and than flush the data again using CcRepinBcb/CcUnpinRepinnedBcb APIs??? I assume that BCB is valid and locked till CcUnpinData is called.

Yes, you can CcSetDirtyPinnedData - CcRepinBcb/CcUnpinRepinnedBcb again.

I think another way to do it, which I described in my previous e-mail, is to keep track of the dirty pages and lazy write them to the disk myself. I should lock the pages via CcMapData and CcPinMappedData APIs, and keep them locked

You use phrase “lock the pages”, but in Windows world this means- “resedent pages”. CcPinMappedData doesn’t lock the page frames. CcPinMappedData only guarantees that the portion of the mapped file will be mapped untill CcUnpinData. The Cache Manager or the Memory Manager may decide to flush the pages mapping the file if there is a page frames shortage in the system, so you will have to cope with this write requests for which you are not originator.

wrote in message news:xxxxx@ntfsd…
Slava,

thanks for you reply. While returning the STATUS_FILE_LOCK_CONFLICT processing the lazy-writer threads is an interesting option, it is not really applicable in my case. My goal here is to keep data in shared memory even after a successful lazy-writer request is complete, and if neccessary, mark this shared data as dirty again and force the CM to requeue the lazy-writer thread.

A call to CcRepinBcb/CcUnpinRepinnedBcb APIs will trigger a lazy-writer thread to flush the data described by BCB to disk. Obviously the data had to be marked as dirty previously via CcSetDirtyPinnedData API. Note that even if lazy-writer threads complete the operation successfully, let’s assume they always do, it is still possible that the data will not get committed on the server side - this is similar to client side “write behind failed” error/message. If I find out that the server side write behind failed, can I mark the relevant BCBs as dirty again, and than flush the data again using CcRepinBcb/CcUnpinRepinnedBcb APIs??? I assume that BCB is valid and locked till CcUnpinData is called.

I think another way to do it, which I described in my previous e-mail, is to keep track of the dirty pages and lazy write them to the disk myself. I should lock the pages via CcMapData and CcPinMappedData APIs, and keep them locked till I receive a confirmation from the server that the data was committed. I will use the returned buffer from CcMapData to modify the data in-memory; and asynchronously write it to the server using my worker (lazy-writer) thread. The downside with this method is that I may loose all optimizations that CM already has built in. e.g. small file writes.

Any thoughts?

Thanks,

Ilya.

-------------- Original message --------------
From: “Slava Imameyev”

> Q. Is it possible to assynchroneously process the lazy-writer threads, and notify the CM that the specified page is not dirty any more only after it is committed to disk?

I think you want to avoid the popup message “write behind failed”.
The page frames are marked as not dirty before write and if the write failed the page frames are marked again as dirty. If the lazy writer fails to write pages the system creates the popup msg “write behind failed”, but there is one exception - if STATUS_FILE_LOCK_CONFLICT is returned the lazy writer thread requeues the request and will repeat an attempt to write pages from the shared cache map. So simply return STATUS_FILE_LOCK_CONFLICT.

>Some time later I force the flush of these BCBs using the CcRepinBcb/CcUnpinRepinnedBcb

If you set WriteThrough to TRUE, then you mark the page frames as dirty and calls MmFlushSection.
And again, if STATUS_FILE_LOCK_CONFLICT is returned the system will repeat an attempt to write data from the lazy writer thread.
Also, CcSetDirtyPinnedData doesn’t mark the page frames as dirty - it marks the BCB as dirty.

>I should lock the pages via CcMapData and CcPinMappedData

This APIs don’t lock page frames associated with a file mapped in the cache!

wrote in message news:xxxxx@ntfsd…
Hi All,

I’ll start with a question and than follow with a more specific description of what I am trying to achieve.

Q. Is it possible to assynchroneously process the lazy-writer threads, and notify the CM that the specified page is not dirty any more only after it is committed to disk?

I am writting a netwrok file system that has both client and server side caching. Even after the client side cache is written to the server I want to keep it around till I receive a confirmation from the server that data was committed to disk. In case of a server side error I want to re-send the client data.

Here is what I do:

On a client side I utilize the Cc pinning interface to cache the user writes. I map and pin the write buffers processing the user write requests using the CcMapData and CcPinMappedData Apis. I mark the data as dirty using CcSetDirtyPinnedData, and I add the resulting BCB to the FCB link list.

Some time later I force the flush of these BCBs using the CcRepinBcb/CcUnpinRepinnedBcb, which writes out all data that was previously was marked as dirty to the server. Next, if I receive notification that the server encountered an error committing the data to disk, I want to mark the data as dirty again via CcSetDirtyPinnedData, but it does not seem to work.

Here is amore simplified description of above using a sequence of API calls:

In FsdWrite thread:
1. CcMapData
2. CcPinMappedData
3. CcSetDirtyPinnedData
4. Add BCB to the link list of FCB and Return

In a worker thread per each BCB in the link list of FCB:
1. CcRepinBcb
2. CcUnpinRepinnedBcb - this triggers the write of dirty data
…
3. Send message to the server to commit the data to the disk - in case of an error do the following.
4. CcSetDirtyPinnedData, and than repeat steps 1-3. However, the problem is that I do not see any writes going through when I call CcRepinBcb/CcUnpinRepinnedBcb Apis next time, which is at the core of my problem.

Thanks in advance for all your responses.

-Ilya.

—
Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

OSR_Community_User · December 13, 2006, 3:22am

> What I am trying to do is to keep the data in memory after

the lazy write went through - this way all future IOs will be
satisfied with in-memory pages.

Hang on, hang on, hang on

Aren’t you trying to develop a product that does nothing ?

A very quick experiment with FileTest and FileSpy:

CreateFile
WriteFile(offset 0, length 0x100000 bytes)
wait a bit
ReadFile(offset 0, length 0x10000 bytes)
CloseHandle

will show this:

IRP_MJ_CREATE C:\TestFile.bin
IRP_MJ_WRITE (cached) Offset 00000000-00000000 ToWrite 100000 Written: 100000
IRP_MJ_WRITE (paging) Offset 00000000-00000000 ToWrite 10000 Written: 10000
IRP_MJ_WRITE (paging) Offset 00000000-00010000 ToWrite 10000 Written: 10000
IRP_MJ_WRITE (paging) Offset 00000000-00020000 ToWrite 10000 Written: 10000
IRP_MJ_WRITE (paging) Offset 00000000-00030000 ToWrite 10000 Written: 10000
IRP_MJ_WRITE (paging) Offset 00000000-00040000 ToWrite 10000 Written: 10000
IRP_MJ_WRITE (paging) Offset 00000000-00050000 ToWrite 10000 Written: 10000
IRP_MJ_WRITE (paging) Offset 00000000-00060000 ToWrite 10000 Written: 10000
IRP_MJ_WRITE (paging) Offset 00000000-00070000 ToWrite 10000 Written: 10000
IRP_MJ_WRITE (paging) Offset 00000000-00080000 ToWrite 10000 Written: 10000
IRP_MJ_WRITE (paging) Offset 00000000-00090000 ToWrite 10000 Written: 10000
IRP_MJ_WRITE (paging) Offset 00000000-000A0000 ToWrite 10000 Written: 10000
IRP_MJ_WRITE (paging) Offset 00000000-000B0000 ToWrite 10000 Written: 10000
FASTIO_READ Offset 00000000-00000000 ToRead 10000 Read 10000
IRP_MJ_CLEANUP C:\TestFile.bin
IRP_MJ_CLOSE C:\TestFile.bin

We see here both writes - the “normal write” (cached) and the paging I/O.
After that, the read request has been satisfied by the system cache.
Note that the paging I/O has been performed on different file object than
the original one, so the IRP_MJ_CLOSE does not matter in this example.

L.

OSR_Community_User · December 13, 2006, 11:44am

>>Hang on, hang on, hang on

>Aren’t you trying to develop a product that does nothing ?
I am trying to develop the network FS that is as reliable as any local FS. Users are expecting to loose data when their workstation is crashing. What they do not expect ( or should not expect ) is to loose data because some obscure network server crashed.
What I am trying to do is to protect users from loosing their data that they have flushed to the server, because server crashed before committing the data to the disk.

Here is a description of what I want to do:

I want to keep a list of dirty pages.
I want to control the time these pages stay in system cache.
I want to control when these data is lazy-written back to disk.
I want to be able to mark the data in system cache as dirty again and to force the data to be lazy-written back to disk.

Do you have an alternative suggestion how to do it, in addition to what I’ve described in this thread already?

Thanks in advance,

Ilya.

-------------- Original message --------------
From: “Ladislav Zezula”

> What I am trying to do is to keep the data in memory after
> the lazy write went through - this way all future IOs will be
> satisfied with in-memory pages.

Hang on, hang on, hang on

Aren’t you trying to develop a product that does nothing ?

A very quick experiment with FileTest and FileSpy:

CreateFile
WriteFile(offset 0, length 0x100000 bytes)
wait a bit
ReadFile(offset 0, length 0x10000 bytes)
CloseHandle

will show this:

IRP_MJ_CREATE C:\TestFile.bin
IRP_MJ_WRITE (cached) Offset 00000000-00000000 ToWrite 100000 Written: 100000
IRP_MJ_WRITE (paging) Offset 00000000-00000000 ToWrite 10000 Written: 10000
IRP_MJ_WRITE (paging) Offset 00000000-00010000 ToWrite 10000 Written: 10000
IRP_MJ_WRITE (paging) Offset 00000000-00020000 ToWrite 10000 Written: 10000
IRP_MJ_WRITE (paging) Offset 00000000-00030000 ToWrite 10000 Written: 10000
IRP_MJ_WRITE (paging) Offset 00000000-00040000 ToWrite 10000 Written: 10000
IRP_MJ_WRITE (paging) Offset 00000000-00050000 ToWrite 10000 Written: 10000
IRP_MJ_WRITE (paging) Offset 00000000-00060000 ToWrite 10000 Written: 10000
IRP_MJ_WRITE (paging) Offset 00000000-00070000 ToWrite 10000 Written: 10000
IRP_MJ_WRITE (paging) Offset 00000000-00080000 ToWrite 10000 Written: 10000
IRP_MJ_WRITE (paging) Offset 00000000-00090000 ToWrite 10000 Written: 10000
IRP_MJ_WRITE (paging) Offset 00000000-000A0000 ToWrite 10000 Written: 10000
IRP_MJ_WRITE (paging) Offset 00000000-000B0000 ToWrite 10000 Written: 10000
FASTIO_READ Offset 00000000-00000000 ToRead 10000 Read 10000
IRP_MJ_CLEANUP C:\TestFile.bin
IRP_MJ_CLOSE C:\TestFile.bin

We see here both writes - the “normal write” (cached) and the paging I/O.
After that, the read request has been satisfied by the system cache.
Note that the paging I/O has been performed on different file object than
the original one, so the IRP_MJ_CLOSE does not matter in this example.

L.

—
Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

OSR_Community_User · December 13, 2006, 11:59am

>>>>Yes, you can CcSetDirtyPinnedData - CcRepinBcb/CcUnpinRepinnedBcb again.
The problem here is that I do not see any paging writes after the above calls - the data does not get re-written.

>>>You use phrase “lock the pages”, but in Windows world this means- “resedent pages”. CcPinMappedData doesn’t lock the page frames. CcPinMappedData only guarantees that the portion of the mapped file will be mapped untill CcUnpinData.
Sorry for my poor use of words. What I meant is that after calling CcPinMappedData and untill calling the CcUnpinData the data will be mapped into the system cache virtual address space. I understand that the data can get paged out, and when referenced again, it will result in a page fault, and the data will be read from the system page file. Again, can you or somebody else verify that the data stays mapped in this example untill the CcUnpinData is called. Or something else can cause the data to unmap. e.g calls to CcRepinBcb/CcUnpinRepinnedBcb APIs.

Thanks,

Ilya.

-------------- Original message --------------
From: “Slava Imameyev”

> If I find out that the server side write behind failed, can I mark the relevant BCBs as dirty again, and than flush the data again using CcRepinBcb/CcUnpinRepinnedBcb APIs??? I assume that BCB is valid and locked till CcUnpinData is called.

Yes, you can CcSetDirtyPinnedData - CcRepinBcb/CcUnpinRepinnedBcb again.

>I think another way to do it, which I described in my previous e-mail, is to keep track of the dirty pages and lazy write them to the disk myself. I should lock the pages via CcMapData and CcPinMappedData APIs, and keep them locked

You use phrase “lock the pages”, but in Windows world this means- “resedent pages”. CcPinMappedData doesn’t lock the page frames. CcPinMappedData only guarantees that the portion of the mapped file will be mapped untill CcUnpinData. The Cache Manager or the Memory Manager may decide to flush the pages mapping the file if there is a page frames shortage in the system, so you will have to cope with this write requests for which you are not originator.

wrote in message news:xxxxx@ntfsd…
Slava,

thanks for you reply. While returning the STATUS_FILE_LOCK_CONFLICT processing the lazy-writer threads is an interesting option, it is not really applicable in my case. My goal here is to keep data in shared memory even after a successful lazy-writer request is complete, and if neccessary, mark this shared data as dirty again and force the CM to requeue the lazy-writer thread.

A call to CcRepinBcb/CcUnpinRepinnedBcb APIs will trigger a lazy-writer thread to flush the data described by BCB to disk. Obviously the data had to be marked as dirty previously via CcSetDirtyPinnedData API. Note that even if lazy-writer threads complete the operation successfully, let’s assume they always do, it is still possible that the data will not get committed on the server side - this is similar to client side “write behind failed” error/message. If I find out that the server side write behind failed, can I mark the relevant BCBs as dirty again, and than flush the data again using CcRepinBcb/CcUnpinRepinnedBcb APIs??? I assume that BCB is valid and locked till CcUnpinData is called.

I think another way to do it, which I described in my previous e-mail, is to keep track of the dirty pages and lazy write them to the disk myself. I should lock the pages via CcMapData and CcPinMappedData APIs, and keep them locked till I receive a confirmation from the server that the data was committed. I will use the returned buffer from CcMapData to modify the data in-memory; and asynchronously write it to the server using my worker (lazy-writer) thread. The downside with this method is that I may loose all optimizations that CM already has built in. e.g. small file writes.

Any thoughts?

Thanks,

Ilya.

-------------- Original message --------------
From: “Slava Imameyev”

> Q. Is it possible to assynchroneously process the lazy-writer threads, and notify the CM that the specified page is not dirty any more only after it is committed to disk?

I think you want to avoid the popup message “write behind failed”.
The page frames are marked as not dirty before write and if the write failed the page frames are marked again as dirty. If the lazy writer fails to write pages the system creates the popup msg “write behind failed”, but there is one exception - if STATUS_FILE_LOCK_CONFLICT is returned the lazy writer thread requeues the request and will repeat an attempt to write pages from the shared cache map. So simply return STATUS_FILE_LOCK_CONFLICT.

>Some time later I force the flush of these BCBs using the CcRepinBcb/CcUnpinRepinnedBcb

If you set WriteThrough to TRUE, then you mark the page frames as dirty and calls MmFlushSection.
And again, if STATUS_FILE_LOCK_CONFLICT is returned the system will repeat an attempt to write data from the lazy writer thread.
Also, CcSetDirtyPinnedData doesn’t mark the page frames as dirty - it marks the BCB as dirty.

>I should lock the pages via CcMapData and CcPinMappedData

This APIs don’t lock page frames associated with a file mapped in the cache!

wrote in message news:xxxxx@ntfsd…
Hi All,

I’ll start with a question and than follow with a more specific description of what I am trying to achieve.

Q. Is it possible to assynchroneously process the lazy-writer threads, and notify the CM that the specified page is not dirty any more only after it is committed to disk?

I am writting a netwrok file system that has both client and server side caching. Even after the client side cache is written to the server I want to keep it around till I receive a confirmation from the server that data was committed to disk. In case of a server side error I want to re-send the client data.

Here is what I do:

On a client side I utilize the Cc pinning interface to cache the user writes. I map and pin the write buffers processing the user write requests using the CcMapData and CcPinMappedData Apis. I mark the data as dirty using CcSetDirtyPinnedData, and I add the resulting BCB to the FCB link list.

Some time later I force the flush of these BCBs using the CcRepinBcb/CcUnpinRepinnedBcb, which writes out all data that was previously was marked as dirty to the server. Next, if I receive notification that the server encountered an error committing the data to disk, I want to mark the data as dirty again via CcSetDirtyPinnedData, but it does not seem to work.

Here is amore simplified description of above using a sequence of API calls:

In FsdWrite thread:
1. CcMapData
2. CcPinMappedData
3. CcSetDirtyPinnedData
4. Add BCB to the link list of FCB and Return

In a worker thread per each BCB in the link list of FCB:
1. CcRepinBcb
2. CcUnpinRepinnedBcb - this triggers the write of dirty data
…
3. Send message to the server to commit the data to the disk - in case of an error do the following.
4. CcSetDirtyPinnedData, and than repeat steps 1-3. However, the problem is that I do not see any writes going through when I call CcRepinBcb/CcUnpinRepinnedBcb Apis next time, which is at the core of my problem.

Thanks in advance for all your responses.

-Ilya.

—
Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

—
Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

Slava_Imameyev · December 14, 2006, 6:02pm

>it will result in a page fault, and the data will be read from the system page file

data will be read from the file that backs the memory mapping the file( i.e. the file that backs the segment object ), this is not the page file, this is common file, i.e. this is your file. And if the page is dirty it will be written in your file!

Again, can you or somebody else verify that the data stays mapped in this example untill the CcUnpinData is called

yes, there must be a reference counter incremented on each CcPinMappedData, CcRepinBcb etc. and decremented in CcUnpinData.

wrote in message news:xxxxx@ntfsd…
>>>>Yes, you can CcSetDirtyPinnedData - CcRepinBcb/CcUnpinRepinnedBcb again.
The problem here is that I do not see any paging writes after the above calls - the data does not get re-written.

>>>>You use phrase “lock the pages”, but in Windows world this means- “resedent pages”. CcPinMappedData doesn’t lock the page frames. CcPinMappedData only guarantees that the portion of the mapped file will be mapped untill CcUnpinData.
Sorry for my poor use of words. What I meant is that after calling CcPinMappedData and untill calling the CcUnpinData the data will be mapped into the system cache virtual address space. I understand that the data can get paged out, and when referenced again, it will result in a page fault, and the data will be read from the system page file. Again, can you or somebody else verify that the data stays mapped in this example untill the CcUnpinData is called. Or something else can cause the data to unmap. e.g calls to CcRepinBcb/CcUnpinRepinnedBcb APIs.

Thanks,

Ilya.

-------------- Original message --------------
From: “Slava Imameyev”

> If I find out that the server side write behind failed, can I mark the relevant BCBs as dirty again, and than flush the data again using CcRepinBcb/CcUnpinRepinnedBcb APIs??? I assume that BCB is valid and locked till CcUnpinData is called.

Yes, you can CcSetDirtyPinnedData - CcRepinBcb/CcUnpinRepinnedBcb again.

>I think another way to do it, which I described in my previous e-mail, is to keep track of the dirty pages and lazy write them to the disk myself. I should lock the pages via CcMapData and CcPinMappedData APIs, and keep them locked

You use phrase “lock the pages”, but in Windows world this means- “resedent pages”. CcPinMappedData doesn’t lock the page frames. CcPinMappedData only guarantees that the portion of the mapped file will be mapped untill CcUnpinData. The Cache Manager or the Memory Manager may decide to flush the pages mapping the file if there is a page frames shortage in the system, so you will have to cope with this write requests for which you are not originator.

wrote in message news:xxxxx@ntfsd…
Slava,

thanks for you reply. While returning the STATUS_FILE_LOCK_CONFLICT processing the lazy-writer threads is an interesting option, it is not really applicable in my case. My goal here is to keep data in shared memory even after a successful lazy-writer request is complete, and if neccessary, mark this shared data as dirty again and force the CM to requeue the lazy-writer thread.

A call to CcRepinBcb/CcUnpinRepinnedBcb APIs will trigger a lazy-writer thread to flush the data described by BCB to disk. Obviously the data had to be marked as dirty previously via CcSetDirtyPinnedData API. Note that even if lazy-writer threads complete the operation successfully, let’s assume they always do, it is still possible that the data will not get committed on the server side - this is similar to client side “write behind failed” error/message. If I find out that the server side write behind failed, can I mark the relevant BCBs as dirty again, and than flush the data again using CcRepinBcb/CcUnpinRepinnedBcb APIs??? I assume that BCB is valid and locked till CcUnpinData is called.

I think another way to do it, which I described in my previous e-mail, is to keep track of the dirty pages and lazy write them to the disk myself. I should lock the pages via CcMapData and CcPinMappedData APIs, and keep them locked till I receive a confirmation from the server that the data was committed. I will use the returned buffer from CcMapData to modify the data in-memory; and asynchronously write it to the server using my worker (lazy-writer) thread. The downside with this method is that I may loose all optimizations that CM already has built in. e.g. small file writes.

Any thoughts?

Thanks,

Ilya.

-------------- Original message --------------
From: “Slava Imameyev”

> Q. Is it possible to assynchroneously process the lazy-writer threads, and notify the CM that the specified page is not dirty any more only after it is committed to disk?

I think you want to avoid the popup message “write behind failed”.
The page frames are marked as not dirty before write and if the write failed the page frames are marked again as dirty. If the lazy writer fails to write pages the system creates the popup msg “write behind failed”, but there is one exception - if STATUS_FILE_LOCK_CONFLICT is returned the lazy writer thread requeues the request and will repeat an attempt to write pages from the shared cache map. So simply return STATUS_FILE_LOCK_CONFLICT.

>Some time later I force the flush of these BCBs using the CcRepinBcb/CcUnpinRepinnedBcb

If you set WriteThrough to TRUE, then you mark the page frames as dirty and calls MmFlushSection.
And again, if STATUS_FILE_LOCK_CONFLICT is returned the system will repeat an attempt to write data from the lazy writer thread.
Also, CcSetDirtyPinnedData doesn’t mark the page frames as dirty - it marks the BCB as dirty.

>I should lock the pages via CcMapData and CcPinMappedData

This APIs don’t lock page frames associated with a file mapped in the cache!

wrote in message news:xxxxx@ntfsd…
Hi All,

I’ll start with a question and than follow with a more specific description of what I am trying to achieve.

Q. Is it possible to assynchroneously process the lazy-writer threads, and notify the CM that the specified page is not dirty any more only after it is committed to disk?

I am writting a netwrok file system that has both client and server side caching. Even after the client side cache is written to the server I want to keep it around till I receive a confirmation from the server that data was committed to disk. In case of a server side error I want to re-send the client data.

Here is what I do:

On a client side I utilize the Cc pinning interface to cache the user writes. I map and pin the write buffers processing the user write requests using the CcMapData and CcPinMappedData Apis. I mark the data as dirty using CcSetDirtyPinnedData, and I add the resulting BCB to the FCB link list.

Some time later I force the flush of these BCBs using the CcRepinBcb/CcUnpinRepinnedBcb, which writes out all data that was previously was marked as dirty to the server. Next, if I receive notification that the server encountered an error committing the data to disk, I want to mark the data as dirty again via CcSetDirtyPinnedData, but it does not seem to work.

Here is amore simplified description of above using a sequence of API calls:

In FsdWrite thread:
1. CcMapData
2. CcPinMappedData
3. CcSetDirtyPinnedData
4. Add BCB to the link list of FCB and Return

In a worker thread per each BCB in the link list of FCB:
1. CcRepinBcb
2. CcUnpinRepinnedBcb - this triggers the write of dirty data
…
3. Send message to the server to commit the data to the disk - in case of an error do the following.
4. CcSetDirtyPinnedData, and than repeat steps 1-3. However, the problem is that I do not see any writes going through when I call CcRepinBcb/CcUnpinRepinnedBcb Apis next time, which is at the core of my problem.

Thanks in advance for all your responses.

-Ilya.

—
Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

—
Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

OSR_Community_User · December 15, 2006, 12:16pm

Correct me if I am wrong …
CcMapData maps the data into system cache memory, where the file that backs the segmen object is my file. This is why after calling the CcMapData, it is not safe to modify the data using the returned buffer pointer, since the data can get paged out without saving the contents. CcPinMappedData ensures that the segment object will be backed by the system paging file. Therefore after calling the CcPinMappedData it is safe to modify the data using the buffer returnmed from previous CcMapData call.

-Ilya.

-------------- Original message --------------
From: “Slava Imameyev”

>it will result in a page fault, and the data will be read from the system page file

data will be read from the file that backs the memory mapping the file( i.e. the file that backs the segment object ), this is not the page file, this is common file, i.e. this is your file. And if the page is dirty it will be written in your file!

>Again, can you or somebody else verify that the data stays mapped in this example untill the CcUnpinData is called

yes, there must be a reference counter incremented on each CcPinMappedData, CcRepinBcb etc. and decremented in CcUnpinData.

wrote in message news:xxxxx@ntfsd…
>>>>Yes, you can CcSetDirtyPinnedData - CcRepinBcb/CcUnpinRepinnedBcb again.
The problem here is that I do not see any paging writes after the above calls - the data does not get re-written.

>>>>You use phrase “lock the pages”, but in Windows world this means- “resedent pages”. CcPinMappedData doesn’t lock the page frames. CcPinMappedData only guarantees that the portion of the mapped file will be mapped untill CcUnpinData.
Sorry for my poor use of words. What I meant is that after calling CcPinMappedData and untill calling the CcUnpinData the data will be mapped into the system cache virtual address space. I understand that the data can get paged out, and when referenced again, it will result in a page fault, and the data will be read from the system page file. Again, can you or somebody else verify that the data stays mapped in this example untill the CcUnpinData is called. Or something else can cause the data to unmap. e.g calls to CcRepinBcb/CcUnpinRepinnedBcb APIs.

Thanks,

Ilya.

-------------- Original message --------------
From: “Slava Imameyev”

> If I find out that the server side write behind failed, can I mark the relevant BCBs as dirty again, and than flush the data again using CcRepinBcb/CcUnpinRepinnedBcb APIs??? I assume that BCB is valid and locked till CcUnpinData is called.

Yes, you can CcSetDirtyPinnedData - CcRepinBcb/CcUnpinRepinnedBcb again.

>I think another way to do it, which I described in my previous e-mail, is to keep track of the dirty pages and lazy write them to the disk myself. I should lock the pages via CcMapData and CcPinMappedData APIs, and keep them locked

You use phrase “lock the pages”, but in Windows world this means- “resedent pages”. CcPinMappedData doesn’t lock the page frames. CcPinMappedData only guarantees that the portion of the mapped file will be mapped untill CcUnpinData. The Cache Manager or the Memory Manager may decide to flush the pages mapping the file if there is a page frames shortage in the system, so you will have to cope with this write requests for which you are not originator.

wrote in message news:xxxxx@ntfsd…
Slava,

thanks for you reply. While returning the STATUS_FILE_LOCK_CONFLICT processing the lazy-writer threads is an interesting option, it is not really applicable in my case. My goal here is to keep data in shared memory even after a successful lazy-writer request is complete, and if neccessary, mark this shared data as dirty again and force the CM to requeue the lazy-writer thread.

A call to CcRepinBcb/CcUnpinRepinnedBcb APIs will trigger a lazy-writer thread to flush the data described by BCB to disk. Obviously the data had to be marked as dirty previously via CcSetDirtyPinnedData API. Note that even if lazy-writer threads complete the operation successfully, let’s assume they always do, it is still possible that the data will not get committed on the server side - this is similar to client side “write behind failed” error/message. If I find out that the server side write behind failed, can I mark the relevant BCBs as dirty again, and than flush the data again using CcRepinBcb/CcUnpinRepinnedBcb APIs??? I assume that BCB is valid and locked till CcUnpinData is called.

I think another way to do it, which I described in my previous e-mail, is to keep track of the dirty pages and lazy write them to the disk myself. I should lock the pages via CcMapData and CcPinMappedData APIs, and keep them locked till I receive a confirmation from the server that the data was committed. I will use the returned buffer from CcMapData to modify the data in-memory; and asynchronously write it to the server using my worker (lazy-writer) thread. The downside with this method is that I may loose all optimizations that CM already has built in. e.g. small file writes.

Any thoughts?

Thanks,

Ilya.

-------------- Original message --------------
From: “Slava Imameyev”

> Q. Is it possible to assynchroneously process the lazy-writer threads, and notify the CM that the specified page is not dirty any more only after it is committed to disk?

I think you want to avoid the popup message “write behind failed”.
The page frames are marked as not dirty before write and if the write failed the page frames are marked again as dirty. If the lazy writer fails to write pages the system creates the popup msg “write behind failed”, but there is one exception - if STATUS_FILE_LOCK_CONFLICT is returned the lazy writer thread requeues the request and will repeat an attempt to write pages from the shared cache map. So simply return STATUS_FILE_LOCK_CONFLICT.

>Some time later I force the flush of these BCBs using the CcRepinBcb/CcUnpinRepinnedBcb

If you set WriteThrough to TRUE, then you mark the page frames as dirty and calls MmFlushSection.
And again, if STATUS_FILE_LOCK_CONFLICT is returned the system will repeat an attempt to write data from the lazy writer thread.
Also, CcSetDirtyPinnedData doesn’t mark the page frames as dirty - it marks the BCB as dirty.

>I should lock the pages via CcMapData and CcPinMappedData

This APIs don’t lock page frames associated with a file mapped in the cache!

wrote in message news:xxxxx@ntfsd…
Hi All,

I’ll start with a question and than follow with a more specific description of what I am trying to achieve.

Q. Is it possible to assynchroneously process the lazy-writer threads, and notify the CM that the specified page is not dirty any more only after it is committed to disk?

I am writting a netwrok file system that has both client and server side caching. Even after the client side cache is written to the server I want to keep it around till I receive a confirmation from the server that data was committed to disk. In case of a server side error I want to re-send the client data.

Here is what I do:

On a client side I utilize the Cc pinning interface to cache the user writes. I map and pin the write buffers processing the user write requests using the CcMapData and CcPinMappedData Apis. I mark the data as dirty using CcSetDirtyPinnedData, and I add the resulting BCB to the FCB link list.

Some time later I force the flush of these BCBs using the CcRepinBcb/CcUnpinRepinnedBcb, which writes out all data that was previously was marked as dirty to the server. Next, if I receive notification that the server encountered an error committing the data to disk, I want to mark the data as dirty again via CcSetDirtyPinnedData, but it does not seem to work.

Here is amore simplified description of above using a sequence of API calls:

In FsdWrite thread:
1. CcMapData
2. CcPinMappedData
3. CcSetDirtyPinnedData
4. Add BCB to the link list of FCB and Return

In a worker thread per each BCB in the link list of FCB:
1. CcRepinBcb
2. CcUnpinRepinnedBcb - this triggers the write of dirty data
…
3. Send message to the server to commit the data to the disk - in case of an error do the following.
4. CcSetDirtyPinnedData, and than repeat steps 1-3. However, the problem is that I do not see any writes going through when I call CcRepinBcb/CcUnpinRepinnedBcb Apis next time, which is at the core of my problem.

Thanks in advance for all your responses.

-Ilya.

—
Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

—
Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

—
Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

Slava_Imameyev · December 15, 2006, 7:38pm

CcMapData maps the data into system cache memory, where the file that backs the segmen object is my file.

Correct, the VACB is allocated.

This is why after calling the CcMapData, it is not safe to modify the data using the returned buffer pointer, since the data can get paged out without saving the contents.

No. This is only a requirement of the Cache Manager developers - you are not allowed to write in the mapped region if it has not been pinned. The CcMapData may be optimized because it maps data for read access only. For example, the Cache Manager developers might design the CcMapData function in such a way that even if the PTE is marked as dirty( i.e. the page has been modified ) but the region is not pinned then this page will not be written to disk because the page frame data base descriptor( PFN descriptor ) is not marked as dirty due to optimization. But if the mapped region is pinned the Cache Manager developers guarantee that all dirty pages will be written to disk before being paged out. In other words, if you modify data that has been only mapped but not pinned then you are breaching the contract.

CcPinMappedData ensures that the segment object will be backed by the system paging file.

No. Forget about pagefile.

Therefore after calling the CcPinMappedData it is safe to modify the data using the buffer returnmed from previous CcMapData call.

See 2)

wrote in message news:xxxxx@ntfsd…
Correct me if I am wrong …
CcMapData maps the data into system cache memory, where the file that backs the segmen object is my file. This is why after calling the CcMapData, it is not safe to modify the data using the returned buffer pointer, since the data can get paged out without saving the contents. CcPinMappedData ensures that the segment object will be backed by the system paging file. Therefore after calling the CcPinMappedData it is safe to modify the data using the buffer returnmed from previous CcMapData call.

-Ilya.

-------------- Original message --------------
From: “Slava Imameyev”

>it will result in a page fault, and the data will be read from the system page file

data will be read from the file that backs the memory mapping the file( i.e. the file that backs the segment object ), this is not the page file, this is common file, i.e. this is your file. And if the page is dirty it will be written in your file!

>Again, can you or somebody else verify that the data stays mapped in this example untill the CcUnpinData is called

yes, there must be a reference counter incremented on each CcPinMappedData, CcRepinBcb etc. and decremented in CcUnpinData.

wrote in message news:xxxxx@ntfsd…
>>>>Yes, you can CcSetDirtyPinnedData - CcRepinBcb/CcUnpinRepinnedBcb again.
The problem here is that I do not see any paging writes after the above calls - the data does not get re-written.

>>>>You use phrase “lock the pages”, but in Windows world this means- “resedent pages”. CcPinMappedData doesn’t lock the page frames. CcPinMappedData only guarantees that the portion of the mapped file will be mapped untill CcUnpinData.
Sorry for my poor use of words. What I meant is that after calling CcPinMappedData and untill calling the CcUnpinData the data will be mapped into the system cache virtual address space. I understand that the data can get paged out, and when referenced again, it will result in a page fault, and the data will be read from the system page file. Again, can you or somebody else verify that the data stays mapped in this example untill the CcUnpinData is called. Or something else can cause the data to unmap. e.g calls to CcRepinBcb/CcUnpinRepinnedBcb APIs.

Thanks,

Ilya.

-------------- Original message --------------
From: “Slava Imameyev”

> If I find out that the server side write behind failed, can I mark the relevant BCBs as dirty again, and than flush the data again using CcRepinBcb/CcUnpinRepinnedBcb APIs??? I assume that BCB is valid and locked till CcUnpinData is called.

Yes, you can CcSetDirtyPinnedData - CcRepinBcb/CcUnpinRepinnedBcb again.

>I think another way to do it, which I described in my previous e-mail, is to keep track of the dirty pages and lazy write them to the disk myself. I should lock the pages via CcMapData and CcPinMappedData APIs, and keep them locked

You use phrase “lock the pages”, but in Windows world this means- “resedent pages”. CcPinMappedData doesn’t lock the page frames. CcPinMappedData only guarantees that the portion of the mapped file will be mapped untill CcUnpinData. The Cache Manager or the Memory Manager may decide to flush the pages mapping the file if there is a page frames shortage in the system, so you will have to cope with this write requests for which you are not originator.

wrote in message news:xxxxx@ntfsd…
Slava,

thanks for you reply. While returning the STATUS_FILE_LOCK_CONFLICT processing the lazy-writer threads is an interesting option, it is not really applicable in my case. My goal here is to keep data in shared memory even after a successful lazy-writer request is complete, and if neccessary, mark this shared data as dirty again and force the CM to requeue the lazy-writer thread.

A call to CcRepinBcb/CcUnpinRepinnedBcb APIs will trigger a lazy-writer thread to flush the data described by BCB to disk. Obviously the data had to be marked as dirty previously via CcSetDirtyPinnedData API. Note that even if lazy-writer threads complete the operation successfully, let’s assume they always do, it is still possible that the data will not get committed on the server side - this is similar to client side “write behind failed” error/message. If I find out that the server side write behind failed, can I mark the relevant BCBs as dirty again, and than flush the data again using CcRepinBcb/CcUnpinRepinnedBcb APIs??? I assume that BCB is valid and locked till CcUnpinData is called.

I think another way to do it, which I described in my previous e-mail, is to keep track of the dirty pages and lazy write them to the disk myself. I should lock the pages via CcMapData and CcPinMappedData APIs, and keep them locked till I receive a confirmation from the server that the data was committed. I will use the returned buffer from CcMapData to modify the data in-memory; and asynchronously write it to the server using my worker (lazy-writer) thread. The downside with this method is that I may loose all optimizations that CM already has built in. e.g. small file writes.

Any thoughts?

Thanks,

Ilya.

-------------- Original message --------------
From: “Slava Imameyev”

> Q. Is it possible to assynchroneously process the lazy-writer threads, and notify the CM that the specified page is not dirty any more only after it is committed to disk?

I think you want to avoid the popup message “write behind failed”.
The page frames are marked as not dirty before write and if the write failed the page frames are marked again as dirty. If the lazy writer fails to write pages the system creates the popup msg “write behind failed”, but there is one exception - if STATUS_FILE_LOCK_CONFLICT is returned the lazy writer thread requeues the request and will repeat an attempt to write pages from the shared cache map. So simply return STATUS_FILE_LOCK_CONFLICT.

>Some time later I force the flush of these BCBs using the CcRepinBcb/CcUnpinRepinnedBcb

If you set WriteThrough to TRUE, then you mark the page frames as dirty and calls MmFlushSection.
And again, if STATUS_FILE_LOCK_CONFLICT is returned the system will repeat an attempt to write data from the lazy writer thread.
Also, CcSetDirtyPinnedData doesn’t mark the page frames as dirty - it marks the BCB as dirty.

>I should lock the pages via CcMapData and CcPinMappedData

This APIs don’t lock page frames associated with a file mapped in the cache!

wrote in message news:xxxxx@ntfsd…
Hi All,

I’ll start with a question and than follow with a more specific description of what I am trying to achieve.

Q. Is it possible to assynchroneously process the lazy-writer threads, and notify the CM that the specified page is not dirty any more only after it is committed to disk?

I am writting a netwrok file system that has both client and server side caching. Even after the client side cache is written to the server I want to keep it around till I receive a confirmation from the server that data was committed to disk. In case of a server side error I want to re-send the client data.

Here is what I do:

On a client side I utilize the Cc pinning interface to cache the user writes. I map and pin the write buffers processing the user write requests using the CcMapData and CcPinMappedData Apis. I mark the data as dirty using CcSetDirtyPinnedData, and I add the resulting BCB to the FCB link list.

Some time later I force the flush of these BCBs using the CcRepinBcb/CcUnpinRepinnedBcb, which writes out all data that was previously was marked as dirty to the server. Next, if I receive notification that the server encountered an error committing the data to disk, I want to mark the data as dirty again via CcSetDirtyPinnedData, but it does not seem to work.

Here is amore simplified description of above using a sequence of API calls:

In FsdWrite thread:
1. CcMapData
2. CcPinMappedData
3. CcSetDirtyPinnedData
4. Add BCB to the link list of FCB and Return

In a worker thread per each BCB in the link list of FCB:
1. CcRepinBcb
2. CcUnpinRepinnedBcb - this triggers the write of dirty data
…
3. Send message to the server to commit the data to the disk - in case of an error do the following.
4. CcSetDirtyPinnedData, and than repeat steps 1-3. However, the problem is that I do not see any writes going through when I call CcRepinBcb/CcUnpinRepinnedBcb Apis next time, which is at the core of my problem.

Thanks in advance for all your responses.

-Ilya.

—
Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

—
Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

—
Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com