OT -- Re: Paged or Non Paged ???

NTDEV
1

Okay!

Now I got the context …

Before I said " It is true that sys file goes to pagefile" in response
to Max’s reply, I briefly read the NT insider’s article and it even says
that the pagefile is named pagefile.sys ( or something by default ).
Would not go in this direction ( sorry for being lazy now ).

Then Jake’s answer was in that context, but reading just that stmt in
isolation made me to belief, wow may be other sys files are using
themselves for backing store … Peter W. embolden and got me into right
path. Thanks to you all.

What was back in my head ???

Very simple, if there are hundreds of sys files, totaling 100+ MB,
what is the point of enlarging the pagefile. Not that it is a helpful
information to me right now, and disk space is usually big, but then
again …

-pro

Prokash Sinha wrote:

Thanks Peter. Seems like I did not parse well Jake’s stmt :-).

So just an arbit question, for sys files that are loaded after Mm is
active, what are the other reasons that they have to be paged out ( if
pageable sections presents) to pagefile. Or is it just a mere
convenience to have them all use the pagefile as backing store ?

-pro

Peter Wieland wrote:
>
> You assume incorrectly. Drivers are backed by the page file, not the
> original on-disk file, no matter when they’re loaded.
>
>
>
> -p
>
>
>
> *From:* xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com] *On Behalf Of *Prokash Sinha
> *Sent:* Wednesday, April 29, 2009 4:51 PM
> *To:* Windows System Software Devs Interest List
> *Subject:* Re: [ntdev] Paged or Non Paged ???
>
>
>
> Thanks a bunch… That clarifies stuff.
>
>
> From your stmt, Jake, now I assume that any driver that are loaded
> much later in the system bring up ( meaning the Mm is active now )
> phase are indeed not using pagefile !!!.
>
>
>
> -pro
>
> On Wed, Apr 29, 2009 at 4:24 PM, Jake Oshins
> >
>> wrote:
>>
>> The answer is simpler than you might expect. The memory manager can
>> only page code from its original image if it knows where that image
>> is. Boot drivers (which are the ones which are paged to the page
>> file) are put into memory by the bootloader. Thus the memory manager
>> has no idea where they came from on disk.
>>
>> –
>> Jake Oshins
>> Hyper-V I/O Architect
>> Windows Kernel Group
>>
>> This post implies no warranties and confers no rights.
>>
>> --------------------------------------------------------------
>>
>> “Prokash Sinha” >
>> wrote in message news:xxxxx@ntdev…
>>
>>
>> Yes it is true that sys file goes to pagefile …
>>
>>
>> Now the most non-obvious question is why they are jumbled together
>> into pagefile ???
>>
>>
>> If I’ve to guess, then (1) because they are relatively small in size
>> (2) security (3) what else ???
>>
>>
>> There are bits and pieces of informations here or there about this
>> whole business of ( image format, in core process structure, who
>> loads, paged vs. non-paged, privileges and all sorts )…
>>
>>
>> In the past I’ve seen two articles on Mm & VA on insider - nicely
>> written, could be nicer for newb ( and very very oldb, like me :).
>> Something that would match with the general concepts one gets out of
>> college ( and does not delve into it on a daily basis, but sometime
>> need to get on with it ). This is exactly the
>> poor-soul-from-the-driver-land needs !!!
>>
>>
>> -pro
>>
>>
>> On Wed, Apr 29, 2009 at 1:56 PM, Maxim S. Shatskih
>> > wrote:
>>
>> Read something about NT file ( including image) mapping …
>>
>>
>>
>> User and kernel mode image mapping do differs a lot.
>>
>> User mapped image holds the file open, and pages in the pages from
>> the image file itself on faults.
>>
>> Kernel driver does not hold the .SYS file open, and pages the pages
>> in from the pagefile on faults.
>>
>> –
>>
>> Maxim S. Shatskih
>> Windows DDK MVP
>> xxxxx@storagecraft.com mailto:xxxxx
>> http://www.storagecraft.com
>>
>>
>> —
>>
>> NTDEV is sponsored by OSR
>>
>> For our schedule of WDF, WDM, debugging and other seminars visit:
>> http://www.osr.com/seminars
>>
>> To unsubscribe, visit the List Server section of OSR Online at
>> http://www.osronline.com/page.cfm?name=ListServer
>>
>> —
>> NTDEV is sponsored by OSR
>>
>> For our schedule of WDF, WDM, debugging and other seminars visit:
>> http://www.osr.com/seminars
>>
>> To unsubscribe, visit the List Server section of OSR Online at
>> http://www.osronline.com/page.cfm?name=ListServer
>>
>>
>>
>> — NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging
>> and other seminars visit: http://www.osr.com/seminars To unsubscribe,
>> visit the List Server section of OSR Online at
>> http://www.osronline.com/page.cfm?name=ListServer
>>
>>
>> —
>> NTDEV is sponsored by OSR
>>
>> For our schedule of WDF, WDM, debugging and other seminars visit:
>> http://www.osr.com/seminars
>>
>> To unsubscribe, visit the List Server section of OSR Online at
>> http://www.osronline.com/page.cfm?name=ListServer
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
></mailto:xxxxx>

2

Thanks Peter,

That is a good reason ( i.e not to lock a driver ), for online update
etc. And IIRC, back in 1996-98 time frame, driver.sys file used to be
locked when it was loaded and running, myself just started using windbg
around that time.

-pro

xxxxx@osr.com wrote:

I know that at one time, drivers were always backed by the page file. This was implemented to allow the driver to be replaced while it was running, without having to stop the device.

Before this was implemented, the damn driver.sys file was locked until the driver was stopped. Made it a PITA for debugging… though I suspect it was actually implemented to allow drivers to take updates more easily.

Peter
OSR

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

3

>OK, since we’re playing “guess the issue”, here’s my guess, My guess

is that your PNACSLog structure is some kind of a buffer that contains
log messages, and you are writing off the end of that structure and
trashing the KSPIN_LOCK.

Why don’t you show us the definition of PNACSLog?

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

static struct
{
PNACSLog data ;
KSPIN_LOCK guard ;
ULONG head ;
ULONG tail ;
HANDLE file ;
HANDLE write_thread ;
HANDLE timer_thread ;
KEVENT write_event ;
KEVENT timer_event;
BOOLEAN b_exit ;
} g_queue ;

PNACSLog data;

Here data is a pointer and its memory got allocated dynamically (from non paged pool ) .its got allocated in DriverEntry routine.Since this memory is from non paged pool (not from non paged system memory),there is no question of , writing off the end of that structure and
trashing the KSPIN_LOCK(its in non-paged system memory).

Here is NACSLog strucute (excuse me ,because i changed the variable names)

typedef struct Log
{

TSSDate variable_name1 ;
TSSTime Variable_name2 ;
ULONG Variable_name3 ;
USHORT Variable_name4 ;
ULONG Variable_name5 ;
USHORT Variable_name6 ;
char Variable_name7 ;
USHORT Variable_name8 ;

} *PNACSLog, NACSLog;

Here is TSSDate structure

typedef struct TSSDate
{
unsigned short Variable_name1;
unsigned short Variable_name2;
unsigned short Variable_name3;
} TSSDate, FAR * LPTSSDate ;

Here is TSSTime structure

typedef struct TSSTime
{
unsigned short Variable_name1 ;
unsigned short Variable_name2 ;
unsigned short Variable_name3 ;
} TSSTime, FAR * LPTSSTime ;

Thanks and Regards

Navaneeth Madahavan

4

Which is why you can delete them even when loaded.

mm

Peter Wieland wrote:

You assume incorrectly. Drivers are backed by the page file, not the
original on-disk file, no matter when they’re loaded.

-p

*From:* xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] *On Behalf Of *Prokash Sinha
*Sent:* Wednesday, April 29, 2009 4:51 PM
*To:* Windows System Software Devs Interest List
*Subject:* Re: [ntdev] Paged or Non Paged ???

Thanks a bunch… That clarifies stuff.

From your stmt, Jake, now I assume that any driver that are loaded much
later in the system bring up ( meaning the Mm is active now ) phase are
indeed not using pagefile !!!.

-pro

On Wed, Apr 29, 2009 at 4:24 PM, Jake Oshins
> wrote:
>
> The answer is simpler than you might expect. The memory manager can
> only page code from its original image if it knows where that image is.
> Boot drivers (which are the ones which are paged to the page file) are
> put into memory by the bootloader. Thus the memory manager has no idea
> where they came from on disk.
>
> –
> Jake Oshins
> Hyper-V I/O Architect
> Windows Kernel Group
>
> This post implies no warranties and confers no rights.
>
> --------------------------------------------------------------
>
> “Prokash Sinha” > wrote
> in message news:xxxxx@ntdev…
>
>
> Yes it is true that sys file goes to pagefile …
>
>
> Now the most non-obvious question is why they are jumbled together into
> pagefile ???
>
>
> If I’ve to guess, then (1) because they are relatively small in size (2)
> security (3) what else ???
>
>
> There are bits and pieces of informations here or there about this whole
> business of ( image format, in core process structure, who loads, paged
> vs. non-paged, privileges and all sorts )…
>
>
> In the past I’ve seen two articles on Mm & VA on insider - nicely
> written, could be nicer for newb ( and very very oldb, like me :).
> Something that would match with the general concepts one gets out of
> college ( and does not delve into it on a daily basis, but sometime need
> to get on with it ). This is exactly the poor-soul-from-the-driver-land
> needs !!!
>
>
> -pro
>
>
> On Wed, Apr 29, 2009 at 1:56 PM, Maxim S. Shatskih
> > wrote:
>
> Read something about NT file ( including image) mapping …
>
>
>
> User and kernel mode image mapping do differs a lot.
>
> User mapped image holds the file open, and pages in the pages from the
> image file itself on faults.
>
> Kernel driver does not hold the .SYS file open, and pages the pages in
> from the pagefile on faults.
>
> –
>
> Maxim S. Shatskih
> Windows DDK MVP
> xxxxx@storagecraft.com mailto:xxxxx
> http://www.storagecraft.com
>
>
> —
>
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
>
>
> — NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging
> and other seminars visit: http://www.osr.com/seminars To unsubscribe,
> visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
></mailto:xxxxx>

5

I remember that when I added event logging capability to the driver and
if any event was generated, I had issues in replacing driver binary.
I might be wrong.

Harish

-----Original Message-----
From: xxxxx@osr.com [mailto:xxxxx@osr.com]
Sent: Wednesday, April 29, 2009 8:11 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Paged or Non Paged ???

I know that at one time, drivers were always backed by the page file.
This was implemented to allow the driver to be replaced while it was
running, without having to stop the device.

Before this was implemented, the damn driver.sys file was locked until
the driver was stopped. Made it a PITA for debugging… though I
suspect it was actually implemented to allow drivers to take updates
more easily.

Peter
OSR

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

6

Remember that MM does some other strange things with driver images, like locking portions of them into memory, allowing drivers to page themselves out, discarding portions of them after DriverEntry has completed, map them before paging has been turned on, etc…

I don’t know for certain, but I suspect backing kernel images by the page file makes a number of things like this easier to manage.

-p

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Prokash Sinha
Sent: Wednesday, April 29, 2009 8:44 PM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] Paged or Non Paged ???

Thanks Peter,

That is a good reason ( i.e not to lock a driver ), for online update
etc. And IIRC, back in 1996-98 time frame, driver.sys file used to be
locked when it was loaded and running, myself just started using windbg
around that time.

-pro

xxxxx@osr.com wrote:

I know that at one time, drivers were always backed by the page file. This was implemented to allow the driver to be replaced while it was running, without having to stop the device.

Before this was implemented, the damn driver.sys file was locked until the driver was stopped. Made it a PITA for debugging… though I suspect it was actually implemented to allow drivers to take updates more easily.

Peter
OSR

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

7

That’s different. When you register your driver’s resources with the event logging service(s) then the service has to map your binary into its user-mode address space to access those resources and format the events. User-mode mappings are backed by the original file, not the page file, and so will prevent your driver from being removed.

-p

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Arora, Harish
Sent: Thursday, April 30, 2009 10:07 AM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] Paged or Non Paged ???

I remember that when I added event logging capability to the driver and
if any event was generated, I had issues in replacing driver binary.
I might be wrong.

Harish

-----Original Message-----
From: xxxxx@osr.com [mailto:xxxxx@osr.com]
Sent: Wednesday, April 29, 2009 8:11 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Paged or Non Paged ???

I know that at one time, drivers were always backed by the page file.
This was implemented to allow the driver to be replaced while it was
running, without having to stop the device.

Before this was implemented, the damn driver.sys file was locked until
the driver was stopped. Made it a PITA for debugging… though I
suspect it was actually implemented to allow drivers to take updates
more easily.

Peter
OSR

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

8

Thanks once again Peter. Now my foggy brain is shaping up :-). I knew there
are some reasons to it :-).

-pro

On Thu, Apr 30, 2009 at 10:23 AM, Peter Wieland <
xxxxx@windows.microsoft.com> wrote:

Remember that MM does some other strange things with driver images, like
locking portions of them into memory, allowing drivers to page themselves
out, discarding portions of them after DriverEntry has completed, map them
before paging has been turned on, etc…

I don’t know for certain, but I suspect backing kernel images by the page
file makes a number of things like this easier to manage.

-p

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:
xxxxx@lists.osr.com] On Behalf Of Prokash Sinha
Sent: Wednesday, April 29, 2009 8:44 PM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] Paged or Non Paged ???

Thanks Peter,

That is a good reason ( i.e not to lock a driver ), for online update
etc. And IIRC, back in 1996-98 time frame, driver.sys file used to be
locked when it was loaded and running, myself just started using windbg
around that time.

-pro

xxxxx@osr.com wrote:
> [quote]
> what are the other reasons that they have to be paged out ( if pageable
sections presents) to pagefile
> [/quote]
>
> I know that at one time, drivers were always backed by the page file.
This was implemented to allow the driver to be replaced while it was
running, without having to stop the device.
>
> Before this was implemented, the damn driver.sys file was locked until
the driver was stopped. Made it a PITA for debugging… though I suspect it
was actually implemented to allow drivers to take updates more easily.
>
> Peter
> OSR
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
>
>

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

9

>if any event was generated, I had issues in replacing driver binary.

If you have Event Viewer opened - then yes, since advapi32’s event log reading paths load the EventMessageFile as a datafile DLL to get to the .MC resource.

Close Event Viewer and all will be fine.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com