64-bit issues/handles

NTFSD
1

I’ve got a mini-filter which calls PsSetCreateProcessNotifyRoutine() to set
up a callback to monitor process start/stop.

The callback header is:

VOID
(*PCREATE_PROCESS_NOTIFY_ROUTINE) (
IN HANDLE ParentId,
IN HANDLE ProcessId,
IN BOOLEAN Create
);

Key issue here is the ‘handle’ to specify process id values. Handles are
pointers. Process ids are, as far as I know, integers in the range 0 (idle),
4, 8, A, etc. to ???. (Perhaps they’re added to the base address of a table
to generate real pointers…)

When porting the mini-filter to AMD64, the handle variable changes size. My
interface code casts this information to a ULONG which doesn’t change. The
compiler gripes about this.

I’m not at all sure I want to use a 64-bit interface for something that as
far as I know never gets much bigger than 4,000 or so.

Question: Is there a paper somewhere that will tell me more than I ever
wanted to know about process ids? What’s the largest process id that will
ever exist? Will it fit in 32 bits? Will there be 32-bit plus ids on 64-bit
systems?

Regards,
Mickey.

2

AFAIK the value will wrap at HANDLE size. I’m not sure where you get your
4000 estimate, my workstation easily passes that in a day, and I know of
servers that have been up for years! Use a ULONG_PTR and be done with it.

–
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Remove StopSpam from the email to reply

“Mickey Lane” wrote in message news:xxxxx@ntfsd…
I’ve got a mini-filter which calls PsSetCreateProcessNotifyRoutine() to set
up a callback to monitor process start/stop.

The callback header is:

VOID
(*PCREATE_PROCESS_NOTIFY_ROUTINE) (
IN HANDLE ParentId,
IN HANDLE ProcessId,
IN BOOLEAN Create
);

Key issue here is the ‘handle’ to specify process id values. Handles are
pointers. Process ids are, as far as I know, integers in the range 0 (idle),
4, 8, A, etc. to ???. (Perhaps they’re added to the base address of a table
to generate real pointers…)

When porting the mini-filter to AMD64, the handle variable changes size. My
interface code casts this information to a ULONG which doesn’t change. The
compiler gripes about this.

I’m not at all sure I want to use a 64-bit interface for something that as
far as I know never gets much bigger than 4,000 or so.

Question: Is there a paper somewhere that will tell me more than I ever
wanted to know about process ids? What’s the largest process id that will
ever exist? Will it fit in 32 bits? Will there be 32-bit plus ids on 64-bit
systems?

Regards,
Mickey.

3

It’s a HANDLE. It’s opaque for a reason. The only use other than passing
it into the OS is testing for equality to see if two instances match. Any
other use is asking for trouble.

Why do you think you need to cast it to any other type?

If you insist on doing something this bug-inviting, at least cast it to a
ULONG_PTR, which is *supposed* to be the size of a pointer on all
platforms. Again, why do you think you need to do this?

Phil

Philip D. Barila
Seagate Technology LLC
(720) 684-1842

“Mickey Lane”
net> To
Sent by: “Windows File Systems Devs Interest
bounce-214402-739 List”
xxxxx@lists.osr.com cc
No Phone Info
Available Subject
[ntfsd] 64-bit issues/handles

07/19/2005 11:49
AM

Please respond to
“Windows File
Systems Devs
Interest List”
com>

I’ve got a mini-filter which calls PsSetCreateProcessNotifyRoutine() to set
up a callback to monitor process start/stop.

The callback header is:

VOID
(*PCREATE_PROCESS_NOTIFY_ROUTINE) (
IN HANDLE ParentId,
IN HANDLE ProcessId,
IN BOOLEAN Create
);

Key issue here is the ‘handle’ to specify process id values. Handles are
pointers. Process ids are, as far as I know, integers in the range 0
(idle),
4, 8, A, etc. to ???. (Perhaps they’re added to the base address of a
table
to generate real pointers…)

When porting the mini-filter to AMD64, the handle variable changes size. My
interface code casts this information to a ULONG which doesn’t change. The
compiler gripes about this.

I’m not at all sure I want to use a 64-bit interface for something that as
far as I know never gets much bigger than 4,000 or so.

Question: Is there a paper somewhere that will tell me more than I ever
wanted to know about process ids? What’s the largest process id that will
ever exist? Will it fit in 32 bits? Will there be 32-bit plus ids on 64-bit
systems?

Regards,
Mickey.

—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

4

Sorry, but process and thread IDs are handles, not USHORT or ULONG.
When the OS is given a process/thread ID and needs the corresponding
structure, it looks them up in a handle table.

The largest process ID that could ever exist is going to be bounded by
the largest possible handle table.

Pragmatically, you are unlikely to see process or thread IDs that
overflow 16 bits, but it is allowed within the current model. You
should change your interface code to take ULONG_PTR or HANDLE data
types. Either prohibit returning the wrong size (by checking if this is
a 32 bit process) or explicitly translate between them, keeping in mind
you might (theoretically) see a PID that doesn’t fit in a ULONG.

Hmm. That suggests an interesting verifier check to me - instead of
returning low numbered handles, the handle table package could return
HIGH numbered values (like (Handle ^ ~0)) to the caller to make sure
that the caller properly deals with really big system configurations
(and if you think that’s impossible, I’m already bumping into people
with 1TB of physical memory - exactly the kind of system where we might
see very large numbers of handles).

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Mickey Lane
Sent: Tuesday, July 19, 2005 1:50 PM
To: ntfsd redirect
Subject: [ntfsd] 64-bit issues/handles

I’ve got a mini-filter which calls PsSetCreateProcessNotifyRoutine() to
set
up a callback to monitor process start/stop.

The callback header is:

VOID
(*PCREATE_PROCESS_NOTIFY_ROUTINE) (
IN HANDLE ParentId,
IN HANDLE ProcessId,
IN BOOLEAN Create
);

Key issue here is the ‘handle’ to specify process id values. Handles are
pointers. Process ids are, as far as I know, integers in the range 0
(idle),
4, 8, A, etc. to ???. (Perhaps they’re added to the base address of a
table
to generate real pointers…)

When porting the mini-filter to AMD64, the handle variable changes size.
My
interface code casts this information to a ULONG which doesn’t change.
The
compiler gripes about this.

I’m not at all sure I want to use a 64-bit interface for something that
as
far as I know never gets much bigger than 4,000 or so.

Question: Is there a paper somewhere that will tell me more than I ever
wanted to know about process ids? What’s the largest process id that
will
ever exist? Will it fit in 32 bits? Will there be 32-bit plus ids on
64-bit
systems?

Regards,
Mickey.

Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: unknown lmsubst tag argument:
‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

5

Well…

The WIN32 function OpenProcess() takes a process ID as it’s 3rd argument.
It’s specified as a DWORD or ‘unsigned long’.

The kernel callback routine provides two 32/64 PVOID handles as process ids.

Up until today, my filter code would send the two process ids up to user
space as ULONGs where they would eventually be passed into OpenProcess().

This gives us 32/64 bits on the kernel end and 32 bits on the use end…

I understand opaque. I don’t really care what the value means. I just want
to know what the rules are so I can deal with stuffing 32/64 bits into a
32-bit only hole.

Mickey.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@seagate.com
Sent: Tuesday, July 19, 2005 2:11 PM
To: Windows File Systems Devs Interest List
Subject: Re: [ntfsd] 64-bit issues/handles

It’s a HANDLE. It’s opaque for a reason. The only use other than passing
it into the OS is testing for equality to see if two instances match. Any
other use is asking for trouble.

Why do you think you need to cast it to any other type?

If you insist on doing something this bug-inviting, at least cast it to a
ULONG_PTR, which is *supposed* to be the size of a pointer on all platforms.
Again, why do you think you need to do this?

Phil

Philip D. Barila
Seagate Technology LLC
(720) 684-1842

“Mickey Lane”
net> To
Sent by: “Windows File Systems Devs Interest
bounce-214402-739 List”
xxxxx@lists.osr.com cc
No Phone Info
Available Subject
[ntfsd] 64-bit issues/handles

07/19/2005 11:49
AM

Please respond to
“Windows File
Systems Devs
Interest List”
com>

I’ve got a mini-filter which calls PsSetCreateProcessNotifyRoutine() to set
up a callback to monitor process start/stop.

The callback header is:

VOID
(*PCREATE_PROCESS_NOTIFY_ROUTINE) (
IN HANDLE ParentId,
IN HANDLE ProcessId,
IN BOOLEAN Create
);

Key issue here is the ‘handle’ to specify process id values. Handles are
pointers. Process ids are, as far as I know, integers in the range 0 (idle),
4, 8, A, etc. to ???. (Perhaps they’re added to the base address of a table
to generate real pointers…)

When porting the mini-filter to AMD64, the handle variable changes size. My
interface code casts this information to a ULONG which doesn’t change. The
compiler gripes about this.

I’m not at all sure I want to use a 64-bit interface for something that as
far as I know never gets much bigger than 4,000 or so.

Question: Is there a paper somewhere that will tell me more than I ever
wanted to know about process ids? What’s the largest process id that will
ever exist? Will it fit in 32 bits? Will there be 32-bit plus ids on 64-bit
systems?

Regards,
Mickey.

—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@earthlink.net To unsubscribe
send a blank email to xxxxx@lists.osr.com

6

BTW, an interesting observation: You would think that
you can straight up compare handle values. At least
with handles passed to registry functions they change
some times. It turns out that when the system looks
up a handle it ignores the first two bits of the
handle value. So, to the system, handles with values
of 4, 5, 6, and 7 are equivalent. If you watch the
handles handed back from Open and Create and compare
with those passed to Close they are different
sometimes (usually by 2)!

Boggled my mind for a long time…

It seemed that this happened more often with MS apps,
so I wonder if they know that these bits aren’t used
and are actually using them for some purpose…don’t
know.

— xxxxx@seagate.com wrote:
…snip…

is testing for equality to see if two
instances match.

7

If the HANDLE you get in KM is indeed the DWORD you use in KM, with no
modification, that’s an unfortunate OS design defect.

But I do see why you would need to cast it, if that’s the case. Perhaps
there is another means of relating the process HANDLE to the process ID?
One which is less size dependent?

Phil

Philip D. Barila
Seagate Technology LLC
(720) 684-1842

“Mickey Lane”
net> To
Sent by: “Windows File Systems Devs Interest
bounce-214407-739 List”
xxxxx@lists.osr.com cc
No Phone Info
Available Subject
RE: [ntfsd] 64-bit issues/handles

07/19/2005 12:27
PM

Please respond to
“Windows File
Systems Devs
Interest List”
com>

Well…

The WIN32 function OpenProcess() takes a process ID as it’s 3rd argument.
It’s specified as a DWORD or ‘unsigned long’.

The kernel callback routine provides two 32/64 PVOID handles as process
ids.

Up until today, my filter code would send the two process ids up to user
space as ULONGs where they would eventually be passed into OpenProcess().

This gives us 32/64 bits on the kernel end and 32 bits on the use end…

I understand opaque. I don’t really care what the value means. I just want
to know what the rules are so I can deal with stuffing 32/64 bits into a
32-bit only hole.

Mickey.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@seagate.com
Sent: Tuesday, July 19, 2005 2:11 PM
To: Windows File Systems Devs Interest List
Subject: Re: [ntfsd] 64-bit issues/handles

It’s a HANDLE. It’s opaque for a reason. The only use other than passing
it into the OS is testing for equality to see if two instances match. Any
other use is asking for trouble.

Why do you think you need to cast it to any other type?

If you insist on doing something this bug-inviting, at least cast it to a
ULONG_PTR, which is supposed to be the size of a pointer on all
platforms.
Again, why do you think you need to do this?

Phil

Philip D. Barila
Seagate Technology LLC
(720) 684-1842

“Mickey Lane”
net> To
Sent by: “Windows File Systems Devs Interest
bounce-214402-739 List”
xxxxx@lists.osr.com cc
No Phone Info
Available Subject
[ntfsd] 64-bit issues/handles

07/19/2005 11:49
AM

Please respond to
“Windows File
Systems Devs
Interest List”
com>

I’ve got a mini-filter which calls PsSetCreateProcessNotifyRoutine() to set
up a callback to monitor process start/stop.

The callback header is:

VOID
(*PCREATE_PROCESS_NOTIFY_ROUTINE) (
IN HANDLE ParentId,
IN HANDLE ProcessId,
IN BOOLEAN Create
);

Key issue here is the ‘handle’ to specify process id values. Handles are
pointers. Process ids are, as far as I know, integers in the range 0
(idle),
4, 8, A, etc. to ???. (Perhaps they’re added to the base address of a
table
to generate real pointers…)

When porting the mini-filter to AMD64, the handle variable changes size. My
interface code casts this information to a ULONG which doesn’t change. The
compiler gripes about this.

I’m not at all sure I want to use a 64-bit interface for something that as
far as I know never gets much bigger than 4,000 or so.

Question: Is there a paper somewhere that will tell me more than I ever
wanted to know about process ids? What’s the largest process id that will
ever exist? Will it fit in 32 bits? Will there be 32-bit plus ids on 64-bit
systems?

Regards,
Mickey.

—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@earthlink.net To
unsubscribe
send a blank email to xxxxx@lists.osr.com

—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

8

OK, so even comparing for equality isn’t a valid use of them, apparently.
Thanks for the posting the warning about the quicksand.

Phil

Philip D. Barila
Seagate Technology LLC
(720) 684-1842

Randy Cook
.com> To
Sent by: “Windows File Systems Devs Interest
bounce-214409-739 List”
xxxxx@lists.osr.com cc
No Phone Info
Available Subject
Re: [ntfsd] 64-bit issues/handles

07/19/2005 12:36
PM

Please respond to
“Windows File
Systems Devs
Interest List”
com>

BTW, an interesting observation: You would think that
you can straight up compare handle values. At least
with handles passed to registry functions they change
some times. It turns out that when the system looks
up a handle it ignores the first two bits of the
handle value. So, to the system, handles with values
of 4, 5, 6, and 7 are equivalent. If you watch the
handles handed back from Open and Create and compare
with those passed to Close they are different
sometimes (usually by 2)!

Boggled my mind for a long time…

It seemed that this happened more often with MS apps,
so I wonder if they know that these bits aren’t used
and are actually using them for some purpose…don’t
know.

— xxxxx@seagate.com wrote:
…snip…
> is testing for equality to see if two
> instances match.

—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@seagate.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

9

As I understand it, the callback *is* providing process ids and they’re
*prototyped* as PVOID handles.

I don’t do any sort of conversion - I just pass the bits.

My code is still very much under development with assorted bugs, etc. but
it’ll run for many hours doing hundreds of thousands of said operations with
no observed problems. If someone can point out a fault here, I’d be *real*
interested…

Mickey.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Philip D Barila
Sent: Tuesday, July 19, 2005 2:42 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] 64-bit issues/handles

If the HANDLE you get in KM is indeed the DWORD you use in KM, with no
modification, that’s an unfortunate OS design defect.

But I do see why you would need to cast it, if that’s the case. Perhaps
there is another means of relating the process HANDLE to the process ID? One
which is less size dependent?

Phil

Philip D. Barila
Seagate Technology LLC
(720) 684-1842

“Mickey Lane”
net> To
Sent by: “Windows File Systems Devs Interest
bounce-214407-739 List”
xxxxx@lists.osr.com cc
No Phone Info
Available Subject
RE: [ntfsd] 64-bit issues/handles

07/19/2005 12:27
PM

Please respond to
“Windows File
Systems Devs
Interest List”
com>

Well…

The WIN32 function OpenProcess() takes a process ID as it’s 3rd argument.
It’s specified as a DWORD or ‘unsigned long’.

The kernel callback routine provides two 32/64 PVOID handles as process ids.

Up until today, my filter code would send the two process ids up to user
space as ULONGs where they would eventually be passed into OpenProcess().

This gives us 32/64 bits on the kernel end and 32 bits on the use end…

I understand opaque. I don’t really care what the value means. I just want
to know what the rules are so I can deal with stuffing 32/64 bits into a
32-bit only hole.

Mickey.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@seagate.com
Sent: Tuesday, July 19, 2005 2:11 PM
To: Windows File Systems Devs Interest List
Subject: Re: [ntfsd] 64-bit issues/handles

It’s a HANDLE. It’s opaque for a reason. The only use other than passing
it into the OS is testing for equality to see if two instances match. Any
other use is asking for trouble.

Why do you think you need to cast it to any other type?

If you insist on doing something this bug-inviting, at least cast it to a
ULONG_PTR, which is supposed to be the size of a pointer on all platforms.
Again, why do you think you need to do this?

Phil

Philip D. Barila
Seagate Technology LLC
(720) 684-1842

“Mickey Lane”
net> To
Sent by: “Windows File Systems Devs Interest
bounce-214402-739 List”
xxxxx@lists.osr.com cc
No Phone Info
Available Subject
[ntfsd] 64-bit issues/handles

07/19/2005 11:49
AM

Please respond to
“Windows File
Systems Devs
Interest List”
com>

I’ve got a mini-filter which calls PsSetCreateProcessNotifyRoutine() to set
up a callback to monitor process start/stop.

The callback header is:

VOID
(*PCREATE_PROCESS_NOTIFY_ROUTINE) (
IN HANDLE ParentId,
IN HANDLE ProcessId,
IN BOOLEAN Create
);

Key issue here is the ‘handle’ to specify process id values. Handles are
pointers. Process ids are, as far as I know, integers in the range 0 (idle),
4, 8, A, etc. to ???. (Perhaps they’re added to the base address of a table
to generate real pointers…)

When porting the mini-filter to AMD64, the handle variable changes size. My
interface code casts this information to a ULONG which doesn’t change. The
compiler gripes about this.

I’m not at all sure I want to use a 64-bit interface for something that as
far as I know never gets much bigger than 4,000 or so.

Question: Is there a paper somewhere that will tell me more than I ever
wanted to know about process ids? What’s the largest process id that will
ever exist? Will it fit in 32 bits? Will there be 32-bit plus ids on 64-bit
systems?

Regards,
Mickey.

—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@earthlink.net To unsubscribe
send a blank email to xxxxx@lists.osr.com

—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@earthlink.net To unsubscribe
send a blank email to xxxxx@lists.osr.com

10

My 4,000 came from superficial observation of a lightly used system. I’ll
accept any alternative statement on the matter.

Mickey.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Don Burn
Sent: Tuesday, July 19, 2005 2:04 PM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] 64-bit issues/handles

AFAIK the value will wrap at HANDLE size. I’m not sure where you get your
4000 estimate, my workstation easily passes that in a day, and I know of
servers that have been up for years! Use a ULONG_PTR and be done with it.

–
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Remove StopSpam from the email to reply

11

From ntifs.h:

NTSYSCALLAPI
NTSTATUS
NTAPI
NtOpenProcess (
__out PHANDLE ProcessHandle,
__in ACCESS_MASK DesiredAccess,
__in POBJECT_ATTRIBUTES ObjectAttributes,
__in_opt PCLIENT_ID ClientId
);

My point: this is the OS function to open a process; Win32 is NOT the
OS, and if the Win32 developers decided to hard code their handle sizes
to 32 bits, that’s their issue. Then again, I’d have to check the
Platform SDK to see if this is different for 64 bit clients.

My point: don’t assume that just because something is done in a
particular way in Win32 that it bears any resemblance to what is going
on in Windows. Or perhaps it would be better to say “any similarity
between Win32 and Windows is purely coincidental…”

The bit-borrowing technique is one that was once (NT 3.1) very common -
it was used in the file system code for example. They would exploit the
knowledge that pool memory is allocated on 8-byte boundaries. It sounds
like they used a comparable trick in Win32.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Mickey Lane
Sent: Tuesday, July 19, 2005 2:27 PM
To: ntfsd redirect
Subject: RE: [ntfsd] 64-bit issues/handles

Well…

The WIN32 function OpenProcess() takes a process ID as it’s 3rd
argument.
It’s specified as a DWORD or ‘unsigned long’.

The kernel callback routine provides two 32/64 PVOID handles as process
ids.

Up until today, my filter code would send the two process ids up to user
space as ULONGs where they would eventually be passed into
OpenProcess().

This gives us 32/64 bits on the kernel end and 32 bits on the use end…

I understand opaque. I don’t really care what the value means. I just
want
to know what the rules are so I can deal with stuffing 32/64 bits into a
32-bit only hole.

Mickey.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@seagate.com
Sent: Tuesday, July 19, 2005 2:11 PM
To: Windows File Systems Devs Interest List
Subject: Re: [ntfsd] 64-bit issues/handles

It’s a HANDLE. It’s opaque for a reason. The only use other than
passing
it into the OS is testing for equality to see if two instances match.
Any
other use is asking for trouble.

Why do you think you need to cast it to any other type?

If you insist on doing something this bug-inviting, at least cast it to
a
ULONG_PTR, which is *supposed* to be the size of a pointer on all
platforms.
Again, why do you think you need to do this?

Phil

Philip D. Barila
Seagate Technology LLC
(720) 684-1842

“Mickey Lane”


net>
To
Sent by: “Windows File Systems Devs
Interest
bounce-214402-739 List”

xxxxx@lists.osr.com
cc
No Phone Info

Available
Subject
[ntfsd] 64-bit issues/handles

07/19/2005 11:49

AM

Please respond to

“Windows File

Systems Devs

Interest List”


com>

I’ve got a mini-filter which calls PsSetCreateProcessNotifyRoutine() to
set
up a callback to monitor process start/stop.

The callback header is:

VOID
(*PCREATE_PROCESS_NOTIFY_ROUTINE) (
IN HANDLE ParentId,
IN HANDLE ProcessId,
IN BOOLEAN Create
);

Key issue here is the ‘handle’ to specify process id values. Handles are
pointers. Process ids are, as far as I know, integers in the range 0
(idle),
4, 8, A, etc. to ???. (Perhaps they’re added to the base address of a
table
to generate real pointers…)

When porting the mini-filter to AMD64, the handle variable changes size.
My
interface code casts this information to a ULONG which doesn’t change.
The
compiler gripes about this.

I’m not at all sure I want to use a 64-bit interface for something that
as
far as I know never gets much bigger than 4,000 or so.

Question: Is there a paper somewhere that will tell me more than I ever
wanted to know about process ids? What’s the largest process id that
will
ever exist? Will it fit in 32 bits? Will there be 32-bit plus ids on
64-bit
systems?

Regards,
Mickey.

—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: unknown lmsubst tag argument:
‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@earthlink.net To
unsubscribe
send a blank email to xxxxx@lists.osr.com

—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: unknown lmsubst tag argument:
‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

12

Tony Mason wrote:

My point: this is the OS function to open a process; Win32 is NOT the
OS, and if the Win32 developers decided to hard code their handle sizes
to 32 bits, that’s their issue. Then again, I’d have to check the
Platform SDK to see if this is different for 64 bit clients.

From the platform SDK I have installed (fairly recent… possibly not
the latest, but compiles win64):

WINBASEAPI
HANDLE
WINAPI
OpenProcess(
IN DWORD dwDesiredAccess,
IN BOOL bInheritHandle,
IN DWORD dwProcessId
);

This suggestes that this function is no longer useful in Win64 (or
someone needs to LART the Platform SDK developers).

A better solution is probably to use OpenProcessToken instead.

Tony

13

Most likely nobody will notice this issue because the practical reality
is that you are unlikely to see a system with more than 269 million
processes and threads (roughly the number required on a 64-bit system to
need more than the low order 32 bits of the handle) - that’s a 4GB
handle table, with 16 bytes per handle (8 bytes for the object, 4 bytes
for the access mask, 4 bytes for the inevitable padding to get to the
next aligned entry).

However, imagine that someone figures out a way to exploit the layout of
handles as a “security hole” (I’m not saying it is, I’m merely trying to
establish a semi-plausible scenario in which the OS team might decide to
change the handle table management in Windows). To thwart such an
effort, they might choose to randomize the handle table so that it uses
an arbitrary offset for the table. Now suddenly the process and thread
IDs are scattered throughout the entire namespace.

A constant struggle in this business is to deal with “what works” versus
“what is supported/allowed”. The fact that something works today does
not mean that doing something in a fashion that does not match the OS
architecture is advisable.

And, of course, when someone points to Win32, I point out that the Win32
people also brought us “Explorer” - one of the most bizarre application
programs any of us in the file systems space are ever likely to see…

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Tony Hoyle
Sent: Tuesday, July 19, 2005 7:34 PM
To: ntfsd redirect
Subject: Re: [ntfsd] 64-bit issues/handles

Tony Mason wrote:

My point: this is the OS function to open a process; Win32 is NOT the
OS, and if the Win32 developers decided to hard code their handle
sizes
to 32 bits, that’s their issue. Then again, I’d have to check the
Platform SDK to see if this is different for 64 bit clients.

From the platform SDK I have installed (fairly recent… possibly not
the latest, but compiles win64):

WINBASEAPI
HANDLE
WINAPI
OpenProcess(
IN DWORD dwDesiredAccess,
IN BOOL bInheritHandle,
IN DWORD dwProcessId
);

This suggestes that this function is no longer useful in Win64 (or
someone needs to LART the Platform SDK developers).

A better solution is probably to use OpenProcessToken instead.

Tony

Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

14

PsLookupProcessByProcessId will find you the EPROCESS pointer from the
process ID.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

----- Original Message -----
From: “Mickey Lane”
To: “Windows File Systems Devs Interest List”
Sent: Tuesday, July 19, 2005 9:49 PM
Subject: [ntfsd] 64-bit issues/handles

I’ve got a mini-filter which calls PsSetCreateProcessNotifyRoutine() to set
up a callback to monitor process start/stop.

The callback header is:

VOID
(*PCREATE_PROCESS_NOTIFY_ROUTINE) (
IN HANDLE ParentId,
IN HANDLE ProcessId,
IN BOOLEAN Create
);

Key issue here is the ‘handle’ to specify process id values. Handles are
pointers. Process ids are, as far as I know, integers in the range 0 (idle),
4, 8, A, etc. to ???. (Perhaps they’re added to the base address of a table
to generate real pointers…)

When porting the mini-filter to AMD64, the handle variable changes size. My
interface code casts this information to a ULONG which doesn’t change. The
compiler gripes about this.

I’m not at all sure I want to use a 64-bit interface for something that as
far as I know never gets much bigger than 4,000 or so.

Question: Is there a paper somewhere that will tell me more than I ever
wanted to know about process ids? What’s the largest process id that will
ever exist? Will it fit in 32 bits? Will there be 32-bit plus ids on 64-bit
systems?

Regards,
Mickey.

—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

15

Tony Hoyle wrote:

From the platform SDK I have installed (fairly recent… possibly not
the latest, but compiles win64):

WINBASEAPI
HANDLE
WINAPI
OpenProcess(
IN DWORD dwDesiredAccess,
IN BOOL bInheritHandle,
IN DWORD dwProcessId
);

This suggestes that this function is no longer useful in Win64 (or
someone needs to LART the Platform SDK developers).

While messing around with process ids and 32/64-bit differences, I happened
on an odd thing.

I made a little console program out of the enumerate-all-processes example
that’s been around for a while. (Uses EnumProcesses(), OpenProcess() and
EnumProcessModules().) I built it on a 32-bit machine and just copied the
executable to the 64-bit machine.

The user space OpenProcess() function seems to work the same in either
situation. On 32-bits, the EnumProcessModules() call will return
ERROR_PARTIAL_COPY once or twice. On 64-bits, it almost always returns the
error.

There doesn’t seem to be much info available about ERROR_PARTIAL_COPY.

What it boils down to is if your user space filter component currently gets
a list of running processes, it’ll break when you go 64-bit.

Regards,
Mickey.

16

Tony Mason comments:

Win32 is NOT the OS, and if the Win32 developers decided to
hard code their handle sizes to 32 bits, that’s their issue.

Just for the record:

ULONG
FltGetRequestorProcessId(
IN PFLT_CALLBACK_DATA CallbackData
);

The term ‘process id’ leads me to believe that the thing is a numeric
identifier. That it is sometimes expressed as a ‘handle’ would lead me to
believe that it is a pointer of some type. Since it seems to switch back and
forth between the two, I don’t know what to call it.

For the time being, I’m casting/truncating ids expressed as handles to 32
bit integers and sticking a note in the application documentation to that
effect.

In another note, Tony comments:

A constant struggle in this business is to deal with “what works”
versus “what is supported/allowed”. The fact that something works
today does not mean that doing something in a fashion that does
not match the OS architecture is advisable.

Very true. Unfortunately, ‘architectural integrity’ is usually not in a
project manager’s vocabulary. If it is, it’s hiding behind terms like
‘schedule’ and questions like “Is it done yet?”

Mickey.

17

That is a filter manager API bug; the appropriate people read the list
here and can decide if/how they wish to resolve this particular problem.
As I pointed out, it is not one that would show up under normal testing,
but could cause subtle issues for the very high end (read: “expensive
and with customers who get snippy when it doesn’t work”) machines.

Again, it argues for adding randomization to the handle table package in
order to find such lurking bugs.

Management teams that decide “get it done as fast as possible” is more
important than “get it done quicky, but don’t compromise on quality”
will ensure that the experienced developers in this field will have
gainful employment for many years into the future.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Mickey Lane
Sent: Wednesday, July 20, 2005 2:42 PM
To: ntfsd redirect
Subject: RE: [ntfsd] 64-bit issues/handles

Tony Mason comments:

Win32 is NOT the OS, and if the Win32 developers decided to
hard code their handle sizes to 32 bits, that’s their issue.

Just for the record:

ULONG
FltGetRequestorProcessId(
IN PFLT_CALLBACK_DATA CallbackData
);

The term ‘process id’ leads me to believe that the thing is a numeric
identifier. That it is sometimes expressed as a ‘handle’ would lead me
to
believe that it is a pointer of some type. Since it seems to switch back
and
forth between the two, I don’t know what to call it.

For the time being, I’m casting/truncating ids expressed as handles to
32
bit integers and sticking a note in the application documentation to
that
effect.

In another note, Tony comments:

A constant struggle in this business is to deal with “what works”
versus “what is supported/allowed”. The fact that something works
today does not mean that doing something in a fashion that does
not match the OS architecture is advisable.

Very true. Unfortunately, ‘architectural integrity’ is usually not in a
project manager’s vocabulary. If it is, it’s hiding behind terms like
‘schedule’ and questions like “Is it done yet?”

Mickey.

Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: unknown lmsubst tag argument:
‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

18

“Tony Mason” wrote:
>> Management teams that decide “get it done as fast as possible” is more
>> important than “get it done quicky, but don’t compromise on quality”
<< will ensure that the experienced developers in this field will have
>> gainful employment for many years into the future.

Tony,

I wish that was the case, unfortunately I am seeing the opposite in the
industry. Locally the two biggest storage employers in my area, want to pay
< $50/hr for a consultant, and have no interest in doing it right. Of
course they both will fire the consultant, blaming him when the bugs roll
in.

I know of WHQL’d shipping drivers, that will not last 5 seconds with
driver verifier turned on, and with driver verifier turned off PoolMon will
show the steady leak of memory. What makes this really bad is some of these
are DataCenter qualified!

It is refreshing when I encounter a customer who is excited that I
develop code that is clean under /W4, PREfast, and PC-Lint. Who is
appreciative that I debug under the check build with driver verifier and CUV
enabled, and who doesn’t look at me cross-eyed when I talk of using
code-coverage and having a code review.

Unfortunately, this is not the case for the most part, so I’ve watched
my consulting revenue shrink each year. If this crap doesn’t change I will
have to leave the industry since I need to make money to live.

–
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Remove StopSpam from the email to reply

19

Maybe those companies see how little they need to pay in other parts of the
world for developers, many of whom come here for answers to their questions.
Consulting is rapidly becoming a lost profession for us. I am glad I am out
of it now.

“Don Burn” wrote in message news:xxxxx@ntfsd…
> “Tony Mason” wrote:
>>> Management teams that decide “get it done as fast as possible” is more
>>> important than “get it done quicky, but don’t compromise on quality”
> << will ensure that the experienced developers in this field will have
>>> gainful employment for many years into the future.
>
> Tony,
>
> I wish that was the case, unfortunately I am seeing the opposite in
> the industry. Locally the two biggest storage employers in my area, want
> to pay < $50/hr for a consultant, and have no interest in doing it right.
> Of course they both will fire the consultant, blaming him when the bugs
> roll in.
>
> I know of WHQL’d shipping drivers, that will not last 5 seconds with
> driver verifier turned on, and with driver verifier turned off PoolMon
> will show the steady leak of memory. What makes this really bad is some
> of these are DataCenter qualified!
>
> It is refreshing when I encounter a customer who is excited that I
> develop code that is clean under /W4, PREfast, and PC-Lint. Who is
> appreciative that I debug under the check build with driver verifier and
> CUV enabled, and who doesn’t look at me cross-eyed when I talk of using
> code-coverage and having a code review.
>
> Unfortunately, this is not the case for the most part, so I’ve watched
> my consulting revenue shrink each year. If this crap doesn’t change I
> will have to leave the industry since I need to make money to live.
>
>
> –
> Don Burn (MVP, Windows DDK)
> Windows 2k/XP/2k3 Filesystem and Driver Consulting
> Remove StopSpam from the email to reply
>
>
>
>
>

20

David J. Craig wrote:

Consulting is rapidly becoming a lost profession for us.

For SOME of us, perhaps. From The NT Insider in 2003:

http://www.osronline.com/article.cfm?name=offshore.pdf&id=260

(free membership required)

Peter
OSR