RE: SPAM-LOW: RE: Referencing a RegKey Object from handle

NTDEV
1

Outside of debugging, the instantaneous reference count is worthless,
because it is not stable with respect to synchronization. And, the only
thing that you should be doing to a reference count is raising or lowering
it.

I do agree with your list of fallbacks. I’m not saying that undocumented
calls or structures should never be used – only that they should be a last
resort, and that the developer has a responsibility to mitigate the damage
this could cause.

– arlie

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Tony Mason
Sent: Monday, August 08, 2005 2:40 PM
To: Windows System Software Devs Interest List
Subject: SPAM-LOW: RE: [ntdev] Referencing a RegKey Object from handle

Not everything. For example, the reference count on the object is not
available. The usual trick I’ve seen used is to call
ObReferenceObject/ObDereferenceObject, which are documented as VOID
functions in the DDK but actually return LONG_PTR values (side note: I
actually logged a bug against the DDK documentation and was told that they
wouldn’t fix it because the developer didn’t want to tell people what the
meaning of the values returned by these functions was!)

Sometimes some of this information is obscured and results in bugs, which is
not a good thing (but not justification for using undocumented features).
For example, a general rule here is that inside a filter you should NOT call
ObReferenceObject and ObDereferenceObject in the IRP_MJ_CLEANUP handler
before passing the request to the underlying driver - that triggers the
IRP_MJ_CLOSE call!

But the bottom line is: if you can do it in a documented and supported
fashion, do it that way. If you have no choice, log a bug. If you
can’t wait for a bug fix, fix it and make damned sure you handle the failure
case gracefully (refuse to load, disable your functionality, log a nice
message, send an e-mail to yourself, whatever…)

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Arlie Davis
Sent: Monday, August 08, 2005 2:30 PM
To: ntdev redirect
Subject: Re:[ntdev] Referencing a RegKey Object from handle

But not zero, and you provide absolutely no protection for your clients
if/when the change does occur.

And again, you have not provided any evidence that you *need* direct access
to the object header. Everything in object header is exposed through
documented, supported APIs, all of which give an indication of what sort of
semantic change you want to occur (such as changing the reference count on
an object).

Do you think data encapsulation is a bad thing? Should the DDK just be a
list of #defines of variable addresses?

– arlie

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Maxim S.
Shatskih
Sent: Monday, August 08, 2005 2:15 PM
To: Windows System Software Devs Interest List
Subject: SPAM-LOW: Re: Re:[ntdev] Referencing a RegKey Object from handle

Any chances that OBJECT_HEADER will change? Very, very minor.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@osr.com To unsubscribe send
a blank email to xxxxx@lists.osr.com

Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

2

>> any chanche OBJECT_HEADER will change?

IIRC, it already did.

> Everything in object header is exposed through
> documented

Uhh. Not really , really no. There are things there which are not exposed
through any API in a object header.

And speaking about object headers, I still simply hate when MS is not
consitent with itself, and object manager philosophy, and you get in a
filter driver to see phony file objects whithout
a valid object header, and allocated on stacks :stuck_out_tongue:

>And again, you have not provided any evidence that you *need* direct
>access
>to the object header.

Sorry, but why should I convince you that I *need* access to it ? A question
was posted. If you know the answer , answer to it. Would you tell me the
format of a OBJECT_HEADER if you know it, or simply say: “Well … I know
it, but you dont need to know about it and its format. So ,
I wont tell you”. If this is so… why do you know its format in the first
place ? And second , why dont you share ?

Lately I see a lot of bull**** on many lists (not only here)on discussing
undocumented stuff. There was a time when ppl could discuss undocumented
aspects and techniques whithout jumping at each other , and accusing “you
are a malware writter”, “you dont need to know this” , and so on.
In fact, there where books written on subject, I recall in particular a book
about Windows 98 written by Matt Pietrek which had a lot of unorthodox stuff
in it.

> Should the DDK just be a list of #defines of variable addresses?

Hahahaha. You are funny. I dont think anybody said that wants the DDK that
way.

Dan

----- Original Message -----
From: “Arlie Davis”
To: “Windows System Software Devs Interest List”
Sent: Monday, August 08, 2005 9:30 PM
Subject: Re:[ntdev] Referencing a RegKey Object from handle

> But not zero, and you provide absolutely no protection for your clients
> if/when the change does occur.
>
> And again, you have not provided any evidence that you need direct
> access
> to the object header. Everything in object header is exposed through
> documented, supported APIs, all of which give an indication of what sort
> of
> semantic change you want to occur (such as changing the reference count on
> an object).
>
> Do you think data encapsulation is a bad thing? Should the DDK just be a
> list of #defines of variable addresses?
>
> – arlie
>
>
>
> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com] On Behalf Of Maxim S. Shatskih
> Sent: Monday, August 08, 2005 2:15 PM
> To: Windows System Software Devs Interest List
> Subject: SPAM-LOW: Re: Re:[ntdev] Referencing a RegKey Object from handle
>
> Any chances that OBJECT_HEADER will change? Very, very minor.
>
> Maxim Shatskih, Windows DDK MVP
> StorageCraft Corporation
> xxxxx@storagecraft.com
> http://www.storagecraft.com
>
>
>
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: xxxxx@rdsor.ro
> To unsubscribe send a blank email to xxxxx@lists.osr.com

3

A great many of the questions that show up on this list and on the
newsgroups about how system internals work are posted by developers who
have decided to write feature X a certain way, then stuck to that path
with pitbull like tenacity until they finally got to the point where it
wouldn’t work (depending their skill level that point can be pretty high
or pretty low). They they decide the only way to continue on this path
is to get some piece of information that’s not obviously available
through the DDI so they ask a very exact question (“How do I do X”)
instead of wondering if perhaps they went down a bad path to begin with.

And quite frequently when they back up and rethink their goal the
problem becomes much easier and their driver is better.

I’m sorry that this interferes with your abilty to learn more about the
OS. I’d love to work in a world where I we could share everything about
how our OS works without worrying that someone would (a) integrate that
knowledge into a competitor’s product, (b) build a better tool to break
it based on that info or (c) build a time-bomb of a driver to sell to
customers.

But until that happens I’m going to keep asking people why they want to
know what they’re asking. I’m going to continue to tell people not to
use undocumented APIs or data structures. And I’m going to have to
continue to be warry of how information I give out could be used for
evil. If you feel this is “bull****” I’ll have to live with that.

Sorry,
-p

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Dan Partelly
Sent: Monday, August 08, 2005 12:47 PM
To: Windows System Software Devs Interest List
Subject: Re: Re:[ntdev] Referencing a RegKey Object from handle

> any chanche OBJECT_HEADER will change?

IIRC, it already did.

> Everything in object header is exposed through documented

Uhh. Not really , really no. There are things there which are not
exposed through any API in a object header.

And speaking about object headers, I still simply hate when MS is not
consitent with itself, and object manager philosophy, and you get in a
filter driver to see phony file objects whithout a valid object header,
and allocated on stacks :stuck_out_tongue:

>And again, you have not provided any evidence that you *need* direct
>access to the object header.

Sorry, but why should I convince you that I *need* access to it ? A
question was posted. If you know the answer , answer to it. Would you
tell me the format of a OBJECT_HEADER if you know it, or simply say:
“Well … I know it, but you dont need to know about it and its format.
So , I wont tell you”. If this is so… why do you know its format in
the first place ? And second , why dont you share ?

Lately I see a lot of bull**** on many lists (not only here)on
discussing undocumented stuff. There was a time when ppl could discuss
undocumented aspects and techniques whithout jumping at each other , and
accusing “you are a malware writter”, “you dont need to know this” , and
so on.
In fact, there where books written on subject, I recall in particular a
book about Windows 98 written by Matt Pietrek which had a lot of
unorthodox stuff in it.

> Should the DDK just be a list of #defines of variable addresses?

Hahahaha. You are funny. I dont think anybody said that wants the DDK
that way.

Dan

----- Original Message -----
From: “Arlie Davis”
To: “Windows System Software Devs Interest List”
Sent: Monday, August 08, 2005 9:30 PM
Subject: Re:[ntdev] Referencing a RegKey Object from handle

> But not zero, and you provide absolutely no protection for your
clients
> if/when the change does occur.
>
> And again, you have not provided any evidence that you need direct
> access
> to the object header. Everything in object header is exposed through
> documented, supported APIs, all of which give an indication of what
sort
> of
> semantic change you want to occur (such as changing the reference
count on
> an object).
>
> Do you think data encapsulation is a bad thing? Should the DDK just
be a
> list of #defines of variable addresses?
>
> – arlie
>
>
>
> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com] On Behalf Of Maxim S.
Shatskih
> Sent: Monday, August 08, 2005 2:15 PM
> To: Windows System Software Devs Interest List
> Subject: SPAM-LOW: Re: Re:[ntdev] Referencing a RegKey Object from
handle
>
> Any chances that OBJECT_HEADER will change? Very, very minor.
>
> Maxim Shatskih, Windows DDK MVP
> StorageCraft Corporation
> xxxxx@storagecraft.com
> http://www.storagecraft.com
>
>
>
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: xxxxx@rdsor.ro
> To unsubscribe send a blank email to xxxxx@lists.osr.com

—
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

4

>>I’m sorry that this interferes with your abilty to learn more about the

>OS.

Im sorry , but all this has nothing to do with my ability to learn something
:stuck_out_tongue:
Fortunately, for me I can find my way inside your OS regardless you make
stuff
public or not.

>But until that happens I’m going to keep asking people why they want to
>know what they’re asking

You have 3 points illustrate here. 2 of them are purely your problem because
you are a Microsoft employee, and thus you have to obey company policy and
not reveal data which would be integrated into a “competitor product”, or a
"
better tool to break it" , I asume you talk about activation and XP keys
here …
but unfortunately the code which handle those tasks is what … 0.00000001 %
linked
with what ppl usually want to know about undocumented asspects of NT.

Ms could reaveal and expose a lot more APIs , and improve on existing one
whithout fearing that the data will be “integrated” into a competitor
product.
Let’s face it, there would be only some decalrations of structs and APIs.
Do you really think that a person determined to learn them really needs
official header to understand this ?

> build a time-bomb of a driver to sell to customers.

I dont want to be evil, and god is my whitness that I love MS’s operating
systems and products. I
apreciate their value. But did you heard about Sasser, MsSQL worm, Codered
and whatever ?
Isnt MS which sold a time bomb to his customers here ? Ugly weakneses and
“security through obscurity”
made this possible. So If Micorsoft doest it, time and again, why is so
concerned about 3rd party companies ?
Solve your problems first, and make sure security is not implemented through
obscurity , then youll have
less problems revealing more aspects of the OS.

I hope you will take this message as is, and not consider at it a attack to
your person, or MS’s products.

Sorry, Dan

----- Original Message -----
From: “Peter Wieland”
To: “Windows System Software Devs Interest List”
Sent: Monday, August 08, 2005 11:17 PM
Subject: RE: Re:[ntdev] Referencing a RegKey Object from handle

A great many of the questions that show up on this list and on the
newsgroups about how system internals work are posted by developers who
have decided to write feature X a certain way, then stuck to that path
with pitbull like tenacity until they finally got to the point where it
wouldn’t work (depending their skill level that point can be pretty high
or pretty low). They they decide the only way to continue on this path
is to get some piece of information that’s not obviously available
through the DDI so they ask a very exact question (“How do I do X”)
instead of wondering if perhaps they went down a bad path to begin with.

And quite frequently when they back up and rethink their goal the
problem becomes much easier and their driver is better.

I’m sorry that this interferes with your abilty to learn more about the
OS. I’d love to work in a world where I we could share everything about
how our OS works without worrying that someone would (a) integrate that
knowledge into a competitor’s product, (b) build a better tool to break
it based on that info or (c) build a time-bomb of a driver to sell to
customers.

But until that happens I’m going to keep asking people why they want to
know what they’re asking. I’m going to continue to tell people not to
use undocumented APIs or data structures. And I’m going to have to
continue to be warry of how information I give out could be used for
evil. If you feel this is “bull****” I’ll have to live with that.

Sorry,
-p

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Dan Partelly
Sent: Monday, August 08, 2005 12:47 PM
To: Windows System Software Devs Interest List
Subject: Re: Re:[ntdev] Referencing a RegKey Object from handle

>> any chanche OBJECT_HEADER will change?

IIRC, it already did.

>> Everything in object header is exposed through documented

Uhh. Not really , really no. There are things there which are not
exposed through any API in a object header.

And speaking about object headers, I still simply hate when MS is not
consitent with itself, and object manager philosophy, and you get in a
filter driver to see phony file objects whithout a valid object header,
and allocated on stacks :stuck_out_tongue:

>>And again, you have not provided any evidence that you need direct
>>access to the object header.

Sorry, but why should I convince you that I need access to it ? A
question was posted. If you know the answer , answer to it. Would you
tell me the format of a OBJECT_HEADER if you know it, or simply say:
“Well … I know it, but you dont need to know about it and its format.
So , I wont tell you”. If this is so… why do you know its format in
the first place ? And second , why dont you share ?

Lately I see a lot of bull**** on many lists (not only here)on
discussing undocumented stuff. There was a time when ppl could discuss
undocumented aspects and techniques whithout jumping at each other , and
accusing “you are a malware writter”, “you dont need to know this” , and
so on.
In fact, there where books written on subject, I recall in particular a
book about Windows 98 written by Matt Pietrek which had a lot of
unorthodox stuff in it.

>> Should the DDK just be a list of #defines of variable addresses?

Hahahaha. You are funny. I dont think anybody said that wants the DDK
that way.

Dan

----- Original Message -----
From: “Arlie Davis”
To: “Windows System Software Devs Interest List”
Sent: Monday, August 08, 2005 9:30 PM
Subject: Re:[ntdev] Referencing a RegKey Object from handle

> But not zero, and you provide absolutely no protection for your
clients
> if/when the change does occur.
>
> And again, you have not provided any evidence that you need direct
> access
> to the object header. Everything in object header is exposed through
> documented, supported APIs, all of which give an indication of what
sort
> of
> semantic change you want to occur (such as changing the reference
count on
> an object).
>
> Do you think data encapsulation is a bad thing? Should the DDK just
be a
> list of #defines of variable addresses?
>
> – arlie
>
>
>
> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com] On Behalf Of Maxim S.
Shatskih
> Sent: Monday, August 08, 2005 2:15 PM
> To: Windows System Software Devs Interest List
> Subject: SPAM-LOW: Re: Re:[ntdev] Referencing a RegKey Object from
handle
>
> Any chances that OBJECT_HEADER will change? Very, very minor.
>
> Maxim Shatskih, Windows DDK MVP
> StorageCraft Corporation
> xxxxx@storagecraft.com
> http://www.storagecraft.com
>
>
>
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: xxxxx@rdsor.ro
> To unsubscribe send a blank email to xxxxx@lists.osr.com

—
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

—
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

5

> Ms could reaveal and expose a lot more APIs ,

Then they will have to give up the rights of changing
it without worrying about backward compatibility.

Calvin Guan (Windows DDK MVP)
NetXtreme Longhorn Miniport Prime
Broadcom Corp. www.broadcom.com

Find your next car at http://autos.yahoo.ca

6

I agree Windows is a time bomb by itself. I have two drivers which were
broken by Win2003 SP1, one because of a newly added bug in classpnp.sys,
another one due to screwed SP1 DDK headers for 1394. Longhorn/Vista is going
to break a lot more software than all undocumented features together.

Of course it’s Microsoft’s fault that people use undocumented features which
get broken later. I do use undocumented stuff. I hate it, but I have no
other choice.

http://www.cristalink.com

“Arlie Davis” wrote in message
news:xxxxx@ntdev…
> Will you also install the new binary when the customer upgrades server X
> from NT build N to N+1, and the machine bugchecks on boot because your
> driver is now doing the wrong thing? Will you document the fragility of
> your driver – will you tell your customer? Will you check the OS version
> in DriverEntry and refuse to start if you are not running on the OS that
> you
> reverse engineered? Can you guarantee that OS patches don’t alter the
> behavior or data structures you are relying on?
>
>>Verifying that this particular object pointer is a registry key.
>
> This was already discussed, and there are easy and reliable ways to
> accomplish this. Peeking into the object header is not necessary.
>
> Issues like this are the difference between software engineering and
> just
> programming.
>
> – arlie
>
>
> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com] On Behalf Of Maxim S. Shatskih
> Sent: Monday, August 08, 2005 12:09 PM
> To: Windows System Software Devs Interest List
> Subject: SPAM-LOW: Re: SPAM-LOW: Re: SPAM-LOW: Re: SPAM-LOW: Re:
> Re:[ntdev]
> Referencing a RegKey Object from handle
>
>> I can’t believe this is even up for discussion.
>>
>> It hasn’t changed – YET. Do you want to bet on it never changing?
>> Do you want to bet on [random data structure X] never changing?
>
> OK, I will create the new build of the binary to support the next Windows
> version :slight_smile: is this bad?
>
>> Show me a solid example of a common problem that you simply cannot
>> solve without relying on undocumented structures.
>
> Verifying that this particular object pointer is a registry key.
>
> BTW - using undocumented functions seems to be absolutely fine in most
> cases (especially in the cases when any other way is too complex).
>
> For instance, PsLookupProcessByProcessId never changed since NT4. Some of
> these functions are documented in IFS kit :slight_smile: why not copy-paste this
> definition from the IFS kit to my own source and build it? or just require
> ntifs.h for building?
>
> Sometimes using the undocumented stuff is really bad. NDIS internals, for
> instance, or EPROCESS - it is too huge and thus changes from SP to SP.
>
> Maxim Shatskih, Windows DDK MVP
> StorageCraft Corporation
> xxxxx@storagecraft.com
> http://www.storagecraft.com
>
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: xxxxx@stonestreetone.com To
> unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
>
>

7

One solution to this problem of code failing due to the use of undocumented
resources is to not purchase the products that employ such strategies to
produce such products. Case in point: I will not use, nor recommend to
anyone, Norton products.Why? Mostly because of their practice of using
unsafe coding practices to produce what I believe is a rather offal product,
which I believe caused the only system crashes and re-installs I have had
for about 3 years.

Using undocumented procedures in-house in a limited capacity to end-run OS
deficiencies is one thing, but using it to produce a marketable product that
is shrink wrapped and then sold to the masses verges on some mad-scientist
in a James Bond movie. The basic problem is they are undocumented, can
change without notice, and there is no control as to who uses them how,
leading to a scenario where Widgets Inc. usage is diametrically opposed to
Framis Inc. Anytime such practices are justifiably used, the poor schmuck
responsible for maintaining that software MUST know the facts, so they can
call the schmuck that caused the problem instead of Microsoft.

–
The personal opinion of
Gary G. Little

“Dan Partelly” wrote in message news:xxxxx@ntdev…
>>>I’m sorry that this interferes with your abilty to learn more about the
>>>OS.
>
> Im sorry , but all this has nothing to do with my ability to learn
> something :stuck_out_tongue:
> Fortunately, for me I can find my way inside your OS regardless you make
> stuff
> public or not.
>
>>>But until that happens I’m going to keep asking people why they want to
>>>know what they’re asking
>
> You have 3 points illustrate here. 2 of them are purely your problem
> because
> you are a Microsoft employee, and thus you have to obey company policy and
> not reveal data which would be integrated into a “competitor product”, or
> a “
> better tool to break it” , I asume you talk about activation and XP
> keys here …
> but unfortunately the code which handle those tasks is what … 0.00000001
> % linked
> with what ppl usually want to know about undocumented asspects of NT.
>
> Ms could reaveal and expose a lot more APIs , and improve on existing one
> whithout fearing that the data will be “integrated” into a competitor
> product.
> Let’s face it, there would be only some decalrations of structs and APIs.
> Do you really think that a person determined to learn them really needs
> official header to understand this ?
>
>
>>> build a time-bomb of a driver to sell to customers.
>
> I dont want to be evil, and god is my whitness that I love MS’s operating
> systems and products. I
> apreciate their value. But did you heard about Sasser, MsSQL worm, Codered
> and whatever ?
> Isnt MS which sold a time bomb to his customers here ? Ugly weakneses and
> “security through obscurity”
> made this possible. So If Micorsoft doest it, time and again, why is so
> concerned about 3rd party companies ?
> Solve your problems first, and make sure security is not implemented
> through obscurity , then youll have
> less problems revealing more aspects of the OS.
>
> I hope you will take this message as is, and not consider at it a attack
> to your person, or MS’s products.
>
> Sorry, Dan
>
>
>
>
>
>
>
> ----- Original Message -----
> From: “Peter Wieland”
> To: “Windows System Software Devs Interest List”
> Sent: Monday, August 08, 2005 11:17 PM
> Subject: RE: Re:[ntdev] Referencing a RegKey Object from handle
>
>
> A great many of the questions that show up on this list and on the
> newsgroups about how system internals work are posted by developers who
> have decided to write feature X a certain way, then stuck to that path
> with pitbull like tenacity until they finally got to the point where it
> wouldn’t work (depending their skill level that point can be pretty high
> or pretty low). They they decide the only way to continue on this path
> is to get some piece of information that’s not obviously available
> through the DDI so they ask a very exact question (“How do I do X”)
> instead of wondering if perhaps they went down a bad path to begin with.
>
> And quite frequently when they back up and rethink their goal the
> problem becomes much easier and their driver is better.
>
> I’m sorry that this interferes with your abilty to learn more about the
> OS. I’d love to work in a world where I we could share everything about
> how our OS works without worrying that someone would (a) integrate that
> knowledge into a competitor’s product, (b) build a better tool to break
> it based on that info or (c) build a time-bomb of a driver to sell to
> customers.
>
> But until that happens I’m going to keep asking people why they want to
> know what they’re asking. I’m going to continue to tell people not to
> use undocumented APIs or data structures. And I’m going to have to
> continue to be warry of how information I give out could be used for
> evil. If you feel this is “bull****” I’ll have to live with that.
>
> Sorry,
> -p
>
>
> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com] On Behalf Of Dan Partelly
> Sent: Monday, August 08, 2005 12:47 PM
> To: Windows System Software Devs Interest List
> Subject: Re: Re:[ntdev] Referencing a RegKey Object from handle
>
>>> any chanche OBJECT_HEADER will change?
>
> IIRC, it already did.
>
>>> Everything in object header is exposed through documented
>
> Uhh. Not really , really no. There are things there which are not
> exposed through any API in a object header.
>
> And speaking about object headers, I still simply hate when MS is not
> consitent with itself, and object manager philosophy, and you get in a
> filter driver to see phony file objects whithout a valid object header,
> and allocated on stacks :stuck_out_tongue:
>
>>>And again, you have not provided any evidence that you need direct
>>>access to the object header.
>
> Sorry, but why should I convince you that I need access to it ? A
> question was posted. If you know the answer , answer to it. Would you
> tell me the format of a OBJECT_HEADER if you know it, or simply say:
> “Well … I know it, but you dont need to know about it and its format.
> So , I wont tell you”. If this is so… why do you know its format in
> the first place ? And second , why dont you share ?
>
> Lately I see a lot of bull**** on many lists (not only here)on
> discussing undocumented stuff. There was a time when ppl could discuss
> undocumented aspects and techniques whithout jumping at each other , and
> accusing “you are a malware writter”, “you dont need to know this” , and
> so on.
> In fact, there where books written on subject, I recall in particular a
> book about Windows 98 written by Matt Pietrek which had a lot of
> unorthodox stuff in it.
>
>>> Should the DDK just be a list of #defines of variable addresses?
>
> Hahahaha. You are funny. I dont think anybody said that wants the DDK
> that way.
>
> Dan
>
>
> ----- Original Message -----
> From: “Arlie Davis”
> To: “Windows System Software Devs Interest List”
> Sent: Monday, August 08, 2005 9:30 PM
> Subject: Re:[ntdev] Referencing a RegKey Object from handle
>
>
>> But not zero, and you provide absolutely no protection for your
> clients
>> if/when the change does occur.
>>
>> And again, you have not provided any evidence that you need direct
>> access
>> to the object header. Everything in object header is exposed through
>> documented, supported APIs, all of which give an indication of what
> sort
>> of
>> semantic change you want to occur (such as changing the reference
> count on
>> an object).
>>
>> Do you think data encapsulation is a bad thing? Should the DDK just
> be a
>> list of #defines of variable addresses?
>>
>> – arlie
>>
>>
>>
>> -----Original Message-----
>> From: xxxxx@lists.osr.com
>> [mailto:xxxxx@lists.osr.com] On Behalf Of Maxim S.
> Shatskih
>> Sent: Monday, August 08, 2005 2:15 PM
>> To: Windows System Software Devs Interest List
>> Subject: SPAM-LOW: Re: Re:[ntdev] Referencing a RegKey Object from
> handle
>>
>> Any chances that OBJECT_HEADER will change? Very, very minor.
>>
>> Maxim Shatskih, Windows DDK MVP
>> StorageCraft Corporation
>> xxxxx@storagecraft.com
>> http://www.storagecraft.com
>>
>>
>>
>>
>> —
>> Questions? First check the Kernel Driver FAQ at
>> http://www.osronline.com/article.cfm?id=256
>>
>> You are currently subscribed to ntdev as: xxxxx@rdsor.ro
>> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: xxxxx@windows.microsoft.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: unknown lmsubst tag argument: ‘’
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>

8

It’s Microsoft’s fault that people use undocumented features which get
broken. Now how ridiculous is THAT comment. If I write anything that abides
strictly to the guidelines and it breaks with a given SP, then I have
grounds to complain to Microsoft. However, if I come up with a supercilious
“fix” based on undocumented structures that then break in the next release,
I’m sorry but that ain’t Microsoft’s fault. It’s my fault, and my
responsibility to respond to my customers, NOT Microsoft’s or any other OS
producer for that matter.

That’s as asinine an argument as suing MacDonald’s because their hot coffee
really is hot.

–
The personal opinion of
Gary G. Little

“cristalink” wrote in message news:xxxxx@ntdev…
>I agree Windows is a time bomb by itself. I have two drivers which were
>broken by Win2003 SP1, one because of a newly added bug in classpnp.sys,
>another one due to screwed SP1 DDK headers for 1394. Longhorn/Vista is
>going to break a lot more software than all undocumented features together.
>
> Of course it’s Microsoft’s fault that people use undocumented features
> which get broken later. I do use undocumented stuff. I hate it, but I have
> no other choice.
> –
> http://www.cristalink.com
>
>
> “Arlie Davis” wrote in message
> news:xxxxx@ntdev…
>> Will you also install the new binary when the customer upgrades server X
>> from NT build N to N+1, and the machine bugchecks on boot because your
>> driver is now doing the wrong thing? Will you document the fragility of
>> your driver – will you tell your customer? Will you check the OS
>> version
>> in DriverEntry and refuse to start if you are not running on the OS that
>> you
>> reverse engineered? Can you guarantee that OS patches don’t alter the
>> behavior or data structures you are relying on?
>>
>>>Verifying that this particular object pointer is a registry key.
>>
>> This was already discussed, and there are easy and reliable ways to
>> accomplish this. Peeking into the object header is not necessary.
>>
>> Issues like this are the difference between software engineering and
>> just
>> programming.
>>
>> – arlie
>>
>>
>> -----Original Message-----
>> From: xxxxx@lists.osr.com
>> [mailto:xxxxx@lists.osr.com] On Behalf Of Maxim S. Shatskih
>> Sent: Monday, August 08, 2005 12:09 PM
>> To: Windows System Software Devs Interest List
>> Subject: SPAM-LOW: Re: SPAM-LOW: Re: SPAM-LOW: Re: SPAM-LOW: Re:
>> Re:[ntdev]
>> Referencing a RegKey Object from handle
>>
>>> I can’t believe this is even up for discussion.
>>>
>>> It hasn’t changed – YET. Do you want to bet on it never changing?
>>> Do you want to bet on [random data structure X] never changing?
>>
>> OK, I will create the new build of the binary to support the next Windows
>> version :slight_smile: is this bad?
>>
>>> Show me a solid example of a common problem that you simply cannot
>>> solve without relying on undocumented structures.
>>
>> Verifying that this particular object pointer is a registry key.
>>
>> BTW - using undocumented functions seems to be absolutely fine in most
>> cases (especially in the cases when any other way is too complex).
>>
>> For instance, PsLookupProcessByProcessId never changed since NT4. Some of
>> these functions are documented in IFS kit :slight_smile: why not copy-paste this
>> definition from the IFS kit to my own source and build it? or just
>> require
>> ntifs.h for building?
>>
>> Sometimes using the undocumented stuff is really bad. NDIS internals, for
>> instance, or EPROCESS - it is too huge and thus changes from SP to SP.
>>
>> Maxim Shatskih, Windows DDK MVP
>> StorageCraft Corporation
>> xxxxx@storagecraft.com
>> http://www.storagecraft.com
>>
>>
>> —
>> Questions? First check the Kernel Driver FAQ at
>> http://www.osronline.com/article.cfm?id=256
>>
>> You are currently subscribed to ntdev as: xxxxx@stonestreetone.com To
>> unsubscribe send a blank email to xxxxx@lists.osr.com
>>
>>
>>
>>
>
>
>

9

Microsoft’s fault is, they don’t provide documented ways to do the things
people need to do. The dilemma is, either you have the software that does
what you need, even though with undocumented ways, or you don’t have it.

What’s really ridiculous are the statements like these:

  • “If Microsoft publishes an undocumented interface, they would need to
    support it in the future”. Bullsh*t. Have a look at the “obsolete APIs” list
    in the latest DDKs. They were all once documented… Every major release of
    Windows obsoletes a great bunch of “compliant” drivers. Some stupid guys
    from Microsoft even want to obsolete innocent strcpy(). That’s of course an
    extremely dangerous function, comparing with any of the DDK APIs :slight_smile: :open_mouth: .
    Besides, there’s nothing in MS license agreements that obliges them to do
    anything specific in the future.

  • “Those who use undocumented features usually make cr*ppy drivers”. “Don’t
    tell them about undocumented ways, they’ll write a driver that will crash
    the system, then Microsoft will be blamed”. Those writing cr*ppy software
    will be producing cr*ppy software regardless of whether they use
    undocumented features or not.

  • “Your undocumented solution will [probably] be broken with the next major
    release, hotfix, or service pack”. As I said before, I had two “compliant”
    drivers which were broken due to Microsoft’s new *bugs* introduced in W2K3
    SP1 and W2K3 SP1 DDK. How many “compliant” NT4 drivers were broken with the
    release of Win2000? So the danger of having undocumented features in your
    driver, if you use them thoughtfully of course, is no bigger than the danger
    of writing “documented” drivers for Windows.

–
http://www.cristalink.com

“Gary G. Little” wrote in message news:xxxxx@ntdev…
> It’s Microsoft’s fault that people use undocumented features which get
> broken. Now how ridiculous is THAT comment. If I write anything that
> abides strictly to the guidelines and it breaks with a given SP, then I
> have grounds to complain to Microsoft. However, if I come up with a
> supercilious “fix” based on undocumented structures that then break in the
> next release, I’m sorry but that ain’t Microsoft’s fault. It’s my fault,
> and my responsibility to respond to my customers, NOT Microsoft’s or any
> other OS producer for that matter.
>
> That’s as asinine an argument as suing MacDonald’s because their hot
> coffee really is hot.
>
> –
> The personal opinion of
> Gary G. Little
>
> “cristalink” wrote in message news:xxxxx@ntdev…
>>I agree Windows is a time bomb by itself. I have two drivers which were
>>broken by Win2003 SP1, one because of a newly added bug in classpnp.sys,
>>another one due to screwed SP1 DDK headers for 1394. Longhorn/Vista is
>>going to break a lot more software than all undocumented features
>>together.
>>
>> Of course it’s Microsoft’s fault that people use undocumented features
>> which get broken later. I do use undocumented stuff. I hate it, but I
>> have no other choice.
>> –
>> http://www.cristalink.com
>>
>>
>> “Arlie Davis” wrote in message
>> news:xxxxx@ntdev…
>>> Will you also install the new binary when the customer upgrades server X
>>> from NT build N to N+1, and the machine bugchecks on boot because your
>>> driver is now doing the wrong thing? Will you document the fragility of
>>> your driver – will you tell your customer? Will you check the OS
>>> version
>>> in DriverEntry and refuse to start if you are not running on the OS that
>>> you
>>> reverse engineered? Can you guarantee that OS patches don’t alter the
>>> behavior or data structures you are relying on?
>>>
>>>>Verifying that this particular object pointer is a registry key.
>>>
>>> This was already discussed, and there are easy and reliable ways to
>>> accomplish this. Peeking into the object header is not necessary.
>>>
>>> Issues like this are the difference between software engineering and
>>> just
>>> programming.
>>>
>>> – arlie
>>>
>>>
>>> -----Original Message-----
>>> From: xxxxx@lists.osr.com
>>> [mailto:xxxxx@lists.osr.com] On Behalf Of Maxim S.
>>> Shatskih
>>> Sent: Monday, August 08, 2005 12:09 PM
>>> To: Windows System Software Devs Interest List
>>> Subject: SPAM-LOW: Re: SPAM-LOW: Re: SPAM-LOW: Re: SPAM-LOW: Re:
>>> Re:[ntdev]
>>> Referencing a RegKey Object from handle
>>>
>>>> I can’t believe this is even up for discussion.
>>>>
>>>> It hasn’t changed – YET. Do you want to bet on it never changing?
>>>> Do you want to bet on [random data structure X] never changing?
>>>
>>> OK, I will create the new build of the binary to support the next
>>> Windows
>>> version :slight_smile: is this bad?
>>>
>>>> Show me a solid example of a common problem that you simply cannot
>>>> solve without relying on undocumented structures.
>>>
>>> Verifying that this particular object pointer is a registry key.
>>>
>>> BTW - using undocumented functions seems to be absolutely fine in most
>>> cases (especially in the cases when any other way is too complex).
>>>
>>> For instance, PsLookupProcessByProcessId never changed since NT4. Some
>>> of
>>> these functions are documented in IFS kit :slight_smile: why not copy-paste this
>>> definition from the IFS kit to my own source and build it? or just
>>> require
>>> ntifs.h for building?
>>>
>>> Sometimes using the undocumented stuff is really bad. NDIS internals,
>>> for
>>> instance, or EPROCESS - it is too huge and thus changes from SP to SP.
>>>
>>> Maxim Shatskih, Windows DDK MVP
>>> StorageCraft Corporation
>>> xxxxx@storagecraft.com
>>> http://www.storagecraft.com
>>>
>>>
>>> —
>>> Questions? First check the Kernel Driver FAQ at
>>> http://www.osronline.com/article.cfm?id=256
>>>
>>> You are currently subscribed to ntdev as: xxxxx@stonestreetone.com To
>>> unsubscribe send a blank email to xxxxx@lists.osr.com
>>>
>>>
>>>
>>>
>>
>>
>>
>
>
>

10

In fact, Windows changes from version to version are SO huge, so the danger
of “this code will cease to work in next Windows version” is nearly not an
issue. The reason is that you will usually need to update the code for the next
Windows version anyway. :slight_smile:

Undocumented functions are by far lesser evil then hooking, un-exported
functions, updating the OS’s data structures without grabbing the corresponding
locks etc. Especially if the undocumented NTOSKRNL export is used in one of the
MS’s own kernel-mode binaries :slight_smile: Yes, the functions invented for private
communication between NTOSKRNL and HAL are usually a no-no - they serve some
very special purpose and called only once in the whole kernel in a very
“interesting” context. But things like PsLookupProcessByProcessId? Things like
ZwQuerySystemInformation (surely called by user-mode Pefmon DLLs)? Things like
ZwQuerySymbolicLinkObject?

Things like RtlCompressBuffer? The latter routine is heavily used by NTFS
for compressed files. Is it a realistic situation that MS will update NTFS.SYS
in some service pack to the old OS like w2k so that the new NTFS will not use
this function? BTW - this function is in user-mode NTDLL also :slight_smile: MS has the
CryptoAPI, and has no CompressionAPI - well, it also has the CompressionAPI,
just undocumented :slight_smile:

Some functions are undocumented because MS’s position seems to be - “hey,
why must we disclose this? it is always better to keep more information
secret”. :slight_smile:

More so. I saw an NT function (sorry, forgotten the name) which is
documented to be present from XP up. This very function exists in w2k too
(undocumented) and works like a charm. So why not use it in a binary which must
work on w2k up? Any chances that w2k SP5 will remove this function?

Undocumented structures - especially huge ones like EPROCESS - are, well,
worse a bit. So, if there are means of object validation without touching
OBJECT_HEADER - then there is no need in touching the OBJECT_HEADER.

Undocumented magic constants are a no-no for obvious reasons.

There are some border cases. For instance, kernel-mode crypto. Way 1 - to
pull the OpenSSL (public domain, just note in the product About box and helps)
source there. Way 2 - to call IOCTLs (yes, undocumented magic constants) to
FIPS.SYS.

What way is better? FIPS is here to stay, KSecDD and EFS rely on it. My
personal opinion is - to borrow the public domain code (it is a reference
implementation after all).

Also I have a question to more experienced elder collegues.

Was VMS also undocumented to this degree as NT? Was RSX-11 such? I know
that both of these OSes were well-designed and well-structured (better then
most commercial UNIXen), and system-level development for them was not this
painful - RSX-11 was maybe a most popular OS in the USSR under the name of “The
Realtime OS” :-), and I know several people which are in their 40ies now with
experience of writing RSX kernel add-ons. Yes, assembler. But everything was
very much structured and well-designed. Also a much better assembler then x86
one.

Yes, MS-DOS had lots of undocumented stuff. Well, this OS was just plain
one huge kludge. Patch over patch, each patch added for some rare case, and
undocumented stuff abuse will re-use this patch in another context - not a wise
decision. It’s obvious that using the undocumented code which looks like a
kludge is a bad idea - the side effects are unpredictable. But if the code
looks - like RtlCompressBuffer - like a well-designed API, just hidden from us?

NT is not MS-DOS. NT is well-designed OS, so I expect that most routines
will work as designed. So, for me, it looks like most undocumented features of
NT is safer to use then in MS-DOS.

So, for now, my opinion is - undocumented stuff usage is possible, but with
great care and with keeping in mind all possible consequences.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

11

I completly agree with you Max.

Dan

----- Original Message -----
From: “Maxim S. Shatskih”
To: “Windows System Software Devs Interest List”
Sent: Wednesday, August 10, 2005 12:44 AM
Subject: Re: Re:[ntdev] Re:Referencing a RegKey Object from handle

> In fact, Windows changes from version to version are SO huge, so the
> danger
> of “this code will cease to work in next Windows version” is nearly not an
> issue. The reason is that you will usually need to update the code for the
> next
> Windows version anyway. :slight_smile:
>
> Undocumented functions are by far lesser evil then hooking, un-exported
> functions, updating the OS’s data structures without grabbing the
> corresponding
> locks etc. Especially if the undocumented NTOSKRNL export is used in one
> of the
> MS’s own kernel-mode binaries :slight_smile: Yes, the functions invented for private
> communication between NTOSKRNL and HAL are usually a no-no - they serve
> some
> very special purpose and called only once in the whole kernel in a very
> “interesting” context. But things like PsLookupProcessByProcessId? Things
> like
> ZwQuerySystemInformation (surely called by user-mode Pefmon DLLs)? Things
> like
> ZwQuerySymbolicLinkObject?
>
> Things like RtlCompressBuffer? The latter routine is heavily used by
> NTFS
> for compressed files. Is it a realistic situation that MS will update
> NTFS.SYS
> in some service pack to the old OS like w2k so that the new NTFS will
> not use
> this function? BTW - this function is in user-mode NTDLL also :slight_smile: MS has
> the
> CryptoAPI, and has no CompressionAPI - well, it also has the
> CompressionAPI,
> just undocumented :slight_smile:
>
> Some functions are undocumented because MS’s position seems to be -
> “hey,
> why must we disclose this? it is always better to keep more information
> secret”. :slight_smile:
>
> More so. I saw an NT function (sorry, forgotten the name) which is
> documented to be present from XP up. This very function exists in w2k too
> (undocumented) and works like a charm. So why not use it in a binary which
> must
> work on w2k up? Any chances that w2k SP5 will remove this function?
>
> Undocumented structures - especially huge ones like EPROCESS - are,
> well,
> worse a bit. So, if there are means of object validation without touching
> OBJECT_HEADER - then there is no need in touching the OBJECT_HEADER.
>
> Undocumented magic constants are a no-no for obvious reasons.
>
> There are some border cases. For instance, kernel-mode crypto. Way 1 -
> to
> pull the OpenSSL (public domain, just note in the product About box and
> helps)
> source there. Way 2 - to call IOCTLs (yes, undocumented magic constants)
> to
> FIPS.SYS.
>
> What way is better? FIPS is here to stay, KSecDD and EFS rely on it. My
> personal opinion is - to borrow the public domain code (it is a reference
> implementation after all).
>
> Also I have a question to more experienced elder collegues.
>
> Was VMS also undocumented to this degree as NT? Was RSX-11 such? I know
> that both of these OSes were well-designed and well-structured (better
> then
> most commercial UNIXen), and system-level development for them was not
> this
> painful - RSX-11 was maybe a most popular OS in the USSR under the name of
> “The
> Realtime OS” :-), and I know several people which are in their 40ies now
> with
> experience of writing RSX kernel add-ons. Yes, assembler. But everything
> was
> very much structured and well-designed. Also a much better assembler then
> x86
> one.
>
> Yes, MS-DOS had lots of undocumented stuff. Well, this OS was just
> plain
> one huge kludge. Patch over patch, each patch added for some rare case,
> and
> undocumented stuff abuse will re-use this patch in another context - not a
> wise
> decision. It’s obvious that using the undocumented code which looks like a
> kludge is a bad idea - the side effects are unpredictable. But if the code
> looks - like RtlCompressBuffer - like a well-designed API, just hidden
> from us?
>
> NT is not MS-DOS. NT is well-designed OS, so I expect that most
> routines
> will work as designed. So, for me, it looks like most undocumented
> features of
> NT is safer to use then in MS-DOS.
>
> So, for now, my opinion is - undocumented stuff usage is possible, but
> with
> great care and with keeping in mind all possible consequences.
>
> Maxim Shatskih, Windows DDK MVP
> StorageCraft Corporation
> xxxxx@storagecraft.com
> http://www.storagecraft.com
>
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: xxxxx@rdsor.ro
> To unsubscribe send a blank email to xxxxx@lists.osr.com

12

> Windows obsoletes a great bunch of “compliant” drivers. Some stupid guys

from Microsoft even want to obsolete innocent strcpy(). That’s of course an

Well, strcpy() is obsoleted for a reason, but why RtlCompressBuffer is
undocumented?

MS wants to make the life harder for people which need compression in their
software? Sorry, but this software will be Windows-only, so, Windows platform
(and thus MS) will benefit from it.

Or MS wants to prevent Windows clone development? Sorry, but for now, this is
unrealistic task. Toys like ReactOS do exist, but not market-grade software.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

13

I suppose this issue is a bit more contentious with NT than a lot of other
OSes because we have this “dual API” split, between the native API and
Win32. The native API is, overall, so much cleaner than the public API, and
it is tempting to call it directly when it works better, or does things that
Win32 doesn’t do at all. (For example, try creating a file called “aux.c”.)

[…] Things like ZwQuerySymbolicLinkObject? […]
[…] lesser evil then hooking, un-exported functions […]
So, for now, my opinion is - undocumented stuff usage
is possible, but with great care and with keeping in mind
all possible consequences.

Agreed and agreed. I’m sure everyone is sick to death of this by now, and
I’m sorry if this has been an annoyance. I’ve just had to deal with way too
much broken code, written by people who reach for the internals without even
bothering to see if there is a “right” way to do it first.

– arlie

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Maxim S. Shatskih
Sent: Tuesday, August 09, 2005 5:45 PM
To: Windows System Software Devs Interest List
Subject: Re: Re:[ntdev] Re:Referencing a RegKey Object from handle

In fact, Windows changes from version to version are SO huge, so the
danger of “this code will cease to work in next Windows version” is nearly
not an issue. The reason is that you will usually need to update the code
for the next Windows version anyway. :slight_smile:

Undocumented functions are by far lesser evil then hooking, un-exported
functions, updating the OS’s data structures without grabbing the
corresponding locks etc. Especially if the undocumented NTOSKRNL export is
used in one of the MS’s own kernel-mode binaries :slight_smile: Yes, the functions
invented for private communication between NTOSKRNL and HAL are usually a
no-no - they serve some very special purpose and called only once in the
whole kernel in a very “interesting” context. But things like
PsLookupProcessByProcessId? Things like ZwQuerySystemInformation (surely
called by user-mode Pefmon DLLs)? Things like ZwQuerySymbolicLinkObject?

Things like RtlCompressBuffer? The latter routine is heavily used by
NTFS for compressed files. Is it a realistic situation that MS will update
NTFS.SYS in some service pack to the old OS like w2k so that the new NTFS
will not use this function? BTW - this function is in user-mode NTDLL also
:slight_smile: MS has the CryptoAPI, and has no CompressionAPI - well, it also has the
CompressionAPI, just undocumented :slight_smile:

Some functions are undocumented because MS’s position seems to be -
“hey, why must we disclose this? it is always better to keep more
information secret”. :slight_smile:

More so. I saw an NT function (sorry, forgotten the name) which is
documented to be present from XP up. This very function exists in w2k too
(undocumented) and works like a charm. So why not use it in a binary which
must work on w2k up? Any chances that w2k SP5 will remove this function?

Undocumented structures - especially huge ones like EPROCESS - are,
well, worse a bit. So, if there are means of object validation without
touching OBJECT_HEADER - then there is no need in touching the
OBJECT_HEADER.

Undocumented magic constants are a no-no for obvious reasons.

There are some border cases. For instance, kernel-mode crypto. Way 1 -
to pull the OpenSSL (public domain, just note in the product About box and
helps) source there. Way 2 - to call IOCTLs (yes, undocumented magic
constants) to FIPS.SYS.

What way is better? FIPS is here to stay, KSecDD and EFS rely on it. My
personal opinion is - to borrow the public domain code (it is a reference
implementation after all).

Also I have a question to more experienced elder collegues.

Was VMS also undocumented to this degree as NT? Was RSX-11 such? I know
that both of these OSes were well-designed and well-structured (better then
most commercial UNIXen), and system-level development for them was not this
painful - RSX-11 was maybe a most popular OS in the USSR under the name of
“The Realtime OS” :-), and I know several people which are in their 40ies
now with experience of writing RSX kernel add-ons. Yes, assembler. But
everything was very much structured and well-designed. Also a much better
assembler then x86 one.

Yes, MS-DOS had lots of undocumented stuff. Well, this OS was just plain
one huge kludge. Patch over patch, each patch added for some rare case, and
undocumented stuff abuse will re-use this patch in another context - not a
wise decision. It’s obvious that using the undocumented code which looks
like a kludge is a bad idea - the side effects are unpredictable. But if the
code looks - like RtlCompressBuffer - like a well-designed API, just hidden
from us?

NT is not MS-DOS. NT is well-designed OS, so I expect that most routines
will work as designed. So, for me, it looks like most undocumented features
of NT is safer to use then in MS-DOS.

So, for now, my opinion is - undocumented stuff usage is possible, but
with great care and with keeping in mind all possible consequences.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@stonestreetone.com To
unsubscribe send a blank email to xxxxx@lists.osr.com

14

Maxim S. Shatskih wrote:

>Windows obsoletes a great bunch of “compliant” drivers. Some stupid guys
>from Microsoft even want to obsolete innocent strcpy(). That’s of course an
>
>

Well, strcpy() is obsoleted for a reason, but why RtlCompressBuffer is
undocumented?

Actually, the strcpy() thing is, in my opinion, strong evidence of
arrogance (perhaps accidental) on the part of Microsoft.

Microsoft has every right to obsolete SetPixel if they want. They
invented it. strcpy, on the other hand, does not belong to Microsoft,
and never has. It is part of the ISO C and C++ Standard run-time
libraries. Microsoft cannot arbitrarily deprecate strcpy. They can
scold us for using it, but the Whidbey plan to issue “deprecation”
warning errors when strcpy is used shows one heck of a lot of chutzpah.

I wrote to the Whidbey project manager when this was announced in MSDN
Magazine a few months ago. He basically patted me kindly on the head
and told me not to worry so much. (That’s not actually what he said, of
course. He was politically correct.)

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

15

>> strcpy, on the other hand, does not belong to Microsoft, and never has.

Well said, Tim!

–
http://www.cristalink.com

“Tim Roberts” wrote in message news:xxxxx@ntdev…
> Maxim S. Shatskih wrote:
>
>>>Windows obsoletes a great bunch of “compliant” drivers. Some stupid guys
>>>from Microsoft even want to obsolete innocent strcpy(). That’s of course
>>>an
>>>
>>
>>Well, strcpy() is obsoleted for a reason, but why RtlCompressBuffer is
>>undocumented?
>>
>
> Actually, the strcpy() thing is, in my opinion, strong evidence of
> arrogance (perhaps accidental) on the part of Microsoft.
>
> Microsoft has every right to obsolete SetPixel if they want. They
> invented it. strcpy, on the other hand, does not belong to Microsoft, and
> never has. It is part of the ISO C and C++ Standard run-time libraries.
> Microsoft cannot arbitrarily deprecate strcpy. They can scold us for
> using it, but the Whidbey plan to issue “deprecation” warning errors when
> strcpy is used shows one heck of a lot of chutzpah.
>
> I wrote to the Whidbey project manager when this was announced in MSDN
> Magazine a few months ago. He basically patted me kindly on the head and
> told me not to worry so much. (That’s not actually what he said, of
> course. He was politically correct.)
>
> –
> Tim Roberts, xxxxx@probo.com
> Providenza & Boekelheide, Inc.
>
>
>

16

> I suppose this issue is a bit more contentious with NT than a lot of other

OSes because we have this “dual API” split, between the native API and
Win32. The native API is, overall, so much cleaner than the public API, and
it is tempting to call it directly when it works better, or does things that
Win32 doesn’t do at all. (For example, try creating a file called “aux.c”.)

Yes. For instance - why there is no kernel32!DeviceIoControlEx? And why not
call ntdll!NtDeviceIoControlFile instead?

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

17

It is dangerous for the many coders out there who won’t make sure it is
safe. It is also their compiler and DDK/WDK/IFS Kit, so they get to block
whatever they want. A monopoly has that power and since we all need their
OS to make a living, we may just have to live with it. When you’re in the
military and they tell you to use a razor blade to clean the shower walls
and floors, you just have to do it. You can always retire or something, but
sometimes it does not achieve any thing to fight against an irresistible
force.

Am I getting mellow?

“Tim Roberts” wrote in message news:xxxxx@ntdev…
> Maxim S. Shatskih wrote:
>
>>>Windows obsoletes a great bunch of “compliant” drivers. Some stupid guys
>>>from Microsoft even want to obsolete innocent strcpy(). That’s of course
>>>an
>>>
>>
>>Well, strcpy() is obsoleted for a reason, but why RtlCompressBuffer is
>>undocumented?
>>
>
> Actually, the strcpy() thing is, in my opinion, strong evidence of
> arrogance (perhaps accidental) on the part of Microsoft.
>
> Microsoft has every right to obsolete SetPixel if they want. They
> invented it. strcpy, on the other hand, does not belong to Microsoft, and
> never has. It is part of the ISO C and C++ Standard run-time libraries.
> Microsoft cannot arbitrarily deprecate strcpy. They can scold us for
> using it, but the Whidbey plan to issue “deprecation” warning errors when
> strcpy is used shows one heck of a lot of chutzpah.
>
> I wrote to the Whidbey project manager when this was announced in MSDN
> Magazine a few months ago. He basically patted me kindly on the head and
> told me not to worry so much. (That’s not actually what he said, of
> course. He was politically correct.)
>
> –
> Tim Roberts, xxxxx@probo.com
> Providenza & Boekelheide, Inc.
>
>

18

Take a look at the Rollup for Windows 2000 SP4. It adds, according to what
I hear, several of the XP and later functions are present in that new
kernel. I suspect they were required for the Filter Manager, but sometimes
something that is supposedly only an ‘update’ might be closer to a service
pack or even a new version.

I do wish though that Microsoft would provide the OS calls to achieve what
WE need and not what they want to give us. Sometimes you can get to a
developer at Microsoft and get something you really need, but it sure would
be nice if it was as easy as it is for Linux. All you have to do there is
convince Linus that it is needed and if he agrees, it will appear or you
will get the chance to write it and have it included.

“Maxim S. Shatskih” wrote in message
news:xxxxx@ntdev…
> In fact, Windows changes from version to version are SO huge, so the
> danger
> of “this code will cease to work in next Windows version” is nearly not an
> issue. The reason is that you will usually need to update the code for the
> next
> Windows version anyway. :slight_smile:
>
> Undocumented functions are by far lesser evil then hooking, un-exported
> functions, updating the OS’s data structures without grabbing the
> corresponding
> locks etc. Especially if the undocumented NTOSKRNL export is used in one
> of the
> MS’s own kernel-mode binaries :slight_smile: Yes, the functions invented for private
> communication between NTOSKRNL and HAL are usually a no-no - they serve
> some
> very special purpose and called only once in the whole kernel in a very
> “interesting” context. But things like PsLookupProcessByProcessId? Things
> like
> ZwQuerySystemInformation (surely called by user-mode Pefmon DLLs)? Things
> like
> ZwQuerySymbolicLinkObject?
>
> Things like RtlCompressBuffer? The latter routine is heavily used by
> NTFS
> for compressed files. Is it a realistic situation that MS will update
> NTFS.SYS
> in some service pack to the old OS like w2k so that the new NTFS will
> not use
> this function? BTW - this function is in user-mode NTDLL also :slight_smile: MS has
> the
> CryptoAPI, and has no CompressionAPI - well, it also has the
> CompressionAPI,
> just undocumented :slight_smile:
>
> Some functions are undocumented because MS’s position seems to be -
> “hey,
> why must we disclose this? it is always better to keep more information
> secret”. :slight_smile:
>
> More so. I saw an NT function (sorry, forgotten the name) which is
> documented to be present from XP up. This very function exists in w2k too
> (undocumented) and works like a charm. So why not use it in a binary which
> must
> work on w2k up? Any chances that w2k SP5 will remove this function?
>
> Undocumented structures - especially huge ones like EPROCESS - are,
> well,
> worse a bit. So, if there are means of object validation without touching
> OBJECT_HEADER - then there is no need in touching the OBJECT_HEADER.
>
> Undocumented magic constants are a no-no for obvious reasons.
>
> There are some border cases. For instance, kernel-mode crypto. Way 1 -
> to
> pull the OpenSSL (public domain, just note in the product About box and
> helps)
> source there. Way 2 - to call IOCTLs (yes, undocumented magic constants)
> to
> FIPS.SYS.
>
> What way is better? FIPS is here to stay, KSecDD and EFS rely on it. My
> personal opinion is - to borrow the public domain code (it is a reference
> implementation after all).
>
> Also I have a question to more experienced elder collegues.
>
> Was VMS also undocumented to this degree as NT? Was RSX-11 such? I know
> that both of these OSes were well-designed and well-structured (better
> then
> most commercial UNIXen), and system-level development for them was not
> this
> painful - RSX-11 was maybe a most popular OS in the USSR under the name of
> “The
> Realtime OS” :-), and I know several people which are in their 40ies now
> with
> experience of writing RSX kernel add-ons. Yes, assembler. But everything
> was
> very much structured and well-designed. Also a much better assembler then
> x86
> one.
>
> Yes, MS-DOS had lots of undocumented stuff. Well, this OS was just
> plain
> one huge kludge. Patch over patch, each patch added for some rare case,
> and
> undocumented stuff abuse will re-use this patch in another context - not a
> wise
> decision. It’s obvious that using the undocumented code which looks like a
> kludge is a bad idea - the side effects are unpredictable. But if the code
> looks - like RtlCompressBuffer - like a well-designed API, just hidden
> from us?
>
> NT is not MS-DOS. NT is well-designed OS, so I expect that most
> routines
> will work as designed. So, for me, it looks like most undocumented
> features of
> NT is safer to use then in MS-DOS.
>
> So, for now, my opinion is - undocumented stuff usage is possible, but
> with
> great care and with keeping in mind all possible consequences.
>
> Maxim Shatskih, Windows DDK MVP
> StorageCraft Corporation
> xxxxx@storagecraft.com
> http://www.storagecraft.com
>
>

19

Have you or any of the other who complain about the undocumented calls or
missing calls asked Microsoft to fix it? I don’t mean complain, but lay out
in a rational manner why the is a need for a capability. The more specifics
on the need and the more flexibility on the interface the better. This is
best done when you know the person, but there are enough feedback emails
that will listen also.

I have asked over the years, and actually have had reasonably good results.
No they will not retro fit the interface to every version of the system, but
it you can truly articulate a need, many of the Microsoft folks are highly
sympathetic and responsive.

–
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Remove StopSpam from the email to reply

“David J. Craig” wrote in message
news:xxxxx@ntdev…
> I do wish though that Microsoft would provide the OS calls to achieve what
> WE need and not what they want to give us. Sometimes you can get to a
> developer at Microsoft and get something you really need, but it sure
> would be nice if it was as easy as it is for Linux. All you have to do
> there is convince Linus that it is needed and if he agrees, it will appear
> or you will get the chance to write it and have it included.

20

Oliver,

This is true, except that once hardware is “done” it is rarely
touched. Drivers and software have a maintaince cost that is a killer. It
has been found by a number of firms, that well designed hardware costs the
same as poorly designed hardware (yes if you add a ton of bells and whistles
the cost can change, but in general). What does change is the cost of the
software support for the hardware.

No matter what one does there is a build cost X for the basic
hardware. Now consider that a commodity PCI board is selling in some case
for $10, and have a profit of less than a $1. Now a good hardware designer
will cost you around $100K in my neck of the woods. Assuming a man-year for
the design, you have to sell over 100K boards to cover the designer.
Interestingly enough the software guys are similar priced, so with a
complete package (not just a driver) this is easily a man-year so another
100K+ boards. But, in many cases the software support costs can for poor
software (or poor hardware forcing games in software), be close to or higher
than the initial software cost! If that is the case you have to be doing
over $1,000,000 in revunes on the boards just to break even!

–
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Remove StopSpam from the email to reply

“Oliver Schneider” wrote in message
news:xxxxx@ntdev…
>> Sorry to disalusion you, but there have been a number of studies
>> that have shown that for most hardware the costs of the associated
>> software is 5 to 10 times higher than the hardware. The concept that
>> the hardware is more expensive went out with discrete packages.
> With only a minor difference. For any class of hardware (class refers to
> a
> prototype, e.g. special chipset or so) you have to write the software once
> and can copy it completely lossless. Yet the hardware needs to be
> produced -
> which is definitely more costive than just copying some bits and bytes.
>
> Just my 2 cents,
>
> Oliver
>
> –
> ---------------------------------------------------
> May the source be with you, stranger :wink:
>
> ICQ: #281645
> URL: http://assarbad.net
>