I see a strange problem when trying to open a regular file using CreateFile(), it returns NULL. CreateFile() is being done by a user mode application on Windows 2000 server.
The usermode component then passes the returned (NULL) handle to a kernel mode component via a ioctl. The thing is, my kernel mode component cannot find a corresponding kernel mode object representing the handle (our driver will crash the
system in this case with a invalid memory access).
msdn documentation clearly states that:
“If the function fails, the return value is INVALID_HANDLE_VALUE. To get extended error information, call GetLastError.”
winbase.h in platform sdk has
#define INVALID_HANDLE_VALUE ((HANDLE)(LONG_PTR)-1).
From the behaviour I have seen, NULL handle seems to be invalid too. Has anyone experienced this problem. Is there an explanation to the mystery?
Thanks in advance for any help.
-Shyam
xxxxx@vormetric.com wrote:
I see a strange problem when trying to open a regular file using CreateFile(), it returns NULL. CreateFile() is being done by a user mode application on Windows 2000 server.
The usermode component then passes the returned (NULL) handle to a kernel mode component via a ioctl. The thing is, my kernel mode component cannot find a corresponding kernel mode object representing the handle (our driver will crash the
system in this case with a invalid memory access).
msdn documentation clearly states that:
“If the function fails, the return value is INVALID_HANDLE_VALUE. To get extended error information, call GetLastError.”
winbase.h in platform sdk has
#define INVALID_HANDLE_VALUE ((HANDLE)(LONG_PTR)-1).
>From the behaviour I have seen, NULL handle seems to be invalid too. Has anyone experienced this problem. Is there an explanation to the mystery?
That’s very odd. I have never seen CreateFile return NULL. This is a
normal disk file? Can you post the CreateFile call so we can see the
parameters?
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
Are you sure that it returned NULL vs an error path you initialized the value as NULL and CreateFile was not called?
d
I think Doron’s hypothesis is the most likely explanation.
However, it must be said that you need to fix your driver. It needs to be
robust enough to handle any ioctl input without crashing.
ObReferenceObjectByHandle and DC2 are your friends here.
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@vormetric.com
Sent: Tuesday, November 14, 2006 12:18 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] CreateFile() returns NULL on Win2k
I see a strange problem when trying to open a regular file using
CreateFile(), it returns NULL. CreateFile() is being done by a user mode
application on Windows 2000 server. The usermode component then passes the
returned (NULL) handle to a kernel mode component via a ioctl. The thing
is, my kernel mode component cannot find a corresponding kernel mode object
representing the handle (our driver will crash the
system in this case with a invalid memory access).
msdn documentation clearly states that:
“If the function fails, the return value is INVALID_HANDLE_VALUE. To get
extended error information, call GetLastError.”
winbase.h in platform sdk has
#define INVALID_HANDLE_VALUE ((HANDLE)(LONG_PTR)-1).
From the behaviour I have seen, NULL handle seems to be invalid too. Has
anyone experienced this problem. Is there an explanation to the mystery?
Thanks in advance for any help. -Shyam
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
In addition, application should be fixed, too. It should be robust enough to not accept NULL handles. I always check for both INVALID_HANDLE_VALUE and NULL. It makes my panaroid thinking (and probably also lint) happy 
Best regards,
Michal Vodicka
UPEK, Inc.
[xxxxx@upek.com, http://www.upek.com]
From: xxxxx@lists.osr.com[SMTP:xxxxx@lists.osr.com] on behalf of Dan Kyler[SMTP:xxxxx@privtek.com]
Reply To: Windows System Software Devs Interest List
Sent: Wednesday, November 15, 2006 12:08 AM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] CreateFile() returns NULL on Win2k
I think Doron’s hypothesis is the most likely explanation.
However, it must be said that you need to fix your driver. It needs to be
robust enough to handle any ioctl input without crashing.
ObReferenceObjectByHandle and DC2 are your friends here.
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@vormetric.com
Sent: Tuesday, November 14, 2006 12:18 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] CreateFile() returns NULL on Win2k
I see a strange problem when trying to open a regular file using
CreateFile(), it returns NULL. CreateFile() is being done by a user mode
application on Windows 2000 server. The usermode component then passes the
returned (NULL) handle to a kernel mode component via a ioctl. The thing
is, my kernel mode component cannot find a corresponding kernel mode object
representing the handle (our driver will crash the
system in this case with a invalid memory access).
msdn documentation clearly states that:
“If the function fails, the return value is INVALID_HANDLE_VALUE. To get
extended error information, call GetLastError.”
winbase.h in platform sdk has
#define INVALID_HANDLE_VALUE ((HANDLE)(LONG_PTR)-1).
From the behaviour I have seen, NULL handle seems to be invalid too. Has
anyone experienced this problem. Is there an explanation to the mystery?
Thanks in advance for any help. -Shyam
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
Questions? First check the Kernel Driver FAQ at http://www.osronline.com/article.cfm?id=256
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
Thanks for your prompt response. I went back and looked some more at what the app was doing.
I had jumped the gun on my inference of some local variables in the driver (I was looking
at a system crash dump file from a blue screen at remote site) to figure out what was
happening. It turns out tha APP was indeed seeing INVALID_HANDLE_VALUE returned from
CreateFile(). But I am facing another problem now in my application, which I will post in a separate message.