Windows System Software -- Consulting, Training, Development -- Unique Expertise, Guaranteed Results

Before Posting... Please check out the Community Guidelines in the
Announcements and Administration Category, below.

Help Stamp Out Sensless WDM Usage

Peter_ViscarolaPeter_Viscarola Posts: 6,649
<RANT>

Folks... including and especially Microsoft folks,

One of the common threads in this list is newbs writing WDM drivers. They ask us questions they get some answers. They go away. Their device drivers and especially filters live on as one of the more common causes of end user system crashes in modern times.

We need a concerted effort to stomp out the senseless writing of WDM drivers.

We need to scrub the samples to make sure there are no WDM drivers around (other than software only ?kernel services?). If you host example on GitHub or someplace else, if it?s a WDM driver , for heavens sakes make the readme say it?s a deprecated model.

We need the WDK docs to very clear say, everywhere, that people should be using WDM
as a last resort only if they are not writing a file system or a kernel service. Shit, put it on every WDM doc page: IoXxxx, KeXxxx, etc.

WDK doc folks... please take some time to focus on this goal. It?ll be time we?ll spent.

People who mistakenly start with a WDM sample are not well served. They would be better off with no sample... though I doubt they would see this. Starting with some hideous shit from CodeProject, or an ancient sample from the WDK, is just an invitation to (a) frustrating the dev, (b) injecting bugs into kernel-mode.

Whew, I feel better now.

Peter
OSR
@OSRDrivers

</RANT>

Peter Viscarola
OSR
@OSRDrivers

«1

Comments

  • R0b0t1R0b0t1 Posts: 129
    On Wed, Mar 7, 2018 at 7:04 AM, xxxxx@osr.com wrote:
    >
    >
    > Folks... including and especially Microsoft folks,
    >
    > One of the common threads in this list is newbs writing WDM drivers. They
    ask us questions they get some answers. They go away. Their device drivers
    and especially filters live on as one of the more common causes of end user
    system crashes in modern times.
    >
    > We need a concerted effort to stomp out the senseless writing of WDM
    drivers.
    >
    > We need to scrub the samples to make sure there are no WDM drivers around
    (other than software only ?kernel services?). If you host example on
    GitHub or someplace else, if it?s a WDM driver , for heavens sakes make the
    readme say it?s a deprecated model.
    >
    > We need the WDK docs to very clear say, everywhere, that people should be
    using WDM
    > as a last resort only if they are not writing a file system or a kernel
    service. Shit, put it on every WDM doc page: IoXxxx, KeXxxx, etc.
    >
    > WDK doc folks... please take some time to focus on this goal. It?ll be
    time we?ll spent.
    >
    > People who mistakenly start with a WDM sample are not well served. They
    would be better off with no sample... though I doubt they would see this.
    Starting with some hideous shit from CodeProject, or an ancient sample from
    the WDK, is just an invitation to (a) frustrating the dev, (b) injecting
    bugs into kernel-mode.
    >
    > Whew, I feel better now.
    >
    > Peter
    > OSR
    > @OSRDrivers
    >

    People are doing this because there are very approachable non-MS curated
    examples. The official examples are, to put it bluntly, crap.

    The WDM driver examples aren't bad by any means. They closely mirror how
    drivers work on *nix OSes. If people should use the new APIs Microsoft
    should show they are just as capable and well supported.

    Historically, adopting new MS technologies is a horrible business decision,
    unless you have millions of dollars to spend. MS is paid to write software,
    something most companies are not paid to do, even if software is an
    integral part of their business. Playing follow the leader can easily
    bankrupt a small to medium sized business.

    If it's stupid and it works, it's not stupid.


    Please don't get me wrong - I understand tge value pf moving forward. But
    effectively I can't. The WDK and EWDK are opaque and expect some workflow
    to ge followed that just isn't well articulated.


    Cheers,
    R0b0t1
  • Don_Burn-2Don_Burn-2 Posts: 1,620
    Calling KMDF "new technology" is idiotic, it has been in use over 10 years. A heck of a lot of the non-Microsoft samples out there would have to do a major rewrite to work up to the level you could call them crap.

    Yes Microsoft and others need to produce more KMDF samples. The last Driver Developer Conference had a presentation on how great KMDF was in the storage stack, but we never got a sample (yes I and others have done them commercially but we cannot give those out). I keep thinking about writing some samples as I phase into retirement, but the demand for drivers is strong enough that retirement is phasing in very slowly.

    We need to work to replace a number of the Microsoft samples that are still WDM.


    Don Burn
    Windows Driver Consulting
    Website: http://www.windrvr.com



    -----Original Message-----
    From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@gmail.com
    Sent: Wednesday, March 07, 2018 9:34 AM
    To: Windows System Software Devs Interest List <xxxxx@lists.osr.com>
    Subject: [ntdev] Help Stamp Out Sensless WDM Usage

    On Wed, Mar 7, 2018 at 7:04 AM, xxxxx@osr.com <mailto:xxxxx@osr.com> <xxxxx@lists.osr.com wrote:
    > <RANT>
    >
    > Folks... including and especially Microsoft folks,
    >
    > One of the common threads in this list is newbs writing WDM drivers. They ask us questions they get some answers. They go away. Their device drivers and especially filters live on as one of the more common causes of end user system crashes in modern times.
    >
    > We need a concerted effort to stomp out the senseless writing of WDM drivers.
    >
    > We need to scrub the samples to make sure there are no WDM drivers around (other than software only ?kernel services?). If you host example on GitHub or someplace else, if it?s a WDM driver , for heavens sakes make the readme say it?s a deprecated model.
    >
    > We need the WDK docs to very clear say, everywhere, that people should
    > be using WDM as a last resort only if they are not writing a file system or a kernel service. Shit, put it on every WDM doc page: IoXxxx, KeXxxx, etc.
    >
    > WDK doc folks... please take some time to focus on this goal. It?ll be time we?ll spent.
    >
    > People who mistakenly start with a WDM sample are not well served. They would be better off with no sample... though I doubt they would see this. Starting with some hideous shit from CodeProject, or an ancient sample from the WDK, is just an invitation to (a) frustrating the dev, (b) injecting bugs into kernel-mode.
    >
    > Whew, I feel better now.
    >
    > Peter
    > OSR
    > @OSRDrivers
    >

    People are doing this because there are very approachable non-MS curated examples. The official examples are, to put it bluntly, crap.

    The WDM driver examples aren't bad by any means. They closely mirror how drivers work on *nix OSes. If people should use the new APIs Microsoft should show they are just as capable and well supported.

    Historically, adopting new MS technologies is a horrible business decision, unless you have millions of dollars to spend. MS is paid to write software, something most companies are not paid to do, even if software is an integral part of their business. Playing follow the leader can easily bankrupt a small to medium sized business.

    If it's stupid and it works, it's not stupid.


    Please don't get me wrong - I understand tge value pf moving forward. But effectively I can't. The WDK and EWDK are opaque and expect some workflow to ge followed that just isn't well articulated.


    Cheers,
    R0b0t1
    --- NTDEV is sponsored by OSR Visit the list online at: MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers! Details at To unsubscribe, visit the List Server section of OSR Online at
  • Peter_ViscarolaPeter_Viscarola Posts: 6,649
    <quote>
    People are doing this because there are very approachable non-MS curated
    examples.
    </quote>

    Well, I think that's right. And a generally helpful comment.

    I also think, however, it is overly generous to call these samples "curated" if that implies they generally do not suck. Because to a large extent, they most certainly do.

    <quote>
    If it's stupid and it works, it's not stupid.
    </quote>

    If it's stupid, and you THINK it works but it doesn't REALLY work, it's WORSE than stupid. It's both stupid and dangerous. THAT'S THE PRIMARY PROBLEM WITH WDM.

    <quote>
    Calling KMDF "new technology" is idiotic, it has been in use over 10 years.
    </quote>

    This. KMDF was introduced in 2005, after a loong period of collaborative design and development with the Driver Development Community. Dozens of devs from the community took part. It was really a landmark project, because I am not aware of any major OS internals development effort undertaken by MSFT before or since, to take input from the community so seriously.

    <quote>
    The WDK and EWDK are opaque and expect some workflow
    to ge followed that just isn't well articulated.
    </quote>

    Oh, stop it. That's just silly on its face.

    No workflow can be simpler than VS "Create Project" -- a template project gets created -- "Build Solution" -- the project gets built. The you copy it to the target machine, install it, and you're done.

    And the workflow doesn't change based on the driver model, so the whole argument is specious.

    I've been writing drivers for Windows for about 20 years. I think I'm pretty good at it. WDM? Yeah, I know how to write a reasonably reliable WDM driver at this point. But it's really, really, painful, and it requires dragging soooo much ancient shit around. There is no chance -- none --
    exactly zero probability -- of a newb writing a properly reliable WDM driver from scratch. That is not true for WDF.

    LOOK... back when WDM was the only option, I was personally involved, working with MSFT, for YEARS trying to get the community to write reliable drivers. We tried shaming vendors, we tried praising vendors, we tried educating vendors, we tried writing samples, we tried bringing vendors in to labs and showing them the errors of their ways. Nothing, nothing, was successful in moving the needle in terms of driver reliability.

    It got so bad Microsoft finally acknowledged the problem, and invested in designing and implementing an entirely new driver model, that was modern, easier to use, and made writing reliable drivers something a reasonable engineer can accomplish. That new model was WDF.

    People need to stop fucking with a model that's been proven to be impractical, and use what we all know works.

    Peter
    OSR
    @OSRDrivers

    Peter Viscarola
    OSR
    @OSRDrivers

  • R0b0t1R0b0t1 Posts: 129
    On Wed, Mar 7, 2018 at 1:36 PM, xxxxx@osr.com <xxxxx@lists.osr.com> wrote:
    > <quote>
    > People are doing this because there are very approachable non-MS curated
    > examples.
    > </quote>
    >
    > Well, I think that's right. And a generally helpful comment.
    >
    > I also think, however, it is overly generous to call these samples "curated" if that implies they generally do not suck. Because to a large extent, they most certainly do.
    >
    > <quote>
    > If it's stupid and it works, it's not stupid.
    > </quote>
    >
    > If it's stupid, and you THINK it works but it doesn't REALLY work, it's WORSE than stupid. It's both stupid and dangerous. THAT'S THE PRIMARY PROBLEM WITH WDM.
    >

    "Kernel code is dangerous" should be no surprise.

    > <quote>
    > Calling KMDF "new technology" is idiotic, it has been in use over 10 years.
    > </quote>
    >
    > This. KMDF was introduced in 2005, after a loong period of collaborative design and development with the Driver Development Community. Dozens of devs from the community took part. It was really a landmark project, because I am not aware of any major OS internals development effort undertaken by MSFT before or since, to take input from the community so seriously.
    >
    > <quote>
    > The WDK and EWDK are opaque and expect some workflow
    > to ge followed that just isn't well articulated.
    > </quote>
    >
    > Oh, stop it. That's just silly on its face.
    >
    > No workflow can be simpler than VS "Create Project" -- a template project gets created -- "Build Solution" -- the project gets built. The you copy it to the target machine, install it, and you're done.
    >
    > And the workflow doesn't change based on the driver model, so the whole argument is specious.
    >

    My entire life I have been very unlucky when it comes to computers. I
    will or would try these things, and in this case I have tried these
    things, and had them not work. It is impossible for me to explain why
    or to troubleshoot the issue as I can't see the source code of the
    components involved. I've tried asking for help in this regard but all
    people can tell me to do is follow the instructions. I repeat the
    procedure with the same result. (At this point I've decided I will
    wait until I finish more of my hardware and can buy a separate test
    computer, but the issue is still unresolved. For now I do all driver
    testing in Linux, which I may have done anyway.)

    Some of the WDM driver tutorials use older EWDK (or maybe just WDK)
    releases that are more approachable than the Visual Studio workflow.
    The individual components are obvious and in some cases instructions
    are given for producing functioning drivers with MinGW.

    I don't expect that level of detail but it does show how well WDK
    drivers were/are understood. At this point those samples do seem
    ancient, and connecting them to newer APIs is hard.

    > I've been writing drivers for Windows for about 20 years. I think I'm pretty good at it. WDM? Yeah, I know how to write a reasonably reliable WDM driver at this point. But it's really, really, painful, and it requires dragging soooo much ancient shit around. There is no chance -- none --
    > exactly zero probability -- of a newb writing a properly reliable WDM driver from scratch. That is not true for WDF.
    >
    > LOOK... back when WDM was the only option, I was personally involved, working with MSFT, for YEARS trying to get the community to write reliable drivers. We tried shaming vendors, we tried praising vendors, we tried educating vendors, we tried writing samples, we tried bringing vendors in to labs and showing them the errors of their ways. Nothing, nothing, was successful in moving the needle in terms of driver reliability.
    >
    > It got so bad Microsoft finally acknowledged the problem, and invested in designing and implementing an entirely new driver model, that was modern, easier to use, and made writing reliable drivers something a reasonable engineer can accomplish. That new model was WDF.
    >
    > People need to stop fucking with a model that's been proven to be impractical, and use what we all know works.
    >

    The model isn't impractical. It is used by Linux. The difference is
    most drivers are not written at that level: there is a strong tendency
    to push everything into userspace as quickly as possible. If I want to
    create a fake hardware device, I can do so relatively safely by
    talking to a driver. If I want to monitor kernel driver events, I can
    usually do that in userspace. In some cases these things are ported
    backwards into kernel drivers, but not always. When they are, there is
    a very transparent framework bolted on top of the WDM-like model.

    WDF is not very transparent. Sadly I don't know what to ask for. It's
    not like I expect someone to write precisely what I want so I can copy
    it, but at the same time, knowing *what you need* is harder than
    knowing *what exists.*


    I will take your advice to heart as best I can and give up on trying
    to do anything with WDM.

    Cheers,
    R0b0t1
  • anton_bassovanton_bassov Posts: 4,777
    >> People need to stop fucking with a model that's been proven to be impractical,
    >>and use what we all know works.



    > The model isn't impractical. It is used by Linux.

    If you don't mind, could you please expand it a bit - I am just desperate to learn about Linux drivers
    relying upon IO Manager and IRPs, as well as upon asynch IO completion routines in the kernel, device stacks with well-defined rules, and all other Windows-specific features. I am holding my breath in anticipation....


    Anton Bassov
  • R0b0t1R0b0t1 Posts: 129
    On Wed, Mar 7, 2018 at 6:34 PM, xxxxx@hotmail.com
    <xxxxx@lists.osr.com> wrote:
    >>> People need to stop fucking with a model that's been proven to be impractical,
    >>>and use what we all know works.
    >
    >
    >
    >> The model isn't impractical. It is used by Linux.
    >
    > If you don't mind, could you please expand it a bit - I am just desperate to learn about Linux drivers
    > relying upon IO Manager and IRPs, as well as upon asynch IO completion routines in the kernel, device stacks with well-defined rules, and all other Windows-specific features. I am holding my breath in anticipation....
    >

    The names are different, the patterns not so much. E.g. does Microsoft
    have some kind of monopoly on asynchronous IO?

    Cheers,
    R0b0t1
  • anton_bassovanton_bassov Posts: 4,777
    > The names are different, the patterns not so much.


    ....which means that your knowledge of both Linux and Windows kernel-mode programming is critically close to zero.In fact, it is hard too imagine two systems that are THAT drastically different from one another, both architecturally and philosophically....


    > E.g. does Microsoft have some kind of monopoly on asynchronous IO?

    Actually, you are almost there.Indeed, Windows NT was the first OS to introduce an asynch IO model that was absolutely generic for all device types.UNIX-like systems have not had had something like that for AT LEAST one more decade. Historically they had been making a distinction between asynch pipe/socket IO with select()/poll() semantics, and asynch disk IO that is concerned about the actual IO completion, rather than data/buffer space availability the way pipes and sockets do. Although the former was the same across all UNIX-like systems, the latter was vendor-specific.

    Solaris was the first UNIX-like system to introduce Windows-like IO completion ports, and FreeBSD followed the same path with introducing kqueue that relies upon the same principles.
    Linux still does not really have it - its epoll() does not offer the same functionality that kqueue and Solaris completion ports do.


    BTW, once we are at it, check the Linux kernel and see how many devices other than pipes and sockets implement asynch IO handlers. Out of those who do, check how many of them actually make use of kernel asynch IO model (i.e. make use of iocb) , rather than just delegating the whole thing to a dedicated thread that, in actuality, works on synchronous basis (i.e. implement it the same way that GNU C library does in the userland)



    Anton Bassov
  • matt_sykesmatt_sykes Posts: 233
    WDM documentation has been disparaging from MSDN recently, only WDF, so it looks like this is happening Peter.


    Of course when someone has to write a WDM driver they are going to struggle to find the info...
  • R0b0t1R0b0t1 Posts: 129
    On Thu, Mar 8, 2018 at 2:46 AM, xxxxx@hotmail.com
    <xxxxx@lists.osr.com> wrote:
    >> The names are different, the patterns not so much.
    >
    >
    > ....which means that your knowledge of both Linux and Windows kernel-mode programming is critically close to zero.In fact, it is hard too imagine two systems that are THAT drastically different from one another, both architecturally and philosophically....
    >
    >

    Can you provide a concrete example? I've had what might be a similar
    argument presented to me about IO completion ports, or
    WaitForMultipleObjects, but despite the specifics of the API
    equivalent information is available on both systems. In the case of
    IO, they tend to view things from the opposite side.

    >> E.g. does Microsoft have some kind of monopoly on asynchronous IO?
    >
    > Actually, you are almost there.Indeed, Windows NT was the first OS to introduce an asynch IO model that was absolutely generic for all device types.UNIX-like systems have not had had something like that for AT LEAST one more decade. Historically they had been making a distinction between asynch pipe/socket IO with select()/poll() semantics, and asynch disk IO that is concerned about the actual IO completion, rather than data/buffer space availability the way pipes and sockets do. Although the former was the same across all UNIX-like systems, the latter was vendor-specific.
    >
    > Solaris was the first UNIX-like system to introduce Windows-like IO completion ports, and FreeBSD followed the same path with introducing kqueue that relies upon the same principles.
    > Linux still does not really have it - its epoll() does not offer the same functionality that kqueue and Solaris completion ports do.
    >

    Importantly epoll can support these facilities. For some of them it
    does, such as timerfds. In other cases the functionality is elsewhere
    for historical reasons.

    >
    > BTW, once we are at it, check the Linux kernel and see how many devices other than pipes and sockets implement asynch IO handlers. Out of those who do, check how many of them actually make use of kernel asynch IO model (i.e. make use of iocb) , rather than just delegating the whole thing to a dedicated thread that, in actuality, works on synchronous basis (i.e. implement it the same way that GNU C library does in the userland)
    >

    So are you able to comment on the Windows implementation as a comparison?
  • Peter_ViscarolaPeter_Viscarola Posts: 6,649
    <quote>
    WDM documentation has been disparaging from MSDN recently,
    </quote>

    Don't know if you meant "disappearing" instead of "disparaging"... but not nearly enough. I'd like MORE disappearing AND active disparaging of WDM. There's still WAAAAY too much legacy documentation, and as our colleague Mr. R0b0t1 stated, people file "curated" WDM samples (that mostly suck, but whatever) and then have enough MSDN to move those samples further into the world of fucked-up.

    <quote>
    Of course when someone has to write a WDM driver
    </quote>

    When would someone HAVE to write a WDM driver, exactly?

    <quote>
    they are going to struggle to find the info
    </quote>

    Good. AFAIC, they should need a secret decoder ring and a personal permission slip signed in Joe Belfiore's blood before they can write a WDM driver.

    OK, maybe I'm exaggerating a LITTLE...

    But shitty filters written in WDM by are now an actual, quantifiable, problem in terms of system stability (and upgrade-ability). I'm soooo fucking tired of seeing newbs posting WDM questions and getting a specific, pointed, answer, and then going away to wreck unintentional havoc. It's not that they WANT to do the wrong thing. They just have no way of knowing that's what they're doing. No reasonable dev WOULD know. That's the fatal flaw of WDM in the 21st Century, and why people need to learn to "just say no" to WDM. Hence my (continued) rant.

    Peter
    OSR
    @OSRDrivers

    Peter Viscarola
    OSR
    @OSRDrivers

  • Peter_ViscarolaPeter_Viscarola Posts: 6,649
    <quote>
    So are you able to comment on the Windows implementation as a comparison
    </quote>

    I can -- And I'll even stop ranting for a moment while I do it.

    In Windows, it's generally considered to be "bad form" to synchronously wait for a device to do your bidding. The driver model is designed to be basically asynchronous and event-driven:

    1) You get a request (read, write, IOCTL).

    2a) If the device is free, you program the device to perform the requested I/O operation and save some context for the pending request.

    2b) If the device is not free, you queue the request and RETURN from your driver.

    3) Assuming you start the request on the device, your driver code then RETURNS, leaving the request in progress on the device.

    4) The device interrupts to indicate the request has been completed. Your driver queues a DpcForIsr from this ISR.

    5) In your DpcForIsr you retrieve the context for the pending request that you saved in step 2a, above. You complete whatever processing is required for the request (if you have a DMA type device, there's probably very little to do... if you have a PIO type device and this is a READ operation from the user, you probably have to move the data from the device to the user's data buffer). Then you complete the request.

    6) With the request completed, and your device now "idle", you look to see if you have any pending requests queued for your device that you can now start.

    I think the model is simplicity itself ;-)

    Peter
    OSR
    @OSRDrivers

    Peter Viscarola
    OSR
    @OSRDrivers

  • Don_Burn-2Don_Burn-2 Posts: 1,620
    The only time in recent memory I have had to write a WDM driver was to
    develop a filter for a bus driver. But these are so rare that it should not
    matter.


    Don Burn
    Windows Driver Consulting
    Website: http://www.windrvr.com



    -----Original Message-----
    From: xxxxx@lists.osr.com
    [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@osr.com
    Sent: Friday, March 09, 2018 10:21 AM
    To: Windows System Software Devs Interest List <xxxxx@lists.osr.com>
    Subject: RE:[ntdev] Help Stamp Out Sensless WDM Usage

    <quote>
    WDM documentation has been disparaging from MSDN recently, </quote>

    Don't know if you meant "disappearing" instead of "disparaging"... but not
    nearly enough. I'd like MORE disappearing AND active disparaging of WDM.
    There's still WAAAAY too much legacy documentation, and as our colleague Mr.
    R0b0t1 stated, people file "curated" WDM samples (that mostly suck, but
    whatever) and then have enough MSDN to move those samples further into the
    world of fucked-up.

    <quote>
    Of course when someone has to write a WDM driver </quote>

    When would someone HAVE to write a WDM driver, exactly?

    <quote>
    they are going to struggle to find the info </quote>

    Good. AFAIC, they should need a secret decoder ring and a personal
    permission slip signed in Joe Belfiore's blood before they can write a WDM
    driver.

    OK, maybe I'm exaggerating a LITTLE...

    But shitty filters written in WDM by are now an actual, quantifiable,
    problem in terms of system stability (and upgrade-ability). I'm soooo
    fucking tired of seeing newbs posting WDM questions and getting a specific,
    pointed, answer, and then going away to wreck unintentional havoc. It's not
    that they WANT to do the wrong thing. They just have no way of knowing
    that's what they're doing. No reasonable dev WOULD know. That's the fatal
    flaw of WDM in the 21st Century, and why people need to learn to "just say
    no" to WDM. Hence my (continued) rant.

    Peter
    OSR
    @OSRDrivers


    ---
    NTDEV is sponsored by OSR

    Visit the list online at:
    <http://www.osronline.com/showlists.cfm?list=ntdev>;

    MONTHLY seminars on crash dump analysis, WDF, Windows internals and software
    drivers!
    Details at <http://www.osr.com/seminars>;

    To unsubscribe, visit the List Server section of OSR Online at
    <http://www.osronline.com/page.cfm?name=ListServer>;
  • Peter_ViscarolaPeter_Viscarola Posts: 6,649
    <quote>
    The only time in recent memory I have had to write a WDM driver was to
    develop a filter for a bus driver. But these are so rare that it should not
    matter.
    </quote>

    Good one, thank you for that. We did that here recently as well.

    Rare and unsupported as well, we should note.

    Peter
    OSR
    @OSRDrivers

    Peter Viscarola
    OSR
    @OSRDrivers

  • Mark_RoddyMark_Roddy Posts: 4,262
    shitty devs " wreck[ing] unintentional havoc" would be a good thing. It is
    when they wreak havoc that things go wrong.

    Mark Roddy

    On Fri, Mar 9, 2018 at 10:50 AM, xxxxx@osr.com
    wrote:

    >
    > The only time in recent memory I have had to write a WDM driver was to
    > develop a filter for a bus driver. But these are so rare that it should
    > not
    > matter.
    >
    >
    > Good one, thank you for that. We did that here recently as well.
    >
    > Rare and unsupported as well, we should note.
    >
    > Peter
    > OSR
    > @OSRDrivers
    >
    >
    > ---
    > NTDEV is sponsored by OSR
    >
    > Visit the list online at: showlists.cfm?list=ntdev>
    >
    > MONTHLY seminars on crash dump analysis, WDF, Windows internals and
    > software drivers!
    > Details at
    >
    > To unsubscribe, visit the List Server section of OSR Online at <
    > http://www.osronline.com/page.cfm?name=ListServer>;
    >
  • Peter_ViscarolaPeter_Viscarola Posts: 6,649
    <quote>
    shitty devs " wreck[ing] unintentional havoc" would be a good thing. It is
    when they wreak havoc that things go wrong.
    </quote>

    Ducking auto-correct, I swear. I'll go back and edit that post. Oh, wait...

    Peter
    OSR
    @OSRDrivers

    Peter Viscarola
    OSR
    @OSRDrivers

  • Mark_RoddyMark_Roddy Posts: 4,262
    registry filter drivers. I suppose you could write one using KMDF but it
    would add unneeded complexity rather than reducing it. Also you could write
    a bus filter driver using KMDF and then escape into WDM to do all the bus
    filtering :-)




    Mark Roddy

    On Fri, Mar 9, 2018 at 10:50 AM, xxxxx@osr.com
    wrote:

    >
    > The only time in recent memory I have had to write a WDM driver was to
    > develop a filter for a bus driver. But these are so rare that it should
    > not
    > matter.
    >
    >
    > Good one, thank you for that. We did that here recently as well.
    >
    > Rare and unsupported as well, we should note.
    >
    > Peter
    > OSR
    > @OSRDrivers
    >
    >
    > ---
    > NTDEV is sponsored by OSR
    >
    > Visit the list online at: showlists.cfm?list=ntdev>
    >
    > MONTHLY seminars on crash dump analysis, WDF, Windows internals and
    > software drivers!
    > Details at
    >
    > To unsubscribe, visit the List Server section of OSR Online at <
    > http://www.osronline.com/page.cfm?name=ListServer>;
    >
  • Registry Filters, Object Manager Filters, Process Manager Filters... these
    are all Kernel Service, which I already mentioned are an entirely separate
    category of beast. WDM for this use is entirely fine.

    Peter
    OSR
    @OSRDrivers




    On Sat, Mar 10, 2018 at 10:43 AM, xxxxx@gmail.com <
    xxxxx@lists.osr.com> wrote:

    > registry filter drivers. I suppose you could write one using KMDF but it
    > would add unneeded complexity rather than reducing it. Also you could write
    > a bus filter driver using KMDF and then escape into WDM to do all the bus
    > filtering :-)
    >
    >
    >
    >
    > Mark Roddy
    >
    > On Fri, Mar 9, 2018 at 10:50 AM, xxxxx@osr.com
    > wrote:
    >
    >>
    >> The only time in recent memory I have had to write a WDM driver was to
    >> develop a filter for a bus driver. But these are so rare that it should
    >> not
    >> matter.
    >>
    >>
    >> Good one, thank you for that. We did that here recently as well.
    >>
    >> Rare and unsupported as well, we should note.
    >>
    >> Peter
    >> OSR
    >> @OSRDrivers
    >>
    >>
    >> ---
    >> NTDEV is sponsored by OSR
    >>
    >> Visit the list online at: > lists.cfm?list=ntdev>
    >>
    >> MONTHLY seminars on crash dump analysis, WDF, Windows internals and
    >> software drivers!
    >> Details at
    >>
    >> To unsubscribe, visit the List Server section of OSR Online at <
    >> http://www.osronline.com/page.cfm?name=ListServer>;
    >>
    >
    > --- NTDEV is sponsored by OSR Visit the list online at: MONTHLY seminars
    > on crash dump analysis, WDF, Windows internals and software drivers!
    > Details at To unsubscribe, visit the List Server section of OSR Online at
  • Jan_BottorffJan_Bottorff Posts: 464
    I think a HUGE problem for a newbie Window’s driver writer’s is figuring out he correct kind of driver to write. Other platforms tend not to have all the different flavors, they just have “a driver”. Even an experienced developer like myself sometimes has to look carefully and do a lot of digging what kind of driver is needed. For example, a network kernel winsock client that does not process IRPs would be more of a kernel service, but one that can be opened by an application (meaning it can process IRPS) would more likely be something like a root-enumerated PnP driver. If I had a kernel service that I want to control from powershell via WMI method calls, I would need to make it a driver with a do almost nothing virtual device object, just so the OS has someplace to send WMI IRPS.

    Perhaps something like a decision tree in the WDK documentation on how to pick the correct kind of driver would help. A newbie will look at this thread and be clueless how a “driver”, written to the WDM API (to be avoided), is a different thing than a “kernel service” written to the WDM API (which is acceptable). This terminology is not spelled out, it’s just a set of slightly fuzzy categories that have informally formed over the years.

    Another thing that could help newbie driver developers would be if the VisualStudio driver wizard knew how to make a variety of specialized class drivers, based on the latest thinking on what’s “optimal”. So then there would be no need to hunt through the samples to find something sort of similar. You would just say dear VS Wizard, make me generic storage controller driver, and it would (today) spit out a storport driver for a ramdisk or something, and tomorrow it might spit our a KMDF driver with the extra storage support.

    Even though I see lots of value in miniports, and other class specific frameworks, I wonder if these just create confusion and ignorance in the long term. Windows kernel developers tend to have boundaries on their driver interface knowledge, significantly due to the miniport model. If you work for a large corporation on the same project for many years, the miniport model tends to keep you ignorant about anything outside your little world. There might be some shift occurring in this, as it seems like Microsoft is moving toward a NIC miniport being a KMDF driver, with some network specific queue ring interfaces.

    Jan


    From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@gmail.com
    Sent: Saturday, March 10, 2018 9:52 AM
    To: Windows System Software Devs Interest List
    Subject: Re: [ntdev] Help Stamp Out Sensless WDM Usage

    Registry Filters, Object Manager Filters, Process Manager Filters... these are all Kernel Service, which I already mentioned are an entirely separate category of beast. WDM for this use is entirely fine.

    Peter
    OSR
    @OSRDrivers




    On Sat, Mar 10, 2018 at 10:43 AM, xxxxx@gmail.com > wrote:
    registry filter drivers. I suppose you could write one using KMDF but it would add unneeded complexity rather than reducing it. Also you could write a bus filter driver using KMDF and then escape into WDM to do all the bus filtering :-)




    Mark Roddy

    On Fri, Mar 9, 2018 at 10:50 AM, xxxxx@osr.com > wrote:

    The only time in recent memory I have had to write a WDM driver was to
    develop a filter for a bus driver. But these are so rare that it should not
    matter.


    Good one, thank you for that. We did that here recently as well.

    Rare and unsupported as well, we should note.

    Peter
    OSR
    @OSRDrivers


    ---
    NTDEV is sponsored by OSR

    Visit the list online at:

    MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers!
    Details at

    To unsubscribe, visit the List Server section of OSR Online at

    --- NTDEV is sponsored by OSR Visit the list online at: MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers! Details at To unsubscribe, visit the List Server section of OSR Online at

    --- NTDEV is sponsored by OSR Visit the list online at: MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers! Details at To unsubscribe, visit the List Server section of OSR Online at
  • Mark_RoddyMark_Roddy Posts: 4,262
    On Sat, Mar 10, 2018 at 6:03 PM, xxxxx@pmatrix.com wrote:

    >
    >
    >
    > Perhaps something like a decision tree in the WDK documentation on how to
    > pick the correct kind of driver would help. A newbie will look at this
    > thread and be clueless how a “driver”, written to the WDM API (to be
    > avoided), is a different thing than a “kernel service” written to the WDM
    > API (which is acceptable). This terminology is not spelled out, it’s just a
    > set of slightly fuzzy categories that have informally formed over the years.
    >
    >
    >
    >
    >
    Um, you mean like this: Choosing a driver model


    Mark Roddy
  • matt_sykesmatt_sykes Posts: 233
    Peter wrote: "When would someone HAVE to write a WDM driver, exactly?"

    Kernel Service drivers. I have had to write a few in my time, in fact wrote one last year.
  • Jamey_KirbyJamey_Kirby Posts: 363
    I left software development about 12 years ago (before WDF was a thing). Since I have returned, I've written one new driver that was a virtual storport miniport. These beasts live somewhere between a WDM and a miniport.

    Recently I have been working on a older WDM driver written by someone else about 10 years ago. I have been considering the idea of porting the driver to a WDF/KMDF driver, but have put that on the back burner because of the demands to get code shipped.

    However, after reading Peter's post, I have decided to move the port to the foreground. How difficult can it be; right?

    After reading the documentation, I see that if you want to receive PNP notifications, you must be a WDM driver. In the driver I am working on, it filters IRP_MJ_PNP in order to detect special files on the volume (this is a volume filter driver, not an FS volume filter). This is done to maintain the power pageable bit while the filter is active.

    Does this requirement prevent me from porting to WDF/KMDF?

    -- Jamey

    BTW: When I reply via email, the emails get lost. I sent this same post via email, and the email is nowhere to be found, not even in my outbox. Strange. So I have to come here to the website to post. If this gets posted twice, I apologize.
  • Peter_ViscarolaPeter_Viscarola Posts: 6,649
    >>When would someone HAVE to write a WDM driver, exactly

    >Kernel Service drivers.

    Yes, absolutely agree that writing Kernel Services using WDM is both entirely appropriate and a Best Practice. I *thought* I said that in my long initial rant, but perhaps it was somewhere else that I ranted this carve-out.

    But, in any case, agreed: Kernel Services should be WDM (though they do not HAVE to be WDM).

    Peter
    OSR
    @OSRDrivers

    Peter Viscarola
    OSR
    @OSRDrivers

  • Don_Burn-2Don_Burn-2 Posts: 1,620
    Peter,

    You did say it, and for a simple kernel service I totally agree. I
    recently did rewrite one kernel service driver to KMDF, the original WDM had
    something like 20 cancel safe queues to do weird things. Moving it to KMDF
    queues made the code so much nicer!


    Don Burn
    Windows Driver Consulting
    Website: http://www.windrvr.com




    -----Original Message-----
    From: xxxxx@lists.osr.com
    [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@osr.com
    Sent: Monday, March 12, 2018 11:55 AM
    To: Windows System Software Devs Interest List <xxxxx@lists.osr.com>
    Subject: RE:[ntdev] Help Stamp Out Sensless WDM Usage

    >>When would someone HAVE to write a WDM driver, exactly

    >Kernel Service drivers.

    Yes, absolutely agree that writing Kernel Services using WDM is both
    entirely appropriate and a Best Practice. I *thought* I said that in my
    long initial rant, but perhaps it was somewhere else that I ranted this
    carve-out.

    But, in any case, agreed: Kernel Services should be WDM (though they do not
    HAVE to be WDM).

    Peter
    OSR
    @OSRDrivers


    ---
    NTDEV is sponsored by OSR

    Visit the list online at:
    <http://www.osronline.com/showlists.cfm?list=ntdev>;

    MONTHLY seminars on crash dump analysis, WDF, Windows internals and software
    drivers!
    Details at <http://www.osr.com/seminars>;

    To unsubscribe, visit the List Server section of OSR Online at
    <http://www.osronline.com/page.cfm?name=ListServer>;
  • Peter_ViscarolaPeter_Viscarola Posts: 6,649
    Warning: Thread Drift Ahead

    <quote>
    I see that if you want to receive PNP notifications, you must be a WDM driver
    </quote>

    No! You can still filter and receive these. WdfDeviceInitAssignWdmIrpPreprocessCallback is your friend.

    <quote>
    In the driver I am working on, it
    filters IRP_MJ_PNP in order to detect special files on the volume (this is a
    volume filter driver, not an FS volume filter). This is done to maintain the
    power pageable bit while the filter is active.
    </quote>

    You are aware that there's WdfDeviceSetSpecialFileSupport ?

    Peter
    OSR
    @OSRDrivers

    Peter Viscarola
    OSR
    @OSRDrivers

  • Tim_RobertsTim_Roberts Posts: 12,567
    xxxxx@osr.com wrote:
    > <quote>
    > I see that if you want to receive PNP notifications, you must be a WDM driver
    > </quote>
    >
    > No! You can still filter and receive these. WdfDeviceInitAssignWdmIrpPreprocessCallback is your friend.

    I suspect the author of the original quote really meant "at least a WDM
    driver".  All KMDF drivers are also WDM drivers.

    --
    Tim Roberts, xxxxx@probo.com
    Providenza & Boekelheide, Inc.

    Tim Roberts, [email protected]
    Providenza & Boekelheide, Inc.

  • Don_Burn-2Don_Burn-2 Posts: 1,620
    Peter,

    Does KMDF support normal flag handling for filter drivers? I thought
    it did, so since the original question was on a filter, this should be let
    the framework do it.


    Don Burn
    Windows Driver Consulting
    Website: http://www.windrvr.com




    -----Original Message-----
    From: xxxxx@lists.osr.com
    [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@osr.com
    Sent: Monday, March 12, 2018 12:01 PM
    To: Windows System Software Devs Interest List <xxxxx@lists.osr.com>
    Subject: RE:[ntdev] Help Stamp Out Sensless WDM Usage

    Warning: Thread Drift Ahead

    <quote>
    I see that if you want to receive PNP notifications, you must be a WDM
    driver </quote>

    No! You can still filter and receive these.
    WdfDeviceInitAssignWdmIrpPreprocessCallback is your friend.

    <quote>
    In the driver I am working on, it
    filters IRP_MJ_PNP in order to detect special files on the volume (this is a
    volume filter driver, not an FS volume filter). This is done to maintain the
    power pageable bit while the filter is active.
    </quote>

    You are aware that there's WdfDeviceSetSpecialFileSupport ?

    Peter
    OSR
    @OSRDrivers


    ---
    NTDEV is sponsored by OSR

    Visit the list online at:
    <http://www.osronline.com/showlists.cfm?list=ntdev>;

    MONTHLY seminars on crash dump analysis, WDF, Windows internals and software
    drivers!
    Details at <http://www.osr.com/seminars>;

    To unsubscribe, visit the List Server section of OSR Online at
    <http://www.osronline.com/page.cfm?name=ListServer>;
  • Jamey_KirbyJamey_Kirby Posts: 363
    Peter, no. I just started digging into the documentation last night. I have zero WDF/KMDF experience. It will take me a few days to come up to speed.
  • Jamey_KirbyJamey_Kirby Posts: 363
    I've run into my first issue :)

    In the WDM driver, I would check device characteristics to see if the device was something we were not interested in filtering. Here is a snippet of WDM code:

    NTSTATUS FilterAddDevice(_In_ PDRIVER_OBJECT Driver,
    _In_ PDEVICE_OBJECT PhysicalDevice) {
    if (FlagOn(PhysicalDevice->Characteristics, FILE_REMOVABLE_MEDIA) ||
    FlagOn(PhysicalDevice->Characteristics, FILE_READ_ONLY_DEVICE) ||
    FlagOn(PhysicalDevice->Characteristics, FILE_READ_ONLY_VOLUME)) {
    // We are not interested in removable media. Exit with nothing to do.
    return STATUS_SUCCESS;
    }

    This would skip filtering read-only and removable devices. In WDF, I cannot get access to the lower device object to get the characteristics. I assume WdfDeviceCreate() attaches to the lower device by me having called WdfFdoInitSetFilter() prior to calling WdfDeviceCreate().

    So, let's say I have called WdfDeviceCreate(), I check the characteristics after calling WdfGetDeviceCharacteristsic(), and determine I really do not want to filter this device.

    How do I go about detaching from the lower device before exiting EvtDevicedd() function? I don't want to have a null filter sitting there just passing data along; do I?

    Maybe I am missing something. I am reading the documentation while trying to do the port.

    -- Jamey
  • Doron_HolanDoron_Holan Posts: 10,353
    WdfFdoInitWdmGetPhysicalDevice and you have the same code as below without creating a WDFDEVICE

    -----Original Message-----
    From: xxxxx@lists.osr.com <xxxxx@lists.osr.com> On Behalf Of xxxxx@gmail.com
    Sent: Monday, March 12, 2018 3:09 PM
    To: Windows System Software Devs Interest List <xxxxx@lists.osr.com>
    Subject: RE:[ntdev] Help Stamp Out Sensless WDM Usage

    I've run into my first issue :)

    In the WDM driver, I would check device characteristics to see if the device was something we were not interested in filtering. Here is a snippet of WDM code:

    NTSTATUS FilterAddDevice(_In_ PDRIVER_OBJECT Driver,
    _In_ PDEVICE_OBJECT PhysicalDevice) {
    if (FlagOn(PhysicalDevice->Characteristics, FILE_REMOVABLE_MEDIA) ||
    FlagOn(PhysicalDevice->Characteristics, FILE_READ_ONLY_DEVICE) ||
    FlagOn(PhysicalDevice->Characteristics, FILE_READ_ONLY_VOLUME)) {
    // We are not interested in removable media. Exit with nothing to do.
    return STATUS_SUCCESS;
    }

    This would skip filtering read-only and removable devices. In WDF, I cannot get access to the lower device object to get the characteristics. I assume WdfDeviceCreate() attaches to the lower device by me having called WdfFdoInitSetFilter() prior to calling WdfDeviceCreate().

    So, let's say I have called WdfDeviceCreate(), I check the characteristics after calling WdfGetDeviceCharacteristsic(), and determine I really do not want to filter this device.

    How do I go about detaching from the lower device before exiting EvtDevicedd() function? I don't want to have a null filter sitting there just passing data along; do I?

    Maybe I am missing something. I am reading the documentation while trying to do the port.

    -- Jamey


    ---
    NTDEV is sponsored by OSR

    Visit the list online at: <https://na01.safelinks.protection.outlook.com/?url=http://www.osronline.com/showlists.cfm?list=ntdev&amp;data=04|01|[email protected]|b170acf17feb44653aa408d58865ec55|72f988bf86f141af91ab2d7cd011db47|1|0|636564893692879141|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwifQ==|-1&amp;sdata=TaGnZx0gH69krNZX6tR0bF/BHdGNw1oFFWsA59N5xQ8=&amp;reserved=0&gt;

    MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers!
    Details at <https://na01.safelinks.protection.outlook.com/?url=http://www.osr.com/seminars&amp;data=04|01|[email protected]|b170acf17feb44653aa408d58865ec55|72f988bf86f141af91ab2d7cd011db47|1|0|636564893692879141|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwifQ==|-1&amp;sdata=PiGdWxQiB0r7NM/BMgCl1Y7ynNC+HXlWlOdpLzpJHQE=&amp;reserved=0&gt;

    To unsubscribe, visit the List Server section of OSR Online at <https://na01.safelinks.protection.outlook.com/?url=http://www.osronline.com/page.cfm?name=ListServer&amp;data=04|01|[email protected]|b170acf17feb44653aa408d58865ec55|72f988bf86f141af91ab2d7cd011db47|1|0|636564893692879141|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwifQ==|-1&amp;sdata=SQYNsVINWkkP+Nek6jjxlgWDJ/QCSWrreOK3Rso+CPc=&amp;reserved=0&gt;
  • Jamey_KirbyJamey_Kirby Posts: 363
    Thanks! I hadn't found that function yet.

    One more thing: When I call WdfFoInitSetFilter(), does it also propagate the device type? In my WDM code, I had to propagate the device type to the filter device object by hand.
    The WDK says it "propagates flags & characteristics". However, it seems reasonable to expect the device type to be propagated in the context of a filter driver. In the WDK sample kbdfilter driver, it sets the device type as follows:

    WdfDeviceInitSetDeviceType(DeviceInit, FILE_DEVICE_KEYBOARD);

    Otherwise, I have to have code like this:

    // Get the device property class GUID.
    ULONG ResultLength = 0;
    WCHAR PropertyBuffer[MAX_UNICODE_STACK_BUFFER_LENGTH];
    ULONG BufferLength = MAX_UNICODE_STACK_BUFFER_LENGTH;
    if (NT_ERROR(WdfFdoInitQueryProperty(DeviceInit, DevicePropertyClassGuid,
    BufferLength, PropertyBuffer, &ResultLength))) {
    // We cannot attach to this device, so do nothing.
    return STATUS_SUCCESS;
    }

    // Double check that someone has not tried to load this driver
    // on an unsupported device type.
    UNICODE_STRING DevGuid;
    RtlInitUnicodeString(&DevGuid, PropertyBuffer);
    if (RtlEqualUnicodeString(&DevGuid, &g_VolumeSnapshotGUID, TRUE)) {
    // If this is a snapshot, we flag it as a virtual disk.
    WdfDeviceInitSetDeviceType(DeviceInit, FILE_DEVICE_VIRTUAL_DISK);
    }
    else if (RtlEqualUnicodeString(&DevGuid, &g_VolumeGUID, TRUE)) {
    // If this is a volume, we flag it as a disk.
    WdfDeviceInitSetDeviceType(DeviceInit, FILE_DEVICE_DISK);
    }
    else {
    // Unsupported device type, so do nothing.
    return STATUS_SUCCESS;
    }


    -- Jamey
Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!