Windows System Software -- Consulting, Training, Development -- Unique Expertise, Guaranteed Results

Home NTDEV
Before Posting...
Please check out the Community Guidelines in the Announcements and Administration Category.

More Info on Driver Writing and Debugging


The free OSR Learning Library has more than 50 articles on a wide variety of topics about writing and debugging device drivers and Minifilters. From introductory level to advanced. All the articles have been recently reviewed and updated, and are written using the clear and definitive style you've come to expect from OSR over the years.


Check out The OSR Learning Library at: https://www.osr.com/osr-learning-library/


С++ in kernel

13»

Comments

  • Prokash_SinhaProkash_Sinha Member - All Emails Posts: 210
    via Email
    I read this somewhere before, thanks for putting it up here, Anton ...
    Perhaps things changed in Modern C++, waiting to read the upcoming NT
    insider articles to see...
    Pretty much everything is tied to std::. And it is
    still there, so runtime still think it can generate exceptions...

    Exceptional C++, and More exceptional C++ ( little outdated) but a
    good reference to understand what could go wrong, how to handle those
    situations.

    Back in those days, we were writing NDIS based drivers on Windows 98,
    we were using all C memory mgmt API. Then RTL*() came and it was a
    real joy, seriously. For C++ there are a lot of features that I would
    love to use, if they take the excption_handler out or a way to disable
    it all together :-)

    -Pro
  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 8,108

    Mr @MBond2 wrote:

    no one has presented any compelling argument to me why any part of C++ represents 'super C' or anything except a regression

    Really? So, you’re going to argue that strong type checking is a regression? Likewise, #define is better than constexpr? That having to code the number of elements in an array, and keep that consistent, is better than range-based for?

    I’d love to hear that.

    Peter

    Peter Viscarola
    OSR
    @OSRDrivers

  • Yan_Vugenfirer-4Yan_Vugenfirer-4 Member Posts: 11

    Shouldn't the question be "Rust in kernel"?
    We are in the 21st century.

    In any case, I agree with Peter on:
    "3. Full-On OO C++: This approach is what I think most C Language folks envision when they think of "drivers in C++" ... and it makes them shudder. I don't commonly see this approach anymore, except from folks who are on dev teams where OO is a way of life and they really hate thinking in C Language interface terms. Having seen, and tried to understand and debug, fully class-based, object oriented approaches to writing Windows drivers I can personally attest to how very painful this can be."
    It is great until you need to add more members to the team or pass your code to someone else.
    But 1 and 2, make a lot of sense, in my opinion.

    Best regards,
    Yan.

  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 8,108

    Shouldn't the question be "Rust in kernel"?

    Well, we _could _have that discussion, but probably in a separate thread.

    FYI, it's been done. Even I managed to write some mess in Rust and got DriverEntry to be called and DbgPrint to output stuff. But it's a super-big mess right now. And you wind-up with a lot of "unsafe" code.

    Peter

    Peter Viscarola
    OSR
    @OSRDrivers

  • Tim_RobertsTim_Roberts Member - All Emails Posts: 13,641

    I've said in the past that the concept of a "Common Language Runtime for Kernel" makes a great deal of sense. Most drivers are largely plumbing around a core set of boilerplate modules. So, embed the boilerplate in a well-tested and proven CLRK, and now one can use any of the .NET languages to write drivers. C#, F#, IronPython all become viable. KMDF is an excellent prototype for the CLRK.

    Plus, the acronym is already pronounceable...

    Tim Roberts, [email protected]
    Providenza & Boekelheide, Inc.

  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 8,108

    the concept of a "Common Language Runtime for Kernel" makes a great deal of sense.

    Interesting. I'd like to understand better. Would you say more about what this would do and how it would work?

    For example -- in MY mind -- the primary issue for supporting C# in the kernel is garbage collection. And, while mapping the APIs would be painful we DO already have access to the necessary "goo" to enable it.

    Similarly in Rust: To make an interface truly useful and Rusty the OS interface functions need to WORK like Rust functions, and therefore take and return appropriately Rusty parameters. For example, in Rust, the standard way to return status and data from a function is with a Result. So (ignoring the input arguments), we might have:

    match WdfDeviceCreate(...) {
        Ok(device) => device,
        Err(err) => {DbgPrint!("WdfDeviceCreateFailed: {:X}", err);  Return(err)},
    };
    

    or, more colloquially (and foregoing the DbgPrint for the sake of brevity):

    device =WdfDeviceCreate(...)?;
    

    I think that's right. One thing about Rust is that if you don't use for it, like, two weeks, you forget everything you learned.

    Anyhow, my point really is: How would CLRK help here?

    Peter

    Peter Viscarola
    OSR
    @OSRDrivers

  • RourkeRourke Member Posts: 60

    @Peter_Viscarola_(OSR) said:
    So, you’re going to argue that strong type checking is a regression? Likewise, #define is better than constexpr? That having to code the number of elements in an array, and keep that consistent, is better than range-based for?

    Good points. Not surprising this question was greeted with silence. There are still a few out there that categorize all of c++ as evil even though they don't know all that much about this ever improving language. You can show them something better than C like the above excellent points and they'll refuse to think about it. They made a decision long ago not to use c++ and admitting now there is anything good would be an admission they were wrong, their advice is wrong, their code is not as good as it could have been, their productivity is suboptimal, and there were better ways they artificially denied themselves from using their entire career as they railed against it. For these people, c++ has to be worse in every way. Anything else is unthinkable. There seems to be fewer of these people every year and they are not as vocal as before. The tide has turned.

    And as far as what linus thought about c++ remember this is the same guy who said "I don't like debuggers. Never have, probably never will." How many people ditch kernel debugging as evil because of his statement and refuse to touch them? Just imagine what that handicap would do to your productivity. It's the same thing when you limit yourself to C.

  • Prokash_SinhaProkash_Sinha Member - All Emails Posts: 210
    via Email
    NOP, this is absolutely false ... The post was from a veteran , and
    only to indicate that there are concerns, in particular C# pitfalls...
    Nothing is silver bullet, unless they design is correct, know the
    limitations.

    Yes use constant expression instead of #define. Use module import,
    instead #include -- no problem, they have benefits, and they have use
    cases sure ... just like inline etc.

    No one saying C++ worse here, __ this is what I was trying to say,
    don't try to bend someone's head into believing ritual(s)__.

    It's a complex language, and kernel debugging is even way way worse,
    if design is wrong. No one has to be religious about it. Some of us
    object to C++ uses lot more features than C++ stdandards, when the
    situation warrants ...

    -Pro
  • MBond2MBond2 Member Posts: 200

    But type safety is no stronger in C++ than in C. Sure you have more different kinds of casts to screw yourself with, but how does that make it safer? Everything you can do wrong in C, you can still do wrong in C++ along with more stuff? By contrast, C# has actual type safety - yes there are unsafe blocks, but you need special compiler options to even allow them and even when you, the block is explicitly marked as unsafe. For example in C++ you can readily write

    const char szABC[] = "ABC";

    WCHAR* p = (WCHAR*)szABC;
    p[42] = 'Q';

    now you will say, that's not the right thing to do in C++, and of course it isn't, but it compiles! Which means that when checking and maintaining code, you have to consider the possibility that someone has written it this way. I spend almost all of my time reviewing and checking code written by others and doubtless this impacts my point of view

    And so it goes on. Every bad thing you can do in C, you can also do in C++ plus more. Again, I might just be too dense to take this in, but I see no advantage in C++ versus C unless one goes all the way to C# - where the compiler / runtime provide true type safety and actually safe exception based error handling. Interfaces in place of multiple inheritance and generics in place of template classes. And lots of other productivity and reliability improvements too

    but this thread should die. I'm not going to respond again unless we start discussing a proposal for a managed runtime for KM development or something more productive. It could probably be done, but large object garbage collection would be a major issue along with large sections of the system and other namespaces that would have to be restricted

  • 0xrepnz0xrepnz Member Posts: 41
    @Tim_Roberts

    >I've said in the past that the concept of a "Common Language Runtime for Kernel" makes a great deal of sense.

    Have you heard about Singularity? (https://en.m.wikipedia.org/wiki/Singularity_(operating_system))

    There was a time that MSFT had so much hype about C# that they tried to write an OS in a managed language. They even published the sources and docs - https://github.com/lastweek/source-singularity/tree/master/docs/Design%20Notes

    It was discontinued in 2008 though.
    - Ori Damari
  • RourkeRourke Member Posts: 60

    @MBond2 said:
    const char szABC[] = "ABC";

    WCHAR* p = (WCHAR*)szABC;
    p[42] = 'Q';

    C++ has 5 types of casts, C has just 1. The reason c++ casts are better is c++ casts describe exactly what they are doing to the reader and the compiler and thus are easier to understand their purpose and get more safety checks from the compiler. A C cast should never be used in c++ because it's just a quick and dirty blindfold. This really comes through when using your example:

       The C way:  WCHAR* p = (WCHAR*)szABC;             // this compiles and you have a serious bug
       The C++ way: auto p = const_cast<WCHAR*>(szABC);  // this does NOT compile since szABC is 'char' not WCHAR...C++ just saved the day!
    
    

    const_cast tells the reader and compiler the 'const' (or volatile) attribute is being removed, nothing more. const_cast is useful when dealing with libraries that did not properly specify const on function parameters and a number of these cases exist in the WDK. Let's review the 5 powerful c++ casts:

    const_cast
    static_cast
    reinterpret_cast
    dynamic_cast
    bit_cast -- new in c++20 and excellent for systems level work / protocol

    As with anything else in c++ you don't have to use any of these if you don't want, but using the right cast for the job makes code more readable, bullet proof, and catching problems at compile time is excellent. Thus casting is just one of thousands of concrete, nuts and bolts ways c++ makes vastly improved device driver code.

  • Tim_RobertsTim_Roberts Member - All Emails Posts: 13,641

    Interesting. I'd like to understand better. Would you say more about what this would do and how it would work?

    We have already established with KMDF that huge chunks of most drivers are common boilerplate code. That's exactly the kind of thing that fits in the CLR model. A great deal of user-mode C# programming involves writing plumbing between system-provided CLR modules. At the most basic level, I can imagine KMDF itself as a CLR. Instead of WDFQUEUE and WDFDEVICE, you'd have:

        using System.Kernel.IoQueue;
        using System.Kernel.Device;
        using System.Kernel.Request;
        using System.Kernel.Memory;
    
        Memory buffer;
        request.RetrieveOutputBuffer( buffer, minsize );
    

    You're right that the memory model would be an issue. Since it is a common address space, you'd get certain economies of scale, because you wouldn't need to manage a whole bunch of separate memory pools, but there would have to be some timing guarantees.

    Tim Roberts, [email protected]
    Providenza & Boekelheide, Inc.

  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 8,108

    Have you heard about Singularity?

    Singularity was a Microsoft Research project, which begat Midori... and actual OS incubation project, which after several years of aggressive development dies in 2014. Folks interested in OSes, and how languages can be applied to OS design, should enjoy reading Joe Duffy’s series of blog posts about Midori.

    I worked on Midori for quite a while. I helped design the I/O Subsystem, and wrote the USB driver stack. It was one of the most interesting experiences in my career.

    What Duffy’s blog posts highlight best, I think, is that the implementation language used for an OS can have an enormous influence on the OS design. It’s best when the implementation language directly supports fundamental OS concepts. And so it was with Midori, with the asynchrony, decoupling of user/provider, promises, and intrinsic safety.

    Peter

    Peter Viscarola
    OSR
    @OSRDrivers

  • MBond2MBond2 Member Posts: 200

    one way of addressing some issues would be 'stack based garbage collection'. What I mean by that is that rather than a global garbage collection scheme, one that tracks allocations made during a specific call path down and back up the stack that never get shared between threads (as never made visible by being stored in some kind of global data structure)

    this would not address the stop the world problem, but if would help tremendously even in UM with the ephemeral object problem the GC deals with constantly

    automatic reference counting (Aka rundown protection) let's the GC decide when to collect, but another implementation could collect at the time of last reference removal. and this need not be done synchronously - which would be a problem of executing in arbitrary context - but could be done by enqueuing the to be freed block into a list to wait for a worker

  • anton_bassovanton_bassov Member MODERATED Posts: 5,187

    How would CLRK help here?

    Well, if we put all the hype aside and look at the whole thing objectively, this is, indeed,the question that has to be asked.

    For example, how would it help you against the memory corruption that may be caused by improperly-programmed DMA?
    Although it can help you against the one caused by the CPU (for example, a dangling pointer or overwritten buffer), it cannot do anything about a buggy driver that does not program its device's DMA properly. Certainly, one may refer to IOMMU, but it is not a panacea. First of all, not every machine is equipped with the one. Second, and even more important, this protection may work only on the page basis. However, in order to screw the things up, a DMA transfer does not really need to cross the page boundary. All it has to do is write just a little bit beyond the end of the target buffer, and this part can be done without crossing the page boundary in some cases.

    Another point to consider here is interrupt handling. How may CLRK possibly protect you from interrupt storm that is bound to happen
    if an ISR for a level-triggered interrupt returns TRUE without actually disabling interrupts on the device?

    Therefore, a buggy KM driver has all means to bring the system down anyway, so that there is nothing that can be done about it simply by offering its writer a managed language. Certainly, one may point out that, in actuality, there are not so many drivers that actually do DMA and handle interrupts (i.e. normally only the ones for the lowest-level PCI/PCIe devices). However, quite a few of these higher-level drivers (for example, USB ones or installable file systems) can be moved out of the kernel altogether.

    In other words, the very idea of bringing the "safety net" to the components that are meant to be trusted by the very definition does not really seem to make that much sense

    Anton Bassov

  • Tim_RobertsTim_Roberts Member - All Emails Posts: 13,641

    No computer language is going to fix buggy hardware.

    Tim Roberts, [email protected]
    Providenza & Boekelheide, Inc.

  • anton_bassovanton_bassov Member MODERATED Posts: 5,187

    No computer language is going to fix buggy hardware.

    Sorry, but I am speaking about the software bugs here. These may result either from the "slip of the finger" ( i.e. an accidental typo or just a plain negligence), or be introduced by a driver that does not handle the "peculiarities" of its target device properly. In both cases the "results" may well be of a system-wide significance.....

    Anton Bassov

  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 8,108

    a buggy driver that does not program its device's DMA properly

    But we can use the IOMMU for that.

    And not being able to fix one particular class of error isn’t an argument against fixing several other classes of errors. Because we can’t fix DMA with a language construct, doesn’t mean we shouldn’t eliminate many races and use-after-frees.

    a buggy KM driver has all means to bring the system down anyway

    Sure. It’s part of the OS, part of the Trusted Computing Base. That’s doesn’t mean we shouldn’t make drivers easier to write, and less prone to error.

    Your argument could easily be expanded to say “drivers can break shit anyways, and while the rules of WDM may be so impenetrable as to make driver writing nearly impossible, we don’t need KMDF because it can’t fix DMA.”

    Let’s not let the perfect be the enemy of the possible.

    Peter

    This thread is reaching the end of its usefulness. Go ahead and get your closing comments in, folks. Then we’ll let it rest, along with every other C++ in the kernel thread we’ve had here over the past 20+ years.

    Peter Viscarola
    OSR
    @OSRDrivers

Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Upcoming OSR Seminars
OSR has suspended in-person seminars due to the Covid-19 outbreak. But, don't miss your training! Attend via the internet instead!
Writing WDF Drivers 7 Dec 2020 LIVE ONLINE
Internals & Software Drivers 25 Jan 2021 LIVE ONLINE
Developing Minifilters 8 March 2021 LIVE ONLINE