Windows System Software -- Consulting, Training, Development -- Unique Expertise, Guaranteed Results

Before Posting...
Please check out the Community Guidelines in the Announcements and Administration Category.

C++ RTL for NT kernel-mode drivers

OSR_Community_UserOSR_Community_User Member Posts: 110,217
Hello!

The developers interesting in a C++ run-time library for the NT kernel-mode drivers can check out the code here:

svn co https://riouxsvn.com/svn/prj_ntke_cpprtl/trunk
or just browse the location
https://riouxsvn.com/svn/prj_ntke_cpprtl/trunk

The library is available under the MIT license.
The following features are implemented: EH, RTTI, new/delete, global/static ctoring/dtoring.
Supported platforms (both x86 and x64): NT 5.1, 5.2, 6.0, 6.1 (6.2 and 6.3 are not tested yet).
Supported compilers: MS C++ (from v13.00 (ddk2600) and newer), Intel C++ (from v11.1 and newer).

Feel free to discuss or ask ask the questions about the project.
«1

Comments

  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    Does it allow the standard libraries to be utilized like vectors and streams?
  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    My understanding is that C++ exceptions are not supported in the kernel.
    But the standard libraries will raise exceptions under a variety of
    conditions, and as such, the standard C++ implementations are not viable
    in the kernel.
    joe


    > Does it allow the standard libraries to be utilized like vectors and
    > streams?
    >
    >
    > ---
    > NTDEV is sponsored by OSR
    >
    > Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev
    >
    > OSR is HIRING!! See http://www.osr.com/careers
    >
    > For our schedule of WDF, WDM, debugging and other seminars visit:
    > http://www.osr.com/seminars
    >
    > To unsubscribe, visit the List Server section of OSR Online at
    > http://www.osronline.com/page.cfm?name=ListServer
    >
  • Maxim_S._ShatskihMaxim_S._Shatskih Member Posts: 10,396
    I think the authors claim support of exceptions for the kernel.

    Good thing, BTW. std::vector and std::string are the things which we miss in the kernel :-)

    BTW - what about Apple Darwin OS? do they support STL in their IOKit? or IOKit is too limited in terms on C++ to have it, so they only have Core Foundation there?

    --
    Maxim S. Shatskih
    Microsoft MVP on File System And Storage
    xxxxx@storagecraft.com
    http://www.storagecraft.com

    <xxxxx@flounder.com> wrote in message news:xxxxx@ntdev...
    > My understanding is that C++ exceptions are not supported in the kernel.
    > But the standard libraries will raise exceptions under a variety of
    > conditions, and as such, the standard C++ implementations are not viable
    > in the kernel.
    > joe
    >
    >
    >> Does it allow the standard libraries to be utilized like vectors and
    >> streams?
    >>
    >>
    >> ---
    >> NTDEV is sponsored by OSR
    >>
    >> Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev
    >>
    >> OSR is HIRING!! See http://www.osr.com/careers
    >>
    >> For our schedule of WDF, WDM, debugging and other seminars visit:
    >> http://www.osr.com/seminars
    >>
    >> To unsubscribe, visit the List Server section of OSR Online at
    >> http://www.osronline.com/page.cfm?name=ListServer
    >>
    >
    >
    >
  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    > My understanding is that C++ exceptions are not supported in the kernel.

    This is exact. But the kernel provides facilities almost the same as for user-mode applications for the C++ support is to be developed. The only cruel restriction is the stack size available for a kernel executing entity. Inspite of this the current RTL realisation is highly limited while handling the nested exceptions. You can see the tests included with the sources.


    > Does it allow the standard libraries to be utilized like vectors and streams?

    > But the standard libraries will raise exceptions under a variety of
    > conditions, and as such, the standard C++ implementations are not viable
    > in the kernel.

    The current RTL includes almost nothing concerned with the 'C++ standard library' - no STL, no strings, no iostreams, no localisation facilities. It just provides a compiler- and environment-specific run-time support for some basic C++ features.
    But as far as known the STL's containers, iterators and algoritms are widely using exceptions and memory allocations than the STL becomes probably the most simple part of the standard library to be ported for using in the kernel (e.g. taken from the STLPort) in conjunction with RTL. The strings are too, but maybe it worth to be rewritten as internally based on UNICODE_STRING/ANSI_STRING for the best compatibility with the kernel interface. And the iostreams and localisation is the most unclear part of the standard library.


    > I think the authors claim support of exceptions for the kernel.

    You are exactly right.


    > Good thing, BTW. std::vector and std::string are the things which we miss in
    > the kernel :-)

    It seems not to be a good idea to use C++ in the kernel because of at the time it adds much more problems than resolves. But it's just very interesting for some kinds of experimenting -)


    > BTW - what about Apple Darwin OS? do they support STL in their IOKit? or
    > IOKit is too limited in terms on C++ to have it, so they only have Core
    > Foundation there?

    The Darwin IOKit uses not a true C++. It uses Embedded C++ - a highly restricted subset - without exceptions, rtti, only the single inheritance is allowed, no namespaces, no templates using is allowed, etc. The standard library is certainly restricted too in view of above restrictions. And the hand-made rtti-substitution is used in 'IOKit'.
  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    The use of 8-bit strings for exception messages in the STL is profoundly
    offensive. There might be a worse way to do it, but I'm not sure I can
    think of one.

    There are at least two better ways: (1) each exception has an integer
    code, and that is the only representation of an exception. If you want to
    turn it into a message, you can have a component that does this, or you
    can provide your own with localization (2) each exception is a subclass
    (perhaps through several levels) of a base class "exception". This is how
    MFC does it; I can derive CSyntaxException from CException, and
    CMissingSemicolon from CSyntaxException, and I can catch a
    CSyntaxException and get all my exceptions. Then CSyntaxException has
    some virtual methods, such as Report(), that return a string, and I can
    modify this to allow for localization. This is vastly more elegant than
    the horror I find in STL.

    Getting an 8-bit string in the kernel is all-but-useless. And for
    container classes, you need to distinguish between
    STATUS_INSUFFICIENT_RESOURCES and STATUS_BAD_PARAMETER (such as "index out
    of range") types of cases. So even if C++ exceptions are supported in
    the kernel (and what happens if an exception occurs and no suitable
    "catch" exists? BSOD?), the STL is not really well-matched to the needs
    of "embedded" programming (which drivers strongly resemble).

    >> My understanding is that C++ exceptions are not supported in the kernel.
    >
    > This is exact. But the kernel provides facilities almost the same as for
    > user-mode applications for the C++ support is to be developed. The only
    > cruel restriction is the stack size available for a kernel executing
    > entity. Inspite of this the current RTL realisation is highly limited
    > while handling the nested exceptions. You can see the tests included with
    > the sources.
    >
    >
    >> Does it allow the standard libraries to be utilized like vectors and
    >> streams?
    >
    >> But the standard libraries will raise exceptions under a variety of
    >> conditions, and as such, the standard C++ implementations are not viable
    >> in the kernel.
    >
    > The current RTL includes almost nothing concerned with the 'C++ standard
    > library' - no STL, no strings, no iostreams, no localisation facilities.
    > It just provides a compiler- and environment-specific run-time support for
    > some basic C++ features.
    > But as far as known the STL's containers, iterators and algoritms are
    > widely using exceptions and memory allocations than the STL becomes
    > probably the most simple part of the standard library to be ported for
    > using in the kernel (e.g. taken from the STLPort) in conjunction with RTL.
    > The strings are too, but maybe it worth to be rewritten as internally
    > based on UNICODE_STRING/ANSI_STRING for the best compatibility with the
    > kernel interface. And the iostreams and localisation is the most unclear
    > part of the standard library.
    >
    >
    >> I think the authors claim support of exceptions for the kernel.
    >
    > You are exactly right.
    >

    But if exceptions are not supported, then this seems to suggest that
    exceptions are supported. You seem to have confirmed both "yes" and "no"
    in the same email, so I remain confused as to what is true.
    joe

    >
    >> Good thing, BTW. std::vector and std::string are the things which we
    >> miss in
    >> the kernel :-)
    >
    > It seems not to be a good idea to use C++ in the kernel because of at the
    > time it adds much more problems than resolves. But it's just very
    > interesting for some kinds of experimenting -)
    >
    >
    >> BTW - what about Apple Darwin OS? do they support STL in their IOKit? or
    >> IOKit is too limited in terms on C++ to have it, so they only have Core
    >> Foundation there?
    >
    > The Darwin IOKit uses not a true C++. It uses Embedded C++ - a highly
    > restricted subset - without exceptions, rtti, only the single inheritance
    > is allowed, no namespaces, no templates using is allowed, etc. The
    > standard library is certainly restricted too in view of above
    > restrictions. And the hand-made rtti-substitution is used in 'IOKit'.
    >
    >
    > ---
    > NTDEV is sponsored by OSR
    >
    > Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev
    >
    > OSR is HIRING!! See http://www.osr.com/careers
    >
    > For our schedule of WDF, WDM, debugging and other seminars visit:
    > http://www.osr.com/seminars
    >
    > To unsubscribe, visit the List Server section of OSR Online at
    > http://www.osronline.com/page.cfm?name=ListServer
    >
  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    > But if exceptions are not supported, then this seems to suggest that
    > exceptions are supported. You seem to have confirmed both "yes" and "no"
    > in the same email, so I remain confused as to what is true.

    Let's clear some internals - NT environment implements the support of structured exception handling (SEH), available by using __try{}__except{} construction as supported by the MS C compiler.
    Briefly speaking the one part of SEH resides in the kernel: RtlRaiseException() spawns the SEH-exception (processor's software exception) that is handled by the dispatcher placed at the idt trap, the dispatcher in turn uses the unwinding mechanics provided by RtlUnwind()/RtlUnwindEx(). Another part of SEH is provided by a compiler/linker/rtl-library toolset - these are the SEH-handlers and function's stack frames unwind information.
    The full SEH support is available for using in NT kernel by your drivers all the time, because of it is uses SEH-handler '__except_handler()' placed in 'ntoskrnl.lib' or 'ntdll.lib' or 'wdm.lib' or somewhere else standardly linked to the driver and the MS C compiler generates unwind information by defualt.
    But when you use the C++ exceptions the compiler generates the invokes to the compiler internally pre-declared entry points assumed to be implemented by the run-time library (these points are a kind of CxxFrameHandler and CxxThrowException). And the library implementation uses SEH facilities to implement the raising of exception and the stack unwinding.
    As a summary: NT kernel provides SEH, C++ compiler provides all the necessary unwinding information and the RTL library uses both them for the handling C++ exceptions.



    > The use of 8-bit strings for exception messages in the STL is profoundly
    > offensive. There might be a worse way to do it, but I'm not sure I can
    > think of one.

    C++ doesn't force you to strictly use something you don't want to. You can build the exeption architecture to be used in the driver on your own preferences.


    > So even if C++ exceptions are supported in
    > the kernel (and what happens if an exception occurs and no suitable
    > "catch" exists? BSOD?), the STL is not really well-matched to the needs
    > of "embedded" programming (which drivers strongly resemble).

    If the driver allows the C++ exception out of its' scope the BSOD is to occure unless someone executes your driver in __try{}__except(1){} handler, because it is propagated up the stack as the SEH exception with specific code.

    And you are right, the STL in common case seems to be less suitable for using in the kernel being compared with facilities ddk/wdk provides. But in some cases it may be very convenient and useful as not all the drivers are used for high performance and minor resource using tasks.
  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    The string class does not need rewritten to be a UNICODE_STRING internally. If such a methodology were a good idea, then std::string would be based on the good old c string internally, but it isn't. Instead there is a way to get to and from a c string and that's all that is needed. In the kernel that's all we need--just a way to get to and from the handful of string types we need to deal with, UNICODE_STRING being just one of those. A std::wstring would suit the kernel quite nicely. Tossing out all of that tedious RtlInitUnicodeString and such baloney code would be nice.

    Sort of a rant, but I hate LIST_ENTRY. I enjoyed it when it and its manipulation macros were state of the art about 30 years ago, but in the new era they are truly horrid when viewed side by side castless std::list code which also snaps into range based for loops and so forth.

    And the new c++11 threads are just killer--they make multi-threading sinfully easy. Would love that in the kernel asap.

    There is really no limit to the practicality of the standard libraries in the kernel.

    "Embedded c++" sounds like being confined to hell and I pray I never have to deal with it. Disallowing much of c++ because someone might not use it correctly is ridiculous. Those kinds of useless chumps are not worth restricting everyone else's code down to the lowest common denominator over. Kind of like when they decided to make c arrays pass by reference because someone might pass a big one by value. But never mind you can pass a 1 megabyte structure by value, but never a 4 byte array. This embedded c++ sounds like the same dumb thing all over again. Punish the professionals for the sake of the idiots.
  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    > The string class does not need rewritten to be a UNICODE_STRING internally.

    Yes, i'm agree with it. The conversion is better to be done outside the stdlib, it's simple enough.


    > Sort of a rant, but I hate LIST_ENTRY. I enjoyed it when it and its manipulation
    > macros were state of the art about 30 years ago, but in the new era they are
    > truly horrid when viewed side by side castless std::list code which also snaps
    > into range based for loops and so forth.

    it's suggested the 'LIST_ENTRY' and the 'std::list' are the rather different things.
    The LIST_ENTRY is an intrusive structure, so one need to prepare his data to be stored here, when you can just add any copy-constructible data to the 'std::list'
    On the other hand 'LIST_ENTRY' is a non-typed data - you can add here any structure with 'LIST_ENTRY' intrused - it becomes like a C-style tuple but which must be accessed very careful.
    And the 'LIST_ENTRY' is a must-used when dealing with the kernel.
    So if using these things according to their capabilities - then no hate arises -)


    > And the new c++11 threads are just killer--they make multi-threading sinfully
    > easy. Would love that in the kernel asap.

    At first the current RTL doesn't even try to overstep the C++03 basic support, so how, when and by whom the newer stdlib will be developed if at the moment even C++03 std for kernel doesn't exist ?

    What about a 'threads'? I don't think it's to hard to write a wrapper around PsCreateSystemThread, moreover the very simple version is presented in the test driver in the project. The point is to make using of std/boost::function and std/boost::bind - this is a really powerful and amazing facility. Currently there exists such a code in my project based on this RTL. But it uses hand-made replacement of bind/function and is used for dispatching IRPs and spawning the kernel threads and work items. But the library code looks like "can't see without bloody tears" and woudn't be released in the nearest future.


    > There is really no limit to the practicality of the standard libraries in the
    > kernel.

    It seems to be a very optimistic statement -).
  • Maxim_S._ShatskihMaxim_S._Shatskih Member Posts: 10,396
    > The Darwin IOKit uses not a true C++. It uses Embedded C++ - a highly restricted subset - without
    >exceptions, rtti, only the single inheritance is allowed, no namespaces, no templates using is
    >allowed, etc. The standard library is certainly restricted too in view of above restrictions. And the
    >hand-made rtti-substitution is used in 'IOKit'.

    Thanks, this looks like what is used by some developers in the Windows kernel.

    I will not be surprised if KMDF (very, very similar to IOKit) is developed using the same kind of C++ internally.

    Also note: C++ standard RTTI is pathetic kludge, and thus is usually replaced with _better_ hand-made stuff, like DECLARE_DYNAMIC in MFC or OSDynamicCast in IOKit.

    Also, Apple's CF reminds me on some pre-STL C++ contrainer objects, like the MFC's one.

    --
    Maxim S. Shatskih
    Microsoft MVP on File System And Storage
    xxxxx@storagecraft.com
    http://www.storagecraft.com
  • Maxim_S._ShatskihMaxim_S._Shatskih Member Posts: 10,396
    > it's suggested the 'LIST_ENTRY' and the 'std::list' are the rather different things.
    > The LIST_ENTRY is an intrusive structure

    Yes, and the same single structure can be on several lists.

    --
    Maxim S. Shatskih
    Microsoft MVP on File System And Storage
    xxxxx@storagecraft.com
    http://www.storagecraft.com
  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    > I will not be surprised if KMDF (very, very similar to IOKit) is developed using
    > the same kind of C++ internally.

    Yes, the KMDF seems to be internally developed using not only the C but with using of the stuff that can be described as "MS approved 'C with classes' kernel subset" -) .

    And no, IOKit and KMDF are very different in the imlemetation and the interface. KMDF exposes the raw C interface to the client. IOKit exposes the C++ class library - to do something useful one almost always subclasses whatever corresponding from the IOKit and implements the necessary virtual functions. Further the IOKit is implemented as the object-oriented class library whereas the KMDF is believed to be a pile of plane structures-wrappers around the kernel-exposed data types and C-functions. But the result is exactly the same - one just writes his drivers using these toolkits -)


    > Also, Apple's CF reminds me on some pre-STL C++ contrainer objects, like the
    > MFC's one.

    Core Foundation, as far as i remember, is a user-land bridge between C- and ObjC-libraries. It's not in the kernel, ObjC for driver development had been replaced by eC++ when NextStep became MacosX.
    IOKit is based on the 'libkern-c++' - this is a kernel library for 'basic object', collections and strings. But yes - very similar in nature to the CF.


    > Also note: C++ standard RTTI is pathetic kludge, and thus is usually replaced
    > with _better_ hand-made stuff, like DECLARE_DYNAMIC in MFC or OSDynamicCast in
    > IOKit.

    It looks you are excessively strict about RTTI. The RTTI on my opinion is just a low level crutch exposed by statically-typed language for some run-time limited purposes and it does it's duties well enough.
    As well let's just recall that when OSDynamicCast appeared the C++ was being in a slightly chaotic state. Moreover OSDynamicCast provides not just the RTTI in the C++ terms but also it exposes some kind of reflection.
  • Maxim_S._ShatskihMaxim_S._Shatskih Member Posts: 10,396
    > And no, IOKit and KMDF are very different in the imlemetation and the interface. KMDF exposes the
    >raw C interface to the client.

    Object classes and their interfaces are very similar in 2 frameworks.

    Just in IOKit you use C++ virtual method call, and KMDF wraps this to a C function :-)

    > Core Foundation, as far as i remember, is a user-land bridge between C- and ObjC-libraries.

    I remember seeing CF in the Darwin kernel/IOKit, IIRC OSDynamicCast is in CF.

    Probably I'm wrong. It was like 2.5 years when I was looking at it seriously.

    >state. Moreover OSDynamicCast provides not just the RTTI in the C++ terms but also it exposes
    >some kind of reflection.

    This is the feature which C++ lacks. And that's why "lack of standard RTTI" is not a drawback. People use their own.

    --
    Maxim S. Shatskih
    Microsoft MVP on File System And Storage
    xxxxx@storagecraft.com
    http://www.storagecraft.com
  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    > Object classes and their interfaces are very similar in 2 frameworks.

    It doesn't appear strange, because both 'KMDF' and 'IOKit' are exposes the terms for device driver developing. And in turn those terms are just dictated by the hardware architecture. Such an explanation seems to look plausible.



    > I remember seeing CF in the Darwin kernel/IOKit, IIRC OSDynamicCast is in CF.
    > Probably I'm wrong. It was like 2.5 years when I was looking at it seriously.

    Oh, well, I'v just seen the xnu sources and the OSDynamicCast is defined here:
    xnu-2050.22.13\libkern\libkern\c++\OSMetaClass.h
    as the macros going into the internal implementation.

    and as a prove that ObjC is banned from the kernel look at this stuff from xnu -)
    #ifdef KERNEL
    #include <libkern/OSTypes.h>
    #include <libkern/OSReturn.h>
    #else
    #include <CoreFoundation/CoreFoundation.h>
    #include <System/libkern/OSReturn.h>
    #endif /* KERNEL */
    and take into account that 'CoreFoundation.h' is a part of the user-land libs.


    >also it exposes
    >some kind of reflection.
    >
    >This is the feature which C++ lacks. And that's why "lack of standard RTTI" is
    >not a drawback. People use their own.

    Yes, well, RTTI at the moment is not a popular solution. It may be useful for classes with a complex multiple and virtual inheritance but these classes in turn are difficult enough to project and they are hardly to be recommended for a all-round usage.
    But by the momemt it was decided to include RTTI to the kernel RTL, almost all the necessary prerequisites had been ready - the EH-lib, the internal-compiler-data iterators lib, and the basic knowledge of how-to. Moreover MS had MSVS2012RC released at that time, where the rtti sources were included so all the unclears had been gone and the kernel rtti became ready fast enough -)
  • Doron_HolanDoron_Holan Member - All Emails Posts: 10,438
    /kernel, which is the default for compiling drivers in WDK 8 and 8.1 does not allow the use of C++ exceptions or RTTI at compile time (as opposed to previous failures at link time due to unresolved symbols)

    d

    -----Original Message-----
    From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@seznam.cz
    Sent: Monday, November 11, 2013 11:02 AM
    To: Windows System Software Devs Interest List
    Subject: RE:[ntdev] C++ RTL for NT kernel-mode drivers

    > Object classes and their interfaces are very similar in 2 frameworks.

    It doesn't appear strange, because both 'KMDF' and 'IOKit' are exposes the terms for device driver developing. And in turn those terms are just dictated by the hardware architecture. Such an explanation seems to look plausible.



    > I remember seeing CF in the Darwin kernel/IOKit, IIRC OSDynamicCast is in CF.
    > Probably I'm wrong. It was like 2.5 years when I was looking at it seriously.

    Oh, well, I'v just seen the xnu sources and the OSDynamicCast is defined here:
    xnu-2050.22.13\libkern\libkern\c++\OSMetaClass.h
    as the macros going into the internal implementation.

    and as a prove that ObjC is banned from the kernel look at this stuff from xnu -)
    #ifdef KERNEL
    #include <libkern/OSTypes.h>
    #include <libkern/OSReturn.h>
    #else
    #include <CoreFoundation/CoreFoundation.h>
    #include <System/libkern/OSReturn.h>
    #endif /* KERNEL */
    and take into account that 'CoreFoundation.h' is a part of the user-land libs.


    >also it exposes
    >some kind of reflection.
    >
    >This is the feature which C++ lacks. And that's why "lack of standard RTTI" is
    >not a drawback. People use their own.

    Yes, well, RTTI at the moment is not a popular solution. It may be useful for classes with a complex multiple and virtual inheritance but these classes in turn are difficult enough to project and they are hardly to be recommended for a all-round usage.
    But by the momemt it was decided to include RTTI to the kernel RTL, almost all the necessary prerequisites had been ready - the EH-lib, the internal-compiler-data iterators lib, and the basic knowledge of how-to. Moreover MS had MSVS2012RC released at that time, where the rtti sources were included so all the unclears had been gone and the kernel rtti became ready fast enough -)


    ---
    NTDEV is sponsored by OSR

    Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

    OSR is HIRING!! See http://www.osr.com/careers

    For our schedule of WDF, WDM, debugging and other seminars visit:
    http://www.osr.com/seminars

    To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
    d
  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    > /kernel, which is the default for compiling drivers in WDK 8 and 8.1 does not
    > allow the use of C++ exceptions or RTTI at compile time (as opposed to previous
    > failures at link time due to unresolved symbols)


    It appears not to be a problem at all because of the RTL build system doesn't use the WDK-build nor the MSBuild nor the VS projects/solutions and is completely custom and based on gnumake. It sets up the compiler/lib/linker options that are necessary to get C++ driver with eh and rtti.

    As a curious fact this RTL was projected after the first NT driver written by me was started as try{}catch(...){} in the DriverMain() and instantly had led to the 'unresolved symbols' you've mentioned ;-)
  • Mark_RoddyMark_Roddy Member - All Emails Posts: 4,307
    Hmmm... that seems a bit problematic. You might want to rethink
    incompatibility with the only supported kernel compiler tool-chain.

    Mark Roddy


    On Mon, Nov 11, 2013 at 2:39 PM, wrote:

    > > /kernel, which is the default for compiling drivers in WDK 8 and 8.1
    > does not
    > > allow the use of C++ exceptions or RTTI at compile time (as opposed to
    > previous
    > > failures at link time due to unresolved symbols)
    >
    >
    > It appears not to be a problem at all because of the RTL build system
    > doesn't use the WDK-build nor the MSBuild nor the VS projects/solutions and
    > is completely custom and based on gnumake. It sets up the
    > compiler/lib/linker options that are necessary to get C++ driver with eh
    > and rtti.
    >
    > As a curious fact this RTL was projected after the first NT driver written
    > by me was started as try{}catch(...){} in the DriverMain() and instantly
    > had led to the 'unresolved symbols' you've mentioned ;-)
    >
    >
    > ---
    > NTDEV is sponsored by OSR
    >
    > Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev
    >
    > OSR is HIRING!! See http://www.osr.com/careers
    >
    > For our schedule of WDF, WDM, debugging and other seminars visit:
    > http://www.osr.com/seminars
    >
    > To unsubscribe, visit the List Server section of OSR Online at
    > http://www.osronline.com/page.cfm?name=ListServer
    >
  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    >
    >> Also note: C++ standard RTTI is pathetic kludge, and thus is usually
    >> replaced
    >> with _better_ hand-made stuff, like DECLARE_DYNAMIC in MFC or
    >> OSDynamicCast in
    >> IOKit.

    You seem to have a very, very confused grasp of reality. DECLARE_DYNAMIC
    and friends were not "added" because "RTTI is a pathetic kludge". They
    were CREATED because RTTI ***DID NOT EXIST***.

    RTTI was added more than 15 years after DECLARE_DYNAMIC was created. Its
    behavior is slightly different, and replacing it with an RTTI
    implementation would break many existing MFC programs. So we're stuck
    with it.
    joe
    >
    > It looks you are excessively strict about RTTI. The RTTI on my opinion is
    > just a low level crutch exposed by statically-typed language for some
    > run-time limited purposes and it does it's duties well enough.
    > As well let's just recall that when OSDynamicCast appeared the C++ was
    > being in a slightly chaotic state. Moreover OSDynamicCast provides not
    > just the RTTI in the C++ terms but also it exposes some kind of
    > reflection.
    >
    >
    > ---
    > NTDEV is sponsored by OSR
    >
    > Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev
    >
    > OSR is HIRING!! See http://www.osr.com/careers
    >
    > For our schedule of WDF, WDM, debugging and other seminars visit:
    > http://www.osr.com/seminars
    >
    > To unsubscribe, visit the List Server section of OSR Online at
    > http://www.osronline.com/page.cfm?name=ListServer
    >
  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    > Hmmm... that seems a bit problematic. You might want to rethink
    > incompatibility with the only supported kernel compiler tool-chain.

    it seems currently to be no reasons for that.

    If you mean as the problem the '/kernel'-switch - so well, it seems to be just a mirror and smoke screen preventing developers from using the officially unsupported stuff. How can it be used in our particular case if the project intentionally ignores this prevention ?

    Let's look do the custom build system appear to be a problem ?
    The currently released WDK8.1 and the early released WDK8.0 are not complete stand-alone solution for a driver development. Roughly speaking they represent only headers and libraries sets. There are no more compilers/linkers/build. The new WDKs require a VisualStudio where the wizards/projects/solutions to be used. All this stuff is rather annoying and is a great overkill for the small strange project like this kernel RTL. If MS would change mind and will release the modern human-kind WDK with complete toolset for the simple guys without root digital certificates, a great production code base and others black-jacks then it would be worth to write the 'wdk-build' scripts.

    But now using the custom build system you can build everything in the project just using gnumake and one of the supported DDK/WDK:
    ddk2600, ddk3790sp1, wdk6.1sp1, wdk7.1.0, wdk8.0+msvc2012.
    and the wdk8.1+msvc2013 is planned to be added in some future.
  • Mark_RoddyMark_Roddy Member - All Emails Posts: 4,307
    If you want developers to adopt use of your library you really should
    support building within the MSFT defined tool-chain. As is this is a
    non-starter for almost everyone building commercial products. You might
    find this unreasonable, but it will be a barrier to adoption.


    Mark Roddy


    On Mon, Nov 11, 2013 at 4:08 PM, wrote:

    > > Hmmm... that seems a bit problematic. You might want to rethink
    > > incompatibility with the only supported kernel compiler tool-chain.
    >
    > it seems currently to be no reasons for that.
    >
    > If you mean as the problem the '/kernel'-switch - so well, it seems to be
    > just a mirror and smoke screen preventing developers from using the
    > officially unsupported stuff. How can it be used in our particular case if
    > the project intentionally ignores this prevention ?
    >
    > Let's look do the custom build system appear to be a problem ?
    > The currently released WDK8.1 and the early released WDK8.0 are not
    > complete stand-alone solution for a driver development. Roughly speaking
    > they represent only headers and libraries sets. There are no more
    > compilers/linkers/build. The new WDKs require a VisualStudio where the
    > wizards/projects/solutions to be used. All this stuff is rather annoying
    > and is a great overkill for the small strange project like this kernel RTL.
    > If MS would change mind and will release the modern human-kind WDK with
    > complete toolset for the simple guys without root digital certificates, a
    > great production code base and others black-jacks then it would be worth to
    > write the 'wdk-build' scripts.
    >
    > But now using the custom build system you can build everything in the
    > project just using gnumake and one of the supported DDK/WDK:
    > ddk2600, ddk3790sp1, wdk6.1sp1, wdk7.1.0, wdk8.0+msvc2012.
    > and the wdk8.1+msvc2013 is planned to be added in some future.
    >
    >
    > ---
    > NTDEV is sponsored by OSR
    >
    > Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev
    >
    > OSR is HIRING!! See http://www.osr.com/careers
    >
    > For our schedule of WDF, WDM, debugging and other seminars visit:
    > http://www.osr.com/seminars
    >
    > To unsubscribe, visit the List Server section of OSR Online at
    > http://www.osronline.com/page.cfm?name=ListServer
    >
  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    Agreed, working within the current MSFT defined tool-chain is a must. Anything else is of no interest to me. Logo compliance is a must. It was heavy handed that out of the blue Microsoft decided they must make /kernel exclusive of EH. Someone clearly had an agenda there as everyone has known for years that EH is the future of the kernel, not if but when and yours is not the first such EH library. Perhaps Microsoft wants to be the only ones to roll this out which is strange since they still don't bother defining new & delete even today. Hopefully library developers like yourself can find a workaround to this.

    And why are you talking about antiquated boost things? We are approaching the year 2014 so get on board with c++11 and std::bind, std::thread and don't even mention legacy things.

    > There is really no limit to the practicality of the standard libraries in the kernel.
    >> It seems to be a very optimistic statement -).

    It is 100% true. My post gave a flavor of this; std::wstring, std::list and std::thread are just the beginning of the incredibly useful library features that blow away code without them.
  • Doron_HolanDoron_Holan Member - All Emails Posts: 10,438
    I made /kernel happen...so blame me.if you want. It has nothing to do with excluding others from a potential market, rather it is codifying the informal rules as formal. EH is not the future in the kernel as far as MSFT is concerned, the danger of misuse and corrupting system state is great. If that changes, we will update the flag to allow it as appropriate

    d

    Bent from my phone
    ________________________________
    From: xxxxx@gmail.com
    Sent: ?11/?11/?2013 4:22 PM
    To: Windows System Software Devs Interest List
    Subject: RE:[ntdev] C++ RTL for NT kernel-mode drivers

    Agreed, working within the current MSFT defined tool-chain is a must. Anything else is of no interest to me. Logo compliance is a must. It was heavy handed that out of the blue Microsoft decided they must make /kernel exclusive of EH. Someone clearly had an agenda there as everyone has known for years that EH is the future of the kernel, not if but when and yours is not the first such EH library. Perhaps Microsoft wants to be the only ones to roll this out which is strange since they still don't bother defining new & delete even today. Hopefully library developers like yourself can find a workaround to this.

    And why are you talking about antiquated boost things? We are approaching the year 2014 so get on board with c++11 and std::bind, std::thread and don't even mention legacy things.

    > There is really no limit to the practicality of the standard libraries in the kernel.
    >> It seems to be a very optimistic statement -).

    It is 100% true. My post gave a flavor of this; std::wstring, std::list and std::thread are just the beginning of the incredibly useful library features that blow away code without them.


    ---
    NTDEV is sponsored by OSR

    Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

    OSR is HIRING!! See http://www.osr.com/careers

    For our schedule of WDF, WDM, debugging and other seminars visit:
    http://www.osr.com/seminars

    To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
    d
  • Maxim_S._ShatskihMaxim_S._Shatskih Member Posts: 10,396
    > RTTI was added more than 15 years after DECLARE_DYNAMIC was created.

    Simultaneously in early 1990ies.

    No one used C++ standard stuff, just because the layout of the RTTI object was too pathetic and there was no virtual constructor.

    DECLARE_DYNAMIC was just better.

    Nevertheless, RTTI is still needed for C++ to do EH: exceptions are classes inherited from one another.

    Anyway these are all kludges: the full and proper RTTI support can occur in managed code only, like the WinRT-extended C++ :-)

    --
    Maxim S. Shatskih
    Microsoft MVP on File System And Storage
    xxxxx@storagecraft.com
    http://www.storagecraft.com
  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    > Nevertheless, RTTI is still needed for C++ to do EH: exceptions are classes
    > inherited from one another.

    It looks to be not a case. The standard says on the contrary: the rtti operations can rise exceptions. All the rest interdependance between RTTI and EH is an internal specific of a compiler. E.g. in the case of the MS C++ the EH work well even if you've RTTI turned off - the compiler still generates the type-descriptor tables concerned with types involved into the EH-process. And note those tables are not the complete RTTI type-describing structures.
    Well, several years ago the gcc had a problem - when the RTTI was being turned off while compiling the EH became broken - that was a bug.



    > Anyway these are all kludges: the full and proper RTTI support can occur in
    > managed code only, like the WinRT-extended C++

    On my opinion the C++ currently doesn't pretend to be object-oriented in terms as do the C#, Java or ObjC. The C++ is just an instrument to build your own OO-framework, and the RTTI capability is good enough as a part of the language but certainly may not fulfil your framework requirements.



    > If you want developers to adopt use of your library you really should
    > support building within the MSFT defined tool-chain. As is this is a
    > non-starter for almost everyone building commercial products. You might
    > find this unreasonable, but it will be a barrier to adoption.

    > Agreed, working within the current MSFT defined tool-chain is a must.
    > Anything else is of no interest to me. Logo compliance is a must.

    Ok, yes. I'll consider to supply VC2012 solution with the project for convenience.
    But the '/kernel' compiler switch is a kind of a mutual exclusion - it just can not be used to build a driver using EH and RTTI -).

    It hardly beleived this RTL can be considered as a part of some commercial product in some future. Let's just look at the current state of the project:
    1) this is entirely a private investigative initiative;
    2) a whole bunch of undocumented or poorly-observed or reverse-engineered technologies is used,
    3) the officially unsupporetd or unrecommended facilities are used,
    4) initially released with the intention to be of an interest to the enthusiast of the C++ using in the NT kernel environment.
    Although the license is permittive enough for the code to be used in a wide range of side projects including commercial, someone intending to involve this code in his buisness need to be strongly ensured what does he do, because all profits and _risks_ are to be taken on his own. The RTL author is only competent to answer the questions about internal layout of the library and supplied tools. :-)



    > And why are you talking about antiquated boost things?
    > We are approaching the year 2014 so get on board with
    > c++11 and std::bind, std::thread and don't even
    > mention legacy things.

    Oh, boost::bind and boost::function are never considred as antiquated IMO. Their std::-twins just resembles the experience of the corresponding boost facilities.


    > It is 100% true. My post gave a flavor of this; std::wstring,
    > std::list and std::thread are just the beginning of the
    > incredibly useful library features
    > that blow away code without them.

    These are a kind of a good, but the great efforts are to be released for this to become real in the kernel.
  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 7,380
    It seems that every few years somebody surfaces with a new one of these things. Didn't we see something like this, with one particular dev posting with great fervor, a few years back?

    While cute, and an interesting personal learning exercises, they're never suitable for commercial products.

    If you want to use C++ for driver development (I think that's too bad, but) you have whatever Microsoft now supports.

    If you want to use a proper OO language with rich run-time support, pray for the day that we can use managed code in kernel-mode... and don't expect a solution soon.

    Peter
    OSR

    Peter Viscarola
    OSR
    @OSRDrivers

  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    > If you want to use a proper OO language with rich run-time
    > support, pray for the day that we can use managed code
    > in kernel-mode... and don't expect a solution soon.

    It's will be great... and the whole 'dbg_console' instead the current DbgPrint() to write there commands and scripts for the kebash and kepython -)



    > It seems that every few years somebody surfaces with
    > a new one of these things.
    > Didn't we see something like this, with one particular
    > dev posting with great fervor, a few years back?

    And the people who try to use C++ in the kernel and get stuck bad are occur a bit more often. And it supposed to be a good deal if those people are able to get the code, see it and make a decision a kind of "oh, EH in kernel is a peace of junk indeed" or "hmm, maybe it worth try for something", isn't it ?
    If you mean 'one particular dev' Peter Hurley, than i'v seen his library with a great interest.



    > While cute, and an interesting personal learning exercises,
    > they're never suitable for commercial products.

    Yes, it was certainly the very interesting development.
    And now every person got this code can make decision about it's using on her own preferences. Is there something wrong ? If some code exists it's a bit better than otherwise. -)
  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 7,380
    <quote>
    Is there something wrong ?
    </quote>

    Wrong? Not so very wrong, no.

    <quote>
    If some code exists it's a bit better than otherwise. -)
    </quote>

    See... I've personally never believed that. That's one reason why I've never been an adherent of the OSS movement. "Here's a lot of code that, while it works very well in some aspects, is shit in other aspects. Please figure out for yourself which parts of the code are which. But it's free and you can use it. And we had fun writing the parts that we wrote. You're welcome."

    From my own little viewpoint, having code available to the developers that is problematic and/or does not follow best practices, is worse than having no code available. This type of code is just a big hole for unsuspecting devs to fall into.

    It's hard enough to write Windows drivers without having to deal with additional unknowns.

    This isn't meant as a specific criticism of your endeavour. Rather, it's my view on the whole genre.

    Peter
    OSR

    Peter Viscarola
    OSR
    @OSRDrivers

  • Maxim_S._ShatskihMaxim_S._Shatskih Member Posts: 10,396
    > If you want to use C++ for driver development (I think that's too bad, but) you have whatever
    >Microsoft now supports.

    Stack size is one of the issues.

    If somebody will really port STL/boost to kmode - then good luck deal with all these by-value objects.

    --
    Maxim S. Shatskih
    Microsoft MVP on File System And Storage
    xxxxx@storagecraft.com
    http://www.storagecraft.com
  • mmmm Member - All Emails Posts: 1,410
    >From my own little viewpoint, having code available to the developers that
    is problematic and/or does not follow best practices, is worse than having
    no code available. This type of code is >just a big hole for unsuspecting
    devs to fall into.

    Tru dat.

    And if it's a commonly available bad example, it takes on a life of its own
    after a while.


    mm

    -----Original Message-----
    From: xxxxx@lists.osr.com
    [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@osr.com
    Sent: Tuesday, November 12, 2013 9:43 AM
    To: Windows System Software Devs Interest List
    Subject: RE:[ntdev] C++ RTL for NT kernel-mode drivers

    <quote>
    Is there something wrong ?
    </quote>

    Wrong? Not so very wrong, no.

    <quote>
    If some code exists it's a bit better than otherwise. -) </quote>

    See... I've personally never believed that. That's one reason why I've
    never been an adherent of the OSS movement. "Here's a lot of code that,
    while it works very well in some aspects, is shit in other aspects. Please
    figure out for yourself which parts of the code are which. But it's free
    and you can use it. And we had fun writing the parts that we wrote. You're
    welcome."

    From my own little viewpoint, having code available to the developers that
    is problematic and/or does not follow best practices, is worse than having
    no code available. This type of code is just a big hole for unsuspecting
    devs to fall into.

    It's hard enough to write Windows drivers without having to deal with
    additional unknowns.

    This isn't meant as a specific criticism of your endeavour. Rather, it's my
    view on the whole genre.

    Peter
    OSR

    ---
    NTDEV is sponsored by OSR

    Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

    OSR is HIRING!! See http://www.osr.com/careers

    For our schedule of WDF, WDM, debugging and other seminars visit:
    http://www.osr.com/seminars

    To unsubscribe, visit the List Server section of OSR Online at
    http://www.osronline.com/page.cfm?name=ListServer
  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    On 11/12/2013 5:43 PM, xxxxx@osr.com wrote:
    > See... I've personally never believed that. That's one reason why I've never been an adherent of the OSS movement. "Here's a lot of code that, while it works very well in some aspects, is shit in other aspects. Please figure out for yourself which parts of the code are which. But it's free and you can use it. And we had fun writing the parts that we wrote. You're welcome."

    Isn't it more like "Here's a lot of code. If it doesn't work we'll do
    our best to fix it if we have the time and energy, otherwise you have
    the source to figure out a fix or workarounds yourself"?

    --
    Bruce Cran
  • Mark_RoddyMark_Roddy Member - All Emails Posts: 4,307
    On Tue, Nov 12, 2013 at 1:18 PM, Bruce Cran wrote:

    > On 11/12/2013 5:43 PM, xxxxx@osr.com wrote:
    >
    >> See... I've personally never believed that. That's one reason why I've
    >> never been an adherent of the OSS movement. "Here's a lot of code that,
    >> while it works very well in some aspects, is shit in other aspects. Please
    >> figure out for yourself which parts of the code are which. But it's free
    >> and you can use it. And we had fun writing the parts that we wrote.
    >> You're welcome."
    >>
    >
    > Isn't it more like "Here's a lot of code. If it doesn't work we'll do our
    > best to fix it if we have the time and energy, otherwise you have the
    > source to figure out a fix or workarounds yourself"?


    As I work with an open source team of developers, this isn't really what is
    going on. "Works" modulo the usual bugs in any complex software product, is
    on one branch, "Works for shit but has new features" is on another, the
    development branch. We get to leverage the efforts of many and in general
    the release branches of the big projects (Linux for example) are rather
    high quality. The dev branches not so much, but that is a known risk
    factor.

    That is the good news. The real problem is the churn in the release branch.
    Because the dev branch is loose and has people adding cool features etc.
    shit that works just fine regularly gets re-implemented for arbitrary
    reasons. Then that stuff moves to the release branch and people using that
    branch to add value via their own products get to deal with the arbitrary
    churn.

    More specifically people with patch queues on release branches frequently
    get to toss all their nice changes and enhancements out and re-implement
    them, refix re-introduced bugs, etc. all over again. And again. And again.
    And fight with the reigning authorities of this product or that product to
    get their conflicting features into mainline and out of patch status rather
    than somebody else's conflicting features.


    Mark Roddy
Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Upcoming OSR Seminars
Developing Minifilters 29 July 2019 OSR Seminar Space
Writing WDF Drivers 23 Sept 2019 OSR Seminar Space
Kernel Debugging 21 Oct 2019 OSR Seminar Space
Internals & Software Drivers 18 Nov 2019 Dulles, VA