Windows System Software -- Consulting, Training, Development -- Unique Expertise, Guaranteed Results

Before Posting...
Please check out the Community Guidelines in the Announcements and Administration Category.

Re: MmGetMdlByteCount greater than TransferBufferLength

Jamey_KirbyJamey_Kirby Member - All Emails Posts: 433
Alignment

On Thu, Feb 2, 2017 at 2:25 PM wrote:

> I've been through that. This happens when an application buffer is too big
> for a single transfer.
>
> Some class drivers (Tape?) make partial MDLs. Some class drivers just
> adjust TransferBuffer and TransferBufferLength, which you're supposed to
> pass to GetScatterGatherList. In the end, you're supposed to use these from
> SRB/URB, not what's in MDL.
>
> MDL just describes memory. The buffer is described by values in SRB/URB.
>
>
>
> ---
> NTDEV is sponsored by OSR
>
> Visit the list online at: <
> http://www.osronline.com/showlists.cfm?list=ntdev>;
>
> MONTHLY seminars on crash dump analysis, WDF, Windows internals and
> software drivers!
> Details at
>
> To unsubscribe, visit the List Server section of OSR Online at <
> http://www.osronline.com/page.cfm?name=ListServer>;
>
«1

Comments

  • matt_sykesmatt_sykes Member - All Emails Posts: 270
    I'll take a look at that, but the MDL buffer is five times the size of the URB buffer, and the MDL offset is zero.
  • Jamey_KirbyJamey_Kirby Member - All Emails Posts: 433
    Tim is correct. Don't use the Mdl size to determine the bytes to
    read/write. You must get those from the IRP stack. Regardless of what
    you've read in documentation, Tim gave you the correct answer.

    -- Jamey

    On Thu, Feb 2, 2017 at 10:44 PM Tim Roberts wrote:

    > On Feb 2, 2017, at 9:50 AM, xxxxx@hotmail.com wrote:
    > >
    > > Oh come on, thats not the same at all.
    > >
    > > One part of the documentaiton saying that TransferBufferLength specifies
    > "the length, in bytes, of the buffer" while another part documents a
    > function that returns the "the length, in bytes, of the buffer" (to quote
    > the MSDN) and to have two different lengths is ridiculous.
    > >
    > > It needs sorting out.
    >
    > It certainly does not. Consider this code:
    >
    > char buffer[1024];
    >
    > ReadFile( hDevice, &buffer[20], 128, &dwActual, NULL );
    >
    > My buffer is 1024 bytes long, but I only want to read 128 bytes into it.
    > This is the EXACT same situation as what you are describing. The MDL
    > describes a buffer, not an I/O transaction. The TransferBufferLength
    > describes a transaction, which may be into or from PART of a buffer.
    > —
    > Tim Roberts, xxxxx@probo.com
    > Providenza & Boekelheide, Inc.
    >
    >
    > ---
    > NTDEV is sponsored by OSR
    >
    > Visit the list online at: <
    > http://www.osronline.com/showlists.cfm?list=ntdev>;
    >
    > MONTHLY seminars on crash dump analysis, WDF, Windows internals and
    > software drivers!
    > Details at
    >
    > To unsubscribe, visit the List Server section of OSR Online at <
    > http://www.osronline.com/page.cfm?name=ListServer>;
  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 7,345
    <quote>
    the MDL buffer is five times the size of the URB
    buffer, and the MDL offset is zero
    </quote>

    You know, I've started to answer this thread three separate times, and each time I've lost the will to argue and deleted my answer.

    Mr. Sykes... with all due respect... from a Windows architectural perspective, you're just wrong. Take this as an opportunity to learn something, instead of as an opportunity to argue. Mr. Roberts is rarely incorrect about such fundamental concepts, and he's absolutely correct here.

    I will admit: If you spend most of your time in the upper parts of the storage stack (for example), it's pretty easy to see the MDL as "the definitive description of the transfer"... but, really, it's not.

    It turns out that there are *some* places were the MDL size and the I/O operation (read/write) length are identical. But any time there is a separate place to get the length of the operation (URB, I/O Stack, SRB) it is incorrect to rely on the size of the underlying MDL as the size of the operation.

    As Mr. Roberts and Mr. Grig described, the MDL describes the size of a buffer in kernel mode. That's all it represents. When one performs an I/O operation from the described buffer, there is not ALWAYS the implication that the MDL describes the I/O operation.

    Whenever there's a description of the I/O operation... pointer, length, whatever... that's separate from the MDL, you use the separate description. The MDL has to be valid. And the actual data buffer used for the transfer has to be a valid subset of the buffer described by the MDL. That subset can be 100% of the MDL or just part of the MDL.

    This is just the way Windows architecture has been, since.... well.... at least 1992 when I started working with Windows NT.

    Peter
    OSR
    @OSRDrivers

    Peter Viscarola
    OSR
    @OSRDrivers

  • Jamey_KirbyJamey_Kirby Member - All Emails Posts: 433
    Two thumbs up to Peter!

    MDL describe the buffer, not the transfer. The concept is not so difficult
    to understand. There can be many, many reasons why the buffer is larger;
    alignment, separate transfers that break a large IO into smaller IOs, &c.

    On Sat, Feb 4, 2017 at 10:57 AM wrote:

    >
    > the MDL buffer is five times the size of the URB
    > buffer, and the MDL offset is zero
    >
    >
    > You know, I've started to answer this thread three separate times, and
    > each time I've lost the will to argue and deleted my answer.
    >
    > Mr. Sykes... with all due respect... from a Windows architectural
    > perspective, you're just wrong. Take this as an opportunity to learn
    > something, instead of as an opportunity to argue. Mr. Roberts is rarely
    > incorrect about such fundamental concepts, and he's absolutely correct here.
    >
    > I will admit: If you spend most of your time in the upper parts of the
    > storage stack (for example), it's pretty easy to see the MDL as "the
    > definitive description of the transfer"... but, really, it's not.
    >
    > It turns out that there are *some* places were the MDL size and the I/O
    > operation (read/write) length are identical. But any time there is a
    > separate place to get the length of the operation (URB, I/O Stack, SRB) it
    > is incorrect to rely on the size of the underlying MDL as the size of the
    > operation.
    >
    > As Mr. Roberts and Mr. Grig described, the MDL describes the size of a
    > buffer in kernel mode. That's all it represents. When one performs an I/O
    > operation from the described buffer, there is not ALWAYS the implication
    > that the MDL describes the I/O operation.
    >
    > Whenever there's a description of the I/O operation... pointer, length,
    > whatever... that's separate from the MDL, you use the separate
    > description. The MDL has to be valid. And the actual data buffer used for
    > the transfer has to be a valid subset of the buffer described by the MDL.
    > That subset can be 100% of the MDL or just part of the MDL.
    >
    > This is just the way Windows architecture has been, since.... well.... at
    > least 1992 when I started working with Windows NT.
    >
    > Peter
    > OSR
    > @OSRDrivers
    >
    >
    > ---
    > NTDEV is sponsored by OSR
    >
    > Visit the list online at: <
    > http://www.osronline.com/showlists.cfm?list=ntdev>;
    >
    > MONTHLY seminars on crash dump analysis, WDF, Windows internals and
    > software drivers!
    > Details at
    >
    > To unsubscribe, visit the List Server section of OSR Online at <
    > http://www.osronline.com/page.cfm?name=ListServer>;
    >
  • MBondMBond Member - All Emails Posts: 846
    Further to Peter?s answer (and I too have essayed to enter heretofore and also have deleted some nascent works)



    You may have a legitimate issue with the document writers at Microsoft for not making this point more clear by using identical language for two different values, but the main problem is this is obvious. First, from the point of view that when approaching any API (DDI) the presence of two separate parameters indicates that they describe two logically different values (independent or otherwise) and second from the point of view that at a systemic level how would it be possible to implement in a different way.



    You may justly criticise the doc writers for not including the correct references to other concepts or failing to qualify the count of which bytes, but when in this history of computing has any documentation ever been definitive from the point of view of the arbitrarily uninured; or for that matter in the history of writing, as each document must perforce assume something from its audience including at least the literacy to read its words.



    I have often found that it is important to think before blindly blaming others for their miss-work on the design or documentation of hardware or software. This method has stood me well through every kind of technology I have ever encountered except for cryptography. In this area Microsoft assuredly has some of their worst APIs and even poorer documentation, but equally assuredly they are well advanced compared with others who participate.



    Sent from Mail for Windows 10



    From: xxxxx@osr.com
    Sent: February 4, 2017 10:57 AM
    To: Windows System Software Devs Interest List
    Subject: RE:[ntdev] Re: MmGetMdlByteCount greater than TransferBufferLength




    the MDL buffer is five times the size of the URB
    buffer, and the MDL offset is zero


    You know, I've started to answer this thread three separate times, and each time I've lost the will to argue and deleted my answer.

    Mr. Sykes... with all due respect... from a Windows architectural perspective, you're just wrong. Take this as an opportunity to learn something, instead of as an opportunity to argue. Mr. Roberts is rarely incorrect about such fundamental concepts, and he's absolutely correct here.

    I will admit: If you spend most of your time in the upper parts of the storage stack (for example), it's pretty easy to see the MDL as "the definitive description of the transfer"... but, really, it's not.

    It turns out that there are *some* places were the MDL size and the I/O operation (read/write) length are identical. But any time there is a separate place to get the length of the operation (URB, I/O Stack, SRB) it is incorrect to rely on the size of the underlying MDL as the size of the operation.

    As Mr. Roberts and Mr. Grig described, the MDL describes the size of a buffer in kernel mode. That's all it represents. When one performs an I/O operation from the described buffer, there is not ALWAYS the implication that the MDL describes the I/O operation.

    Whenever there's a description of the I/O operation... pointer, length, whatever... that's separate from the MDL, you use the separate description. The MDL has to be valid. And the actual data buffer used for the transfer has to be a valid subset of the buffer described by the MDL. That subset can be 100% of the MDL or just part of the MDL.

    This is just the way Windows architecture has been, since.... well.... at least 1992 when I started working with Windows NT.

    Peter
    OSR
    @OSRDrivers


    ---
    NTDEV is sponsored by OSR

    Visit the list online at:

    MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers!
    Details at

    To unsubscribe, visit the List Server section of OSR Online at
  • Sven_KreamerSven_Kreamer Member Posts: 34
    The replies to this topic have been excellent. The mindset of the OP is incompatible with kernel development and surely makes many hope his drivers never run on their systems. Yes it is true the kernel APIs are convoluted, error prone, and documentation often problematic. You have to put processes together to deal with this to be successful. Over the years the most reliable thing I have found to do when there is ever a doubt is defer to what the samples do. Since they were written by knowledgeable people, get executed and tested and updated, they do things the correct way. So a simple peek at a relevant source file was all that was necessary to confirm any doubt in cases such as this.
  • anton_bassovanton_bassov Member Posts: 5,023
    > This is just the way Windows architecture has been, since.... well.... at least 1992
    > when I started working with Windows NT.


    Oh dear......


    You seem to have had started working with NT before it even got released....

    https://en.wikipedia.org/wiki/Windows_NT

    <quote>

    Windows NT is a family of operating systems produced by Microsoft, the first version of which was released in July 1993.

    </quote>

    To be honest, I had a very remote idea of what the PC looked like at the time anyway....

    Anton Bassov
  • Tim_RobertsTim_Roberts Member - All Emails Posts: 13,022
    xxxxx@hotmail.com wrote:
    > > This is just the way Windows architecture has been, since.... well.... at least 1992
    >> when I started working with Windows NT.
    > Oh dear......
    >
    > You seem to have had started working with NT before it even got released....

    Many people did. The development of what was originally called NT OS/2
    began in 1988 or 1989.

    --
    Tim Roberts, xxxxx@probo.com
    Providenza & Boekelheide, Inc.

    Tim Roberts, [email protected]
    Providenza & Boekelheide, Inc.

  • matt_sykesmatt_sykes Member - All Emails Posts: 270
    @Everyone, I find it staggering that the very simple statement of mine that:
    1) Either the documentation needs improving to describe theses values better, or
    2) The driver above has put a wrong value in one of them

    Is not accepted as a perfectly valid statement, since it clearly is factual.

    And now with the clarification from Tim and Anton we know its the documentaion that is insufficient.

    @Sven. Trusting the sample code is a very bad idea. It is good for ideas, but has always been very buggy. To use your suggestion for example, take a look at this line from the sample code:

    length = min(MmGetMdlByteCount(irp->MdlAddress), pMpRfd->PacketSize);
    RtlCopyMemory(buffer, pMpRfd->Buffer, length);

    from the 7600 DDK, in the pcidrv\wdm\hw source. It is using the byte count as the copy length, not the irp value.
  • Pavel_APavel_A Member Posts: 2,681
    Ok so how can it be that the size of a request is (much) greater than the size of MDL associated with it? Do newer Windows versions (where the ioctl buffer size is not limited by max. MDL size) create something like array of MDLs to pass a large buffer? Then one such request can span several MDLs?

    Regards,
    -- pa
  • matt_sykesmatt_sykes Member - All Emails Posts: 270
    @Pavel, if there are a chain of MDLs does their total byte count look something like the transfer buffer size?
  • Tim_RobertsTim_Roberts Member - All Emails Posts: 13,022
    xxxxx@hotmail.com wrote:
    > @Everyone, I find it staggering that the very simple statement of mine that:
    > 1) Either the documentation needs improving to describe theses values better, or
    > 2) The driver above has put a wrong value in one of them
    >
    > Is not accepted as a perfectly valid statement, since it clearly is factual.
    >
    > And now with the clarification from Tim and Anton we know its the documentaion that is insufficient.

    Which documentation? I don't claim that the documentation is perfect,
    but I'd like to know which documents set you astray.


    > @Sven. Trusting the sample code is a very bad idea. It is good for ideas, but has always been very buggy. To use your suggestion for example, take a look at this line from the sample code:
    >
    > length = min(MmGetMdlByteCount(irp->MdlAddress), pMpRfd->PacketSize);
    > RtlCopyMemory(buffer, pMpRfd->Buffer, length);
    >
    > from the 7600 DDK, in the pcidrv\wdm\hw source. It is using the byte count as the copy length, not the irp value.

    This is a completely different situation. You were specifically talking
    about USB URBs, where requests can come from arbitrary kernel-mode
    drivers. Kernel-mode programmers have the flexibility to do magical
    things with their MDLs, so the lengths don't need to match. In that
    case, the length of the transfer is given in the URB field.

    In the pcidrv example, this is known to be an IRP_MJ_READ, and the
    contract is that it is coming from a user-mode client using direct I/O.
    In that case, we KNOW that the MDL exactly describes the user's buffer,
    because the operating system I/O manager always creates them that way.
    This design alleviates the need to carry the length as a separate piece
    of state.

    --
    Tim Roberts, xxxxx@probo.com
    Providenza & Boekelheide, Inc.

    Tim Roberts, [email protected]
    Providenza & Boekelheide, Inc.

  • Alex_GrigAlex_Grig Member Posts: 3,238
    >In that case, we KNOW that the MDL exactly describes the user's buffer,
    because the operating system I/O manager always creates them that way.
    This design alleviates the need to carry the length as a separate piece
    of state.

    Actually, IO_STACK_LOCATION::Read carries Length, as well (Write does that, too), which takes priority over whatever is in MDL. The MDL will most likely have the same length, though.
  • matt_sykesmatt_sykes Member - All Emails Posts: 270
    @Tim The MSDN documentation on the MDL buffer length, and the URB transfer buffer length.

    https://msdn.microsoft.com/en-us/library/windows/hardware/ff554530(v=vs.85).aspx "MmGetMdlByteCount returns the length, in bytes, of the buffer described by Mdl. "


    https://msdn.microsoft.com/en-us/library/windows/hardware/ff540352(v=vs.85).aspx "TransferBufferLength

    Specifies the length, in bytes, of the buffer specified in TransferBuffer or described in TransferBufferMDL"


    See what I mean? They are documented as being one and the same. If it is legal for a driver to act as you and Anton say then the documentation should be changed since as it stands it doesnt permit it.


    Your comments regarding my example merely compound the lack of documentation. You do resalise what you are saying here Tim, that it depends on who created the MDL-Irp-URB as to whether the two byte count values are the same or not?

    Really? And the code is supposed to know this is it? How about if an upper filter is put between between your driver that trusts the IoManager, and suddenly your driver hasa byte count thats wrong?

    BSOD.


    Come on Tim, this has to be tightened up. I say it again, either the documentation is wrong or the driver that preps an Irp with different byte count values is wrong.

    Trying to have it both ways is chaos, utter chaos.
  • matt_sykesmatt_sykes Member - All Emails Posts: 270
    @Peter, and can I add I am NOT saying Tim is wrong in his description of the way the MDL byte count is handled, I am saying if he is right, then he is wrong to state the documentation describes such a handling. Because it doesn't.

    I don't know how you misread my replies to Tim, I thought my popsition was, and is, quite clear from them.
  • Alex_GrigAlex_Grig Member Posts: 3,238
    You keep confusing two things. An MDL and an I/O request.
    An MDL just describes some memory.
    You do I/O with whatever the I/O request says. If it has Length and Address fields, you use these fields. If it has an MDL, you pass the MDL to GetScatterGatherList, but still use URB/SRB length and address in GetScatterGatherList call.
  • Jamey_KirbyJamey_Kirby Member - All Emails Posts: 433
    And it keeps going...


    On Wed, Feb 8, 2017 at 9:35 AM wrote:

    > You keep confusing two things. An MDL and an I/O request.
    > An MDL just describes some memory.
    > You do I/O with whatever the I/O request says. If it has Length and
    > Address fields, you use these fields. If it has an MDL, you pass the MDL to
    > GetScatterGatherList, but still use URB/SRB length and address in
    > GetScatterGatherList call.
    >
    >
    > ---
    > NTDEV is sponsored by OSR
    >
    > Visit the list online at: <
    > http://www.osronline.com/showlists.cfm?list=ntdev>;
    >
    > MONTHLY seminars on crash dump analysis, WDF, Windows internals and
    > software drivers!
    > Details at
    >
    > To unsubscribe, visit the List Server section of OSR Online at <
    > http://www.osronline.com/page.cfm?name=ListServer>;
    >
  • Tim_RobertsTim_Roberts Member - All Emails Posts: 13,022
    xxxxx@hotmail.com wrote:
    > @Tim The MSDN documentation on the MDL buffer length, and the URB transfer buffer length.
    >
    > https://msdn.microsoft.com/en-us/library/windows/hardware/ff554530(v=vs.85).aspx "MmGetMdlByteCount returns the length, in bytes, of the buffer described by Mdl. "

    That's true.


    > https://msdn.microsoft.com/en-us/library/windows/hardware/ff540352(v=vs.85).aspx "TransferBufferLength
    > specifies the length, in bytes, of the buffer specified in TransferBuffer or described in TransferBufferMDL"

    I will grant you that this could be worded more precisely. It should
    say that TransferBufferLength specifies the length, in bytes, of the
    buffer to be used for this transfer.

    I assume you can understand the difficulty here. I can allocate a large
    buffer, and then use parts of that buffer for different purposes. When
    you get a TransferBuffer in an URB, the length is not carried along, so
    there is no conflict. When you get a TransferBufferMDL, it just so
    happens that the MDL carries knowledge of its own length, but that
    length is not relevant to this transfer.


    > Your comments regarding my example merely compound the lack of documentation. You do resalise what you are saying here Tim, that it depends on who created the MDL-Irp-URB as to whether the two byte count values are the same or not?

    Right. It absolutely does. Different driver stacks have different
    contracts, largely because of history. That's just the way it is.


    > Really? And the code is supposed to know this is it? How about if an upper filter is put between between your driver that trusts the IoManager, and suddenly your driver hasa byte count thats wrong?
    >
    > BSOD.

    Correct. That's what happens when a driver violates the contract, which
    is what such a filter driver would have done. An upper-level driver is
    ALLOWED to assume conditions from I/O Manager. Any upper filter must
    maintain those conditions.

    --
    Tim Roberts, xxxxx@probo.com
    Providenza & Boekelheide, Inc.

    Tim Roberts, [email protected]
    Providenza & Boekelheide, Inc.

  • MBondMBond Member - All Emails Posts: 846
    In fact they even had a partner program for application software. The highlight of which for me was getting a fancy paperweight with the OS/2 logo suspended in a triangular prism of plastic



    Sent from Mail for Windows 10



    From: Tim Roberts
    Sent: February 6, 2017 2:32 PM
    To: Windows System Software Devs Interest List
    Subject: Re: [ntdev] Re: MmGetMdlByteCount greater than TransferBufferLength



    xxxxx@hotmail.com wrote:
    > > This is just the way Windows architecture has been, since.... well.... at least 1992
    >> when I started working with Windows NT.
    > Oh dear......
    >
    > You seem to have had started working with NT before it even got released....

    Many people did. The development of what was originally called NT OS/2
    began in 1988 or 1989.

    --
    Tim Roberts, xxxxx@probo.com
    Providenza & Boekelheide, Inc.


    ---
    NTDEV is sponsored by OSR

    Visit the list online at:

    MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers!
    Details at

    To unsubscribe, visit the List Server section of OSR Online at
  • anton_bassovanton_bassov Member Posts: 5,023
    >Many people did. The development of what was originally called NT OS/2 began in 1988 or 1989


    Well, normally I would expect them to be mainly on the development team, but, OTOH, any OS
    needs some hardware to run on, and,hence, some third-party drivers may be included in the very first OS release. Therefore, a bit of OS work may be,indeed, done by the third-party developers even prior to its first official release..



    Anton Bassov
  • MBondMBond Member - All Emails Posts: 846
    Anton



    For OS/2, they had programs for all kinds of ISVs. They wanted to have applications that actually ran on the new OS when it launched , so they engaged a wide audience. At the time I was working on Line of Business applications for the financial industry (as I am again) rather than drivers and we were assigned a project to build an OS/2 version of our software. Our life became radially simpler once Microsoft decided to implement the Win32 subsystem (or was it called Win16 of something I forget; or I?m just getting old). Anyways, after spending a lot of time and effort to determine what work would be required to migrate our software and producing some prototype code, Microsoft came out with this new fangled NT thing which just ran our existing binaries (and probably still does if I bothered to try them) and the fortunes of Microsoft and IBM have diverged ever since.



    Anyways, even later on Microsoft was very keen on having third parties participate in their BETA testing ? and that hasn?t changed much based on the availability of the plethora of RC builds on MSDN and even publically. I suspense it makes sense as it is cost effective to have others test and validate your software if they are willing to volunteer and there are good reasons why we might want to have some idea of what those in Redmond are planning for us. Anyways I seem to have gotten over my nightmares of working with BETA OS versions ? although I?m sure the scars have made me a better programmer on balance.



    As I said, the best thing I got out of this was a triangular prism of plastic with the OS/2 logo suspended partway through. It was a partners only thing ??







    Sent from Mail for Windows 10



    From: xxxxx@hotmail.com
    Sent: February 9, 2017 7:45 AM
    To: Windows System Software Devs Interest List
    Subject: RE:[ntdev] Re: MmGetMdlByteCount greater than TransferBufferLength



    >Many people did. The development of what was originally called NT OS/2 began in 1988 or 1989


    Well, normally I would expect them to be mainly on the development team, but, OTOH, any OS
    needs some hardware to run on, and,hence, some third-party drivers may be included in the very first OS release. Therefore, a bit of OS work may be,indeed, done by the third-party developers even prior to its first official release..



    Anton Bassov

    ---
    NTDEV is sponsored by OSR

    Visit the list online at:

    MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers!
    Details at

    To unsubscribe, visit the List Server section of OSR Online at
  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    I have once read that the guys who wrote NT came from DEC. I think it was about the Windows PE binary format. This format was said to be used in VAX computers and brought to Windows NT by DEC engineers.

    Mark Russinovich wrote a very interesting article about Windows NT's history (in 1998). Every windows driver writer should read it.

    http://m.windowsitpro.com/windows-client/windows-nt-and-vms-rest-story
  • anton_bassovanton_bassov Member Posts: 5,023
    > I have once read that the guys who wrote NT came from DEC.I have once read that
    > the guys who wrote NT came from DEC.


    This is true - NT chief architect came from DEC and had previously authored VMS. Therefore, I suspect quite a few engineers must have followed him. In fact, quite a few of this list" old timers"
    come from DEC/VMS background. Our host is just one example. Another example is my "buddy" Dan Kyler - he even came up once with his "only proper spinlock implementation in existence"(which, in his opinion, happens to be a tight loop of interlocked operations) in VAX assembly language


    > I think it was about the Windows PE binary format.


    Well, PE is based upon COFF file format, which had been used by quite a few systems - VMS happened (or, probably still happens) to be just one of them. However, architectural similarities
    between VMS and NT go well beyond that - after all,where do you think an abomination known as pageable kernel code is rooted? As they say, increment every letter of 'VMS", and see what you will get....



    Anton Bassov
  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    >Our host is just one example.

    Amazing ! In 1993, we were at the university, learning UNIX on computers running ... VAX!
  • Jamey_KirbyJamey_Kirby Member - All Emails Posts: 433
    Read some old books on VMS and VMS file systems. This is where NT came
    from. I think I still have a copy of the book VMS File System. It gives you
    a good idea on where IRPs came from and how things tie together. We've come
    a long way; sorta.

    On Fri, Feb 10, 2017, 12:54 PM wrote:

    > >Our host is just one example.
    >
    > Amazing ! In 1993, we were at the university, learning UNIX on computers
    > running ... VAX!
    >
    > ---
    > NTDEV is sponsored by OSR
    >
    > Visit the list online at: <
    > http://www.osronline.com/showlists.cfm?list=ntdev>;
    >
    > MONTHLY seminars on crash dump analysis, WDF, Windows internals and
    > software drivers!
    > Details at
    >
    > To unsubscribe, visit the List Server section of OSR Online at <
    > http://www.osronline.com/page.cfm?name=ListServer>;
    >
  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    >Read some old books on VMS and VMS file systems. This is where NT came
    from. I think I still have a copy of the book VMS File System.

    Thank you for the suggestion.
  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 7,345
    We're not going to comprehensively review Windows NT history here.

    The Windows NT project started on November 7th 1988, when a group of former DEC guys (7 of them, IIRC) started at MSFT. This was subsequent to a new OS (Mica) for entirely new silicon (PRISM) both then under development at DEC being cancelled.

    Windows NT is remarkably like Mica (I know this as I was at DEC and privileged to have access to the Mica design documents at the time). This was litigated and settled amicably between DEC and Microsoft. Interestingly, the Alpha processor was also remarkably like PRISM. And part of the "amicable settlement" was MSFT supporting NT on Alpha.

    The rest is history. Very interesting history, but history nonetheless. I swear someday I'm going to write a book about this.

    There are significant architectural similarities between VMS and NT, and even between RSX-11M (another Culter OS) and NT. These reflect Cutler's approach to OS design. He is an intensely practical engineer and just has always done things certain ways that he knows "work."

    Peter
    OSR
    @OSRDrivers

    P.S. Yes, Anton. I was writing NT drivers well before NT was released. Lots and lots of people were. The beta was open and widely accessible.

    Peter Viscarola
    OSR
    @OSRDrivers

  • Bo_BranténBo_Brantén Member Posts: 92
    On Fri, 10 Feb 2017, xxxxx@osr.com wrote:

    > We're not going to comprehensively review Windows NT history here.

    For they who want deeper information I can recomend the book
    "Showstopper!" by G. Pascal Zachary, it is to a part a biography over
    Dave Cutler and then describes the project at Microsoft untill the first
    release of Windows NT.

    Bo Branten
  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 7,345
    <quote>
    I can recomend the book
    "Showstopper!" by G. Pascal Zachary
    </quote>

    I'm not a fan. I found this book to be poorly written, by an author that seemed to have little or no gasp of OS architecture or development.

    Then again, I haven't looked at it since I read it when it was first released. I found it VERY disappointing. As I remember it, it spoke to few of the questions or issues I was interested in reading about.

    Peter
    OSR
    @OSRDrivers

    Peter Viscarola
    OSR
    @OSRDrivers

  • matt_sykesmatt_sykes Member - All Emails Posts: 270
    @ Tim "I will grant you that this could be worded more precisely."

    Fianlly!

    Right, so how do we go about getting this changed. The MDL byte count should be be documented as the "total size of the MDL buffer which might be greater than the length of data to be transfered in the buffer".
Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Upcoming OSR Seminars
Developing Minifilters 29 July 2019 OSR Seminar Space
Writing WDF Drivers 23 Sept 2019 OSR Seminar Space
Kernel Debugging 21 Oct 2019 OSR Seminar Space
Internals & Software Drivers 18 Nov 2019 Dulles, VA