At 17:02 06/09/2008, xxxxx@hotmail.com wrote:
Peter,
>Sometimes I *think* we might have a good discussion, but your
discussion degenerates
> to mere argument.Are you sure this is my fault??? Look at the statements that you
made on this thread:
blah … blah … blah
I just updated my junk filters … bye Anton.
Don,
your idea of designing a driver model withough considering the rest of the OS, shows a lack of >experience in the problems of OS design.
I am afraid you just got it wrong - I have never ever said anything about “designing a driver model withough considering the rest of the OS”. Furthermore, the very term “driver model” itself is relative, because you may have totally different driver models for different device classes. The idea of the experiment we are discussing with Doron its to try yet another driver model with interface of our own choice (if we decide that we don’t want it to be limited to only to POSIX, we can always just introduce the additional system calls the way sockfs and pipefs do), and to make this driver model layered, which will make it different from all existing models (i.e. the oens for character/block/network/TTY devices).
In other words, we are speaking about extending the OS, rather than ignoring it altogether…
Anton Bassov
> I just updated my junk filters … bye Anton.
Bye Mark…
You have found a really wonderful solution - from now on you are not going to be annoyed with my posts, and I am not going to get annoyed with your constant complaints. After all, what is pointless blah-blah-blah for person X make be quite informative for person Y - if you check the archives, you will see quite a few posts saying that some people actually learn a lot on these “rambling threads”. Apparently, you are not among them, so you have a full right to be protected from the information you have no interest in whatsoever, and those who find it informative have a full right to be protected from your complaints. Everyone is going to be happy now…
Anton Bassov
This is the worst thread ever, both of them that is, (SSDT & this one).
mm
Mark S. Edwards wrote:
At 17:02 06/09/2008, xxxxx@hotmail.com wrote:
> Peter,
>
> >Sometimes I *think* we might have a good discussion, but your
> discussion degenerates
> > to mere argument.
>
> Are you sure this is my fault??? Look at the statements that you made
> on this thread:blah … blah … blah
I just updated my junk filters … bye Anton.
Don’t be so quick some wise ass butt-head is going to come and say it
helps to learn a lot and I’m keeping these email in my personal folder…
And same butt-load of shit goes as an advice, and why you moron did not
do it as I propose. It is going to do all the good you never imagine …
-pro
Martin O’Brien wrote:
This is the worst thread ever, both of them that is, (SSDT & this one).
mm
Mark S. Edwards wrote:
> At 17:02 06/09/2008, xxxxx@hotmail.com wrote:
>> Peter,
>>
>> >Sometimes I *think* we might have a good discussion, but your
>> discussion degenerates
>> > to mere argument.
>>
>> Are you sure this is my fault??? Look at the statements that you
>> made on this thread:
>
>
> blah … blah … blah
>
>
> I just updated my junk filters … bye Anton.
>
>
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminarsTo unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
I personally find reading Anton’s posts very entertaining, especially after having a big meal.
–
Calvin Guan
Broadcom Corporation
Connecting Everything(r)
“Mark S. Edwards” wrote in message news:xxxxx@ntdev…
> At 17:02 06/09/2008, xxxxx@hotmail.com wrote:
>>Peter,
>>
>> >Sometimes I think we might have a good discussion, but your
>> discussion degenerates
>> > to mere argument.
>>
>>Are you sure this is my fault??? Look at the statements that you
>>made on this thread:
>
>
> blah … blah … blah
>
>
> I just updated my junk filters … bye Anton.
>
Doron,
why not create a callback based, layered driver model that does sync and async io in linux
to prove your point.
Already investigating this direction…
The very first thoughts are:
As long as we speak about asynch IO (especially to the stack you don’t participate in), I DO admit
that this or that IO request descriptor is needed -even if the whole model is based purely upon the callback mechanism. Furthermore, I do admit that it would be a just great idea to give you a chance to specify your IO completion callback on per-request basis, particularly if you send asynch request to a different stack.
The above statement applies only to asynch requests, and only specific types of request may be asynchronus (i.e. read/write/ioctl/flush). I still think there is no need for a request descriptor for synch operations of any description, so that I still insist that in vast majority of cases it is not-so-great idea. Therefore, I would not build the whole IO model on it, and, instead, reserve it only for a specific types of requests, plus define separate methods for synch and asynch IO paths, so that this descriptor would be just one of the arguments supplied to asynch methods.
If you want to do truly efficient asynch IO, it would be a great idea to give you a chance to split/merge/swap/extend or otherwise modify IO buffers simply by updating few pointers, rather than going through the whole pain of allocating/copying/freeing buffers. Therefore, asynch IO requests should come with something reminiscent of socket buffers or NDIS_PACKETs( with few additional fields , of course) - in any case, no matter what they are like, they should be totally from IRPs as known under Windows, because even MDLs, let alone plain-virtual-address-based buffers, just don’t seem to be adequate here…
Anton Bassov
Did I mention that I *really* need a way to lock threads, to prevent further posting to them after they’ve run their course??
Yeah… Let’s see, a bit of database programming (I’ll do it in C#), and I’ll be all set. Just need the time…
Peter
OSR
Bill McKenzie wrote:
While we are at it why don’t we ask why the x86 architecture is so screwed
up compared to just about any other processor architecture on the market?
The answer is most likely “Binary backward compatibility to the 8086.”
See http://en.wikipedia.org/wiki/Intel_8086
and http://en.wikipedia.org/wiki/X86
At least until some years ago, real-mode code was in any MBR.
Probably still runs on the latest Core 2.
Keeping binary compatibility for an extended period of time will
probably make any architecture - hardware and software - look “screwed
up” after a few years.
This is virtually unavoidable:
You have either to drag around these 10-year-old binary APIs with you,
or some old app will break and not run anymore.
If you need to change parameters or functionality, you can not just
replace the old calls (again, then the old software would break). You
can only add new, “extended”, calls alongside the old interface.
The pity is that you even have to be “bugward compatible” in the old
call interface, otherwise (again!) existing software with workarounds
for them will break.
Another possible approach would be to run old applications in their own
subsystem (cf. Apple “Classic Environment”, WOW, etc.), or to drop
binary compatibility and accept you have to recompile or at least
re-link your application for a new platform (cf. Unix).
With the old API calls then you can have a compatibility wrapper at the
application side or in the subsystem and don’t need to clutter up your
main API.
On the other hand, binary compatibility rocks: it’s cool that I only
need one WDM kernel driver binary for anything from Win98SE to Vista. 
xxxxx@osr.com wrote:
Did I mention that I *really* need a way to lock threads, to prevent
further posting to them after they’ve run their course??
Ahem… sorry, the thread-based display made my miss this post.
Yeah… Let’s see, a bit of database programming […]
Don’t forget the NNTP interface to reject or “park” posts after locking,
too.
Hagen Patzke wrote:
- The pity is that you even have to be “bugward compatible” in the old
call interface, otherwise (again!) existing software with workarounds
for them will break.
In case anyone is interested in a good read, “The Old New Thing” by
Raymond Chen describes a lot of the bugward compatibility hacks still in
the Win32 API. I found it fascinating.
He has a blog with the same name and subject, too.
-Stephen Cleary
Yes. We need a mechanism to lock threads from all 3 forms of access (email, NNTP, and web).
Given that Lyris was initially designed to be “a better listsrv”, it doesn’t natively have a “lock” capability. In fact, you might notice that it BARELY had a thread capability (some “RE: Topic Replied To” posts become their own topic… based on some twisted algorithm or other).
Promisingly, Lyris DOES have a semi-documented facility to run a program to pre-process each submission to a given list. Semi-documented meaning in this case “they can’t provide a single example of how this works, and nobody at Lyris Inc has used this interface for more than 5 years”. So, I’d need to spend a few days figuring it out… and while I can think of nothing that would contribute more to my overall sense of well-being at this point, I don’t think my client would appreciate my missing the next milestone, which is what doing the list server work would talk at this point.
Someday…
Peter
OSR
Well of course the processors are backward compatible…that is one of the
reasons the Itanium had such a rough go. But the 8086, save for its
superior integration with its math coprocessor, was arguably not as good a
processor as its contemporaries when IBM came out with the first PC. My
point was, why ask why some technology made it or not, even if it is an
inferior technology. It doesn’t matter…the best technologies rarely win
and who really cares. Gots to go with a winner.
And yes, non-EFI systems still run 16-bit x86 assembly in the MBR. I don’t
know what EFI does to that part of the boot process…haven’t studied that
yet.
Bill M.
“Hagen Patzke” wrote in message news:xxxxx@ntdev…
> Bill McKenzie wrote:
>> While we are at it why don’t we ask why the x86 architecture is so
>> screwed up compared to just about any other processor architecture on the
>> market?
>
> The answer is most likely “Binary backward compatibility to the 8086.”
>
> See http://en.wikipedia.org/wiki/Intel_8086
> and http://en.wikipedia.org/wiki/X86
>
> At least until some years ago, real-mode code was in any MBR.
> Probably still runs on the latest Core 2.
>
>
>
> Keeping binary compatibility for an extended period of time will probably
> make any architecture - hardware and software - look “screwed up” after a
> few years.
>
> This is virtually unavoidable:
>
> - You have either to drag around these 10-year-old binary APIs with you,
> or some old app will break and not run anymore.
>
> - If you need to change parameters or functionality, you can not just
> replace the old calls (again, then the old software would break). You can
> only add new, “extended”, calls alongside the old interface.
>
> - The pity is that you even have to be “bugward compatible” in the old
> call interface, otherwise (again!) existing software with workarounds for
> them will break.
>
> Another possible approach would be to run old applications in their own
> subsystem (cf. Apple “Classic Environment”, WOW, etc.), or to drop binary
> compatibility and accept you have to recompile or at least re-link your
> application for a new platform (cf. Unix).
> With the old API calls then you can have a compatibility wrapper at the
> application side or in the subsystem and don’t need to clutter up your
> main API.
>
>
>
> On the other hand, binary compatibility rocks: it’s cool that I only need
> one WDM kernel driver binary for anything from Win98SE to Vista.
>
I don’t know much about EFI, but I do know there is a new version called
uEFI. My capitalization may be wrong, but there is a new standard being
pushed.
“Bill McKenzie” wrote in message
news:xxxxx@ntdev…
> Well of course the processors are backward compatible…that is one of the
> reasons the Itanium had such a rough go. But the 8086, save for its
> superior integration with its math coprocessor, was arguably not as good a
> processor as its contemporaries when IBM came out with the first PC. My
> point was, why ask why some technology made it or not, even if it is an
> inferior technology. It doesn’t matter…the best technologies rarely win
> and who really cares. Gots to go with a winner.
>
> And yes, non-EFI systems still run 16-bit x86 assembly in the MBR. I
> don’t know what EFI does to that part of the boot process…haven’t
> studied that yet.
>
> Bill M.
>
>
> “Hagen Patzke” wrote in message news:xxxxx@ntdev…
>> Bill McKenzie wrote:
>>> While we are at it why don’t we ask why the x86 architecture is so
>>> screwed up compared to just about any other processor architecture on
>>> the market?
>>
>> The answer is most likely “Binary backward compatibility to the 8086.”
>>
>> See http://en.wikipedia.org/wiki/Intel_8086
>> and http://en.wikipedia.org/wiki/X86
>>
>> At least until some years ago, real-mode code was in any MBR.
>> Probably still runs on the latest Core 2.
>>
>>
>>
>> Keeping binary compatibility for an extended period of time will probably
>> make any architecture - hardware and software - look “screwed up” after a
>> few years.
>>
>> This is virtually unavoidable:
>>
>> - You have either to drag around these 10-year-old binary APIs with you,
>> or some old app will break and not run anymore.
>>
>> - If you need to change parameters or functionality, you can not just
>> replace the old calls (again, then the old software would break). You can
>> only add new, “extended”, calls alongside the old interface.
>>
>> - The pity is that you even have to be “bugward compatible” in the old
>> call interface, otherwise (again!) existing software with workarounds for
>> them will break.
>>
>> Another possible approach would be to run old applications in their own
>> subsystem (cf. Apple “Classic Environment”, WOW, etc.), or to drop binary
>> compatibility and accept you have to recompile or at least re-link your
>> application for a new platform (cf. Unix).
>> With the old API calls then you can have a compatibility wrapper at the
>> application side or in the subsystem and don’t need to clutter up your
>> main API.
>>
>>
>>
>> On the other hand, binary compatibility rocks: it’s cool that I only need
>> one WDM kernel driver binary for anything from Win98SE to Vista.
>>
>
>
>
Bill McKenzie wrote:
Well of course the processors are backward compatible…that is one of the
reasons the Itanium had such a rough go. But the 8086, save for its
superior integration with its math coprocessor, was arguably not as good a
processor as its contemporaries when IBM came out with the first PC.
Correct. The design if the original IBM PC was as much accident as
engineering. The design group had essentially zero budget, so the
designers were forced to use whatever spare parts were floating around
the lab parts bin. They happened to have a bunch of 8086s.
That’s also why we are stuck with the odd countdown timer frequency –
the parts counter in IBM had a lot of NTSC color television crystals at
3.579545 MHz.
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
Well, it looks like this “rambling thread” just refuses to die out, despite Peter’s objections…
My point was, why ask why some technology made it or not, even if it is an inferior technology.
…
the best technologies rarely win and who really cares.
First of all, you have to give your definition of superior and inferior technology…
What is the criteria??? Is it efficiency? Is it low manufacturing cost (which is particularly important for the hardware technologies)? Is it portability so that the same technology can be used on a wide range of platforms (for example, x86 that you have earlier mentioned as something"inferior" may be used on everything from the pocket PC to the server-grade system)? Please note that everything mentioned above are technical parameters -after all, a technology may become a “winner” for totally non-technical reasons as well…
Anton Bassov
> First of all, you have to give your definition of superior and inferior
technology…
No I don’t.
Bill M.
wrote in message news:xxxxx@ntdev…
> Well, it looks like this “rambling thread” just refuses to die out,
> despite Peter’s objections…
>
>
>
>> My point was, why ask why some technology made it or not, even if it is
>> an inferior technology.
> …
>
>> the best technologies rarely win and who really cares.
>
> First of all, you have to give your definition of superior and inferior
> technology…
>
> What is the criteria??? Is it efficiency? Is it low manufacturing cost
> (which is particularly important for the hardware technologies)? Is it
> portability so that the same technology can be used on a wide range of
> platforms (for example, x86 that you have earlier mentioned as
> something"inferior" may be used on everything from the pocket PC to the
> server-grade system)? Please note that everything mentioned above are
> technical parameters -after all, a technology may become a “winner” for
> totally non-technical reasons as well…
>
>
> Anton Bassov
>
>
I have had some experience with 2) below.
There have been several cases that I am aware of within Windows where a driver stack was built assuming an operation will always be synchronous. I have seen this later become a major pain point where there is some underlying change that requires the operation to become asynchronous. Thus from this experience, I now ensure all such operations are designed to be asynchronous, even though they usually complete synchronously. While this may have some performance penalty (though I haven’t seen it make a measurable difference), it really helps to ensure that the contracts don’t have to change when the technology changes.
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@hotmail.com
Sent: Sunday, September 07, 2008 12:11 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Microkernel discussion continued…
Doron,
why not create a callback based, layered driver model that does sync and async io in linux
to prove your point.
Already investigating this direction…
The very first thoughts are:
As long as we speak about asynch IO (especially to the stack you don’t participate in), I DO admit
that this or that IO request descriptor is needed -even if the whole model is based purely upon the callback mechanism. Furthermore, I do admit that it would be a just great idea to give you a chance to specify your IO completion callback on per-request basis, particularly if you send asynch request to a different stack.
The above statement applies only to asynch requests, and only specific types of request may be asynchronus (i.e. read/write/ioctl/flush). I still think there is no need for a request descriptor for synch operations of any description, so that I still insist that in vast majority of cases it is not-so-great idea. Therefore, I would not build the whole IO model on it, and, instead, reserve it only for a specific types of requests, plus define separate methods for synch and asynch IO paths, so that this descriptor would be just one of the arguments supplied to asynch methods.
If you want to do truly efficient asynch IO, it would be a great idea to give you a chance to split/merge/swap/extend or otherwise modify IO buffers simply by updating few pointers, rather than going through the whole pain of allocating/copying/freeing buffers. Therefore, asynch IO requests should come with something reminiscent of socket buffers or NDIS_PACKETs( with few additional fields , of course) - in any case, no matter what they are like, they should be totally from IRPs as known under Windows, because even MDLs, let alone plain-virtual-address-based buffers, just don’t seem to be adequate here…
Anton Bassov
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
Randy,
There have been several cases that I am aware of within Windows where a driver
stack was built assuming an operation will always be synchronous.
Well, the above seems to be just a generally-accepted practice under Linux. If you grep
the kernel tree for kick_iocb (by calling this function a driver tells AIO core about asynch completion so that it invokes a callback in context of a thread that initialized asynch IO), you will see that there is only ONE (!!!) driver that actually makes any use of it All other drivers seem to implement their asynch IO handlers exactlly the same way they would implement a synchronous one - AIO core mostly seems to be just a dead piece of code.
In any case, there does not seem to be that much demand for asynch IO from the userland either. Standard GNU C library does not make any use of asynch IO-related kernel exports - instead, it provides its own implementation of asynch POSIX calls in the user mode by means of introducing additional thread, so that, although from the caller’s s perspective it may look like something asynchronous, in actuality it is still sychronous IO. If you want to make use of Linux asynch IO kernel exports, you need to link against a separate library.
Therefore, although Doron’s proposal sounds like really exciting intellectual challenge, I am afraid its practical usefulness is critically close to zero - even if you find this or that way to push it into a mainstream kernel, there is no guarantee that someone may actually make any practical use of it…
Anton Bassov