> telling it from whence and to where I want the data. I allocated
the
descriptors from non-paged pool. I need to get the DMA chain’s
address in
Allocate them from the common buffer instead. This is the common
practice.
Max
Asher Hoodin wrote:
Does this mean that you can write a WDM driver for an ISA board?
Yes, of course.
I have been told that WDM is only for PNP PCI devices. (not that I
believe them but I have no other sources that i actually physically
socialize with.)
The only real difference between a WDM driver for a PCI card and an ISA
card is that you put the configuration information for the ISA card into
a LogConfig section in the INF file. The driver just accepts whatever
resources the PnP Manager gives it in an IRP_MN_START_DEVICE. This is
easier than trying to figure out which resources to claim like you would
have in NT 4.
For example, my WDM book has two sample drivers (PCI42 and PKTDMA) for
the AMCC 5933 development kit, which includes a PCI card and an ISA card
that connects to the PCI card via a ribbon cable so you can simulate the
“back end” functionality of some device you’re trying to develop. Those
samples work with a WDM driver (S5933DK.SYS) for the ISA card, and its
INF file has the resource configuration information needed to make the
ISA card work.
(My DMA sample is not directly relevant to the question that got this
thread started, BTW, which is why I didn’t mention it earlier. The
original poster needs to use a common buffer, and several people have
already told him that. He said he had already studied my sample,
anyway.)
–
Walter Oney, Consulting and Training
Check out new US seminar schedule at http://www.oneysoft.com
Yes you can, but you will need to specify LogConfig in the INF.
Max
----- Original Message -----
From: “Asher Hoodin”
To: “NT Developers Interest List”
Sent: Friday, September 27, 2002 4:21 PM
Subject: [ntdev] Re: Looking for a WDM DMA Sample
> Does this mean that you can write a WDM driver for an ISA board?
>
> I have been told that WDM is only for PNP PCI devices. (not that I
> believe them but I have no other sources that i actually physically
> socialize with.)
>
> Asher
>
> >Peter Viscarola of OSR gives a very good class in Windows 2000
Device
> >Drivers. They offer a lab where you have a computer with a board
that
> >presents itself on both the ISA and PCI bus. The PCI side uses DMA.
I
> >remember, but haven’t needed to do it myself, that there are
specific
> >calls using map registers to handle DMA. It just works. It helps if
the
> device follows the PCI specs.
>
> —
> You are currently subscribed to ntdev as: xxxxx@storagecraft.com
> To unsubscribe send a blank email to %%email.unsub%%
>
There’s one more thing worth mentioning here. NT (all versions) runs on
machines which may do address transformations as cycles pass from the
memory bus onto the I/O busses and back again. Most PC-style machines
don’t do this. But your driver may end up running on a machine that
does. This means that, even if the memory is below 4GB, or even if your
adapter and your bus can generate 64-bit addresses, the
processor-relative physical address of the buffer (which is what you get
from MmGetPhysicalAddress) may not be the same as the I/O-bus-relative
physical address of the same buffer. The HAL and the bus drivers know
how to account for this when you call the DMA APIs. The address you get
back from them will take this into account. The address that you get
back from MmGetPhysicalAddress won’t.
Please use the DMA APIs.
Jake Oshins
Windows Kernel Group
This posting is provided “AS IS” with no warranties, and confers no
rights.
-----Original Message-----
Subject: Re: Looking for a WDM DMA Sample
From: “Peter Viscarola”
Date: Thu, 26 Sep 2002 18:25:23 -0400
X-Message-Number: 42
“John Reilly” wrote in message
news:xxxxx@ntdev…
>
> I have a busmaster radar video device. Each time the device gives me
a
> “I’ve got data” interrupt, I need to build a chain of DMA descriptors
> telling it from whence and to where I want the data. I allocated the
> descriptors from non-paged pool. I need to get the DMA chain’s
address in
> such a form that, when I put that 32-bit address into a register on my
> board, it can read the chain from memory and execute it.
>
> What I’m doing now is trying to calculate the PHYSICAL_ADDRESS of the
> chain. Maybe this works; I’m not sure. What I do know is that I’m
> stepping all over memory and bugchecking W2k. What fun!
>
Just to make sure I follow what you’re asking: You have an DMA device
(like
that supported by the PLX 905x), and you want to support scatter/gather.
The s/g descriptors are built in “shared memory” (that is, host memory
that
is shared between the device (doing continuous DMA) and your driver).
Is
that right?
Assuming this is the case, you need to allocate the memory where you’re
building your descriptors by calling AllocateCommonBuffer. This’ll
return
you a hunk of memory that is both virtually and logically (from the DMA
viewpoint) contiguous. It’ll also return you the base “Physical
Address”
(in quotes because this is really a device bus logical address… but,
yes,
it’s an address that you can give to your DMA controller), and a kernel
virtual address (that’ll enable your driver to talk to it).
You can then build the actual SG list with BuildScatterGatherList (qv,
providing this function a pointer to where you want the system to put
the SG
list in your shared memory area). Alternatively, you call
GetScatterGatherList and copy the SG list that’s pass to you to your
shared
memory area.
Does that help any?
Peter
OSR
XXXXXXXXXXXXXXXXXXXXXXXXX said:
So, let me follow up with a question about scatter/gather. I’d really
like to use it. But the device I’m using (PLX 9080-based), can’t
transfer more than 1024 bytes at a time. I need to grab a full radar
sector of 2000 bytes at each interrupt.
What do you mean a PLX9080 can only transfer 1024 at a time? It most
certainly can transfer more then that. The count field is big enough
to transfer several meg from a single cell. You can easily transfer
a page of 4096 bytes in one go.
Or is there some other limitation?
–
Steve Williams “The woods are lovely, dark and deep.
steve at icarus.com But I have promises to keep,
steve at picturel.com and lines to code before I sleep,
http://www.picturel.com And lines to code before I sleep.”
Steve,
Perhaps I should have said that the fellow who designed our board told me
that our board is limited to a max DMA transfer block of 1024. I know the
board is PLX 9080-based, and I know that he told me of the limitation. I
can’t tell you why, because I don’t know.
john.
Peter,
The Virtual to Physical mapping exists inside the processor, independently
of the OS. What goes to the bus, barred some black magic done inside the
North Bridge - and I’m not sure whether that’s possible - should be the same
thing that the Processor itself sees, no ?
Somehow I feel there’s no replacement for the ability to allocate memory
that’s physically contiguous - just virtual or logical contiguity won’t do.
My initial reaction is, get the OS out of the way of the hardware, and you
can do that. But the OS can be intrusive enough to make it impossible, or at
least, a royal pain.
Or am I wrong ?
Alberto.
-----Original Message-----
From: Peter Viscarola [mailto:xxxxx@osr.com]
Sent: Thursday, September 26, 2002 6:33 PM
To: NT Developers Interest List
Subject: [ntdev] Re: Looking for a WDM DMA Sample
“Bi Chen” wrote in message news:xxxxx@ntdev…
> Hi Peter:
>
>
> Also, I don’t understand the 64-bit issue. We are not talking about 64-bit
> OS. I know even 32-bit PCI bus device can be addressed in 64-bit through
> dual address cycle. But the system memory cannot be addressed in 64-bit. I
> think John wants to use system memory to store his scatter-gather-list
> descriptor, not the hardware MMIO memory. If he writes the 32-bit system
> physical address to the device register, the device will generate the
> address which will be passed by PCI-to-host bridge to system memory. If he
> uses hardware MMIO memory, he will use (HalTranslateBusAddress, NT 4.0)
> translated address and MmMapIoSpace on the memory so the driver can write
> descriptors directly to the hardware memory.
>
Bi… Suppose there’s more than 4GB of physical memory installed on the
system? Physical addresses can be 36 bits long on Pentium Pro and later,
through the use of PAE.
In short, everybody should listen to Peter Wieland: Do NOT attempt to
substitute a physical address for a Device Bug Logical Address (provided to
you via the bus driver/HAL). In other words, you MUST NOT use
MmGetPhysicalAddress(…) to get the address of host memory that you want to
use for DMA. I don’t know how many times I’ve said this in The NT Insider
or in class. In this operating system Main Memory Physical Address IS NOT
necessarily EQUAL TO the Device Bus Logical Address which is necessary
address for DMA. This is NOT just some esoteric architectural issue about
correctness.
Oh, I’m ranting again…
Peter
OSR
—
You are currently subscribed to ntdev as: xxxxx@compuware.com
To unsubscribe send a blank email to %%email.unsub%%
The contents of this e-mail are intended for the named addressee only. It
contains information that may be confidential. Unless you are the named
addressee or an authorized designee, you may not copy or use it, or disclose
it to anyone else. If you received it in error please notify us immediately
and then destroy it.
Futoshi-san,
As far as I know, you can use SoftICE 2.7 in Japan. We will have one of our
support guys to contact you and he will tell you how to proceed. Please send
me an email if you need more help ! Sorry I didn’t contact you earlier, I
was traveling.
Alberto Moreira.
xxxxx@compuware.com
-----Original Message-----
From: xxxxx@citrix.co.jp [mailto:xxxxx@citrix.co.jp]
Sent: Tuesday, October 01, 2002 3:35 AM
To: NT Developers Interest List
Subject: [ntdev] RE: [SoftICE2.6 generated BSOD when rebooting server]
Thanks Tomer,
I try to do as you mentioned from now.
By the way, this is the first time for me to use SoftICE, actually I am
looking forward using SoftICE because I have been using windbg so far.
From What I have heard, SoftICE is powerful debugger, though, How was your
feeling for SoftICE?
If you have favorite feature for SoftICE, Could you tell me abut it?
Thanks,
Futoshi
-----Original Message-----
From: Tomer Goldberg [mailto:xxxxx@royatech.com]
Sent: Tuesday, October 01, 2002 5:12 PM
To: NT Developers Interest List
Subject: [ntdev] RE: [SoftICE2.6 generated BSOD when rebooting server]
well there is no hotfix
that I know about and I’m
still using 2.6.
you can shutdown the server instead of rebooting it
-----Original Message-----
From: xxxxx@citrix.co.jp [mailto:xxxxx@citrix.co.jp]
Sent: Tuesday, October 01, 2002 8:59 AM
To: NT Developers Interest List
Subject: [ntdev] RE: [SoftICE2.6 generated BSOD when
rebooting server ]Thanks Tomer,
In Japan, 2.7 has not released yet.
Is there Hotfix for this issue? I have no oppotunity to get
information
because I have not completed registration as user yet.
If you know, could you tell me about it?Thanks,
Futoshi-----Original Message-----
From: Tomer Goldberg [mailto:xxxxx@royatech.com]
Sent: Tuesday, October 01, 2002 4:46 PM
To: NT Developers Interest List
Subject: [ntdev] RE: [SoftICE2.6 generated BSOD when
rebooting server ]Its probably the APIC bug
this version has
I read it will be fixed in 2.7
which is already released> -----Original Message-----
> From: xxxxx@CITRIX.CO.JP [mailto:xxxxx@CITRIX.CO.JP]
> Sent: Monday, September 30, 2002 2:28 PM
> To: NT Developers Interest List
> Subject: [ntdev] [SoftICE2.6 generated BSOD when rebooting server]
>
>
> Hi all,
>
> After I installed SoftICE contains DriverStudio2.6, I have
> BSOD when I am
> about rebooting server.
> I got memory.dmp, however I don’t understand the reason the
> BSOD occured.
> and also I install it, so symbol mismatch occured in windbg
> when I read it.
>
> If anyone have the same experience, Could you tell me about
> the reason?
>
> Thanks in advance,
> Futoshi
> -----
> Microsoft (R) Windows Debugger Version 6.0.0017.0
> Copyright (c) Microsoft Corporation. All rights reserved.
>
>
> Loading Dump File [C:\WINNT\DriverStudio26JMEMORY.DMP]
> Kernel Dump File: Full address space is available
>
> Symbol search path is:
> srv*\debug8j\symsrv*http://msdl.microsoft.com/download/symbols
> Executable search path is:
> *** ERROR: Symbol file could not be found. Defaulted to
> export symbols for
> ntoskrnl.exe -
> Windows 2000 Kernel Version 2195 (Service Pack 2) UP Free x86
> compatible
> Product: Server, suite: TerminalServer
> Kernel base = 0x80400000 PsLoadedModuleList = 0x8046ccf0
> Debug session time: Mon Sep 30 15:43:05 2002
> System Uptime: 0 days 4:19:27.000
> *** ERROR: Symbol file could not be found. Defaulted to
> export symbols for
> ntoskrnl.exe -
> Loading Kernel Symbols
> …
> …
> …
> Loading unloaded module list
> …
> Loading User Symbols
> …Unable to translate address 76930000 with prototype PTE
> .Unable to translate address 76910000 with prototype PTE
> …Unable to translate address 67500000 with prototype PTE
> …Unable to translate address 3fc30000 with prototype PTE
> …Unable to translate address 3fe20000 with prototype PTE
> .Unable to translate address 778f0000 with prototype PTE
> …Unable to translate address 768c0000 with prototype PTE
> …Unable to translate address 76990000 with prototype PTE
> .Unable to translate address 3fe00000 with prototype PTE
> .Unable to translate address 3fe50000 with prototype PTE
> …Unable to translate address 768b0000 with prototype PTE
> .Unable to translate address 768f0000 with prototype PTE
> .Unable to translate address 00eb0000 with prototype PTE
> .Unable to translate address 777d0000 with prototype PTE
> .Unable to translate address 75940000 with prototype PTE
> …Unable to translate address 782d0000 with prototype PTE
> .Unable to translate address 63830000 with prototype PTE
> .Unable to translate address 648c0000 with prototype PTE
> …Unable to translate address 66e80000 with prototype PTE
> …
> **************************************************************
> **************
> ***
> *
> *
> * Bugcheck Analysis
> *
> *
> *
> **************************************************************
> **************
> ***
>
> Use !analyze -v to get detailed debugging information.
>
> BugCheck A, {20, ff, 1, 800690a6}
>
> ***** Kernel symbols are WRONG. Please fix symbols to do analysis.
>
> *** ERROR: Symbol file could not be found. Defaulted to
> export symbols for
> halaacpi.dll -
> Probably caused by : ntoskrnl.exe ( nt!Kei386EoiHelper+2ac9 )
>
> Followup: MachineOwner
> ---------
>
> kd> kv
> ChildEBP RetAddr Args to Child
> WARNING: Stack unwind information not available. Following
> frames may be
> wrong.
> f055fa68 f055fa90 00000001 ff281101 bffc4028
nt!Kei386EoiHelper+0x2ac9
> 8140a8f4 ffffffff b235fa02 fc58a800 00185ab2 0xf055fa90
> 8140a8f4 ffffffff b235fa02 fc58a800 00185ab2 0xffffffff
> 00000020 00000000 00000000 00000000 00000000 0xffffffff
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@royatech.com
> To unsubscribe send a blank email to %%email.unsub%%
>
You are currently subscribed to ntdev as: xxxxx@citrix.co.jp
To unsubscribe send a blank email to %%email.unsub%%
You are currently subscribed to ntdev as: xxxxx@royatech.com
To unsubscribe send a blank email to %%email.unsub%%
You are currently subscribed to ntdev as: xxxxx@citrix.co.jp
To unsubscribe send a blank email to %%email.unsub%%
You are currently subscribed to ntdev as: xxxxx@compuware.com
To unsubscribe send a blank email to %%email.unsub%%
The contents of this e-mail are intended for the named addressee only. It
contains information that may be confidential. Unless you are the named
addressee or an authorized designee, you may not copy or use it, or disclose
it to anyone else. If you received it in error please notify us immediately
and then destroy it.
There are platforms where the logical address space seen by IO
controllers is in some way mapped onto the physical address space of the
system. IO controllers and the processors do not see the same virtual
address space.
“get[ing] the OS out of the way of the hardware” here would probably
involve putting lots of very platform specific code into your driver.
Perhaps you do that already in your drivers - I generally prefer to
avoid that sort of thing as much as possible.
allocating the appropriately memory (in terms of contiguity, alignment,
and boundaries) has always seemed pretty easy to me - call
[Hal]AllocateCommonBuffer and you get a nice chunk. The problem with
just allocating physically contiguous memory is that it, in and of
itself, is also not always enough.
-p
-----Original Message-----
From: Moreira, Alberto [mailto:xxxxx@compuware.com]
Sent: Tuesday, October 01, 2002 7:37 AM
To: NT Developers Interest List
Subject: [ntdev] Re: Looking for a WDM DMA Sample
Peter,
The Virtual to Physical mapping exists inside the processor,
independently of the OS. What goes to the bus, barred some black magic
done inside the North Bridge - and I’m not sure whether that’s possible
- should be the same thing that the Processor itself sees, no ?
Somehow I feel there’s no replacement for the ability to allocate memory
that’s physically contiguous - just virtual or logical contiguity won’t
do. My initial reaction is, get the OS out of the way of the hardware,
and you can do that. But the OS can be intrusive enough to make it
impossible, or at least, a royal pain.
Or am I wrong ?
Alberto.
-----Original Message-----
From: Peter Viscarola [mailto:xxxxx@osr.com]
Sent: Thursday, September 26, 2002 6:33 PM
To: NT Developers Interest List
Subject: [ntdev] Re: Looking for a WDM DMA Sample
“Bi Chen” wrote in message news:xxxxx@ntdev…
> Hi Peter:
>
>
> Also, I don’t understand the 64-bit issue. We are not talking about
> 64-bit OS. I know even 32-bit PCI bus device can be addressed in
> 64-bit through dual address cycle. But the system memory cannot be
> addressed in 64-bit. I think John wants to use system memory to store
> his scatter-gather-list descriptor, not the hardware MMIO memory. If
> he writes the 32-bit system physical address to the device register,
> the device will generate the address which will be passed by
> PCI-to-host bridge to system memory. If he uses hardware MMIO memory,
> he will use (HalTranslateBusAddress, NT 4.0) translated address and
> MmMapIoSpace on the memory so the driver can write descriptors
> directly to the hardware memory.
>
Bi… Suppose there’s more than 4GB of physical memory installed on the
system? Physical addresses can be 36 bits long on Pentium Pro and
later, through the use of PAE.
In short, everybody should listen to Peter Wieland: Do NOT attempt to
substitute a physical address for a Device Bug Logical Address (provided
to you via the bus driver/HAL). In other words, you MUST NOT use
MmGetPhysicalAddress(…) to get the address of host memory that you
want to use for DMA. I don’t know how many times I’ve said this in The
NT Insider or in class. In this operating system Main Memory Physical
Address IS NOT necessarily EQUAL TO the Device Bus Logical Address which
is necessary address for DMA. This is NOT just some esoteric
architectural issue about correctness.
Oh, I’m ranting again…
Peter
OSR
—
You are currently subscribed to ntdev as: xxxxx@compuware.com
To unsubscribe send a blank email to %%email.unsub%%
The contents of this e-mail are intended for the named addressee only.
It contains information that may be confidential. Unless you are the
named addressee or an authorized designee, you may not copy or use it,
or disclose it to anyone else. If you received it in error please notify
us immediately and then destroy it.
—
You are currently subscribed to ntdev as: xxxxx@microsoft.com To
unsubscribe send a blank email to %%email.unsub%%
It depends on what one wants, no ? If I have no intention of running my code
outside an Intel platform, this consideration isn’t very relevant. If I will
be targeting outside the PC space, I’ll have to take care of it.
And what if I want to write a piece of OS-independent Pentium code ?
Meaning, a hardware level handler that works, say, on Windows and on Linux
with minimal or no modification ?
I believe my point is, this is a decision that should be taken by the
developer at the light of his product requirements, and a sensible operating
system API would allow both ways. I believe Windows could do with a slab
allocator !
Alberto.
-----Original Message-----
From: Peter Wieland [mailto:xxxxx@windows.microsoft.com]
Sent: Tuesday, October 01, 2002 2:13 PM
To: NT Developers Interest List
Subject: [ntdev] Re: Looking for a WDM DMA Sample
There are platforms where the logical address space seen by IO
controllers is in some way mapped onto the physical address space of the
system. IO controllers and the processors do not see the same virtual
address space.
“get[ing] the OS out of the way of the hardware” here would probably
involve putting lots of very platform specific code into your driver.
Perhaps you do that already in your drivers - I generally prefer to
avoid that sort of thing as much as possible.
allocating the appropriately memory (in terms of contiguity, alignment,
and boundaries) has always seemed pretty easy to me - call
[Hal]AllocateCommonBuffer and you get a nice chunk. The problem with
just allocating physically contiguous memory is that it, in and of
itself, is also not always enough.
-p
-----Original Message-----
From: Moreira, Alberto [mailto:xxxxx@compuware.com]
Sent: Tuesday, October 01, 2002 7:37 AM
To: NT Developers Interest List
Subject: [ntdev] Re: Looking for a WDM DMA Sample
Peter,
The Virtual to Physical mapping exists inside the processor,
independently of the OS. What goes to the bus, barred some black magic
done inside the North Bridge - and I’m not sure whether that’s possible
- should be the same thing that the Processor itself sees, no ?
Somehow I feel there’s no replacement for the ability to allocate memory
that’s physically contiguous - just virtual or logical contiguity won’t
do. My initial reaction is, get the OS out of the way of the hardware,
and you can do that. But the OS can be intrusive enough to make it
impossible, or at least, a royal pain.
Or am I wrong ?
Alberto.
-----Original Message-----
From: Peter Viscarola [mailto:xxxxx@osr.com]
Sent: Thursday, September 26, 2002 6:33 PM
To: NT Developers Interest List
Subject: [ntdev] Re: Looking for a WDM DMA Sample
“Bi Chen” wrote in message news:xxxxx@ntdev…
> Hi Peter:
>
>
> Also, I don’t understand the 64-bit issue. We are not talking about
> 64-bit OS. I know even 32-bit PCI bus device can be addressed in
> 64-bit through dual address cycle. But the system memory cannot be
> addressed in 64-bit. I think John wants to use system memory to store
> his scatter-gather-list descriptor, not the hardware MMIO memory. If
> he writes the 32-bit system physical address to the device register,
> the device will generate the address which will be passed by
> PCI-to-host bridge to system memory. If he uses hardware MMIO memory,
> he will use (HalTranslateBusAddress, NT 4.0) translated address and
> MmMapIoSpace on the memory so the driver can write descriptors
> directly to the hardware memory.
>
Bi… Suppose there’s more than 4GB of physical memory installed on the
system? Physical addresses can be 36 bits long on Pentium Pro and
later, through the use of PAE.
In short, everybody should listen to Peter Wieland: Do NOT attempt to
substitute a physical address for a Device Bug Logical Address (provided
to you via the bus driver/HAL). In other words, you MUST NOT use
MmGetPhysicalAddress(…) to get the address of host memory that you
want to use for DMA. I don’t know how many times I’ve said this in The
NT Insider or in class. In this operating system Main Memory Physical
Address IS NOT necessarily EQUAL TO the Device Bus Logical Address which
is necessary address for DMA. This is NOT just some esoteric
architectural issue about correctness.
Oh, I’m ranting again…
Peter
OSR
—
You are currently subscribed to ntdev as: xxxxx@compuware.com
To unsubscribe send a blank email to %%email.unsub%%
The contents of this e-mail are intended for the named addressee only.
It contains information that may be confidential. Unless you are the
named addressee or an authorized designee, you may not copy or use it,
or disclose it to anyone else. If you received it in error please notify
us immediately and then destroy it.
—
You are currently subscribed to ntdev as: xxxxx@microsoft.com To
unsubscribe send a blank email to %%email.unsub%%
—
You are currently subscribed to ntdev as: xxxxx@compuware.com
To unsubscribe send a blank email to %%email.unsub%%
The contents of this e-mail are intended for the named addressee only. It
contains information that may be confidential. Unless you are the named
addressee or an authorized designee, you may not copy or use it, or disclose
it to anyone else. If you received it in error please notify us immediately
and then destroy it.
“Moreira, Alberto” wrote in message
news:xxxxx@ntdev…
>
> Peter,
>
> The Virtual to Physical mapping exists inside the processor, independently
> of the OS. What goes to the bus, barred some black magic done inside the
> North Bridge - and I’m not sure whether that’s possible - should be the
same
> thing that the Processor itself sees, no ?
>
> Somehow I feel there’s no replacement for the ability to allocate memory
> that’s physically contiguous - just virtual or logical contiguity won’t
do.
> My initial reaction is, get the OS out of the way of the hardware, and you
> can do that. But the OS can be intrusive enough to make it impossible, or
at
> least, a royal pain.
>
> Or am I wrong ?
>
Yeah, sorry, but you’re wrong. Your mistake is that you are (narturally)
assuming a design with a North Bridge. There are lots of ways to connect
the CPU, Main Memory, and (one or more) device bus(es). Consider MIPS R3000
systems, certain Alphas, and even some VAXen. And while these aren’t X86
systems, there is certainly nothing that constrains how an x86 architecture
CPU connects the main memory and the systems device buses.
Always remember: Physical memory address is not necessarily equal to the
address used for DMA to that physical memory from a device on a given device
bus. The address used on the device bus is a function of how the device bus
is connected to the main memory. Hence, mapping registers.
See figure 8.7, on page 164 of that incomparable tome, Windows NT Device
Driver Development 
Peter
OSR
The physical address comes out on the system bus, eh ? And some kind of
controller separates the system bus from the peripheral bus. Call it “North
Bridge” or whatever, the issue is more or less the same. In fact, one can
make a point that whatever the pre-translated view of the addressing space
is, at I/O level what matters is the physical address, meaning, what comes
out of the processor address pins. That’s what taken to the memory, that’s
what’s taken to the peripheral bus, modulo those magic bridges, or call them
memory controllers or peripheral controllers if you will. Boils down to the
same issue: do I the kernel developer have a handle on the way the hardware
does things, or must I always do it only in the way the OS lets me ? Somehow
I can’t accept this last proposition.
Also, one of the tenets of the i386 architecture has been maintaining
compatilibity at the hardware level, through a well defined set of hardware
interfaces. I would assume, maybe in an utopian way, that an OS would not
prevent a developer from coding to that hardware standard - again, I may
want a piece of code that runs on one platform but under multiple operating
systems, or without any operating system for that matter ! The issue is, do
we want compatibility to be always at OS level ? I believe we should allow
for hardware-level compatibility too, and make sure that the developer has
access to a way of removing the OS from sight if that’s what’s needed.
Alberto.
-----Original Message-----
From: Peter Viscarola [mailto:xxxxx@osr.com]
Sent: Tuesday, October 01, 2002 5:32 PM
To: NT Developers Interest List
Subject: [ntdev] Re: Looking for a WDM DMA Sample
“Moreira, Alberto” wrote in message
news:xxxxx@ntdev…
>
> Peter,
>
> The Virtual to Physical mapping exists inside the processor, independently
> of the OS. What goes to the bus, barred some black magic done inside the
> North Bridge - and I’m not sure whether that’s possible - should be the
same
> thing that the Processor itself sees, no ?
>
> Somehow I feel there’s no replacement for the ability to allocate memory
> that’s physically contiguous - just virtual or logical contiguity won’t
do.
> My initial reaction is, get the OS out of the way of the hardware, and you
> can do that. But the OS can be intrusive enough to make it impossible, or
at
> least, a royal pain.
>
> Or am I wrong ?
>
Yeah, sorry, but you’re wrong. Your mistake is that you are (narturally)
assuming a design with a North Bridge. There are lots of ways to connect
the CPU, Main Memory, and (one or more) device bus(es). Consider MIPS R3000
systems, certain Alphas, and even some VAXen. And while these aren’t X86
systems, there is certainly nothing that constrains how an x86 architecture
CPU connects the main memory and the systems device buses.
Always remember: Physical memory address is not necessarily equal to the
address used for DMA to that physical memory from a device on a given device
bus. The address used on the device bus is a function of how the device bus
is connected to the main memory. Hence, mapping registers.
See figure 8.7, on page 164 of that incomparable tome, Windows NT Device
Driver Development
Peter
OSR
—
You are currently subscribed to ntdev as: xxxxx@compuware.com
To unsubscribe send a blank email to %%email.unsub%%
The contents of this e-mail are intended for the named addressee only. It
contains information that may be confidential. Unless you are the named
addressee or an authorized designee, you may not copy or use it, or disclose
it to anyone else. If you received it in error please notify us immediately
and then destroy it.
“Moreira, Alberto” wrote in message
news:xxxxx@ntdev…
> The issue is, do
> we want compatibility to be always at OS level ? I believe we should allow
> for hardware-level compatibility too, and make sure that the developer has
> access to a way of removing the OS from sight if that’s what’s needed.
Not in a production OS.
–
Philip D. Barila
Seagate Technology, LLC
(720) 684-1842
> The physical address comes out on the system bus, eh ? And some kind
of
controller separates the system bus from the peripheral bus. Call itBoils down to
the
same issue: do I the kernel developer have a handle on the way the
hardware
does things, or must I always do it only in the way the OS lets me ?
You can have a handle on the way hardware does things, but in general you
must do things in the kernel ‘only the way the os lets me’ if you want
your software to work well with other software and on all supported
platforms. You have to work on your ‘works and plays well with others’
skills Alberto
Somehow
I can’t accept this last proposition.
Somehow that is not a shock.
Also, one of the tenets of the i386 architecture has been maintaining
compatilibity at the hardware level, through a well defined set of
hardware
interfaces. I would assume, maybe in an utopian way, that an OS would
not
prevent a developer from coding to that hardware standard -
That may be a ‘tenet’ of the i386 architecture, but the i386
architecture, for example, does not support 36bit memory address while
the later x86 processors do. On a 386, dma from pci device to physical
memory did not have any boundary issues to deal with, on later x86
processors pci devices may indeed have address boundary issues to deal
with. NT provides an architected facility to deal with memory boundary
issues, in a platform independent manner, in a way that is safe for all
drivers doing DMA and in a way that is reasonably efficient.
again, I
may
want a piece of code that runs on one platform but under multiple
operating
systems, or without any operating system for that matter !
People have been designing portability layers into drivers for a long
time, and maintaining full compatability with the device driver
architectures for the targeted operating systems.
Hi, Guys.
I just utter something to see if my understanding on the issue is correct.
I’ll appreciate if something I post here is wrong.
Peter and others are right that in NT/2k/XP one should use HAL/Bus driver
supported DMA functions. NT family OSes are sort of microkernel architecture
and its microkerenl is HAL (maybe with some other part). The microkerenl has
the knowledge of the CPU architecture and northbridge, bus etc and present
an platform architecture independent view to the rest of the kernel.
Nevertheless, I don’t think NT family OSes are pure microkerenl OSes since
its VMM must know CPU MMU’s, which is not in the HAL. I could be wrong since
HAL API I all is from DDK.
Conceivably, some northbridge or its equivalent could add a displacement to
the physical address when the generate address on the (front) system bus are
destination to peripheral bus and subtract the displacement when the R/W
address issued by a device. I have not seen anything like that and wonder
why if anyone wants to do that.
Northbridge is essentially a crossbar router that routes R/W request based
on the address generated by CPU or device. For example, if the address fall
into the window of the X-bus, the r/w request is routed to southbridge. If
the address fall into the windows of system memory, rom/flash, AGP, it will
be routed correspondingly.
In absence of bus bridge that wants to translate an address from upstream or
downstream as aforementioned, the entire system physical address is
essentially flat. The system address can be used by the device if it is a
positive decoding bus device such as PCI device (positive decoding bus
device means that if the address falling into a device window, it will claim
the transaction).
Serial bus is not included in the discussion since the host hub/bridge will
do the the job of converting framed message to proper address destination to
northbridge.
I don’t think the issue has anything to do with CPUs core where MMU resides.
Has anyone seen a real life example that a physical address that is from
MmGetPhysicalAddress of a contiguous system memory different from return
value of the MapTransfer (IoMapTransfer on NT 4.0) of the same memory (I bet
someone did)? On what kind of bus (or what platform?)
Thanks
Bi
-----Original Message-----
From: Moreira, Alberto [mailto:xxxxx@compuware.com]
Sent: Tuesday, October 01, 2002 3:04 PM
To: NT Developers Interest List
Subject: [ntdev] Re: Looking for a WDM DMA Sample
The physical address comes out on the system bus, eh ? And some kind of
controller separates the system bus from the peripheral bus. Call it “North
Bridge” or whatever, the issue is more or less the same. In fact, one can
make a point that whatever the pre-translated view of the addressing space
is, at I/O level what matters is the physical address, meaning, what comes
out of the processor address pins. That’s what taken to the memory, that’s
what’s taken to the peripheral bus, modulo those magic bridges, or call them
memory controllers or peripheral controllers if you will. Boils down to the
same issue: do I the kernel developer have a handle on the way the hardware
does things, or must I always do it only in the way the OS lets me ? Somehow
I can’t accept this last proposition.
Also, one of the tenets of the i386 architecture has been maintaining
compatilibity at the hardware level, through a well defined set of hardware
interfaces. I would assume, maybe in an utopian way, that an OS would not
prevent a developer from coding to that hardware standard - again, I may
want a piece of code that runs on one platform but under multiple operating
systems, or without any operating system for that matter ! The issue is, do
we want compatibility to be always at OS level ? I believe we should allow
for hardware-level compatibility too, and make sure that the developer has
access to a way of removing the OS from sight if that’s what’s needed.
Alberto.
-----Original Message-----
From: Peter Viscarola [mailto:xxxxx@osr.com]
Sent: Tuesday, October 01, 2002 5:32 PM
To: NT Developers Interest List
Subject: [ntdev] Re: Looking for a WDM DMA Sample
“Moreira, Alberto” wrote in message
news:xxxxx@ntdev…
>
> Peter,
>
> The Virtual to Physical mapping exists inside the processor, independently
> of the OS. What goes to the bus, barred some black magic done inside the
> North Bridge - and I’m not sure whether that’s possible - should be the
same
> thing that the Processor itself sees, no ?
>
> Somehow I feel there’s no replacement for the ability to allocate memory
> that’s physically contiguous - just virtual or logical contiguity won’t
do.
> My initial reaction is, get the OS out of the way of the hardware, and you
> can do that. But the OS can be intrusive enough to make it impossible, or
at
> least, a royal pain.
>
> Or am I wrong ?
>
Yeah, sorry, but you’re wrong. Your mistake is that you are (narturally)
assuming a design with a North Bridge. There are lots of ways to connect
the CPU, Main Memory, and (one or more) device bus(es). Consider MIPS R3000
systems, certain Alphas, and even some VAXen. And while these aren’t X86
systems, there is certainly nothing that constrains how an x86 architecture
CPU connects the main memory and the systems device buses.
Always remember: Physical memory address is not necessarily equal to the
address used for DMA to that physical memory from a device on a given device
bus. The address used on the device bus is a function of how the device bus
is connected to the main memory. Hence, mapping registers.
See figure 8.7, on page 164 of that incomparable tome, Windows NT Device
Driver Development
Peter
OSR
—
You are currently subscribed to ntdev as: xxxxx@compuware.com
To unsubscribe send a blank email to %%email.unsub%%
The contents of this e-mail are intended for the named addressee only. It
contains information that may be confidential. Unless you are the named
addressee or an authorized designee, you may not copy or use it, or disclose
it to anyone else. If you received it in error please notify us immediately
and then destroy it.
—
You are currently subscribed to ntdev as: xxxxx@appstream.com
To unsubscribe send a blank email to %%email.unsub%%
Why not ? Hardware level code isn’t necessarily more unstable than any other
code. And I trust my hand better.
Alberto.
-----Original Message-----
From: Phil Barila [mailto:xxxxx@Seagate.com]
Sent: Tuesday, October 01, 2002 6:59 PM
To: NT Developers Interest List
Subject: [ntdev] Re: Looking for a WDM DMA Sample
“Moreira, Alberto” wrote in message
news:xxxxx@ntdev…
> The issue is, do
> we want compatibility to be always at OS level ? I believe we should allow
> for hardware-level compatibility too, and make sure that the developer has
> access to a way of removing the OS from sight if that’s what’s needed.
Not in a production OS.
–
Philip D. Barila
Seagate Technology, LLC
(720) 684-1842
—
You are currently subscribed to ntdev as: xxxxx@compuware.com
To unsubscribe send a blank email to %%email.unsub%%
The contents of this e-mail are intended for the named addressee only. It
contains information that may be confidential. Unless you are the named
addressee or an authorized designee, you may not copy or use it, or disclose
it to anyone else. If you received it in error please notify us immediately
and then destroy it.
A good kernel should run without making any assumptions on the underlying
hardware layer, and it should work just as well if the interface to the
hardware is virtualized. There should be ways of operating independently of
the OS, in cases we either don’t need it or we want to achieve something in
spite of it. I can see many reasons why I should want to write a kernel
module that bridges directly from an app to an OS - direct rendering, for
example, is such a case. A well designed operating system will let me write
kernel modules that expose my hardware to my app bypassing its own
mechanisms.
The i386 architecture today supports 36-bit memory addressing. The fact that
36-bit addressing is implemented as blocks of 32-bit address spaces does
insert some considerations that older hardware didn’t have, and that
requires hardware-level software developers to take it into account - but
that doesn’t mean we necessarily want to punt to the OS when we handle it.
Hardware has been evolving ever since I remember, and hardware-level
programmers had to keep up with it - but again, I find it a terrible idea
that every time the going gets tough I must punt to the OS. Sorry, no, if an
OS does not make strong provision for hardware handling that doesn’t
necessarily comply with its internal peeves, I can’t say I like it.
As for a nice driver subsystem design, look at Linux. It’s at least as
stable as NT, and people don’t have to go through nearly as many hoops to
get things done.
Alberto.
-----Original Message-----
From: Mark Roddy [mailto:xxxxx@hollistech.com]
Sent: Tuesday, October 01, 2002 9:38 PM
To: NT Developers Interest List
Subject: [ntdev] Re: Looking for a WDM DMA Sample
You can have a handle on the way hardware does things, but in general you
must do things in the kernel ‘only the way the os lets me’ if you want
your software to work well with other software and on all supported
platforms. You have to work on your ‘works and plays well with others’
skills Alberto
Somehow
I can’t accept this last proposition.
Somehow that is not a shock.
Also, one of the tenets of the i386 architecture has been maintaining
compatilibity at the hardware level, through a well defined set of
hardware
interfaces. I would assume, maybe in an utopian way, that an OS would
not
prevent a developer from coding to that hardware standard -
That may be a ‘tenet’ of the i386 architecture, but the i386
architecture, for example, does not support 36bit memory address while
the later x86 processors do. On a 386, dma from pci device to physical
memory did not have any boundary issues to deal with, on later x86
processors pci devices may indeed have address boundary issues to deal
with. NT provides an architected facility to deal with memory boundary
issues, in a platform independent manner, in a way that is safe for all
drivers doing DMA and in a way that is reasonably efficient.
again, I
may
want a piece of code that runs on one platform but under multiple
operating
systems, or without any operating system for that matter !
People have been designing portability layers into drivers for a long
time, and maintaining full compatability with the device driver
architectures for the targeted operating systems.
You are currently subscribed to ntdev as: xxxxx@compuware.com
To unsubscribe send a blank email to %%email.unsub%%
The contents of this e-mail are intended for the named addressee only. It
contains information that may be confidential. Unless you are the named
addressee or an authorized designee, you may not copy or use it, or disclose
it to anyone else. If you received it in error please notify us immediately
and then destroy it.
>
Has anyone seen a real life example that a physical address that is
from
MmGetPhysicalAddress of a contiguous system memory different from
return
value of the MapTransfer (IoMapTransfer on NT 4.0) of the same memory
(I bet
someone did)? On what kind of bus (or what platform?)
- ISA bus on any x86 platform. (ISA cannot address all of 32bit
physical memory.) - ALPHA platforms (although only by reputation, I’ve never used one.)
- I think that this happens with 32bit PCI devices on 36bit x86.
- ANY bus where the target device is not capable of scatter/gather
operations. (NT uses MapTransfer to do the copy for you.)
===========================
Mark Roddy
Consultant, Microsoft DDK MVP
Hollis Technology Solutions
xxxxx@hollistech.com
www.hollistech.com
603-321-1032
“Moreira, Alberto” wrote in message
news:xxxxx@ntdev…
>
> As for a nice driver subsystem design, look at Linux. It’s at least as
> stable as NT, and people don’t have to go through nearly as many hoops to
> get things done.
>
>
HA! Cuz Linux is so architecture independent? Give me a BREAK! Basically,
Linux forces ever driver to deal with the complexities of how the I/O bus
and memory bus happen to be wired-together on the specific platform on which
the system is running. I call that “jumping through hoops” even if you
don’t.
That was forward-thinking in the 1970’s… Fortunately, the industry has
learned one or two things since then,
Peter
OSR
But Peter, that’s what a device driver is supposed to do, no ? To handle the
hardware ? In fact, the hardware layer can be designed completely separate
from the kernel, they don’t even need to run at the same protection ring. It
is totally feasible to run the whole I/O subsystem in Ring 1, and much
better too, I believe.
Actually, I/O should be done on the other side of the bus - every peripheral
could be controlled by its own processor, and device drivers should live in
firmware at the I/O subsystem. This was the original intention when IBM
first did the PC: the Bios was intended to be a hardware layer that exposed
a low level, uniform view of the hardware, and that view was supported by
the boards themselves through making board-resident Bios code visible to the
system. The original intention was subverted by years of taking advantage of
dirty compatibility, and also by the industry’s failure to evolve the
mechanism into something amenable to multithreading: the ABios was a good
try, but it didn’t fly for political reasons.
Yet today I see no reason not to adopt that kind of approach: build a
hardware layer that lives underneath the OS and is provided by hardware
vendors, make sure that the OS kernel sees that layer through a well defined
and generic API, insulate the kernel from hardware considerations, and
basically move interrupts to peripheral subsystems where they belong: a
peripheral shouldn’t be able to interrupt but its own local processor, not
the CPUs themselves.
The way it is now, well, it’s a rat’s nest. Wasn’t it Dijkstra who said
something in the direction that the advent of PCs would throw us back into
stone age for many years to come ? Well, he was right.
Alberto.
-----Original Message-----
From: Peter Viscarola [mailto:xxxxx@osr.com]
Sent: Wednesday, October 02, 2002 11:13 AM
To: NT Developers Interest List
Subject: [ntdev] Re: Looking for a WDM DMA Sample
“Moreira, Alberto” wrote in message
news:xxxxx@ntdev…
>
> As for a nice driver subsystem design, look at Linux. It’s at least as
> stable as NT, and people don’t have to go through nearly as many hoops to
> get things done.
>
>
HA! Cuz Linux is so architecture independent? Give me a BREAK! Basically,
Linux forces ever driver to deal with the complexities of how the I/O bus
and memory bus happen to be wired-together on the specific platform on which
the system is running. I call that “jumping through hoops” even if you
don’t.
That was forward-thinking in the 1970’s… Fortunately, the industry has
learned one or two things since then,
Peter
OSR
—
You are currently subscribed to ntdev as: xxxxx@compuware.com
To unsubscribe send a blank email to %%email.unsub%%
The contents of this e-mail are intended for the named addressee only. It
contains information that may be confidential. Unless you are the named
addressee or an authorized designee, you may not copy or use it, or disclose
it to anyone else. If you received it in error please notify us immediately
and then destroy it.