RE: Re-post: Large buffers

OSR_Community_User · September 12, 2000, 12:58pm

> Subject: [ntdev] Large buffers

Is there a limit to the size buffer that can be described by
an MDL? If
so, where is it defined. 32MBs works; 64MBs does not, failing with a
‘no resources’ kind of error. We’re using an IOCTL with
METHOD_IN_DIRECT with plenty of RAM.

Bill Casey
== SCSI Adapters & VirtualSCSI Target Mode Libs ==

OSR_Community_User · September 12, 2000, 1:39pm

Bill,

Looking at the definition of an MDL I see that the MDL size is contained in
a 16 bit field. The contents of that field changes with the size of the
buffer you are describing since it accounts for the PFN array that begins at
the end of the MDL header. Doing some cogitating with my HP calculator I see
that minus the header of an MDL I can have a maximum of 16,377 PFNs. ((65536

size of MDL) / size of PFN). Now multiplying 16377 times 4096 tells me
that the maximum buffer size is 67,080,192, where as 64Meg actually computes
to 67,108,864. This can vary of course since a PFN is not limited to a
single page if the pages are contiguous.

Now the question is: Is this correct or is it really done with mirrors?

Gary

-----Original Message-----
From: Bill Casey [mailto:xxxxx@advstor.com]
Sent: Tuesday, September 12, 2000 9:54 AM
To: NT Developers Interest List
Subject: [ntdev] RE: Re-post: Large buffers

Subject: [ntdev] Large buffers
>
>
> Is there a limit to the size buffer that can be described
by
> an MDL? If
> so, where is it defined. 32MBs works; 64MBs does not,
failing with a
> ‘no resources’ kind of error. We’re using an IOCTL with
> METHOD_IN_DIRECT with plenty of RAM.
>
> Bill Casey
> == SCSI Adapters & VirtualSCSI Target Mode Libs ==

You are currently subscribed to ntdev as:
xxxxx@delphieng.com
To unsubscribe send a blank email to
$subst(‘Email.Unsub’)

OSR_Community_User · September 12, 2000, 1:48pm

The limit on an MDL, if I recall correctly is 64MB-sizeof(USHORT), or
something like that. Effectively it is a little less than 64MB. Actually,
once again if I recall correctly, it is possible to construct an MDL that
describes a much larger region of memory, it just isn’t legitimate.

-----Original Message-----
From: Bill Casey [mailto:xxxxx@advstor.com]
Sent: Tuesday, September 12, 2000 12:54 PM
To: NT Developers Interest List
Subject: [ntdev] RE: Re-post: Large buffers

> Subject: [ntdev] Large buffers
>
>
> Is there a limit to the size buffer that can be described by
> an MDL? If
> so, where is it defined. 32MBs works; 64MBs does not,
failing with a
> ‘no resources’ kind of error. We’re using an IOCTL with
> METHOD_IN_DIRECT with plenty of RAM.
>
> Bill Casey
> == SCSI Adapters & VirtualSCSI Target Mode Libs ==

You are currently subscribed to ntdev as: xxxxx@stratus.com
To unsubscribe send a blank email to $subst(‘Email.Unsub’)

OSR_Community_User · September 12, 2000, 2:14pm

Thanks Mark and Gary. My client and I have an incident going with MS
and I’ll post the ‘real deal’ when I get word.

Bill Casey
== SCSI Adapters & VirtualSCSI Target Mode Libs ==

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Roddy, Mark
Sent: Tuesday, September 12, 2000 12:42 PM
To: NT Developers Interest List
Subject: [ntdev] RE: Re-post: Large buffers

The limit on an MDL, if I recall correctly is 64MB-sizeof(USHORT), or
something like that. Effectively it is a little less than
64MB. Actually,
once again if I recall correctly, it is possible to construct
an MDL that
describes a much larger region of memory, it just isn’t legitimate.

> -----Original Message-----
> From: Bill Casey [mailto:xxxxx@advstor.com]
> Sent: Tuesday, September 12, 2000 12:54 PM
> To: NT Developers Interest List
> Subject: [ntdev] RE: Re-post: Large buffers
>
>
> > Subject: [ntdev] Large buffers
> >
> >
> > Is there a limit to the size buffer that can be described by
> > an MDL? If
> > so, where is it defined. 32MBs works; 64MBs does not,
> failing with a
> > ‘no resources’ kind of error. We’re using an IOCTL with
> > METHOD_IN_DIRECT with plenty of RAM.
> >
> > Bill Casey
> > == SCSI Adapters & VirtualSCSI Target Mode Libs ==
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@stratus.com
> To unsubscribe send a blank email to $subst(‘Email.Unsub’)
>

You are currently subscribed to ntdev as: xxxxx@advstor.com
To unsubscribe send a blank email to $subst(‘Email.Unsub’)

OSR_Community_User · September 12, 2000, 2:47pm

Gary:
No word from MS yet but I had a thought: since we are supposed to be
in a 32-bit world why did the designers choose a 16-bit number for the
size of an MDL? I’m sure there was some rationale. Anybody know what
that might be? Or did they just not plan for the future and large RAM
sizes well enough?

Bill Casey
== SCSI Adapters & VirtualSCSI Target Mode Libs ==

Bill,

Looking at the definition of an MDL I see that the MDL size
is contained in
a 16 bit field. The contents of that field changes with the
size of the
buffer you are describing since it accounts for the PFN array
that begins at
the end of the MDL header. Doing some cogitating with my HP
calculator I see
that minus the header of an MDL I can have a maximum of
16,377 PFNs. ((65536

size of MDL) / size of PFN). Now multiplying 16377 times
4096 tells me
that the maximum buffer size is 67,080,192, where as 64Meg
actually computes
to 67,108,864. This can vary of course since a PFN is not limited to a
single page if the pages are contiguous.

Now the question is: Is this correct or is it really done
with mirrors?

Gary

-----Original Message-----
From: Bill Casey [mailto:xxxxx@advstor.com]
Sent: Tuesday, September 12, 2000 9:54 AM
To: NT Developers Interest List
Subject: [ntdev] RE: Re-post: Large buffers

> Subject: [ntdev] Large buffers
>
>
> Is there a limit to the size buffer that can
be described
by
> an MDL? If
> so, where is it defined. 32MBs works; 64MBs does not,
failing with a
> ‘no resources’ kind of error. We’re using an
IOCTL with
> METHOD_IN_DIRECT with plenty of RAM.
>
> Bill Casey
> == SCSI Adapters & VirtualSCSI Target Mode Libs ==

You are currently subscribed to ntdev as:
xxxxx@delphieng.com
To unsubscribe send a blank email to
$subst(‘Email.Unsub’)

You are currently subscribed to ntdev as: xxxxx@advstor.com
To unsubscribe send a blank email to $subst(‘Email.Unsub’)

OSR_Community_User · September 12, 2000, 3:29pm

> … since we are supposed to begin a 32-bit world why did the designers
choose a 16-> bit number for the size of an MDL?

Other than pure speculation, I couldn’t say. Since the MDL header is fixed
length, why even include it in the size at all? Why not let that field
indicate the total number of PFNs in the array, beginning at *(&MDL +
sizeof(MDL)). But then the boys and girls at Microsoft never ask me about
design stuff.

It may be related to the maximum buffer size that can be used in a DMA
transfer which for SCSIPort, and many SCSI devices is still on the order of
16M (16,777,216). I know our device limits our scatter/gather list to 1024
entries, or hex 400.

Or it could be related to the page size. If your page size is 8k then 16,377
PFNs equates to over 134,169,384.

Most likely it’s defined that way because that’s what the engineer typed
when they entered that field definition for that structure, and no one gave
it a thought in 1988 that we’d be pushing a terabyte in 2000. Remember, most
machines were 16-bit, and many were still 8 bits, and memory was still
primarily measured in kilobytes. To use a Meg of memory you had to do back
flips and load those infamous “DOS Extenders”.

OSR_Community_User · September 12, 2000, 3:37pm

>

Most likely it’s defined that way because that’s what the
engineer typed
when they entered that field definition for that structure,
and no one gave
it a thought in 1988 that we’d be pushing a terabyte in 2000.
Remember, most
machines were 16-bit, and many were still 8 bits, and memory was still
primarily measured in kilobytes. To use a Meg of memory you
had to do back
flips and load those infamous “DOS Extenders”.

Except that the NT team was all from the VMS world and should have known
better.

I’m still fairly sure that you can build/unbuild an MDL by hand that is much
bigger than 64M. The size field is generally ignored.

OSR_Community_User · September 12, 2000, 3:49pm

I agree this is probably a historical remnant but since this is supposed
to be an opaque structure they should be able to just change it - right?
To match current machines and technology?

Bill
== SCSI Adapters & VirtualSCSI Target Mode Libs ==

Subject: [ntdev] RE: Re-post: Large buffers

> … since we are supposed to begin a 32-bit world why did
the designers choose a 16-> bit number for the size of an MDL?

Other than pure speculation, I couldn’t say. Since the MDL
header is fixed
length, why even include it in the size at all? Why not let
that field
indicate the total number of PFNs in the array, beginning at *(&MDL +
sizeof(MDL)). But then the boys and girls at Microsoft never
ask me about
design stuff.

It may be related to the maximum buffer size that can be used in a DMA
transfer which for SCSIPort, and many SCSI devices is still
on the order of
16M (16,777,216). I know our device limits our scatter/gather
list to 1024
entries, or hex 400.

Or it could be related to the page size. If your page size is
8k then 16,377
PFNs equates to over 134,169,384.

Most likely it’s defined that way because that’s what the
engineer typed
when they entered that field definition for that structure,
and no one gave
it a thought in 1988 that we’d be pushing a terabyte in 2000.
Remember, most
machines were 16-bit, and many were still 8 bits, and memory was still
primarily measured in kilobytes. To use a Meg of memory you
had to do back
flips and load those infamous “DOS Extenders”.

You are currently subscribed to ntdev as: xxxxx@advstor.com
To unsubscribe send a blank email to $subst(‘Email.Unsub’)

OSR_Community_User · September 12, 2000, 4:24pm

I’ve been reading my 386 book about LDT, GDT, and segment descriptors. Ok, I
know, the 386 is defunct, but that is the last hardware book I’ve had to
buy, and code written for it will run on today’s architecture. Hell we boot
DOS 5 here on a PII all the time. If the PFN in some manner, closely relates
to the segment base in the descriptor tables then, a 16 bit size field for
the PFN can effectively address several Gig of memory. For instance,
IoMapTransfer takes the next PFN in the MDL, and derives the base and limit
(or size) of the segment from the LDT/GDT.

Of course, that being the case, then Bill’s problem may be more in the way
he has defined the buffer and locked it down, and not in an arbitrary limit
set by the OS. Which is what Mark has been saying.

But again, that is still speculating (and wading into areas outside my
expertise).

OSR_Community_User · September 12, 2000, 5:00pm

Gary:

FYI, the IOCTL with the large buffer never even gets to my driver; the
I/O manager returns it with an out of resources-type error. I thought
it might be because I didn’t have enough MapRegisters allocated but not
only isn’t it that but I’m not even using an AdapterObject. So I think
it something else - like the I/O manager just won’t accept buffers that
big, i.e. 64MB.

Hate to say it but in some industries, seismic for example, 64MB is a
SMALL buffer.

Bill
== SCSI Adapters & VirtualSCSI Target Mode Libs ==

>
> I’ve been reading my 386 book about LDT, GDT, and segment
> descriptors. Ok, I
> know, the 386 is defunct, but that is the last hardware book
> I’ve had to
> buy, and code written for it will run on today’s
> architecture. Hell we boot
> DOS 5 here on a PII all the time. If the PFN in some manner,
> closely relates
> to the segment base in the descriptor tables then, a 16 bit
> size field for
> the PFN can effectively address several Gig of memory. For instance,
> IoMapTransfer takes the next PFN in the MDL, and derives the
> base and limit
> (or size) of the segment from the LDT/GDT.
>
> Of course, that being the case, then Bill’s problem may be
> more in the way
> he has defined the buffer and locked it down, and not in an
> arbitrary limit
> set by the OS. Which is what Mark has been saying.
>
> But again, that is still speculating (and wading into areas outside my
> expertise).
>
> —
> You are currently subscribed to ntdev as: xxxxx@advstor.com
> To unsubscribe send a blank email to $subst(‘Email.Unsub’)
>
>

OSR_Community_User · September 12, 2000, 5:08pm

So you need to redefine the IOCTL to be method neither and then slide an MDL
window across the buffer, or build a series of MDLs, or figure out how to
build your own MDL that ignores the size field limitation. You would have to
do this of course from some sort of highest level driver.

-----Original Message-----
From: Bill Casey [mailto:xxxxx@advstor.com]
Sent: Tuesday, September 12, 2000 4:56 PM
To: NT Developers Interest List
Subject: [ntdev] RE: Large buffers

Gary:

FYI, the IOCTL with the large buffer never even gets to my driver; the
I/O manager returns it with an out of resources-type error. I thought
it might be because I didn’t have enough MapRegisters
allocated but not
only isn’t it that but I’m not even using an AdapterObject.
So I think
it something else - like the I/O manager just won’t accept
buffers that
big, i.e. 64MB.

Hate to say it but in some industries, seismic for example, 64MB is a
SMALL buffer.

Bill
== SCSI Adapters & VirtualSCSI Target Mode Libs ==

>
> >
> > I’ve been reading my 386 book about LDT, GDT, and segment
> > descriptors. Ok, I
> > know, the 386 is defunct, but that is the last hardware book
> > I’ve had to
> > buy, and code written for it will run on today’s
> > architecture. Hell we boot
> > DOS 5 here on a PII all the time. If the PFN in some manner,
> > closely relates
> > to the segment base in the descriptor tables then, a 16 bit
> > size field for
> > the PFN can effectively address several Gig of memory. For instance,
> > IoMapTransfer takes the next PFN in the MDL, and derives the
> > base and limit
> > (or size) of the segment from the LDT/GDT.
> >
> > Of course, that being the case, then Bill’s problem may be
> > more in the way
> > he has defined the buffer and locked it down, and not in an
> > arbitrary limit
> > set by the OS. Which is what Mark has been saying.
> >
> > But again, that is still speculating (and wading into areas
> outside my
> > expertise).
> >
> > —
> > You are currently subscribed to ntdev as: xxxxx@advstor.com
> > To unsubscribe send a blank email to $subst(‘Email.Unsub’)
> >
> >
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@stratus.com
> To unsubscribe send a blank email to $subst(‘Email.Unsub’)
>

Gregory_G_Dyess · September 13, 2000, 7:19am

The origins of the 16-bit size field may be yet another hold-over from NT’s
VMS roots. This is just a guess, but many of the things from the internals
of NT are VERY similar to the equivalent structures from VMS.

Greg

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Gary Little
Sent: Tuesday, September 12, 2000 2:26 PM
To: NT Developers Interest List
Subject: [ntdev] RE: Re-post: Large buffers

> … since we are supposed to begin a 32-bit world why did the
designers
choose a 16-> bit number for the size of an MDL?

Other than pure speculation, I couldn’t say. Since the MDL header is fixed
length, why even include it in the size at all? Why not let that field
indicate the total number of PFNs in the array, beginning at *(&MDL +
sizeof(MDL)). But then the boys and girls at Microsoft never ask me about
design stuff.

It may be related to the maximum buffer size that can be used in a DMA
transfer which for SCSIPort, and many SCSI devices is still on
the order of
16M (16,777,216). I know our device limits our scatter/gather list to 1024
entries, or hex 400.

Or it could be related to the page size. If your page size is 8k
then 16,377
PFNs equates to over 134,169,384.

Most likely it’s defined that way because that’s what the engineer typed
when they entered that field definition for that structure, and
no one gave
it a thought in 1988 that we’d be pushing a terabyte in 2000.
Remember, most
machines were 16-bit, and many were still 8 bits, and memory was still
primarily measured in kilobytes. To use a Meg of memory you had to do back
flips and load those infamous “DOS Extenders”.

You are currently subscribed to ntdev as: xxxxx@pdq.net
To unsubscribe send a blank email to $subst(‘Email.Unsub’)

OSR_Community_User · September 13, 2000, 12:45pm

Mark:
The problem is it never REACHES the driver. The I/O manager is
bouncing it back. I should hear from MS today on this with a
(hopefully) definitive answer.

Bill

== SCSI Adapters & VirtualSCSI Target Mode Libs ==
Advanced Storage Concepts, Inc. (409) 744-2129
2720 Terminal Drive xxxxx@advstor.com
Galveston, TX 77554 USA www.virtualscsi.com

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Roddy, Mark
Sent: Tuesday, September 12, 2000 4:03 PM
To: NT Developers Interest List
Subject: [ntdev] RE: Large buffers

So you need to redefine the IOCTL to be method neither and
then slide an MDL
window across the buffer, or build a series of MDLs, or
figure out how to
build your own MDL that ignores the size field limitation.
You would have to
do this of course from some sort of highest level driver.

> -----Original Message-----
> From: Bill Casey [mailto:xxxxx@advstor.com]
> Sent: Tuesday, September 12, 2000 4:56 PM
> To: NT Developers Interest List
> Subject: [ntdev] RE: Large buffers
>
>
> Gary:
>
> FYI, the IOCTL with the large buffer never even gets to my
driver; the
> I/O manager returns it with an out of resources-type error.
I thought
> it might be because I didn’t have enough MapRegisters
> allocated but not
> only isn’t it that but I’m not even using an AdapterObject.
> So I think
> it something else - like the I/O manager just won’t accept
> buffers that
> big, i.e. 64MB.
>
> Hate to say it but in some industries, seismic for example,
64MB is a
> SMALL buffer.
>
> Bill
> == SCSI Adapters & VirtualSCSI Target Mode Libs ==
>
>
> >
> > >
> > > I’ve been reading my 386 book about LDT, GDT, and segment
> > > descriptors. Ok, I
> > > know, the 386 is defunct, but that is the last hardware book
> > > I’ve had to
> > > buy, and code written for it will run on today’s
> > > architecture. Hell we boot
> > > DOS 5 here on a PII all the time. If the PFN in some manner,
> > > closely relates
> > > to the segment base in the descriptor tables then, a 16 bit
> > > size field for
> > > the PFN can effectively address several Gig of memory.
> For instance,
> > > IoMapTransfer takes the next PFN in the MDL, and derives the
> > > base and limit
> > > (or size) of the segment from the LDT/GDT.
> > >
> > > Of course, that being the case, then Bill’s problem may be
> > > more in the way
> > > he has defined the buffer and locked it down, and not in an
> > > arbitrary limit
> > > set by the OS. Which is what Mark has been saying.
> > >
> > > But again, that is still speculating (and wading into areas
> > outside my
> > > expertise).
> > >
> > > —
> > > You are currently subscribed to ntdev as: xxxxx@advstor.com
> > > To unsubscribe send a blank email to
> $subst(‘Email.Unsub’)
> > >
> > >
> >
> >
> > —
> > You are currently subscribed to ntdev as: xxxxx@stratus.com
> > To unsubscribe send a blank email to $subst(‘Email.Unsub’)
> >
>
> —
> You are currently subscribed to ntdev as: xxxxx@advstor.com
> To unsubscribe send a blank email to $subst(‘Email.Unsub’)
>
>

OSR_Community_User · September 15, 2000, 9:20pm

It never reaches your driver because using METHOD_IN_DIRECT, the IO manager
builds the MDL for you, which it is unable to do. I was not aware of the
limit of 64 megs on an MDL, but the IO manager does have a limit on the
number of pages it will lock into memory ( which an MDL does ) for IO. You
can tune this number in the registry. If you switch to METHOD_NEITHER like
someone already suggested, you should get the request, and then you will
need to build your own MDL(s). Heck, with a data buffer size that large,
you may not want to build MDLs and lock the pages in memory. If you take
the user buffer from a METHOD_NEITHER call, you may want to lock down only
smaller parts of the buffer at a time, and work on those. Better yet, why
not just have the application send down 2 or 3 smaller buffers at a time in
overlapped IO requests? Throughput should not suffer from the overhead of
a few more IRPs that you need to process, and in fact, by not locking down
huge swaths of memory at a time, performance may gain. Why do you feel you
need to use such huge buffers anyway?

At 11:41 AM 9/13/00 -0500, you wrote:

Mark:
The problem is it never REACHES the driver. The I/O manager is
bouncing it back. I should hear from MS today on this with a
(hopefully) definitive answer.

Bill

== SCSI Adapters & VirtualSCSI Target Mode Libs ==
Advanced Storage Concepts, Inc. (409) 744-2129
2720 Terminal Drive xxxxx@advstor.com
Galveston, TX 77554 USA www.virtualscsi.com

> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com]On Behalf Of Roddy, Mark
> Sent: Tuesday, September 12, 2000 4:03 PM
> To: NT Developers Interest List
> Subject: [ntdev] RE: Large buffers
>
>
> So you need to redefine the IOCTL to be method neither and
> then slide an MDL
> window across the buffer, or build a series of MDLs, or
> figure out how to
> build your own MDL that ignores the size field limitation.
> You would have to
> do this of course from some sort of highest level driver.
>
> > -----Original Message-----
> > From: Bill Casey [mailto:xxxxx@advstor.com]
> > Sent: Tuesday, September 12, 2000 4:56 PM
> > To: NT Developers Interest List
> > Subject: [ntdev] RE: Large buffers
> >
> >
> > Gary:
> >
> > FYI, the IOCTL with the large buffer never even gets to my
> driver; the
> > I/O manager returns it with an out of resources-type error.
> I thought
> > it might be because I didn’t have enough MapRegisters
> > allocated but not
> > only isn’t it that but I’m not even using an AdapterObject.
> > So I think
> > it something else - like the I/O manager just won’t accept
> > buffers that
> > big, i.e. 64MB.
> >
> > Hate to say it but in some industries, seismic for example,
> 64MB is a
> > SMALL buffer.
> >
> > Bill
> > == SCSI Adapters & VirtualSCSI Target Mode Libs ==
> >
> >
> > >
> > > >
> > > > I’ve been reading my 386 book about LDT, GDT, and segment
> > > > descriptors. Ok, I
> > > > know, the 386 is defunct, but that is the last hardware book
> > > > I’ve had to
> > > > buy, and code written for it will run on today’s
> > > > architecture. Hell we boot
> > > > DOS 5 here on a PII all the time. If the PFN in some manner,
> > > > closely relates
> > > > to the segment base in the descriptor tables then, a 16 bit
> > > > size field for
> > > > the PFN can effectively address several Gig of memory.
> > For instance,
> > > > IoMapTransfer takes the next PFN in the MDL, and derives the
> > > > base and limit
> > > > (or size) of the segment from the LDT/GDT.
> > > >
> > > > Of course, that being the case, then Bill’s problem may be
> > > > more in the way
> > > > he has defined the buffer and locked it down, and not in an
> > > > arbitrary limit
> > > > set by the OS. Which is what Mark has been saying.
> > > >
> > > > But again, that is still speculating (and wading into areas
> > > outside my
> > > > expertise).
> > > >
> > > > —
> > > > You are currently subscribed to ntdev as: xxxxx@advstor.com
> > > > To unsubscribe send a blank email to
> > $subst(‘Email.Unsub’)
> > > >
> > > >
> > >
> > >
> > > —
> > > You are currently subscribed to ntdev as: xxxxx@stratus.com
> > > To unsubscribe send a blank email to $subst(‘Email.Unsub’)
> > >
> >
> > —
> > You are currently subscribed to ntdev as: xxxxx@advstor.com
> > To unsubscribe send a blank email to $subst(‘Email.Unsub’)
> >
> >
>
>
>—
>You are currently subscribed to ntdev as: xxxxx@iag.net
>To unsubscribe send a blank email to $subst(‘Email.Unsub’)

–>Phillip Susi
xxxxx@iag.net

Mark_Roddy · September 15, 2000, 9:56pm

Actually the decision about whether it is ‘good’ to lock down a huge buffer
really depends on what the intended use is. If for example the intended use
is for a system *dedicated* to moving huge blocks of data to and from user
space, then indeed locking down these huge buffers while the scsi adapter is
DMAing through them might be just the thing to do. On the other hand if the
intended use is within the context of a general purpose system that is
supposed to be useful for other things, then as you say it might be better
to re-think the application’s approach to data transfer.

Mark Roddy
Windows 2000/NT Consultant
Hollis Technology Solutions
www.hollistech.com

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Phillip Susi
Sent: Friday, September 15, 2000 9:14 PM
To: NT Developers Interest List
Subject: [ntdev] RE: Large buffers

It never reaches your driver because using METHOD_IN_DIRECT, the
IO manager
builds the MDL for you, which it is unable to do. I was not aware of the
limit of 64 megs on an MDL, but the IO manager does have a limit on the
number of pages it will lock into memory ( which an MDL does )
for IO. You
can tune this number in the registry. If you switch to
METHOD_NEITHER like
someone already suggested, you should get the request, and then you will
need to build your own MDL(s). Heck, with a data buffer size that large,
you may not want to build MDLs and lock the pages in memory. If you take
the user buffer from a METHOD_NEITHER call, you may want to lock
down only
smaller parts of the buffer at a time, and work on those. Better
yet, why
not just have the application send down 2 or 3 smaller buffers at
a time in
overlapped IO requests? Throughput should not suffer from the
overhead of
a few more IRPs that you need to process, and in fact, by not
locking down
huge swaths of memory at a time, performance may gain. Why do
you feel you
need to use such huge buffers anyway?

At 11:41 AM 9/13/00 -0500, you wrote:
>Mark:
> The problem is it never REACHES the driver. The I/O manager is
>bouncing it back. I should hear from MS today on this with a
>(hopefully) definitive answer.
>
>Bill
>
>== SCSI Adapters & VirtualSCSI Target Mode Libs ==
>Advanced Storage Concepts, Inc. (409) 744-2129
>2720 Terminal Drive xxxxx@advstor.com
>Galveston, TX 77554 USA www.virtualscsi.com
>
>
>
> > -----Original Message-----
> > From: xxxxx@lists.osr.com
> > [mailto:xxxxx@lists.osr.com]On Behalf Of Roddy, Mark
> > Sent: Tuesday, September 12, 2000 4:03 PM
> > To: NT Developers Interest List
> > Subject: [ntdev] RE: Large buffers
> >
> >
> > So you need to redefine the IOCTL to be method neither and
> > then slide an MDL
> > window across the buffer, or build a series of MDLs, or
> > figure out how to
> > build your own MDL that ignores the size field limitation.
> > You would have to
> > do this of course from some sort of highest level driver.
> >
> > > -----Original Message-----
> > > From: Bill Casey [mailto:xxxxx@advstor.com]
> > > Sent: Tuesday, September 12, 2000 4:56 PM
> > > To: NT Developers Interest List
> > > Subject: [ntdev] RE: Large buffers
> > >
> > >
> > > Gary:
> > >
> > > FYI, the IOCTL with the large buffer never even gets to my
> > driver; the
> > > I/O manager returns it with an out of resources-type error.
> > I thought
> > > it might be because I didn’t have enough MapRegisters
> > > allocated but not
> > > only isn’t it that but I’m not even using an AdapterObject.
> > > So I think
> > > it something else - like the I/O manager just won’t accept
> > > buffers that
> > > big, i.e. 64MB.
> > >
> > > Hate to say it but in some industries, seismic for example,
> > 64MB is a
> > > SMALL buffer.
> > >
> > > Bill
> > > == SCSI Adapters & VirtualSCSI Target Mode Libs ==
> > >
> > >
> > > >
> > > > >
> > > > > I’ve been reading my 386 book about LDT, GDT, and segment
> > > > > descriptors. Ok, I
> > > > > know, the 386 is defunct, but that is the last hardware book
> > > > > I’ve had to
> > > > > buy, and code written for it will run on today’s
> > > > > architecture. Hell we boot
> > > > > DOS 5 here on a PII all the time. If the PFN in some manner,
> > > > > closely relates
> > > > > to the segment base in the descriptor tables then, a 16 bit
> > > > > size field for
> > > > > the PFN can effectively address several Gig of memory.
> > > For instance,
> > > > > IoMapTransfer takes the next PFN in the MDL, and derives the
> > > > > base and limit
> > > > > (or size) of the segment from the LDT/GDT.
> > > > >
> > > > > Of course, that being the case, then Bill’s problem may be
> > > > > more in the way
> > > > > he has defined the buffer and locked it down, and not in an
> > > > > arbitrary limit
> > > > > set by the OS. Which is what Mark has been saying.
> > > > >
> > > > > But again, that is still speculating (and wading into areas
> > > > outside my
> > > > > expertise).
> > > > >
> > > > > —
> > > > > You are currently subscribed to ntdev as: xxxxx@advstor.com
> > > > > To unsubscribe send a blank email to
> > > $subst(‘Email.Unsub’)
> > > > >
> > > > >
> > > >
> > > >
> > > > —
> > > > You are currently subscribed to ntdev as: xxxxx@stratus.com
> > > > To unsubscribe send a blank email to $subst(‘Email.Unsub’)
> > > >
> > >
> > > —
> > > You are currently subscribed to ntdev as: xxxxx@advstor.com
> > > To unsubscribe send a blank email to $subst(‘Email.Unsub’)
> > >
> > >
> >
> >
> >—
> >You are currently subscribed to ntdev as: xxxxx@iag.net
> >To unsubscribe send a blank email to $subst(‘Email.Unsub’)
>
> –>Phillip Susi
> xxxxx@iag.net
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@wattanuck.mv.com
> To unsubscribe send a blank email to $subst(‘Email.Unsub’)
>
>

OSR_Community_User · September 16, 2000, 1:33pm

Mark & Phillip:
Thanks for the input. I had hoped to get around all this with the
ReadFileScatter() and WriteFileGather() functions. However, MS has just
informed me that they are meant for ‘file system’ (DISK, CDROM, etc.)
devices only and will not work with a monolithic driver of unknown type.
Of course, this restriction (among others) is not DOCUMENTED in the
descriptions of either function :).

Bill
== SCSI Adapters & VirtualSCSI Target Mode Libs ==

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Mark Roddy
Sent: Friday, September 15, 2000 8:43 PM
To: NT Developers Interest List
Subject: [ntdev] RE: Large buffers

Actually the decision about whether it is ‘good’ to lock down
a huge buffer
really depends on what the intended use is. If for example
the intended use
is for a system *dedicated* to moving huge blocks of data to
and from user
space, then indeed locking down these huge buffers while the
scsi adapter is
DMAing through them might be just the thing to do. On the
other hand if the
intended use is within the context of a general purpose system that is
supposed to be useful for other things, then as you say it
might be better
to re-think the application’s approach to data transfer.

Mark Roddy
Windows 2000/NT Consultant
Hollis Technology Solutions
www.hollistech.com

> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com]On Behalf Of Phillip Susi
> Sent: Friday, September 15, 2000 9:14 PM
> To: NT Developers Interest List
> Subject: [ntdev] RE: Large buffers
>
>
> It never reaches your driver because using METHOD_IN_DIRECT, the
> IO manager
> builds the MDL for you, which it is unable to do. I was
not aware of the
> limit of 64 megs on an MDL, but the IO manager does have a
limit on the
> number of pages it will lock into memory ( which an MDL does )
> for IO. You
> can tune this number in the registry. If you switch to
> METHOD_NEITHER like
> someone already suggested, you should get the request, and
then you will
> need to build your own MDL(s). Heck, with a data buffer
size that large,
> you may not want to build MDLs and lock the pages in
memory. If you take
> the user buffer from a METHOD_NEITHER call, you may want to lock
> down only
> smaller parts of the buffer at a time, and work on those. Better
> yet, why
> not just have the application send down 2 or 3 smaller buffers at
> a time in
> overlapped IO requests? Throughput should not suffer from the
> overhead of
> a few more IRPs that you need to process, and in fact, by not
> locking down
> huge swaths of memory at a time, performance may gain. Why do
> you feel you
> need to use such huge buffers anyway?
>
> At 11:41 AM 9/13/00 -0500, you wrote:
> >Mark:
> > The problem is it never REACHES the driver. The
I/O manager is
> >bouncing it back. I should hear from MS today on this with a
> >(hopefully) definitive answer.
> >
> >Bill
> >
> >== SCSI Adapters & VirtualSCSI Target Mode Libs ==
> >Advanced Storage Concepts, Inc. (409) 744-2129
> >2720 Terminal Drive xxxxx@advstor.com
> >Galveston, TX 77554 USA www.virtualscsi.com
> >
> >
> >
> > > -----Original Message-----
> > > From: xxxxx@lists.osr.com
> > > [mailto:xxxxx@lists.osr.com]On Behalf Of Roddy, Mark
> > > Sent: Tuesday, September 12, 2000 4:03 PM
> > > To: NT Developers Interest List
> > > Subject: [ntdev] RE: Large buffers
> > >
> > >
> > > So you need to redefine the IOCTL to be method neither and
> > > then slide an MDL
> > > window across the buffer, or build a series of MDLs, or
> > > figure out how to
> > > build your own MDL that ignores the size field limitation.
> > > You would have to
> > > do this of course from some sort of highest level driver.
> > >
> > > > -----Original Message-----
> > > > From: Bill Casey [mailto:xxxxx@advstor.com]
> > > > Sent: Tuesday, September 12, 2000 4:56 PM
> > > > To: NT Developers Interest List
> > > > Subject: [ntdev] RE: Large buffers
> > > >
> > > >
> > > > Gary:
> > > >
> > > > FYI, the IOCTL with the large buffer never even gets to my
> > > driver; the
> > > > I/O manager returns it with an out of resources-type error.
> > > I thought
> > > > it might be because I didn’t have enough MapRegisters
> > > > allocated but not
> > > > only isn’t it that but I’m not even using an AdapterObject.
> > > > So I think
> > > > it something else - like the I/O manager just won’t accept
> > > > buffers that
> > > > big, i.e. 64MB.
> > > >
> > > > Hate to say it but in some industries, seismic for example,
> > > 64MB is a
> > > > SMALL buffer.
> > > >
> > > > Bill
> > > > == SCSI Adapters & VirtualSCSI Target Mode Libs ==
> > > >
> > > >
> > > > >
> > > > > >
> > > > > > I’ve been reading my 386 book about LDT, GDT, and segment
> > > > > > descriptors. Ok, I
> > > > > > know, the 386 is defunct, but that is the last hardware book
> > > > > > I’ve had to
> > > > > > buy, and code written for it will run on today’s
> > > > > > architecture. Hell we boot
> > > > > > DOS 5 here on a PII all the time. If the PFN in some manner,
> > > > > > closely relates
> > > > > > to the segment base in the descriptor tables then, a 16 bit
> > > > > > size field for
> > > > > > the PFN can effectively address several Gig of memory.
> > > > For instance,
> > > > > > IoMapTransfer takes the next PFN in the MDL, and derives the
> > > > > > base and limit
> > > > > > (or size) of the segment from the LDT/GDT.
> > > > > >
> > > > > > Of course, that being the case, then Bill’s problem may be
> > > > > > more in the way
> > > > > > he has defined the buffer and locked it down, and not in an
> > > > > > arbitrary limit
> > > > > > set by the OS. Which is what Mark has been saying.
> > > > > >
> > > > > > But again, that is still speculating (and wading into areas
> > > > > outside my
> > > > > > expertise).
> > > > > >
> > > > > > —
> > > > > > You are currently subscribed to ntdev as: xxxxx@advstor.com
> > > > > > To unsubscribe send a blank email to
> > > > $subst(‘Email.Unsub’)
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > —
> > > > > You are currently subscribed to ntdev as:
> xxxxx@stratus.com
> > > > > To unsubscribe send a blank email to
> $subst(‘Email.Unsub’)
> > > > >
> > > >
> > > > —
> > > > You are currently subscribed to ntdev as: xxxxx@advstor.com
> > > > To unsubscribe send a blank email to
> $subst(‘Email.Unsub’)
> > > >
> > > >
> > >
> > >
> > >—
> > >You are currently subscribed to ntdev as: xxxxx@iag.net
> > >To unsubscribe send a blank email to $subst(‘Email.Unsub’)
> >
> > –>Phillip Susi
> > xxxxx@iag.net
> >
> >
> > —
> > You are currently subscribed to ntdev as: xxxxx@wattanuck.mv.com
> > To unsubscribe send a blank email to $subst(‘Email.Unsub’)
> >
> >
>
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@advstor.com
> To unsubscribe send a blank email to $subst(‘Email.Unsub’)
>
>