More of a curiosity than a pressing issue, but why is it that the file mapping API’s support 64-bit sizes for view-size and mapping object sizes, file seek pointers are 64 bits, overlapped reads are 64 bit offsets, yet you can only readfile/writefile a maximum of a 32-bit value even in this 64 bit world?
I figured if everything else was 64 bit-capable, there was a pretty good reason this wasn’t…
Asa
Actually, it is pretty weird that you don’t understand it - an answer to your question seems to be plainly obvious…
When you read/write from/to a file, you do it to/from the buffer in memory from/to the disk, i.e.
all read/write operations are bound by the maximum buffer size. It is understandable that a process that resides in 32-bit virtual address space objectively cannot have a buffer of the size that cannot be expressed as a 32-bit value - even if the target machine has more than 4G of RAM. Therefore, once all read/write operations are bound by the maximum buffer size, the amount of data they deal with can be always expressed as a 32-bit value. However, when it comes to file size, it is bound only by the disk space constraints, so that it is not always possible to present a file offset by 32-bit value…
Anton Bassov
xxxxx@hotmail.com wrote:
Actually, it is pretty weird that you don’t understand it - an answer to your question seems to be plainly obvious…
You are being too dismissive here, because you have overlooked one
legitimate case, which is not typical for you…
When you read/write from/to a file, you do it to/from the buffer in memory from/to the disk, i.e.
all read/write operations are bound by the maximum buffer size. It is understandable that a process that resides in 32-bit virtual address space objectively cannot have a buffer of the size that cannot be expressed as a 32-bit value - even if the target machine has more than 4G of RAM.
This is true. HOWEVER, what you have overlooked is that the ReadFile
and WriteFile APIs still take a 32-bit byte count even in a 64-bit
process. It’s defined as a DWORD, rather than a SIZE_T. Thus, his
question is not unreasonable. It is possible, as far as I know, to
allocate a memory buffer greater than 4GB in a 64-bit process.
I’m not sure it is particularly useful; if I needed to do that kind of
thing I’d just use a memory-mapped file, but it is a fair question.
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
IO_STACK_LOCATION for read and write uses ULONG for length, so the liit is probably well baked into the kernel code.
May just have been a case of not reworking that code until there’s a compelling need for single multi-GB transfers…
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Tim Roberts
Sent: Tuesday, July 17, 2007 9:48 AM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] Read/Write
xxxxx@hotmail.com wrote:
Actually, it is pretty weird that you don’t understand it - an answer to your question seems to be plainly obvious…
You are being too dismissive here, because you have overlooked one
legitimate case, which is not typical for you…
When you read/write from/to a file, you do it to/from the buffer in memory from/to the disk, i.e.
all read/write operations are bound by the maximum buffer size. It is understandable that a process that resides in 32-bit virtual address space objectively cannot have a buffer of the size that cannot be expressed as a 32-bit value - even if the target machine has more than 4G of RAM.
This is true. HOWEVER, what you have overlooked is that the ReadFile
and WriteFile APIs still take a 32-bit byte count even in a 64-bit
process. It’s defined as a DWORD, rather than a SIZE_T. Thus, his
question is not unreasonable. It is possible, as far as I know, to
allocate a memory buffer greater than 4GB in a 64-bit process.
I’m not sure it is particularly useful; if I needed to do that kind of
thing I’d just use a memory-mapped file, but it is a fair question.
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
64bit length? Sorry, but, until rather recent Windows versions, a single
MDL size was limited to 32MB.
So, 32bit length is OK.
–
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com
“Asa Yeamans” wrote in message news:xxxxx@ntdev…
More of a curiosity than a pressing issue, but why is it that the file mapping
API’s support 64-bit sizes for view-size and mapping object sizes, file seek
pointers are 64 bits, overlapped reads are 64 bit offsets, yet you can only
readfile/writefile a maximum of a 32-bit value even in this 64 bit world?
I figured if everything else was 64 bit-capable, there was a pretty good reason
this wasn’t…
Asa
> HOWEVER, what you have overlooked is that the ReadFile and WriteFile APIs still
take a 32-bit byte count even in a 64-bit process. It’s defined as a DWORD, rather
than a SIZE_T. Thus, his question is not unreasonable.
Actually, I just explained the apparent reasoning of those who designed ReadFile() and WriteFile()
functions. Please note that at the time they were designed no one could possibly imagine that one day Windows would become 64-bit OS, so that the reasoning behind introduction of 64-bit parameters, apparently, had nothing to do with plans to make Windows 64-bit at some point. Instead, apparently it was dictated solely by the necesssity of being able to handle large files.
Ironically, with the advent of 64-bit Windows you may read the OP’s question rather differently,
i.e. why MSFT has not yet replaced DWORD parameters with SIZE_T…
Anton Bassov
At the time (NT development) the Alpha was a target platform and was
64bits.
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@hotmail.com
Sent: Tuesday, July 17, 2007 6:22 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Read/Write
HOWEVER, what you have overlooked is that the ReadFile and WriteFile
APIs still
take a 32-bit byte count even in a 64-bit process. It’s defined as a
DWORD, rather
than a SIZE_T. Thus, his question is not unreasonable.
Actually, I just explained the apparent reasoning of those who designed
ReadFile() and WriteFile()
functions. Please note that at the time they were designed no one could
possibly imagine that one day Windows would become 64-bit OS, so that
the reasoning behind introduction of 64-bit parameters, apparently, had
nothing to do with plans to make Windows 64-bit at some point. Instead,
apparently it was dictated solely by the necesssity of being able to
handle large files.
Ironically, with the advent of 64-bit Windows you may read the OP’s
question rather differently,
i.e. why MSFT has not yet replaced DWORD parameters with SIZE_T…
Anton Bassov
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
But no 64 bit support was even thought about until the win2k timeframe,
so while there was a 64 bit platform, it was not a targeted platform at
the time of nt3.1
d
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Roddy, Mark
Sent: Wednesday, July 18, 2007 10:46 AM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] Read/Write
At the time (NT development) the Alpha was a target platform and was
64bits.
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@hotmail.com
Sent: Tuesday, July 17, 2007 6:22 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Read/Write
HOWEVER, what you have overlooked is that the ReadFile and WriteFile
APIs still
take a 32-bit byte count even in a 64-bit process. It’s defined as a
DWORD, rather
than a SIZE_T. Thus, his question is not unreasonable.
Actually, I just explained the apparent reasoning of those who designed
ReadFile() and WriteFile()
functions. Please note that at the time they were designed no one could
possibly imagine that one day Windows would become 64-bit OS, so that
the reasoning behind introduction of 64-bit parameters, apparently, had
nothing to do with plans to make Windows 64-bit at some point. Instead,
apparently it was dictated solely by the necesssity of being able to
handle large files.
Ironically, with the advent of 64-bit Windows you may read the OP’s
question rather differently,
i.e. why MSFT has not yet replaced DWORD parameters with SIZE_T…
Anton Bassov
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
BTW, I worked on 3.1 [started when we turned OS/2 over to IBM], and for the life of me, I’m convinced that the Alpha as a target didn’t happen until 3.5.
I remember MIPS (R4000?), 386/486, and early on the Intel i860 [IIRC- it was being removed about the time I got involved, but there were still references to it].
As an SDET, I had MIPS and 386 test machines, and I’m certain if we were targeting something else for release, I’d have had it [because they routinely used us as STEs in those days].
Of course, that’s like 14-17 years ago, and I don’t entirely trust the accuracy of my memories of last week…
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Doron Holan
Sent: Wednesday, July 18, 2007 11:06 AM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] Read/Write
But no 64 bit support was even thought about until the win2k timeframe,
so while there was a 64 bit platform, it was not a targeted platform at
the time of nt3.1
d
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Roddy, Mark
Sent: Wednesday, July 18, 2007 10:46 AM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] Read/Write
At the time (NT development) the Alpha was a target platform and was
64bits.
>BTW, I worked on 3.1 [started when we turned OS/2 over to IBM], and for the
life
of me, I’m convinced that the Alpha as a target didn’t happen until 3.5.
IIRC 3.1 was x86/MIPS/Alpha, and 3.5 added PowerPC.
–
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com
Yeah, I’m beginning to remember something along the lines of “Alpha will be supported, but DEC is going to have to do all the real work, because we’re all pegged, and behind schedule to boot”- but that was long after this stuff was actually designed. As I am recalling it, it was described as a porting task, so I doubt redesign would have even been countenanced.
If that’s the case, then so as far as I would have been concerned, it didn’t exist, and I didn’t have to think about it.
The list just dodged a bullet- I almost began real reminiscing, there…
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Maxim S. Shatskih
Sent: Wednesday, July 18, 2007 2:07 PM
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] Read/Write
BTW, I worked on 3.1 [started when we turned OS/2 over to IBM], and for the
life
of me, I’m convinced that the Alpha as a target didn’t happen until 3.5.
IIRC 3.1 was x86/MIPS/Alpha, and 3.5 added PowerPC.
–
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com
Didn’t the lack of byte granular memory access dictate some of the
design of the (formerly) user mode component of display drivers? I seem
to recall something about that in a whitepaper in (probably) the NT4
DDK? I have no idea; I just thought I would throw it in for good
measure.
mm
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Bob Kjelgaard
Sent: Wednesday, July 18, 2007 18:03
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] Read/Write
Yeah, I’m beginning to remember something along the lines of “Alpha will
be supported, but DEC is going to have to do all the real work, because
we’re all pegged, and behind schedule to boot”- but that was long after
this stuff was actually designed. As I am recalling it, it was
described as a porting task, so I doubt redesign would have even been
countenanced.
If that’s the case, then so as far as I would have been concerned, it
didn’t exist, and I didn’t have to think about it.
The list just dodged a bullet- I almost began real reminiscing, there…
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Maxim S.
Shatskih
Sent: Wednesday, July 18, 2007 2:07 PM
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] Read/Write
BTW, I worked on 3.1 [started when we turned OS/2 over to IBM], and for
the
life
of me, I’m convinced that the Alpha as a target didn’t happen until
3.5.
IIRC 3.1 was x86/MIPS/Alpha, and 3.5 added PowerPC.
–
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
On early steppings of the Alpha, that is.
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Martin O’Brien
Sent: Wednesday, July 18, 2007 18:31
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] Read/Write
Didn’t the lack of byte granular memory access dictate some of the
design of the (formerly) user mode component of display drivers? I seem
to recall something about that in a whitepaper in (probably) the NT4
DDK? I have no idea; I just thought I would throw it in for good
measure.
mm
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Bob Kjelgaard
Sent: Wednesday, July 18, 2007 18:03
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] Read/Write
Yeah, I’m beginning to remember something along the lines of “Alpha will
be supported, but DEC is going to have to do all the real work, because
we’re all pegged, and behind schedule to boot”- but that was long after
this stuff was actually designed. As I am recalling it, it was
described as a porting task, so I doubt redesign would have even been
countenanced.
If that’s the case, then so as far as I would have been concerned, it
didn’t exist, and I didn’t have to think about it.
The list just dodged a bullet- I almost began real reminiscing, there…
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Maxim S.
Shatskih
Sent: Wednesday, July 18, 2007 2:07 PM
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] Read/Write
BTW, I worked on 3.1 [started when we turned OS/2 over to IBM], and for
the
life
of me, I’m convinced that the Alpha as a target didn’t happen until
3.5.
IIRC 3.1 was x86/MIPS/Alpha, and 3.5 added PowerPC.
–
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
FWIW, I’ve been told that Alpha support happened after RTM (at least as I would have seen it) as an “update” of 3.1. So it sound slike an issue of semantics to me, and it’s all ancient history, anyway…
Martin O’Brien wrote:
Didn’t the lack of byte granular memory access dictate some of the
design of the (formerly) user mode component of display drivers?
The original NT-capable platform for the Alpha was called “Jensen”. It
sucked hugely. One of the biggest problems was the mapping of the PCI
bus into memory space. The Alpha wasn’t designed for PCI, so there was
an incredible amount of hackage to wire it in. In particular, the PCI
address bits were not mapped into the low-order bits of the Alpha’s
memory space. They were shifted up something like 7 bits. They called
it a “sparse address space”. We called it a “pile of excrement”.
What that means is that the dword registers at offset 0x100 and 0x104 in
your device’s PCI memory space were located at offset 0x8000 and 0x8200
in the Alpha’s address space. You couldn’t just write a buffer or a
bitmap using memcpy. You had to write them one at a time. It still
makes me shudder.
The next generation of NT-capable Alphas solved this, but by then the
damage was done.
The Jensen ran the Alpha at 150 MHz, which got you roughly the
performance of a Pentium-50, at roughly 4 times the price.
I seem
to recall something about that in a whitepaper in (probably) the NT4
DDK?
Probably not. The Alphas were dropped in NT4.
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
Thanks, Tim.
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Tim Roberts
Sent: Wednesday, July 18, 2007 20:04
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] Read/Write
Martin O’Brien wrote:
Didn’t the lack of byte granular memory access dictate some of the
design of the (formerly) user mode component of display drivers?
The original NT-capable platform for the Alpha was called “Jensen”. It
sucked hugely. One of the biggest problems was the mapping of the PCI
bus into memory space. The Alpha wasn’t designed for PCI, so there was
an incredible amount of hackage to wire it in. In particular, the PCI
address bits were not mapped into the low-order bits of the Alpha’s
memory space. They were shifted up something like 7 bits. They called
it a “sparse address space”. We called it a “pile of excrement”.
What that means is that the dword registers at offset 0x100 and 0x104 in
your device’s PCI memory space were located at offset 0x8000 and 0x8200
in the Alpha’s address space. You couldn’t just write a buffer or a
bitmap using memcpy. You had to write them one at a time. It still
makes me shudder.
The next generation of NT-capable Alphas solved this, but by then the
damage was done.
The Jensen ran the Alpha at 150 MHz, which got you roughly the
performance of a Pentium-50, at roughly 4 times the price.
I seem
to recall something about that in a whitepaper in (probably) the NT4
DDK?
Probably not. The Alphas were dropped in NT4.
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
>> I seem
> to recall something about that in a whitepaper in (probably) the NT4
> DDK?Probably not. The Alphas were dropped in NT4.
Tim, your technical expertise is impeccable, but I think your knowledge of
history is less than perfect.
Google the KB article for the fixes in NT4 SP6A, and you will see Alpha
specific fixes.
If I booted my DEC AlphaStation 233 today, it would come up running Windows
NT 4.0. Although, I could also boot the second disk into VMS.
Alphas were dropped in W2K, but 64 bit Windows was developed on Alpha,
because Intel was incapable of delivering a working Ia64 system in a timely
manner.
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Tim Roberts
Sent: Wednesday, July 18, 2007 6:04 PM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] Read/Write
Martin O’Brien wrote:
Didn’t the lack of byte granular memory access dictate some of the
design of the (formerly) user mode component of display drivers?
The original NT-capable platform for the Alpha was called “Jensen”. It
sucked hugely. One of the biggest problems was the mapping of the PCI bus
into memory space. The Alpha wasn’t designed for PCI, so there was an
incredible amount of hackage to wire it in. In particular, the PCI address
bits were not mapped into the low-order bits of the Alpha’s memory space.
They were shifted up something like 7 bits. They called it a “sparse
address space”. We called it a “pile of excrement”.
What that means is that the dword registers at offset 0x100 and 0x104 in
your device’s PCI memory space were located at offset 0x8000 and 0x8200 in
the Alpha’s address space. You couldn’t just write a buffer or a bitmap
using memcpy. You had to write them one at a time. It still makes me
shudder.
The next generation of NT-capable Alphas solved this, but by then the damage
was done.
The Jensen ran the Alpha at 150 MHz, which got you roughly the performance
of a Pentium-50, at roughly 4 times the price.
I seem
to recall something about that in a whitepaper in (probably) the NT4
DDK?
Probably not. The Alphas were dropped in NT4.
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
Bob Kjelgaard wrote:
Yeah, I’m beginning to remember something along the lines of “Alpha will be supported, but DEC is going to have to do all the real work, because we’re all pegged, and behind schedule to boot”- but that was long after this stuff was actually designed. As I am recalling it, it was described as a porting task, so I doubt redesign would have even been countenanced.
My recollection is that DEC had their own source base and their own port
on the Alpha and that it was 64bits and that it was never released. The
Microsoft Alpha releases were all 32bit and went back all the way to 3.1.
If that’s the case, then so as far as I would have been concerned, it didn’t exist, and I didn’t have to think about it.
The list just dodged a bullet- I almost began real reminiscing, there…
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Maxim S. Shatskih
Sent: Wednesday, July 18, 2007 2:07 PM
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] Read/Write> BTW, I worked on 3.1 [started when we turned OS/2 over to IBM], and for the
>
life> of me, I’m convinced that the Alpha as a target didn’t happen until 3.5.
>IIRC 3.1 was x86/MIPS/Alpha, and 3.5 added PowerPC.
–
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com
Questions? First check the Kernel Driver FAQ at http://www.osronline.com/article.cfm?id=256
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
Dan Kyler wrote:
Tim, your technical expertise is impeccable, but I think your knowledge of
history is less than perfect.Google the KB article for the fixes in NT4 SP6A, and you will see Alpha
specific fixes.
Duh, yes, I knew that, or at least I would have if I had thought about
it. We did the Cinepak codec for Windows, and I know we were building
RISC versions well into the NT4 era. Thanks.
Alphas were dropped in W2K, but 64 bit Windows was developed on Alpha,
because Intel was incapable of delivering a working Ia64 system in a timely
manner.
Yes, I remember seeing a demo of a 64-bit SQL Server running on a 64-bit
Alpha Windows system at WinHEC some time around the turn of the
millennium. It was either NT4 or Win2K.
I’ve sometimes wondered whether the 64-bit code we have now was derived
from that same code base, or if that was a one-time hack and Microsoft
actually started over.
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.