64 Bit Raid to 32 bit PCI Target Card.

Hi,
I’ve got a 32 Bit PCI Framebuffer card that I want to dump
video to from a raid disk as fast as possible. The raid is
on separate 64 bit pci bus. The PCI framebuffer doesn’t
have dma so i just map the framebuffer into userspace
and use the following snippet of code.

HANDLE fileHandle = CreateFile(“test.yuv”,
GENERIC_READ,
0,NULL,
OPEN_EXISTING,
FILE_ATTRIBUTE_NORMAL,
NULL);

ULWord* frameBuffer;
GetBaseAddress(&frameBuffer);

ReadFile(fileHandle,
frameBuffer,
1920*2*1080,
&nBytesRead,
NULL) ;

So this directly reads from disk to our board. The raid
array can spit data out at 150 Mbytes per second and
our card can take in data at at least 105Mbytes per
second but the transfer gets throttled to about 50
Mbytes per second.

Any ideas of what is going on?
Maybe the Bridge chip is throttling?
By the way, what does FILE_FLAG_WRITE_THROUGH
really do?


You are currently subscribed to ntdev as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com

>and use the following snippet of code.

HANDLE fileHandle = CreateFile(“test.yuv”,
GENERIC_READ,
0,NULL,
OPEN_EXISTING,
FILE_ATTRIBUTE_NORMAL,
NULL);

ULWord* frameBuffer;
GetBaseAddress(&frameBuffer);

ReadFile(fileHandle,
frameBuffer,
1920*2*1080,
&nBytesRead,
NULL) ;

So this directly reads from disk to our board.

If this is your exact code, your NOT doing DMA directly from the frame
buffer to the disk controller. You need to open the file in unbuffered mode
(I forget the flag). This code is doing a memory copy from your frame
buffer to some file system buffers, and then doing DMA from those buffers
to disk. The memory copy is probably not getting good PCI bursts, as it’s
really hard to get bursts for PCI target to memory copies. Disk transfer
are also probably not really large, as the lazy page writer is initiating
them. Note that an unbuffered transfer will have to be an integral number
of disk sectors, which 1920*2*1080 probably is. Also note the file offset
also needs to align on sectors.

Doing the performance analysis math: each frame is 4,147,200 bytes, which
at 30 fps is about 124 MBytes/sec or at 24 fps is about 100 MBytes/sec. If
your capturing HDTV video, your either already too slow (at 30 fps) or else
need to have almost 100% device utilization (at 24 fps). Per frame: the
frame buffer transfer takes about 40 milliseconds, and the disk transfer
takes about 28 milliseconds. If these two do not overlap, the total time
will be 68 milliseconds for 4.14 MBytes or about 15 fps, or about 62
MBytes/sec.

Another question is the architecture of the PCI bus to memory path. If the
32-bit bus ties up the 64-bit bus for the duration of a 32-bit PCI
transaction, your may have a problem. If the 32-bit bus is bridged from the
64-bit bus, this may be the case. I actually don’t know if a PCI bridge
tries to minimize latency, by initiating the destination side transfer in
parallel with the source side transfer. I think there is a high probability
it does, to tell the master that some target is accepting the transaction,
especially since in this case the master will be reading from the target.
Bridges can sometimes read ahead, although all the bridge programming magic
happens in the OS kernel (assuming W2K). You should be able to look at the
bridge parent bus registers and decide the hierarchical structure. What you
want is for both the 32-bit and 64-bit buses to be directly attached to the
memory controller. Perhaps if you told us the machine, somebody here knows
it’s PCI bus architecture. I’m not saying this IS a problem, just it MIGHT
be a problem. This problem also goes away if you actually get direct
transfers from the capture target memory to the disk controller.

  • Jan

  • Jan


You are currently subscribed to ntdev as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com

Hi Jan,
Thanks for the info. I knew there was some hidden flag around.
The MSDN documentation for CreateFile does not list
FILE_FLAG_NO_BUFFERING
as a dwFlagsAndAttributes flag.
That’s kinda bogus.
I found the name of the flag in KB# Q99794 while searching
for FILE_FLAG_WRITE_THROUGH.

Jan Bottorff wrote:

>and use the following snippet of code.
>
> HANDLE fileHandle = CreateFile(“test.yuv”,
> GENERIC_READ,
> 0,NULL,
> OPEN_EXISTING,
> FILE_ATTRIBUTE_NORMAL,
> NULL);
>
> ULWord* frameBuffer;
> GetBaseAddress(&frameBuffer);
>
> ReadFile(fileHandle,
> frameBuffer,
> 1920*2*1080,
> &nBytesRead,
> NULL) ;
>
>So this directly reads from disk to our board.

If this is your exact code, your NOT doing DMA directly from the frame
buffer to the disk controller. You need to open the file in unbuffered mode
(I forget the flag). This code is doing a memory copy from your frame
buffer to some file system buffers, and then doing DMA from those buffers
to disk. The memory copy is probably not getting good PCI bursts, as it’s
really hard to get bursts for PCI target to memory copies. Disk transfer
are also probably not really large, as the lazy page writer is initiating
them. Note that an unbuffered transfer will have to be an integral number
of disk sectors, which 1920*2*1080 probably is. Also note the file offset
also needs to align on sectors.

Doing the performance analysis math: each frame is 4,147,200 bytes, which
at 30 fps is about 124 MBytes/sec or at 24 fps is about 100 MBytes/sec. If
your capturing HDTV video, your either already too slow (at 30 fps) or else
need to have almost 100% device utilization (at 24 fps). Per frame: the
frame buffer transfer takes about 40 milliseconds, and the disk transfer
takes about 28 milliseconds. If these two do not overlap, the total time
will be 68 milliseconds for 4.14 MBytes or about 15 fps, or about 62
MBytes/sec.

Another question is the architecture of the PCI bus to memory path. If the
32-bit bus ties up the 64-bit bus for the duration of a 32-bit PCI
transaction, your may have a problem. If the 32-bit bus is bridged from the
64-bit bus, this may be the case. I actually don’t know if a PCI bridge
tries to minimize latency, by initiating the destination side transfer in
parallel with the source side transfer. I think there is a high probability
it does, to tell the master that some target is accepting the transaction,
especially since in this case the master will be reading from the target.
Bridges can sometimes read ahead, although all the bridge programming magic
happens in the OS kernel (assuming W2K). You should be able to look at the
bridge parent bus registers and decide the hierarchical structure. What you
want is for both the 32-bit and 64-bit buses to be directly attached to the
memory controller. Perhaps if you told us the machine, somebody here knows
it’s PCI bus architecture. I’m not saying this IS a problem, just it MIGHT
be a problem. This problem also goes away if you actually get direct
transfers from the capture target memory to the disk controller.

  • Jan

  • Jan


You are currently subscribed to ntdev as: xxxxx@aja.com
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntdev as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com

>The MSDN documentation for CreateFile does not list

FILE_FLAG_NO_BUFFERING
as a dwFlagsAndAttributes flag.
That’s kinda bogus.

The online version of the MSDN library seems to list FILE_FLAG_NO_BUFFERING
under the CreateFile function reference. Are you saying it’s not in the CD
version?

  • Jan

You are currently subscribed to ntdev as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com

I have the January 2001 MSDN and it does not list
the flag. I will no not to rely on that in the future.

After becoming hip to the flag, I see that is documented
all over the place…except the CD MSDN.

Jan Bottorff wrote:

>The MSDN documentation for CreateFile does not list
>FILE_FLAG_NO_BUFFERING
>as a dwFlagsAndAttributes flag.
>That’s kinda bogus.

The online version of the MSDN library seems to list FILE_FLAG_NO_BUFFERING
under the CreateFile function reference. Are you saying it’s not in the CD
version?

  • Jan

You are currently subscribed to ntdev as: xxxxx@aja.com
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntdev as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com

It is documented in the January MSDN CD, I suspect you
looked up CreateFile and got the Windows CE version,
that does not have the flag.

Don Burn
Windows 2000 Device Driver and Filesystem consulting

----- Original Message -----
From: “Bill Bowen”
To: “NT Developers Interest List”
Sent: Monday, July 30, 2001 12:09 PM
Subject: [ntdev] Re: 64 Bit Raid to 32 bit PCI Target Card.

> I have the January 2001 MSDN and it does not list
> the flag. I will no not to rely on that in the future.
>
> After becoming hip to the flag, I see that is documented
> all over the place…except the CD MSDN.
>
> Jan Bottorff wrote:
>
> > >The MSDN documentation for CreateFile does not list
> > >FILE_FLAG_NO_BUFFERING
> > >as a dwFlagsAndAttributes flag.
> > >That’s kinda bogus.
> >
> > The online version of the MSDN library seems to list
FILE_FLAG_NO_BUFFERING
> > under the CreateFile function reference. Are you saying it’s not in the
CD
> > version?
> >
> > - Jan
> >
> > —
> > You are currently subscribed to ntdev as: xxxxx@aja.com
> > To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@acm.org
> To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com
>


You are currently subscribed to ntdev as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com

>>I have the January 2001 MSDN and it does not list

>the flag. I will no not to rely on that in the future.

I have the January 2001 MSDN too. You might be looking at wrong place “API
Reference”, that’s for WinCE. The “Files and IO: Platform SDK” is the right
place for Win32.

Michael


You are currently subscribed to ntdev as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com

Which would be about the 10,000,000 time somebody got snookered by the wince
docs. Perhaps msoft could make that stuff OPTIONAL?

-----Original Message-----
From: Don Burn [mailto:xxxxx@acm.org]
Sent: Monday, July 30, 2001 12:20 PM
To: NT Developers Interest List
Subject: [ntdev] Re: 64 Bit Raid to 32 bit PCI Target Card.

It is documented in the January MSDN CD, I suspect you
looked up CreateFile and got the Windows CE version,
that does not have the flag.

Don Burn
Windows 2000 Device Driver and Filesystem consulting

----- Original Message -----
From: “Bill Bowen”
To: “NT Developers Interest List”
Sent: Monday, July 30, 2001 12:09 PM
Subject: [ntdev] Re: 64 Bit Raid to 32 bit PCI Target Card.

> I have the January 2001 MSDN and it does not list
> the flag. I will no not to rely on that in the future.
>
> After becoming hip to the flag, I see that is documented
> all over the place…except the CD MSDN.
>
> Jan Bottorff wrote:
>
> > >The MSDN documentation for CreateFile does not list
> > >FILE_FLAG_NO_BUFFERING as a dwFlagsAndAttributes flag.
> > >That’s kinda bogus.
> >
> > The online version of the MSDN library seems to list
FILE_FLAG_NO_BUFFERING
> > under the CreateFile function reference. Are you saying it’s not in
> > the
CD
> > version?
> >
> > - Jan
> >
> > —
> > You are currently subscribed to ntdev as: xxxxx@aja.com
> > To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@acm.org
> To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com
>


You are currently subscribed to ntdev as: xxxxx@stratus.com To
unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntdev as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com

…which is the nicest thing about VisualStudio.Net – the ease of installing
just the dox for the stuff you use.

I mean, who *really* needs to see all those articles about FoxPro or Access
anyway ? :wink:

Regards,

Paul Bunn, UltraBac.com, 425-644-6000
Microsoft MVP - WindowsNT/2000
http://www.ultrabac.com

-----Original Message-----
From: Roddy, Mark [mailto:xxxxx@stratus.com]
Sent: Monday, July 30, 2001 9:32 AM
To: NT Developers Interest List
Subject: [ntdev] Re: 64 Bit Raid to 32 bit PCI Target Card.

Which would be about the 10,000,000 time somebody got snookered by the wince
docs. Perhaps msoft could make that stuff OPTIONAL?


You are currently subscribed to ntdev as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com

If you define a custom subset, which doesn’t include anything you don’t
need, and make that the default, you won’t get those bogus hits. I made
mine by starting with a blank one, then adding all the MSVC++ and PSDK docs.
I haven’t found it missing anything I needed yet. After looking though the
msdn130.col file, and playing just a bit with a copy of it, I think it is
possible to add the docs from the Win2K and/or the WinXP DDKs, though I
haven’t tried very hard, and my initial efforts weren’t successful.

-----Original Message-----
From: Roddy, Mark [mailto:xxxxx@stratus.com]
Sent: Monday, July 30, 2001 9:32 AM
To: NT Developers Interest List
Subject: [ntdev] Re: 64 Bit Raid to 32 bit PCI Target Card.

Which would be about the 10,000,000 time somebody got snookered by the wince
docs. Perhaps msoft could make that stuff OPTIONAL?

-----Original Message-----
From: Don Burn [mailto:xxxxx@acm.org]
Sent: Monday, July 30, 2001 12:20 PM
To: NT Developers Interest List
Subject: [ntdev] Re: 64 Bit Raid to 32 bit PCI Target Card.

It is documented in the January MSDN CD, I suspect you
looked up CreateFile and got the Windows CE version,
that does not have the flag.

Don Burn
Windows 2000 Device Driver and Filesystem consulting

----- Original Message -----
From: “Bill Bowen”
To: “NT Developers Interest List”
Sent: Monday, July 30, 2001 12:09 PM
Subject: [ntdev] Re: 64 Bit Raid to 32 bit PCI Target Card.

> I have the January 2001 MSDN and it does not list
> the flag. I will no not to rely on that in the future.
>
> After becoming hip to the flag, I see that is documented
> all over the place…except the CD MSDN.
>
> Jan Bottorff wrote:
>
> > >The MSDN documentation for CreateFile does not list
> > >FILE_FLAG_NO_BUFFERING as a dwFlagsAndAttributes flag.
> > >That’s kinda bogus.
> >
> > The online version of the MSDN library seems to list
FILE_FLAG_NO_BUFFERING
> > under the CreateFile function reference. Are you saying it’s not in
> > the
CD
> > version?
> >
> > - Jan
> >
> > —
> > You are currently subscribed to ntdev as: xxxxx@aja.com
> > To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@acm.org
> To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com
>


You are currently subscribed to ntdev as: xxxxx@stratus.com To
unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntdev as: xxxxx@intel.com
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntdev as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com

Thanks.
Man that is confusing. Who’da thought the “API Reference” would
only apply to Windows CE. How about suggesting that is bogus:).
I removed the CE documentation so I won’t make that mistake again.

xxxxx@3com.com wrote:

>>I have the January 2001 MSDN and it does not list
>>the flag. I will no not to rely on that in the future.

I have the January 2001 MSDN too. You might be looking at wrong place “API
Reference”, that’s for WinCE. The “Files and IO: Platform SDK” is the right
place for Win32.

Michael


You are currently subscribed to ntdev as: xxxxx@aja.com
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntdev as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com

I’m affraid it is more complicated. HH examines both .COL and systemwide
hhcolreg.dat file. It is probably possible to add DDK docs to MSDN
collection this way but you’d have to change hhcolreg.dat also. BTW, good
programmers editors don’t have this problem, you can use any help collection
you want. I never understood why is VC so limited. Maybe I’m missing
something and there is some easy way how to deal with collections and their
registration… has anybody some info?

Best regards,

Michal Vodicka
Veridicom
(RKK - Skytale)
[WWW: http://www.veridicom.com , http://www.skytale.com]


From: Barila, Phil[SMTP:xxxxx@intel.com]
Reply To: NT Developers Interest List
Sent: Monday, July 30, 2001 6:50 PM
To: NT Developers Interest List
Subject: [ntdev] Re: 64 Bit Raid to 32 bit PCI Target Card.

If you define a custom subset, which doesn’t include anything you don’t
need, and make that the default, you won’t get those bogus hits. I made
mine by starting with a blank one, then adding all the MSVC++ and PSDK
docs.
I haven’t found it missing anything I needed yet. After looking though
the
msdn130.col file, and playing just a bit with a copy of it, I think it is
possible to add the docs from the Win2K and/or the WinXP DDKs, though I
haven’t tried very hard, and my initial efforts weren’t successful.


You are currently subscribed to ntdev as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com