Supposing a CPU you are about to disable like this is already spinning
waiting for an event that it expects to be done by another CPU that you’ve
just disabled. For example two apps doing the process suggested here at the
same time are likely to deadlock the system aren’t they? Shouldn’t there be
a law against this?. … Mike
Basically what you have to do is to turn your MP system into an UP one for
the duration of your operation. Before you start your operation, you create
system threads that run on all CPUs (apart from the one that does an
operation, of course) that disable interrupts and spin in do-nothing while
loop until, by setting a global flag, you indicate them that they can
break out. At this point they re-enable interrupts on CPUs they run on, and
terminate themselves…
Yes sure you can safely corral all the processors and read the
standard config space using CF8, it is just a horrible idea unless
there are no other viable options.
On Jan 17, 2008 10:52 AM, wrote:
> Don,
>
> > NO THIS DOES NOT WORK.
>
> Believe me or not, but it does - you will see it shortly…
>
> > YOU ARE STILL NOT PROTECTING AGAINST
> > THE OS HAVING WRITTEN CF8 BUT THEN NOT ACCESSES CFC.
>
> Sure you do …
>
> Which way??? Let’s think logically. The OS synchronizes an access to IO ports 0xCF8 and 0xCFC with a spinlock. It is understandable that both write to 0xCF8 and read from 0xCFC are made by the OS while spinlock in question is being held - otherwise, it would just defeat the purpose of protecting them with a spinlock, in the first place.
>
> If you think about it very, very carefully, you are going to realize that, by the moment you start your IO access (I assume you have elevated IRQL to DPC level before you proceed to actual IO operation), no one in the entire system can be a spinlock holder. This applies not only the spinlock we are interested it - not a single spinlock may be held in the system by anyone at this moment. Therefore, all operation on the target IO ports have been completed, and the new ones cannot start until you complete yours - you are the only owner of the target resource at the moment…
>
>
> Anton Bassov
>
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
>
–
Mark Roddy
Don,
Except you cannot rely on it being a spinlock, IIRC it was not always.
The quote below is taken from MSD description of IRP_MN_READ_CONFIG
[begin quote]
A driver can access a bus’s configuration space at DISPATCH_LEVEL through a bus interface routine,…
[end quote]
As you can see, it just has to be a spinlock - I really don’t know what you are arguing about…
Sorry people who do crap like this in anything close to a production driver
should have their fingers broken so thay cannot use a keyboard again.
This is already a different story - in practical terms, what is the point of going to go for so convoluted solution at the time when straightforward one is available??? However, no matter how stupid, ugly, inefficient and unreasonable the thing that I have described is, it is still workable one…
Anton Bassov
Actually, that is still an assumption, for instance it could be an internal
lock at higher IRQL etc. Making assumptions based on descriptions or on
disassembly (and assuming things will stay that way) is almost always a road
to disaster.
–
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr
Remove StopSpam to reply
wrote in message news:xxxxx@ntdev…
> Don,
>
>> Except you cannot rely on it being a spinlock, IIRC it was not always.
>
> The quote below is taken from MSD description of IRP_MN_READ_CONFIG
>
> [begin quote]
>
> A driver can access a bus’s configuration space at DISPATCH_LEVEL through
> a bus interface routine,…
>
> [end quote]
>
> As you can see, it just has to be a spinlock - I really don’t know what
> you are arguing about…
>
>> Sorry people who do crap like this in anything close to a production
>> driver
>> should have their fingers broken so thay cannot use a keyboard again.
>
> This is already a different story - in practical terms, what is the point
> of going to go for so convoluted solution at the time when straightforward
> one is available??? However, no matter how stupid, ugly, inefficient and
> unreasonable the thing that I have described is, it is still workable
> one…
>
> Anton Bassov
>
xxxxx@hotmail.com wrote:
If you think about it very, very carefully, you are going to realize that, by the moment you start your IO access (I assume you have elevated IRQL to DPC level before you proceed to actual IO operation), no one in the entire system can be a spinlock holder. This applies not only the spinlock we are interested it - not a *single* spinlock may be held in the system by anyone at this moment.
Why do you think so? If we could gain access to the operating system’s
spinlock for these registers, then it could be made safe. But I don’t
see why you think I can’t sneak in on processor 3 while processor 0
holds the CF8/CFC spinlock.
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
> Not really
As Don mentioned we can not be sure our driver did not interrupt another
driver
after it’s writing to 0cf8
With 100% probability - no.
But, if all MS-provided code touches 0xcf8/cfc only at DISPATCH_LEVEL (not from
ISRs) - the CPU corraling helps 100%.
–
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com
It is safe only if the MS-provided code base never touches the registers at
DISPATCH.
Otherwise, Don is correct.
–
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com
“Mark Roddy” wrote in message news:xxxxx@ntdev…
> Yes sure you can safely corral all the processors and read the
> standard config space using CF8, it is just a horrible idea unless
> there are no other viable options.
>
> On Jan 17, 2008 10:52 AM, wrote:
> > Don,
> >
> > > NO THIS DOES NOT WORK.
> >
> > Believe me or not, but it does - you will see it shortly…
> >
> > > YOU ARE STILL NOT PROTECTING AGAINST
> > > THE OS HAVING WRITTEN CF8 BUT THEN NOT ACCESSES CFC.
> >
> > Sure you do …
> >
> > Which way??? Let’s think logically. The OS synchronizes an access to IO
ports 0xCF8 and 0xCFC with a spinlock. It is understandable that both write to
0xCF8 and read from 0xCFC are made by the OS while spinlock in question is
being held - otherwise, it would just defeat the purpose of protecting them
with a spinlock, in the first place.
> >
> > If you think about it very, very carefully, you are going to realize that,
by the moment you start your IO access (I assume you have elevated IRQL to DPC
level before you proceed to actual IO operation), no one in the entire system
can be a spinlock holder. This applies not only the spinlock we are interested
it - not a single spinlock may be held in the system by anyone at this
moment. Therefore, all operation on the target IO ports have been completed,
and the new ones cannot start until you complete yours - you are the only owner
of the target resource at the moment…
> >
> >
> > Anton Bassov
> >
> >
> >
> > —
> > NTDEV is sponsored by OSR
> >
> > For our schedule of WDF, WDM, debugging and other seminars visit:
> > http://www.osr.com/seminars
> >
> > To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
> >
>
>
>
> –
> Mark Roddy
>
By the way.
What about to use Enhanced Configuration Mechanism for PCI Express for reading ?
In general this technique should be limited to the OS (which
appropriates IPI for this) and platform vendor specific system
management software (which might also do something similar using
management interrupts.) Advocating that random device drivers adopt
this technique would be a really bad idea, and I do not think that
Anton was doing this, he was simply pointing out that it can be done.
The problem is that readers of this list might get the idea that their
random non-platform specific driver that they think just ought to be
able to read config space for some reason using CF8 should now proceed
to do so by writing a processor corral, the implementation of which is
likely to be flawed. So in addition to a bogus CF8 reading driver we
get a bogus CF8 reading driver with a crappy deadlock provoking
processor corral. With any luck at all it is also a root kit 
Between the deprecated HalGetBusData and the documented IRP and Bus
Interface mechanisms I don’t quite understand why any driver needs to
use CF8.
It is possible to write a deadlock safe processor corral.
On Jan 17, 2008 12:56 PM, Mike Kemp wrote:
> Supposing a CPU you are about to disable like this is already spinning
> waiting for an event that it expects to be done by another CPU that you’ve
> just disabled. For example two apps doing the process suggested here at the
> same time are likely to deadlock the system aren’t they? Shouldn’t there be
> a law against this?. … Mike
>
> >Basically what you have to do is to turn your MP system into an UP one for
> >the duration of your operation. Before you start your operation, you create
> >system threads that run on all CPUs (apart from the one that does an
> >operation, of course) that disable interrupts and spin in do-nothing while
> >loop until, by setting a global flag, you indicate them that they can
> >break out. At this point they re-enable interrupts on CPUs they run on, and
> >terminate themselves…
>
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
>
–
Mark Roddy
Tim,
But I don’t see why you think I can’t sneak in on processor 3 while processor
0 holds the CF8/CFC spinlock.
Because at this moment processor 3 spins in do-nothing loop with interrupts disabled. .Look at preudo-code below (I assume we are going to access IO space on the CPU with local APIC’s ID of 0) :
ULONG CanBreakOut, NumberOfLockedCPUs;
SpinThreadRoutine(…)
{
//disable interrupts
KeRaiseIrql(HIGH_LEVEL,&oldIrql);
if (get_ local_ APIC’s_ ID() )
{
NumberOfLockedCPUs++;
while( !CanBreakOut){}
}
//re-enable interrupts
KeLowerIrql(oldIrql);
PsTerminateSystemThread(STATUS_SUCCESS);
}
in the code:
CanBreakOut=0; NumberOfLockedCPUs=0;
while(NumberOfLockedCPUs{
PsCreateSystemThread(…, SpinThreadRoutine,…)
}
By the time the above loop completes (it may take a while) all CPUs, apart from the one with APIC ID of 0, are out of play - they all are spinning in do-nothing loop with interrupts disabled, so that, in practical terms, CPU with APIC ID of 0 is a single functional CPU on the machine.
In other words, we have turned MP machine into UP one. It is understandable that all CPU
have released all spinlocks they could have held by the time they have disabled interrupts by write to TPR - our thread start execution at the PASSIVE level.
At this point we can do the following (we run at the PASSIVE_LEVEL as well):
//disable interrupts
KeRaiseIrql(HIGH_LEVEL,&oldIrql);
At the moment we wrote to TPR IRQL was PASSIVE_LEVEL, so that no spinlock could have been held. Once we have wrote to TPR, our code cannot get interrupted by anyone
( in fact, I don’t think 0xCF8-0xCFC can be accessed in context of ISR, so that raising IRQL to DPC level would suffice here, but just to be 100% sure, let’s disable interrupts completely). Therefore, we can proceed to the actual operation.
access_ configuration_space();
//signal all CPUs that operation is complete, so that they can break out
CanBreakOut=1;
//re-enable interrupts
KeLowerIrql(oldIrql);
No matter how convoluted and inefficient the above code is, it is absolutely reliable…
Anton Bassov
Don,
Actually, that is still an assumption, for instance it could be an internal lock at higher IRQL etc.
So what??? Look at the code in my previous post, and you will see that this is not a problem at all. The only case when problem may arise is if configuration space is protected not by a spinlock but by some construct that allows preemption. According to MSDN quotation that I have provided this is not the case here…
Anton Bassov
Mark,
I do not think that Anton was doing this, he was simply pointing out that it can be done.
Correct - the above code demonstrates how convoluted and inefficient the whole thing is
(although it still does work).
Advocating that random device drivers adopt this technique would be a really bad idea,
Well, I think my sample code will convince any more-or-less reasonable newbie not to do it at the time when documented and straightforward solution is available…
Anton Bassov
“Maxim S. Shatskih” wrote in message news:xxxxx@ntdev…
> “Corraling” all CPUs in the same predictable loop within a DPC - as Anton
> mentioned - is a help.
With a DPC, you say? what if a random device interrupt comes then (at DIRQL)
and breaks into your dispatch level “corral”?
However… it will work most of the time
–PA
> With a DPC, you say? what if a random device interrupt comes then (at DIRQL)
and breaks into your dispatch level “corral”?
However… it will work *most of the time*
If the interrupt will never touch 0xcf8/cfc - then this is OK
Also, you can raise to HIGH_LEVEL in the corral DPC or execute “cli”
instruction.
–
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com
“Don Burn” wrote in message news:xxxxx@ntdev…
> Actually, that is still an assumption, for instance it could be an internal lock at higher IRQL etc. Making assumptions based
> on descriptions or on disassembly (and assuming things will stay that way) is almost always a road to disaster.
>
With all due respect… it seems that disasters become just
too common these days and this word doesn’t scare people too much…
Everybody make up disaster recovery plans and just live with it.
One more, one less.
We’re seeing that disasters hit enterprises that have never
touched any kernel development at all.
Regards,
–PA
Ah, ok. noticed Anton’s replies about high irql.
Regards,
–PA
“Maxim S. Shatskih” wrote in message news:xxxxx@ntdev…
>> With a DPC, you say? what if a random device interrupt comes then (at DIRQL)
>> and breaks into your dispatch level “corral”?
>>
>> However… it will work most of the time
>
> If the interrupt will never touch 0xcf8/cfc - then this is OK
>
> Also, you can raise to HIGH_LEVEL in the corral DPC or execute “cli”
> instruction.
>
> –
> Maxim Shatskih, Windows DDK MVP
> StorageCraft Corporation
> xxxxx@storagecraft.com
> http://www.storagecraft.com
>
>
Hello Tim R.
I want to know how i can configure my driver for such a pci based card
means reading configuration space, getting information about availability of
memory windows, configuring ISR,IRQL.
again i am confused about bus relative address of PCI bus which is parameter
for
*HalGetInterruptVector API *
Thanks
Regards,
Pravin G.
On Jan 17, 2008 11:03 PM, Tim Roberts wrote:
> pravin gawale wrote:
> >
> > I want to configure PCI based device (card) from device driver
> > so i want know about PCI bus enumeration and corresponding API
> > available for driver development
>
> You need to tell us more about what you want to do. The operating
> system’s handling of PCI devices through the normal plug-and-play system
> is very well documented, and backed up through numerous examples in the
> DDK. Tell us exactly what is confusing you, and we can clear that up.
>
> –
> Tim Roberts, xxxxx@probo.com
> Providenza & Boekelheide, Inc.
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
First, what type of device is this ? a NIC? A storage adaptor? Knowing the type will help in recommending what type of driver you should write. I will assume it is not a NIC or storage device, in which case you need to write a pnp enabled driver. This can be WDM or KMDF. I would further recommend KMDF to get things going for you quickly b/c you will have much less to implement. In a pnp driver you do not use the HAL apis to get your resources, they are given to you by the pci bus driver (either in IRP_MN_START_DEVICE for WDM or EvtDevicePrepareHardware for KDMF). You then walk the list of assigned resources (which can include memory, io ports, interrupts, etc) and initialize your device. I doubt that you will have to touch config space in this scenario.
d
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of pravin gawale
Sent: Thursday, January 17, 2008 9:47 PM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] PCI based device enumeration…???
Hello Tim R.
I want to know how i can configure my driver for such a pci based card
means reading configuration space, getting information about availability of memory windows, configuring ISR,IRQL.
again i am confused about bus relative address of PCI bus which is parameter for
HalGetInterruptVector API
Thanks
Regards,
Pravin G.
On Jan 17, 2008 11:03 PM, Tim Roberts > wrote:
pravin gawale wrote:
>
> I want to configure PCI based device (card) from device driver
> so i want know about PCI bus enumeration and corresponding API
> available for driver development
You need to tell us more about what you want to do. The operating
system’s handling of PCI devices through the normal plug-and-play system
is very well documented, and backed up through numerous examples in the
DDK. Tell us exactly what is confusing you, and we can clear that up.
–
Tim Roberts, xxxxx@probo.commailto:xxxxx
Providenza & Boekelheide, Inc.
—
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
— NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and other seminars visit: http://www.osr.com/seminars To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer</mailto:xxxxx>
pravin gawale wrote:
Hello Tim R.
I want to know how i can configure my driver for such a pci based card
means reading configuration space, getting information about
availability of memory windows, configuring ISR,IRQL.
again i am confused about bus relative address of PCI bus which is
parameter for
*HalGetInterruptVector API *
Summarizing Doron’s reply just a bit, YOU don’t handle these tasks. The
operating system handles them, and hands the information to you when it
loads your driver, based on your INF. Your driver never needs to call
any of the Hal APIs.
Tell us the type of device, and we’ll point you to a sample that will
make it clear.
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.