SCSIPORT context?

Gary, you aren’t tampering the dev_ext, are you?

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Gary G. Little
Sent: Wednesday, June 09, 2010 10:14 AM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] SCSIPORT context?

From having spent quite a bit of time stepping into SCSIPORT 10 years ago I
do remember there being a bit that is set when within SCSIPORT and not set
when calling from elsewhere. The operative word there being “REMEMBER” since
you are aware of where I was what I have done since then, so my memory could
have faded with age. During the time I was looking and experimenting I
recall seeing the SCSIPORT functions testing for that “context” and then
simply exiting with whatever would constitute a valid return with error
condition. Hence the only way to get in “legally” was via the timer or ISR,
but the ISR was already held by SCSIPORT.

During that time I did find another way which required a left-handed use of
WDM in DriverEntry to acquire the DeviceObject that held the DPC and
attaching a DO created in this left handed WDM side. Overall that worked
except it required some weird NULL handling in the front of the left-handed
DPC handler, but I was able to handle IO in real time instead of from the
timer.

So yeah, I would say that there really is a “context” in SCSIPORT. I think I
covered this in an article I posted here sometime around 1998 to 2000. As I
said it’s been awhile, though, so if Doran or Max or Peter pop in and say
“WRONGO!” just write it off to gray hair and Swiss cheese memory. :slight_smile:

Gary G. Little

H (952) 223-1349

C (952) 454-4629

xxxxx@comcast.net

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Philip D Barila
Sent: Wednesday, June 09, 2010 11:39 AM
To: Windows System Software Devs Interest List
Subject: [ntdev] SCSIPORT context?

I’ve waffled whether this belongs on NTDEV or NTTALK, since I’m not
currently dealing with this. I decided to put it on NTDEV because it’s a
search for real understanding of a real problem real devs might run into.
List-Slaves, please relocate it if you think otherwise.

The recent discussion of virtual SCSI miniport brought up something I’ve
never quite understood about the well documented performance issues of a
virtual SCSI miniport, namely that any calls from the miniport to SCSIPORT
need to be “in the SCSIPORT context”, or something like that, with the
common solution being to use the SCSIPORT timer, which means you’re going to
incur approximately 10 ms of latency. What I don’t understand is the
definition of “the SCSIPORT context”? Does that mean any call to SCSIPORT
from the miniport must only be in response to a call from SCSIPORT? (i.e.
Any call from miniport to port must have port already in the stack?) Or
does it mean something else? Is it a thread context?

I guess my confusion stems from the “usual” meaning of context in the
kernel. It is unfortunate that the term context is so overloaded as to
offer more confusion than precision at times, when the definition is not
precisely understood. Maybe my confusion is the nature of kernel context?
Is it correct that the kernel address space is a flat address space shared
by all kernel components, and when the “context” is merely that address
space plus the current thread stack, and when the scheduler switches to a
new thread inside the kernel, it’s really just a stack swap? If so, what’s
special about the “SCSIPORT context”? SCSIPORT has to be in that stack
somewhere already before you can call into SCSIPORT from the miniport (as I
said above)?

Whatever the definition of “the SCSIPORT context”, is anyone willing to say
why it is that way?

Thanks,

Phil

Sent from Outlook through exchange using a keyboard, and a spellchecker.
Any mistakes are purely between the operator’s head and the keys.

Philip D. Barila


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

Not at all, I used my own device extension. I do know I created a new device
object and DPC and used it to swap with the DO in SCSIPORTS driver object.
At the end of my left-handed DPC I then invoked SCSIPORTs DO, but I’s have
to dig out the code, if I still have it, blow an inch of dust off of it, and
review it before I get to much more specific.

And there is no “aren’t” to it. I am currently gainfully employed by myself
and working on a Windows Filtering Platform driver.

Gary G. Little

H (952) 223-1349

C (952) 454-4629

xxxxx@comcast.net

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Calvin Guan
Sent: Wednesday, June 09, 2010 12:35 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] SCSIPORT context?

Gary, you aren’t tampering the dev_ext, are you?

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Gary G. Little
Sent: Wednesday, June 09, 2010 10:14 AM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] SCSIPORT context?

From having spent quite a bit of time stepping into SCSIPORT 10 years ago I
do remember there being a bit that is set when within SCSIPORT and not set
when calling from elsewhere. The operative word there being “REMEMBER” since
you are aware of where I was what I have done since then, so my memory could
have faded with age. During the time I was looking and experimenting I
recall seeing the SCSIPORT functions testing for that “context” and then
simply exiting with whatever would constitute a valid return with error
condition. Hence the only way to get in “legally” was via the timer or ISR,
but the ISR was already held by SCSIPORT.

During that time I did find another way which required a left-handed use of
WDM in DriverEntry to acquire the DeviceObject that held the DPC and
attaching a DO created in this left handed WDM side. Overall that worked
except it required some weird NULL handling in the front of the left-handed
DPC handler, but I was able to handle IO in real time instead of from the
timer.

So yeah, I would say that there really is a “context” in SCSIPORT. I think I
covered this in an article I posted here sometime around 1998 to 2000. As I
said it’s been awhile, though, so if Doran or Max or Peter pop in and say
“WRONGO!” just write it off to gray hair and Swiss cheese memory. :slight_smile:

Gary G. Little

H (952) 223-1349

C (952) 454-4629

xxxxx@comcast.net

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Philip D Barila
Sent: Wednesday, June 09, 2010 11:39 AM
To: Windows System Software Devs Interest List
Subject: [ntdev] SCSIPORT context?

I’ve waffled whether this belongs on NTDEV or NTTALK, since I’m not
currently dealing with this. I decided to put it on NTDEV because it’s a
search for real understanding of a real problem real devs might run into.
List-Slaves, please relocate it if you think otherwise.

The recent discussion of virtual SCSI miniport brought up something I’ve
never quite understood about the well documented performance issues of a
virtual SCSI miniport, namely that any calls from the miniport to SCSIPORT
need to be “in the SCSIPORT context”, or something like that, with the
common solution being to use the SCSIPORT timer, which means you’re going to
incur approximately 10 ms of latency. What I don’t understand is the
definition of “the SCSIPORT context”? Does that mean any call to SCSIPORT
from the miniport must only be in response to a call from SCSIPORT? (i.e.
Any call from miniport to port must have port already in the stack?) Or
does it mean something else? Is it a thread context?

I guess my confusion stems from the “usual” meaning of context in the
kernel. It is unfortunate that the term context is so overloaded as to
offer more confusion than precision at times, when the definition is not
precisely understood. Maybe my confusion is the nature of kernel context?
Is it correct that the kernel address space is a flat address space shared
by all kernel components, and when the “context” is merely that address
space plus the current thread stack, and when the scheduler switches to a
new thread inside the kernel, it’s really just a stack swap? If so, what’s
special about the “SCSIPORT context”? SCSIPORT has to be in that stack
somewhere already before you can call into SCSIPORT from the miniport (as I
said above)?

Whatever the definition of “the SCSIPORT context”, is anyone willing to say
why it is that way?

Thanks,

Phil

Sent from Outlook through exchange using a keyboard, and a spellchecker.
Any mistakes are purely between the operator’s head and the keys.

Philip D. Barila


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

Correct. OsrVirtualCdAndDisk (the one available in executable format as a download, NOT the Virtual StorPort sample that accompanies the recent series of articles in The NT Insider) is a full port driver.

Peter
OSR

> -----Original Message-----

From: xxxxx@lists.osr.com [mailto:bounce-414157-
xxxxx@lists.osr.com] On Behalf Of Calvin Guan
Sent: Wednesday, June 09, 2010 11:33 AM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] SCSIPORT context?

[snip]

> with the common solution being to use the SCSIPORT timer, which means
you’re
> going to incur approximately 10 ms of latency.

There are least 2 engineering solutions for this problem without resorting
to the 10ms timer.

Calvin

Calvin,

Thanks for the response. Care to suggest how either, or both of them, might
be done? It’s for curiosity’s sake only, but it would be appreciated.

Philip D. Barila

Gary,

Thanks for groveling through the dusty bits of memory. J

Phil

Philip D. Barila

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Gary G. Little
Sent: Wednesday, June 09, 2010 11:14 AM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] SCSIPORT context?

So yeah, I would say that there really is a “context” in SCSIPORT. I think I
covered this in an article I posted here sometime around 1998 to 2000. As I
said it’s been awhile, though, so if Doran or Max or Peter pop in and say
“WRONGO!” just write it off to gray hair and Swiss cheese memory. :slight_smile:

Gary G. Little

H (952) 223-1349

C (952) 454-4629

xxxxx@comcast.net

> Thanks for the response. Care to suggest how either,

or both of them, might be done? It’s for curiosity’s sake
only, but it would be appreciated.

Not that because I don’t want to, it ain’t rocket science after all. My “code of conduct” prohibits me from saying anything closely related to the work performed during my employment w/o a written approval from my legal department and signed by executive officers.

I certainly wouldn’t want you to put your employment at risk.

Phil

Philip D. Barila

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@yahoo.ca
Sent: Wednesday, June 09, 2010 1:54 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] SCSIPORT context?

Thanks for the response. Care to suggest how either,
or both of them, might be done? It’s for curiosity’s sake
only, but it would be appreciated.

Not that because I don’t want to, it ain’t rocket science after all. My
“code of conduct” prohibits me from saying anything closely related to the
work performed during my employment w/o a written approval from my legal
department and signed by executive officers.


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

How about sending an IOCTL_MINIPORT down through the stack?

“Philip D Barila”
Sent by: xxxxx@lists.osr.com
06/09/2010 02:32 PM
Please respond to
“Windows System Software Devs Interest List”

To
“Windows System Software Devs Interest List”
cc

Subject
RE: [ntdev] SCSIPORT context?

> -----Original Message-----
> From: xxxxx@lists.osr.com [mailto:bounce-414157-
> xxxxx@lists.osr.com] On Behalf Of Calvin Guan
> Sent: Wednesday, June 09, 2010 11:33 AM
> To: Windows System Software Devs Interest List
> Subject: RE: [ntdev] SCSIPORT context?

[snip]

> > with the common solution being to use the SCSIPORT timer, which means
> you’re
> > going to incur approximately 10 ms of latency.
>
> There are least 2 engineering solutions for this problem without
resorting
> to the 10ms timer.
>
> Calvin

Calvin,

Thanks for the response. Care to suggest how either, or both of them,
might
be done? It’s for curiosity’s sake only, but it would be appreciated.

Philip D. Barila


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

Hmmm, can you nod your head up and down if we happen to get close?

Gary G. Little
H (952) 223-1349
C (952) 454-4629
xxxxx@comcast.net

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@yahoo.ca
Sent: Wednesday, June 09, 2010 2:54 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] SCSIPORT context?

Thanks for the response. Care to suggest how either, or both of them,
might be done? It’s for curiosity’s sake only, but it would be
appreciated.

Not that because I don’t want to, it ain’t rocket science after all. My
“code of conduct” prohibits me from saying anything closely related to the
work performed during my employment w/o a written approval from my legal
department and signed by executive officers.


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

Given you’re writing the miniport so you know how to set up the control
packet, that REALLY bounces up the overhead. I tried that while I was
working on the issue and found it unacceptable. Not only is performance to
slow your limited to the buffer sizes allowed by IOCTL_SCSI_MINIPORT. Plus,
that control block is a header that is affixed to the buffer that contains
your data, so it becomes a whose pointing at whom mishmash god awful
internal links.

But . is Calvin nodding or shaking his head? J

Gary G. Little

H (952) 223-1349

C (952) 454-4629

xxxxx@comcast.net

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@attotech.com
Sent: Wednesday, June 09, 2010 3:29 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] SCSIPORT context?

How about sending an IOCTL_MINIPORT down through the stack?

“Philip D Barila”
Sent by: xxxxx@lists.osr.com

06/09/2010 02:32 PM

Please respond to
“Windows System Software Devs Interest List”

To

“Windows System Software Devs Interest List”

cc

Subject

RE: [ntdev] SCSIPORT context?

> -----Original Message-----
> From: xxxxx@lists.osr.com [mailto:bounce-414157-
> xxxxx@lists.osr.com] On Behalf Of Calvin Guan
> Sent: Wednesday, June 09, 2010 11:33 AM
> To: Windows System Software Devs Interest List
> Subject: RE: [ntdev] SCSIPORT context?

[snip]

> > with the common solution being to use the SCSIPORT timer, which means
> you’re
> > going to incur approximately 10 ms of latency.
>
> There are least 2 engineering solutions for this problem without resorting
> to the 10ms timer.
>
> Calvin

Calvin,

Thanks for the response. Care to suggest how either, or both of them, might
be done? It’s for curiosity’s sake only, but it would be appreciated.

Philip D. Barila


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

— NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and
other seminars visit: http://www.osr.com/seminars To unsubscribe, visit the
List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

No, that’s not what I was suggesting.

Let’s say I have SRBs that I want to complete when I’m not within the
context of the port driver. So I put them on a linked list in the
adapter’s Extension. Then I cause a small IOCTL_MINIPORT to be sent down
the stack. Miniport gets the IOCTL, now in the context of the port
driver, cleans off the linked list and calls ScsiPortNotification for
each. We’re not transferring any data here with the IOCTL at all, we’re
just using it as a “signal” to the miniport to complete the requests.

There are probably several variations on this kind of thing; doesn’t have
to be an IOCTL that’s used to signal - could be a special SCSI command
with no data buffer that completes immediately in HwStartIo after posting
the completions.

“Gary G. Little”
Sent by: xxxxx@lists.osr.com
06/09/2010 05:16 PM
Please respond to
“Windows System Software Devs Interest List”

To
“Windows System Software Devs Interest List”
cc

Subject
RE: [ntdev] SCSIPORT context?

Given you?re writing the miniport so you know how to set up the control
packet, that REALLY bounces up the overhead. I tried that while I was
working on the issue and found it unacceptable. Not only is performance to
slow your limited to the buffer sizes allowed by IOCTL_SCSI_MINIPORT.
Plus, that control block is a header that is affixed to the buffer that
contains your data, so it becomes a whose pointing at whom mishmash god
awful internal links.

But ? is Calvin nodding or shaking his head? J

Gary G. Little
H (952) 223-1349
C (952) 454-4629
xxxxx@comcast.net

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@attotech.com
Sent: Wednesday, June 09, 2010 3:29 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] SCSIPORT context?

How about sending an IOCTL_MINIPORT down through the stack?

“Philip D Barila”
Sent by: xxxxx@lists.osr.com
06/09/2010 02:32 PM

Please respond to
“Windows System Software Devs Interest List”

To
“Windows System Software Devs Interest List”
cc

Subject
RE: [ntdev] SCSIPORT context?

> -----Original Message-----
> From: xxxxx@lists.osr.com [mailto:bounce-414157-
> xxxxx@lists.osr.com] On Behalf Of Calvin Guan
> Sent: Wednesday, June 09, 2010 11:33 AM
> To: Windows System Software Devs Interest List
> Subject: RE: [ntdev] SCSIPORT context?

[snip]

> > with the common solution being to use the SCSIPORT timer, which means
> you’re
> > going to incur approximately 10 ms of latency.
>
> There are least 2 engineering solutions for this problem without
resorting
> to the 10ms timer.
>
> Calvin

Calvin,

Thanks for the response. Care to suggest how either, or both of them,
might
be done? It’s for curiosity’s sake only, but it would be appreciated.

Philip D. Barila


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

— NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and
other seminars visit: http://www.osr.com/seminars To unsubscribe, visit
the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

>No, that’s not what I was suggesting. Let’s say I have SRBs that I want to complete when I’m not >within the context of the port driver. So I put them on a linked list in the adapter’s Extension. Then I >cause a small IOCTL_MINIPORT to be sent down=20 the stack. Miniport gets the IOCTL, now in the >context of the port driver, cleans off the linked list and calls ScsiPortNotification for each. We’re not >transferring any data here with the IOCTL at all, we’re just using it as a “signal” to the miniport to >complete the requests. There are probably several variations on this kind of thing; doesn’t have to be >an IOCTL that’s used to signal - could be a special SCSI command with no data buffer that >

completes immediately in HwStartIo after posting the completions.

Could such schema decrease performance? I see some overhead in this design compare to the regular SCSIPort-Miniport path.

Igor Sharovar

It’s WAY WAY better than waiting for a timer tick to complete requests,
don’t you think?

xxxxx@hotmail.com
Sent by: xxxxx@lists.osr.com
06/09/2010 06:08 PM
Please respond to
“Windows System Software Devs Interest List”

To
“Windows System Software Devs Interest List”
cc

Subject
RE:[ntdev] SCSIPORT context?

>No, that’s not what I was suggesting. Let’s say I have SRBs that I want
to complete when I’m not >within the context of the port driver. So I put
them on a linked list in the adapter’s Extension. Then I >cause a small
IOCTL_MINIPORT to be sent down=20 the stack. Miniport gets the IOCTL, now
in the >context of the port driver, cleans off the linked list and calls
ScsiPortNotification for each. We’re not >transferring any data here with
the IOCTL at all, we’re just using it as a “signal” to the miniport to
>complete the requests. There are probably several variations on this kind
of thing; doesn’t have to be >an IOCTL that’s used to signal - could be a
special SCSI command with no data buffer that >
>completes immediately in HwStartIo after posting the completions.

Could such schema decrease performance? I see some overhead in this design
compare to the regular SCSIPort-Miniport path.

Igor Sharovar


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

> some reasons love it and keep it that way until introduction of iscsiprt and

new storport.

I think msiscsi is storport’s virtual miniport. So is storvsc (Hyper-V’s guest virtual SCSI).


Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

> Could such schema decrease performance?

Not much I think. At least it is by far better then timers.

The flow:

  • usual storage command arrives to miniport
  • it is pended there in this list, since you cannot call anything in the kernel in the miniport (i.e. interrupt spinlock) context
  • the special “get command” SCSI command arrives (sent by a second driver).
  • it is completed immediately by the miniport, and returns the pointer to one of the pending commands
  • the second driver executes the command in the usual kernel context
  • the second driver sends the special “complete command” SCSI command to the miniport
  • the miniport completes both this special command and the main command which is pended inside


Maxim S. Shatskih

Windows DDK MVP

xxxxx@storagecraft.com
http://www.storagecraft.com

>> some reasons love it and keep it that way until introduction of iscsiprt and

> new storport.

I think msiscsi is storport’s virtual miniport.
So is storvsc (Hyper-V’s guest virtual SCSI).

IIRC iscsiprt.sys is an interim solution when storport was not able to
support virtual miniport. I don’t know if it has ever made it
to production. They merged back to storport later.

I guess they finally realized how cripple the SCSIPORT was and why outside programmers
need something better than the scsi miniport until
they have to support iscsi.

Yet they refused to back port storport to WXP which forces us who have
to battle with WXP support to mess with the scsiport. Yes, there are elegant
solutions but I feel sorry for burning raw intellengence
to deal with the nonsense and dying technology.

Calvin

>It’s WAY WAY better than waiting for a timer tick to complete requests, don’t you think?
I missed that you compare this approach with timer completion.

Igor Sharovar

The real unfortunate part about this is all microsoft had to do is a simple and transparent code change in a scsiport update that when it gets a RequestTimerCall with a delay value of 0 is to act on it immediately rather than waiting another 10ms. Would have saved about 100 discussion threads and who knows how many drivers. But the way it stands is the timer method is not a real solution and there is just no way that could ever be an acceptable product so just don’t go there. There are a number of efficient algorithms to get back on scsiport context, though some are lesser known and proprietary. I have seen a very elegant one not yet discussed.

Scsiport will continue to play a key role for future drivers as XP is the market today and forecasts say it will be a huge part of the market for years to come.

hi
long time the OsrVirtualCdAndDisk project live as executable. may be now time for put the source at github?

Post to TEN year old thread. Locked.

If you have something to say, say it in a new thread please.

Peter