Windows System Software -- Consulting, Training, Development -- Unique Expertise, Guaranteed Results

Before Posting...
Please check out the Community Guidelines in the Announcements and Administration Category.

Off-loading work from StartIO

Ajit_YagatyAjit_Yagaty Member Posts: 28
Hi All,

We are writing a Virtual Storport Miniport driver for a custom device. There is requirement to off-load the Read/Write SRBs from StartIO onto a driver thread. We are returning SRB_STATUS_PENDING from StartIO and we are completing the SRBs from the driver thread.

We were experimenting the above on a RAM Disk implementation. The observation was that the performance (measured using IOMeter) got halved when we started off-loading the work to a driver thread when compared to completing the requests in StartIO itself. Is this because of the fact that StartIO runs at DISPATCH LEVEL and the driver thread runs at PASSIVE LEVEL and the thread would get very little time to run?

We tried setting the thread's affinity to specific processor and even then it didn't help. Is there anything that we are missing?

Regards,
AY

Comments

  • James_HarperJames_Harper Member Posts: 1,615
    > Hi All,
    >
    > We are writing a Virtual Storport Miniport driver for a custom device.
    There
    > is requirement to off-load the Read/Write SRBs from StartIO onto a
    driver
    > thread. We are returning SRB_STATUS_PENDING from StartIO and we are
    completing
    > the SRBs from the driver thread.
    >
    > We were experimenting the above on a RAM Disk implementation. The
    observation
    > was that the performance (measured using IOMeter) got halved when we
    started
    > off-loading the work to a driver thread when compared to completing
    the
    > requests in StartIO itself. Is this because of the fact that StartIO
    runs at
    > DISPATCH LEVEL and the driver thread runs at PASSIVE LEVEL and the
    thread
    > would get very little time to run?
    >
    > We tried setting the thread's affinity to specific processor and even
    then it
    > didn't help. Is there anything that we are missing?
    >

    If it truly is a ramdisk with zero latency, why would you expect
    anything else? Instead of completing the io operation immediately you
    are waiting for your thread to be scheduled which takes huge numbers of
    clock cycles compared to simply completing the request right there and
    then.

    If you are on a single CPU system, read the TSC
    (KeQueryPerformanceCounter) at the start of your startio, do your io
    work, then read the TSC again at the end before completing and calculate
    the difference. Then try again with the driver thread and do the second
    read of the TSC when completing there. I think you'll be surprised at
    the time it can take for your thread to be scheduled.

    (If you have multiple CPU's you won't get useful results as your thread
    may be running on a different cpu when you take the second reading.)

    James
  • Ajit_YagatyAjit_Yagaty Member Posts: 28
    Hi James,

    Thanks for the response. I was using RAM-Disk just to see if there will be any performance loss incurred while off-loading work to the thread. In the final implementation we would need the thread as we have to do some more processing before performing the read/write operation.

    --AY
  • Ajit_YagatyAjit_Yagaty Member Posts: 28
    Hi again,

    I guess my previous post is a little misleading. I am using RAM-Disk to see what will be the performance loss and whether the miniport will be able to receive another request while the thread is processing the previous one. From the experiments it seems that the overlap is not happening. So I wanted to know if there is a way to achieve this.

    Thanks,
    AY
  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    If I recall you need to be on StorPort context to complete requests which won't be the case in a thread. What mechanism is being used to switch back to call StorPortNotification?
  • Ajit_YagatyAjit_Yagaty Member Posts: 28
    From the thread we are sending an IRP back to the miniport with a pointer to the SRB and in the ProcessServiceRequest function we are completing the SRB. The other question is whether completing the SRB within Storport's context is mandatory. We were able to complete it directly from the thread's context also.
  • Maxim_S._ShatskihMaxim_S._Shatskih Member Posts: 10,396
    > If I recall you need to be on StorPort context to complete requests

    Virtual miniport has no such requirement.

    --
    Maxim S. Shatskih
    Windows DDK MVP
    [email protected]
    http://www.storagecraft.com
  • Maxim_S._ShatskihMaxim_S._Shatskih Member Posts: 10,396
    >StartIO runs at DISPATCH LEVEL and the driver thread runs at PASSIVE LEVEL and the thread
    >would get very little time to run?

    Scheduling delay before the thread is really awaken? try raising its priority.

    --
    Maxim S. Shatskih
    Windows DDK MVP
    [email protected]
    http://www.storagecraft.com
  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    Did you set command depth/tag queue/multiple request per LU properly?

    -----Original Message-----
    From: [email protected] [mailto:[email protected]]
    Sent: Tuesday, June 01, 2010 6:39 PM
    To: Windows System Software Devs Interest List
    Subject: RE:[ntdev] Off-loading work from StartIO

    Hi again,

    I guess my previous post is a little misleading. I am using RAM-Disk to see what will be the performance loss and whether the miniport will be able to receive another request while the thread is processing the previous one. From the experiments it seems that the overlap is not happening. So I wanted to know if there is a way to achieve this.

    Thanks,
    AY

    ---
    NTDEV is sponsored by OSR

    For our schedule of WDF, WDM, debugging and other seminars visit:
    http://www.osr.com/seminars

    To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
  • Mark_RoddyMark_Roddy Member - All Emails Posts: 4,310
    On Tue, Jun 1, 2010 at 4:49 AM, wrote:

    > We are returning SRB_STATUS_PENDING from StartIO


    Really?

    BOOLEAN
    *HwStorStartIo* (
    IN PVOID *DeviceExtension
    *,
    IN PSCSI_REQUEST_BLOCK
    *Srb
    *
    );

    Did you mean that you were calling StorPortNotification for the SRB
    indicating SRB_STATUS_PENDING? Because that would be wrong too.

    You need to indicate an appropriate queue depth and just queue the srbs. The
    IRPs are already "STATUS_PENDING".

    Mark Roddy
  • yuncheng_liyuncheng_li Member Posts: 4

    @Ajit_Yagaty Same problem when I tried the sample and modify it to schedule SRBs from startio routine to a system thread, the read performance is really slow, are there any workaround or suggestions?

  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 7,505

    You know you’re responding to a ten year old post, right?

    Right.

    Peter

    Peter Viscarola
    OSR
    @OSRDrivers

  • yuncheng_liyuncheng_li Member Posts: 4

    Yeah, recently we tried to migrate from virtual scsi miniport(network storage) to storport, is it an wise chosen?

  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 7,505

    Yea... well, what you’re doing is called “necroposting” and is explicitly disallowed in this forum.

    But, you know that, because you read the Read Before Posting link that appears on every page of this forum, right?

    People... is it really too much to expect you to read the guidelines before you seek help here? Really?

    Thread closed.

    Peter

    Peter Viscarola
    OSR
    @OSRDrivers

This discussion has been closed.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Upcoming OSR Seminars
Writing WDF Drivers 21 Oct 2019 OSR Seminar Space & ONLINE
Internals & Software Drivers 18 Nov 2019 Dulles, VA
Kernel Debugging 30 Mar 2020 OSR Seminar Space
Developing Minifilters 27 Apr 2020 OSR Seminar Space & ONLINE