Windows System Software -- Consulting, Training, Development -- Unique Expertise, Guaranteed Results

Sept/Oct 2019 Issue of The NT Insider available


Download PDF here: http://insider.osr.com/2019/ntinsider_2019_01.pdf

It’s a particularly BIG issue, too: 40 pages of technical goodness, ranging from WDF to Minifilters. Check it out.
Before Posting...
Please check out the Community Guidelines in the Announcements and Administration Category.

Function Level Reset (FLR) from user space

Rajeev_RanjanRajeev_Ranjan Member Posts: 19

Hello NTDEV community members,
May I ask your opinion/help on below query -
**
In short** - Is it possible to issue FLR from user application?

In Details - A multi function NVMe drive has changed one of its function IOQ pair and/or Interrupt vector. To get this effected on host the BAR register need to be re-read by host OS. The best way to achieve this is to use FLR on the device.
On Linux I can use 'echo 1 > /sys/bus/pci/devices/BDF..' and rescan to achieve it.
On Windows, I tried to search the same. I could find the below link -
https://docs.microsoft.com/en-us/windows-hardware/drivers/kernel/resetting-and-recovering-a-device
It specifically calls out that FLR can be implemented either by bus driver or by ACPI firmware.

I was giving up the hope that I can do FLR on windows from user space.
But then I came across Discrete Device Assignment (DDA) on Windows -
https://docs.microsoft.com/en-us/windows-server/virtualization/hyper-v/deploy/deploying-storage-devices-using-dda#dismount-the-device
If I follow below steps -
1. Disable (Disable-PnpDevice -InstanceId -Confirm:$false)
2. Unmount (Dismount-VMHostAssignableDevice -LocationPath $locationPath)
3. mount (Mount-VMHostAssignableDevice -LocationPath $locationPath)
4. Enable (Enble-PnpDevice -InstanceId -Confirm:$false)

The device BAR is reread -- essentially FLR is achieved in different way from user space.

Based on above finding I was trying to achieve above effect of steps (1 to 4 above) without having hyper-V services.
My understanding is - if FLR can be achieved by above steps with hyper-V services enabled, there must be a way to achieve FLR by using either devcon.exe or SetupDI* calls without hyper-V.

But my search for this effect has not yielded any result so far.
Does anyone from this group provide some hints on it.

Note: Sorry to posting here though it is not related to driver development directly. But I am not able to find more suitable group for this question.

Comments

  • Igor_SharovarIgor_Sharovar Member Posts: 618

    What do you mean?

    A multi function NVMe drive has changed one of its function IOQ pair and/or Interrupt vector.

    The PnP manager assigns PCI resources not a device itself. In general, a PCI device should not care about exact numbers of assigned resources, which controlled by the PnP manager. Could you give more details what your device exactly do?
    Igor Sharovar

  • Rajeev_RanjanRajeev_Ranjan Member Posts: 19

    Sorry Igor Sharovar, I was not very specific.
    Yes, PnP manager assigns PCI resources. But drive advertise what max it supports. Relevant section in NVme 1.3 spec -
    section 8.5 - Virtualization Enhancements (Optional)

    • Virtual Queue Resource (VQ Resource): a type of controller resource that manages one Submission Queue (SQ) and one Completion Queue (CQ) (refer to section 8.5.1); and
    • Virtual Interrupt Resource (VI Resource): a type of controller resource that manages one interrupt vector (refer to section 8.5.2).

    Please let me know if I can explain more. Sorry for the confusion.

    Thanks,
    Rajeev

  • Igor_SharovarIgor_Sharovar Member Posts: 618

    OK,it is more clear. I don't have a real NVMe experience but undrstand it should be allocated through BAR registers as PCI resources. As soon as it allocated you could not change exact numbers, for example interrupt number which you want. But why do you want to change assigned PCI resourses? Applications and drivers must not care about it. It is not their business.
    Igor Sharovar

  • Rajeev_RanjanRajeev_Ranjan Member Posts: 19
    edited September 5

    As per NVMe spec the firmware/drive wanted to change the interrupt number. To get this effected, one way to reboot the server so that PCI config space will be re-created by bus driver and drivers stack will be loaded with updated interrupt vector by PnP manager. Reboot can be avoided if a FLR can be issued to the function/drive so that bus driver will re-create the config space for that function/drive. This is what I intended to do from application - FLR to particular function/drive (BDF) from user space.

Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Upcoming OSR Seminars
Writing WDF Drivers 21 Oct 2019 OSR Seminar Space & ONLINE
Internals & Software Drivers 18 Nov 2019 Dulles, VA
Kernel Debugging 30 Mar 2020 OSR Seminar Space
Developing Minifilters 27 Apr 2020 OSR Seminar Space & ONLINE