Windows System Software -- Consulting, Training, Development -- Unique Expertise, Guaranteed Results

Home NTDEV
Before Posting...
Please check out the Community Guidelines in the Announcements and Administration Category.

More Info on Driver Writing and Debugging


The free OSR Learning Library has more than 50 articles on a wide variety of topics about writing and debugging device drivers and Minifilters. From introductory level to advanced. All the articles have been recently reviewed and updated, and are written using the clear and definitive style you've come to expect from OSR over the years.


Check out The OSR Learning Library at: https://www.osr.com/osr-learning-library/


Need a Suggestion for a Simple Sample PCIe Device

Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 7,807

In our current WDF seminar, we use a super simple PCI device as an example of a backplane bus type device, and we've written a SUPER simple driver for this device as an instructional illustration.
 
The device we currently use is a PCI device, which is a clone of the venerable Intel 8255. This is a super simple, incredibly common, digital I/O device, with a ridiculously simple register map:

This device, and its accompanying example, has served us well for many years.
 
It is now more than a little obsolete. The primary problem with these devices is that the registers are in Port I/O space (to maintain compat with the old 8255). It also uses a Line Base Interrupt.
 
I'm looking for a replacement device to use as a sample in our learning environment. The device and its register map have to be simple enough that any engineer can understand how to use the device quickly. After all, the goal isn't to teach people how to program this device, it's to teach them how to write WDF drivers for a PCI device. The device details need to "get out of the way" so we can discuss, for example, how you claim resources, connect to interrupts, service I/O Requests, and process interrupts and DPCs.
 
Obviously, a data sheet for the device that includes the register map and control info for the device needs to be available. The device needs to be a PCIe device (not trad PCI... though I'd accept a M.2 or MiniPCI device that can be plugged into a PCIe carrier, if necessary)... because showing how to write a driver for a backplane bus connected device is the entire point of this exercise. I would love the device to support MSI, and multiple MSIs would be great addition.
 
It needs to be a PIO device... We do not have the time to properly discuss DMA in this seminar. One must crawl before one can walk.
 
I don't care what KIND of device it is, though it would be nice if the device could do something, or be connected to something, where it could be SEEN working. And, as I previously mentioned, it needs to be easy to understand (even for those who are not necessarily device experts).
 
Any suggestions?
 
IF somebody in the community suggests a device that we wind up using, I'll commit to:
 
1. Write a WDF driver for the device that illustrates best practices
2. Write an accompanying tutorial, describing how to write a WDF driver, using this device as an example
3. Make the driver source code available on GitHub
4. Use the example in our seminar
 
Anybody have any good device suggestions to share???
 
Peter

Peter Viscarola
OSR
@OSRDrivers

Comments

  • Calvin_Guan-2Calvin_Guan-2 Member Posts: 314

    Hi Peter,

    I don't know if such a super simple device exists for PCIe device unless you want to build one with FPGA.

    Alternatively, I would suggest Realtek Gbe PCIe NIC like RTL8111. It's not super simple as your backplane device but it is simple for a GigE device. It meets all your "needs" and "wants". Register level document is publicly available along with open source drivers too. They are super inexpensive and widely available.

    One doesn't have to write a NIDS miniport driver for it. In fact, when I learned programming general IRP based KMDs back in the NT3.x days, I was using a NE2000 NIC as a target device.

    For samples, you could selectively support functionalities based on how much you want to overwhelm your poor students:

    0) R/W PCI config
    1) Reading IDs, MAC address, blinking LEDs.
    2) Bring up the Link with or without MSI interrupt.
    3) Send a packet, and use a sniffer on the other side to catch it.
    4) Receive a packet with MSI.
    5) Receive packets with RSS (MSIx and intr/DPC steering across CPU cores)
    6) Power management, Sleep, Hibernation, Wakeup. Wait/Wake etc.
    7) Combinations of above as bonus topic.

    Good luck finding your devices!

    Regards,
    Calvin
    p.s. I have never worked for RealTek.

  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 7,807

    Hi Calvin... Haven't "talked" with you in ages! How've you been??

    unless you want to build one with FPGA

    Dude... That was EXACTLY what I was thinking, too. But before I resort to that, given that I'm no wizard at using Vivado, I thought I'd ask the community for help.

    There are some ADC + DIO boards that I'm currently exploring right now. They look simple, and (of course) they're implemented with an FPGA (too funny... the manufacturer of the board puts a sticker on the Spartan FPGA... but the VID/DID are still Xilinx).

    Using a NIC is a good idea, and that's not a "complicated" device to your or me... but to really new learners, I think it'll be way "too much" -- I really need a super simple PIO device.

    Good to hear from you Calvin.

    And to ALL: Please pass along your ideas...

    Peter

    Peter Viscarola
    OSR
    @OSRDrivers

  • Pavel_APavel_A Member Posts: 2,708
    edited May 14

    FPGA is a very good idea. How about a PCIe evaluation board of a major FPGA vendor. These usually come with free example FPGA designs. Advantage: can play with both "user" functionality and configuration space features.
    IMHO NIC isn't a good choice because of NDIS specifics.

    Another idea - eval. board for a PCIe bridge like CMD (don't remember who acquired them) - they have a lot to play with, and instead of a FPGA can connect to something easier to program on the other side.

    -- pa

  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 7,807

    FPGA is a very good idea

    Thanks for that Pavel.

    The problem with using an FPGA to create The Example Board of My Dreams (as much as I would enjoy the opportunity to Vivado-my-brains-out making it work) is that it's a one-off board with a contrived example. Sure, folks could buy the same Eval Board and sure I could give them the code to flash (or, heaven forbid, provide the Verilog source code). But I'd still need to explain in class "this is a device I invented"... as opposed to "go to this web site, and you'll find this device... you can see this very simple driver actually controls a real device that's in the world... and if you're so inclined you can buy it and continue to play yourself."

    Writing a driver for a commercially available board makes the exercise "feel" more "real" to the student, don't you think?

    Peter

    Peter Viscarola
    OSR
    @OSRDrivers

  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 7,807

    BTW, I'm looking at this board right now as a good possibility.

    4 channel digital input, 4 channel digital output... plus 8 (single ended) analog input channels. I haven't decided if I want to even USE the AD channels yet... but the 4 DI + 4 DO channels are pretty simple. And, thank heavens, the vendor supplies the Register Map!

    Peter

    Peter Viscarola
    OSR
    @OSRDrivers

  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 7,807

    (following up my own post)

    The disadvantages of the above card are:
    1. The DIOs don't have any interrupt capability. Which is unfortunate.
    2. The register documentation for the ADC is... sketchy... at best.

    So, this card might work, but I'm still looking...

    Peter

    Peter Viscarola
    OSR
    @OSRDrivers

  • David_R._CattleyDavid_R._Cattley Member - All Emails Posts: 2,114
    via Email
    Peter,

    Consider the student use-case:

    Are they likely to work with whatever device you pick or are they actually likely to be working on a project that is a custom device (initially or forever) realized via an FPGA based design?

    And yeah, give them the Vivado or Quartus project along with it 😊

    Perhaps that will head off the inevitable of creating PCIe devices that have “issues” with Windows ….

    (The FPGA world and Verilog / SystemVerilog is my new love-hate friends these days)

    Good Luck,
    Dave Cattley
  • Pavel_APavel_A Member Posts: 2,708

    It's so XX century. Forgive me. What could be demonstrated with this card? If the goal is to show bare PCI (enumeration, BAR mapping) then OK.
    Ability to customize the device side adds interesting possibilities: for one, demo of attack on the host via DMA and thus, the role of IOMMU ;)

    -- pa

  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 7,807

    If the goal is to show bare PCI (enumeration, BAR mapping) then OK

    The folks who come to learn how to write drivers need to learn how to map a block of registers into memory, receive I/O Requests, process those requests by writing to and reading from device registers, how to get an interrupt and handle a DpcForIsr.

    Some of them need to learn what registers and interrupts ARE... having only a very sketchy recollection from their Computer Architecture calls in university.

    Nothing even remotely like DMA or IOMMUs even comes CLOSE to getting into the picture. That's all for the ADVANCED class.

    This class is for total newbies. The one who have never heard of WdfRequestComplete.

    Peter

    Peter Viscarola
    OSR
    @OSRDrivers

  • Prokash_SinhaProkash_Sinha Member - All Emails Posts: 201
    via Email
    Peter,

    I've been out of Windows kernel for some time now. And it may not make sense !

    But would it be possible to write a driver with APIs -- To emulate the
    devices you want ??

    _prokash
  • anton_bassovanton_bassov Member Posts: 5,158

    But would it be possible to write a driver with APIs -- To emulate the devices you want ??

    Unless we are speaking about a virtual device (which clearly does not apply here, for the obvious reasons), you would need to run your target OS on some extensible emulator like QEMU if you want to do something like that. Although this approach,indeed, gives you a chance to create a truly custom device that suits your particular needs and can be made as simple or complex as you want it to be, I don't really think this is what they are looking for. After all, Peter made it clear that they are looking for some simplistic real-life device that is commercially available, so that even a FPGA-based custom device is not the optimal option.

    Anton Bassov

  • Prokash_SinhaProkash_Sinha Member - All Emails Posts: 201
    via Email
    Yeah, Peter mentioned those including MSIs. I could be totally wrong,
    MS has now emulated USB !!

    I was basically thinking to abstract those interface like : Mapping
    Registers, Interrupts ( with timeouts to cover the DPCforIsr etc )
    without using polling etc.

    Perhaps, it way too complicated, but yes, once done, it could be
    morphed to act as Network, Removable Media etc...

    Let others give practical feed back :-)

    -pro
  • anton_bassovanton_bassov Member Posts: 5,158

    I was basically thinking to abstract those interface like : Mapping Registers, Interrupts ( with timeouts to cover the DPCforIsr etc )
    without using polling etc.

    Well, the task in itself is not infeasible, but implementing something like that would require developing some "Virtual PCI/PCIe" bus, plus, apparently, making certain modifications to the existing system services (IoConnectInterrupt() is the very first thing that gets into my head) in order to integrate it with the existing architecture.

    The very first real-life analogy that comes to my mind here is advising Peter to invest in few oil rigs, as well as in oil refinery, at the time when the only thing he asks us about is a direction to the nearest petrol station....

    Let others give practical feed back :-)

    In my opinion, Calvin's suggestion seems to be the most reasonable one in so far. All you have to do is to get some dirt-cheap low-end network adapter. Once we are speaking about a simple class exercise, you don't really need to think of it as of a real-life network interface that is utilised by the bound protocol drivers. Therefore, you don't really need to expose it to NDIS.

    Once you haven't got any existing clients, apart from your test application, there are no compulsory interfaces that you are required to implement on the upper edge, which means it is up to you to decide upon the API that it exposes to its clients. Therefore, you don't need to make use of all its potential capabilities. For the purpose of this exercise, you can think of it just as of a simple device that is capable of sending and receiving data, so that you can make it as simple as you wish.

    To summarise, the key points here are wide availability, low cost, flexibility of choosing its programming model, availability of register-level documentation, as well of an open-source implementation that you may use as a potential reference....

    Anton Bassov

  • Prokash_SinhaProkash_Sinha Member - All Emails Posts: 201
    via Email
    On Sat, May 16, 2020 at 2:15 AM anton_bassov
    wrote:

    > OSR https://community.osr.com/
    >
    > anton_bassov commented on Need a Suggestion for a Simple Sample PCIe Device
    >
    > > I was basically thinking to abstract those interface like : Mapping
    > Registers, Interrupts ( with timeouts to cover the DPCforIsr etc )
    > >
    > > without using polling etc.
    >
    > Well, the task in itself is not infeasible, but implementing something
    > like that would require developing some "Virtual PCI/PCIe" bus, plus,
    > apparently, making certain modifications to the existing system services
    > (IoConnectInterrupt() is the very first thing that gets into my head) in
    > order to integrate it with the existing architecture.
    >
    > The very first real-life analogy that comes to my mind here is advising
    > Peter to invest in few oil rigs, as well as in oil refinery, at the time
    > when the only thing he asks us about is a direction to the nearest petrol
    > station....
    > >>> oil now dirt cheap
    > > Let others give practical feed back :-)
    >
    > In my opinion, Calvin's suggestion seems to be the most reasonable one in
    > so far. All you have to do is to get some dirt-cheap low-end network
    > adapter. Once we are speaking about a simple class exercise, you don't
    > really need to think of it as of a real-life network interface that is
    > utilised by the bound protocol drivers. Therefore, you don't really need to
    > expose it to NDIS.
    >
    >>> This is same path I took back around 2000 , had a ddk example. That was
    pci days. Hope there is one pci-e based
  • Prokash_SinhaProkash_Sinha Member - All Emails Posts: 201
    via Email
    I did not look this article before I proposed the idea --
    http://htfsoftware.com/wp-content/uploads/2016/01/PciDevSim1.pdf

    I know it easier said than done :-)

    Calvin, along with all the veterans here are prolific windows kernel
    engineer, so ...

    -pro
  • Tim_RobertsTim_Roberts Member - All Emails Posts: 13,403

    This thing has 3 LEDs, 4 GPIO lines, and a couple of high-speed connectors, onboard flash, and JTAG, $199. Xilinx supplies a driver that even does DMA, but you can always do your own driver.
    https://picoevb.com/

    Tim Roberts, [email protected]
    Providenza & Boekelheide, Inc.

  • Tim_RobertsTim_Roberts Member - All Emails Posts: 13,403

    I have to say Xilinx has grown smarter about their drivers. Their previous PCIe IP, from Northwest Logic, worked fine, but they supplied an enormously complicated driver suite that involved a surprisingly large set of interoperating drivers. I threw it out and wrote my own. Their newer PCIe IP block comes with a driver that is simple and flexible, so that I might actually just use theirs instead of writing one.

    Tim Roberts, [email protected]
    Providenza & Boekelheide, Inc.

  • Pavel_APavel_A Member Posts: 2,708
    edited May 16

    $200 seems like a lot for the stated goal. I'd look for something cheap and common, which has a Linux driver.
    For example these serial cards for $24.
    Then write an exemplary correct and didactic Windows driver ;)
    They are easy to use: send it some data and it will show on a serial terminal on the PC. Type something in the terminal and it'll get an interrupt.
    -- pa

  • anton_bassovanton_bassov Member Posts: 5,158

    I did not look this article before I proposed the idea --
    http://htfsoftware.com/wp-content/uploads/2016/01/PciDevSim1.pdf

    This is a really interesting one....

    Thanks a lot, Pro

    Anton Bassov

  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 7,807
    edited May 16

    This thing has 3 LEDs, 4 GPIO lines...

    Oh, Mr. Roberts... that is extremely intriguing. Yes, I have to order one of those. I mean, just to fool with.

    Thank you very much for that.

    Xilinx has grown smarter about their drivers

    In some ways. We have had multiple occasions to work with the Xilinx XDMA library and sample driver. It has many very serious architectural issues. Just sayin’...

    Thank you again for the pointer that that little Pico thingy. That and an Express to M.2 adapter and I might be good to go.

    Fascinating board. And I already have Vivado installed on my system at home...

    ETA: I ordered one, and a pile of m.2 to PCIe adapters. BTW, they sell the Pico thingy on Amazon (though in my excitement I ordered it from the link Mr. Roberts posted).

    BUt... friends... keep the device suggestions coming. Please.

    Peter

    Post edited by Peter_Viscarola_(OSR) on

    Peter Viscarola
    OSR
    @OSRDrivers

  • Tim_RobertsTim_Roberts Member - All Emails Posts: 13,403

    And I already have Vivado installed on my system at home...

    Anyone who has complained about Visual Studio and the WDK tools will generally stop doing so after a little exposure to Vivado and the other Xilinx tools.

    Tim Roberts, [email protected]
    Providenza & Boekelheide, Inc.

  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 7,807

    LOL... too true.

    The biggest problem I’m having solid ideas about the PicoEVB is that there are no good examples... even the GitHub sample uses the infamous Xilinx XDMA library. I know that driver very well. Ugh. And there’s absolutely nothing tutorial in terms of Vivado projects.

    They provide the constraints file. But... I don’t know what if I have the Verilog chops to create my own “memory mapped DIO, with MSI on Change of State” application entirely from scratch, which appears to be what will be required. I guess I shall see.

    The other thing that made me laugh (literally) about the picoEVB is that it apparently takes 30 minutes to program the bitstream into it.

    I’m still waiting for the other, industrial, DIO and ADC card I linked above to arrive. That, sadly, doesn’t interrupt on COS on the DIOs. Sigh.

    Peter

    Peter Viscarola
    OSR
    @OSRDrivers

  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 7,807

    Sigh!

    PicoEVB: The example is in VHDL, and is super-complicated relative to my abilities.

    Peter

    Peter Viscarola
    OSR
    @OSRDrivers

  • anton_bassovanton_bassov Member Posts: 5,158

    PicoEVB: The example is in VHDL, and is super-complicated relative to my abilities.

    What about the following one

    https://numato.com/product/galatea-pci-express-spartan-6-fpga-development-board

    Although it is Spartan 6 -based (which means you would need XILINX ISE, which is much less user-friendly, compared to Vivado, ifyou decide to try this option)
    here is a brief tutorial available. Judging from its screenshot, a sample design in Verilog is available. To be honest, just cannot be asked to experimentally establish it, although I've got ISE 14.7 installed, but I trust their words.

    https://numato.com/kb/getting-started-with-pcie-on-galatea/

    OTOH, it does not sound too Windows-friendly

    Anton Bassov

  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 7,807

    What about the following one

    Oh, that's a good thought. But, you noted the issue: You have to use ISE.

    Well, after days of searching (and a whole day spent in Vivado trying to create my own design targeted at the PicoEVB), I think I may have found a board: National Instruments, to the rescue!

    The board is the PCIe-6509. It meets all the criteria: DIO, simple interface, MSI on line change of state.

    The only downside is that it support 96 DIO lines, which is... you know... about 94 more than I need for this example. But I can hide some of that complexity for the purposes of the class, I think.

    The register map is public, and their docs look to be sufficient to actually be able to use the device.

    I was excited to see that they have a (now long deprecated) WDM driver that's public for the device... until I realized that what their WDM driver does is map the device's register space into user-space (via \device\PhysicalMemory, no less). Now I know why the WDM driver has been deprecated. But, no matter, there are user-mode examples that show how to program the registers to make the device do things.

    We have NI folks who occasionally visit this Community. I'm kinda surprised nobody suggested their board.

    So, it looks like I can skip the FPGA programming (leaving my skills at the "flip THIS switch, light THAT light!" level)... and concentrate on what I sort of know how to do.

    Peter

    Peter Viscarola
    OSR
    @OSRDrivers

  • rob18767rob18767 Member Posts: 25

    I started on this.

    https://www.intel.com/content/www/us/en/programmable/products/boards_and_kits/dev-kits/altera/kit-cyclone-iv-starter.html

    That said I have a strong hardware background so I can open up the platform designer/QSys/Sopc builder tools from the relevant Intel FPGA (formally Altera) and figure out the BAR offsets for myself.

    If you want something specific you can get it design. Avid Tech in Cleveland were taken over by Avnet.

    https://www.avid-tech.com/services/turnkey-design-and-manufacturing/hardware-design/

    They are brilliant went it comes to PCIe hardware design.

  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 7,807

    Thanks.

    I’ve gone ahead with the NI PCIe-6509. Lovely device, actually. Very flexible. I’d love to write a real driver for it some day (one for actual production use that fully exploits all its features, as opposed to one that designed for noobs to learn from).

    Peter

    Peter Viscarola
    OSR
    @OSRDrivers

Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Upcoming OSR Seminars
OSR has suspended in-person seminars due to the Covid-19 outbreak. But, don't miss your training! Attend via the internet instead!
Kernel Debugging 30 Mar 2020 OSR Seminar Space
Developing Minifilters 15 Jun 2020 LIVE ONLINE
Writing WDF Drivers 22 June 2020 LIVE ONLINE
Internals & Software Drivers 28 Sept 2020 Dulles, VA