Need a Suggestion for a Simple Sample PCIe Device

In our current WDF seminar, we use a super simple PCI device as an example of a backplane bus type device, and we’ve written a SUPER simple driver for this device as an instructional illustration.

The device we currently use is a PCI device, which is a clone of the venerable Intel 8255. This is a super simple, incredibly common, digital I/O device, with a ridiculously simple register map:

This device, and its accompanying example, has served us well for many years.

It is now more than a little obsolete. The primary problem with these devices is that the registers are in Port I/O space (to maintain compat with the old 8255). It also uses a Line Base Interrupt.

I’m looking for a replacement device to use as a sample in our learning environment. The device and its register map have to be simple enough that any engineer can understand how to use the device quickly. After all, the goal isn’t to teach people how to program this device, it’s to teach them how to write WDF drivers for a PCI device. The device details need to “get out of the way” so we can discuss, for example, how you claim resources, connect to interrupts, service I/O Requests, and process interrupts and DPCs.

Obviously, a data sheet for the device that includes the register map and control info for the device needs to be available. The device needs to be a PCIe device (not trad PCI… though I’d accept a M.2 or MiniPCI device that can be plugged into a PCIe carrier, if necessary)… because showing how to write a driver for a backplane bus connected device is the entire point of this exercise. I would love the device to support MSI, and multiple MSIs would be great addition.

It needs to be a PIO device… We do not have the time to properly discuss DMA in this seminar. One must crawl before one can walk.

I don’t care what KIND of device it is, though it would be nice if the device could do something, or be connected to something, where it could be SEEN working. And, as I previously mentioned, it needs to be easy to understand (even for those who are not necessarily device experts).

Any suggestions?

IF somebody in the community suggests a device that we wind up using, I’ll commit to:

  1. Write a WDF driver for the device that illustrates best practices
  2. Write an accompanying tutorial, describing how to write a WDF driver, using this device as an example
  3. Make the driver source code available on GitHub
  4. Use the example in our seminar

Anybody have any good device suggestions to share???

Peter

Hi Peter,

I don’t know if such a super simple device exists for PCIe device unless you want to build one with FPGA.

Alternatively, I would suggest Realtek Gbe PCIe NIC like RTL8111. It’s not super simple as your backplane device but it is simple for a GigE device. It meets all your “needs” and “wants”. Register level document is publicly available along with open source drivers too. They are super inexpensive and widely available.

One doesn’t have to write a NIDS miniport driver for it. In fact, when I learned programming general IRP based KMDs back in the NT3.x days, I was using a NE2000 NIC as a target device.

For samples, you could selectively support functionalities based on how much you want to overwhelm your poor students:

  1. R/W PCI config
  2. Reading IDs, MAC address, blinking LEDs.
  3. Bring up the Link with or without MSI interrupt.
  4. Send a packet, and use a sniffer on the other side to catch it.
  5. Receive a packet with MSI.
  6. Receive packets with RSS (MSIx and intr/DPC steering across CPU cores)
  7. Power management, Sleep, Hibernation, Wakeup. Wait/Wake etc.
  8. Combinations of above as bonus topic.

Good luck finding your devices!

Regards,
Calvin
p.s. I have never worked for RealTek.

1 Like

Hi Calvin… Haven’t “talked” with you in ages! How’ve you been??

unless you want to build one with FPGA

Dude… That was EXACTLY what I was thinking, too. But before I resort to that, given that I’m no wizard at using Vivado, I thought I’d ask the community for help.

There are some ADC + DIO boards that I’m currently exploring right now. They look simple, and (of course) they’re implemented with an FPGA (too funny… the manufacturer of the board puts a sticker on the Spartan FPGA… but the VID/DID are still Xilinx).

Using a NIC is a good idea, and that’s not a “complicated” device to your or me… but to really new learners, I think it’ll be way “too much” – I really need a super simple PIO device.

Good to hear from you Calvin.

And to ALL: Please pass along your ideas…

Peter

FPGA is a very good idea. How about a PCIe evaluation board of a major FPGA vendor. These usually come with free example FPGA designs. Advantage: can play with both “user” functionality and configuration space features.
IMHO NIC isn’t a good choice because of NDIS specifics.

Another idea - eval. board for a PCIe bridge like CMD (don’t remember who acquired them) - they have a lot to play with, and instead of a FPGA can connect to something easier to program on the other side.

– pa

FPGA is a very good idea

Thanks for that Pavel.

The problem with using an FPGA to create The Example Board of My Dreams (as much as I would enjoy the opportunity to Vivado-my-brains-out making it work) is that it’s a one-off board with a contrived example. Sure, folks could buy the same Eval Board and sure I could give them the code to flash (or, heaven forbid, provide the Verilog source code). But I’d still need to explain in class “this is a device I invented”… as opposed to “go to this web site, and you’ll find this device… you can see this very simple driver actually controls a real device that’s in the world… and if you’re so inclined you can buy it and continue to play yourself.”

Writing a driver for a commercially available board makes the exercise “feel” more “real” to the student, don’t you think?

Peter

BTW, I’m looking at this board right now as a good possibility.

4 channel digital input, 4 channel digital output… plus 8 (single ended) analog input channels. I haven’t decided if I want to even USE the AD channels yet… but the 4 DI + 4 DO channels are pretty simple. And, thank heavens, the vendor supplies the Register Map!

Peter

(following up my own post)

The disadvantages of the above card are:

  1. The DIOs don’t have any interrupt capability. Which is unfortunate.
  2. The register documentation for the ADC is… sketchy… at best.

So, this card might work, but I’m still looking…

Peter

Peter,

Consider the student use-case:

Are they likely to work with whatever device you pick or are they actually likely to be working on a project that is a custom device (initially or forever) realized via an FPGA based design?

And yeah, give them the Vivado or Quartus project along with it ?

Perhaps that will head off the inevitable of creating PCIe devices that have “issues” with Windows ….

(The FPGA world and Verilog / SystemVerilog is my new love-hate friends these days)

Good Luck,
Dave Cattley

1 Like

It’s so XX century. Forgive me. What could be demonstrated with this card? If the goal is to show bare PCI (enumeration, BAR mapping) then OK.
Ability to customize the device side adds interesting possibilities: for one, demo of attack on the host via DMA and thus, the role of IOMMU :wink:

– pa

If the goal is to show bare PCI (enumeration, BAR mapping) then OK

The folks who come to learn how to write drivers need to learn how to map a block of registers into memory, receive I/O Requests, process those requests by writing to and reading from device registers, how to get an interrupt and handle a DpcForIsr.

Some of them need to learn what registers and interrupts ARE… having only a very sketchy recollection from their Computer Architecture calls in university.

Nothing even remotely like DMA or IOMMUs even comes CLOSE to getting into the picture. That’s all for the ADVANCED class.

This class is for total newbies. The one who have never heard of WdfRequestComplete.

Peter

Peter,

I’ve been out of Windows kernel for some time now. And it may not make sense !

But would it be possible to write a driver with APIs – To emulate the
devices you want ??

_prokash

But would it be possible to write a driver with APIs – To emulate the devices you want ??

Unless we are speaking about a virtual device (which clearly does not apply here, for the obvious reasons), you would need to run your target OS on some extensible emulator like QEMU if you want to do something like that. Although this approach,indeed, gives you a chance to create a truly custom device that suits your particular needs and can be made as simple or complex as you want it to be, I don’t really think this is what they are looking for. After all, Peter made it clear that they are looking for some simplistic real-life device that is commercially available, so that even a FPGA-based custom device is not the optimal option.

Anton Bassov

Yeah, Peter mentioned those including MSIs. I could be totally wrong,
MS has now emulated USB !!

I was basically thinking to abstract those interface like : Mapping
Registers, Interrupts ( with timeouts to cover the DPCforIsr etc )
without using polling etc.

Perhaps, it way too complicated, but yes, once done, it could be
morphed to act as Network, Removable Media etc…

Let others give practical feed back :slight_smile:

-pro

I was basically thinking to abstract those interface like : Mapping Registers, Interrupts ( with timeouts to cover the DPCforIsr etc )
without using polling etc.

Well, the task in itself is not infeasible, but implementing something like that would require developing some “Virtual PCI/PCIe” bus, plus, apparently, making certain modifications to the existing system services (IoConnectInterrupt() is the very first thing that gets into my head) in order to integrate it with the existing architecture.

The very first real-life analogy that comes to my mind here is advising Peter to invest in few oil rigs, as well as in oil refinery, at the time when the only thing he asks us about is a direction to the nearest petrol station…

Let others give practical feed back :slight_smile:

In my opinion, Calvin’s suggestion seems to be the most reasonable one in so far. All you have to do is to get some dirt-cheap low-end network adapter. Once we are speaking about a simple class exercise, you don’t really need to think of it as of a real-life network interface that is utilised by the bound protocol drivers. Therefore, you don’t really need to expose it to NDIS.

Once you haven’t got any existing clients, apart from your test application, there are no compulsory interfaces that you are required to implement on the upper edge, which means it is up to you to decide upon the API that it exposes to its clients. Therefore, you don’t need to make use of all its potential capabilities. For the purpose of this exercise, you can think of it just as of a simple device that is capable of sending and receiving data, so that you can make it as simple as you wish.

To summarise, the key points here are wide availability, low cost, flexibility of choosing its programming model, availability of register-level documentation, as well of an open-source implementation that you may use as a potential reference…

Anton Bassov

On Sat, May 16, 2020 at 2:15 AM anton_bassov
wrote:

> OSR https://community.osr.com/
>
> anton_bassov commented on Need a Suggestion for a Simple Sample PCIe Device
>
> > I was basically thinking to abstract those interface like : Mapping
> Registers, Interrupts ( with timeouts to cover the DPCforIsr etc )
> >
> > without using polling etc.
>
> Well, the task in itself is not infeasible, but implementing something
> like that would require developing some “Virtual PCI/PCIe” bus, plus,
> apparently, making certain modifications to the existing system services
> (IoConnectInterrupt() is the very first thing that gets into my head) in
> order to integrate it with the existing architecture.
>
> The very first real-life analogy that comes to my mind here is advising
> Peter to invest in few oil rigs, as well as in oil refinery, at the time
> when the only thing he asks us about is a direction to the nearest petrol
> station…
> >>> oil now dirt cheap
> > Let others give practical feed back :slight_smile:
>
> In my opinion, Calvin’s suggestion seems to be the most reasonable one in
> so far. All you have to do is to get some dirt-cheap low-end network
> adapter. Once we are speaking about a simple class exercise, you don’t
> really need to think of it as of a real-life network interface that is
> utilised by the bound protocol drivers. Therefore, you don’t really need to
> expose it to NDIS.
>
>>> This is same path I took back around 2000 , had a ddk example. That was
pci days. Hope there is one pci-e based

I did not look this article before I proposed the idea –
http://htfsoftware.com/wp-content/uploads/2016/01/PciDevSim1.pdf

I know it easier said than done :slight_smile:

Calvin, along with all the veterans here are prolific windows kernel
engineer, so …

-pro

This thing has 3 LEDs, 4 GPIO lines, and a couple of high-speed connectors, onboard flash, and JTAG, $199. Xilinx supplies a driver that even does DMA, but you can always do your own driver.
https://picoevb.com/

I have to say Xilinx has grown smarter about their drivers. Their previous PCIe IP, from Northwest Logic, worked fine, but they supplied an enormously complicated driver suite that involved a surprisingly large set of interoperating drivers. I threw it out and wrote my own. Their newer PCIe IP block comes with a driver that is simple and flexible, so that I might actually just use theirs instead of writing one.

$200 seems like a lot for the stated goal. I’d look for something cheap and common, which has a Linux driver.
For example these serial cards for $24.
Then write an exemplary correct and didactic Windows driver :wink:
They are easy to use: send it some data and it will show on a serial terminal on the PC. Type something in the terminal and it’ll get an interrupt.
– pa

I did not look this article before I proposed the idea –
http://htfsoftware.com/wp-content/uploads/2016/01/PciDevSim1.pdf

This is a really interesting one…

Thanks a lot, Pro

Anton Bassov