Need a Suggestion for a Simple Sample PCIe Device

It’s so XX century. Forgive me. What could be demonstrated with this card? If the goal is to show bare PCI (enumeration, BAR mapping) then OK.
Ability to customize the device side adds interesting possibilities: for one, demo of attack on the host via DMA and thus, the role of IOMMU :wink:

– pa

If the goal is to show bare PCI (enumeration, BAR mapping) then OK

The folks who come to learn how to write drivers need to learn how to map a block of registers into memory, receive I/O Requests, process those requests by writing to and reading from device registers, how to get an interrupt and handle a DpcForIsr.

Some of them need to learn what registers and interrupts ARE… having only a very sketchy recollection from their Computer Architecture calls in university.

Nothing even remotely like DMA or IOMMUs even comes CLOSE to getting into the picture. That’s all for the ADVANCED class.

This class is for total newbies. The one who have never heard of WdfRequestComplete.

Peter

Peter,

I’ve been out of Windows kernel for some time now. And it may not make sense !

But would it be possible to write a driver with APIs – To emulate the
devices you want ??

_prokash

But would it be possible to write a driver with APIs – To emulate the devices you want ??

Unless we are speaking about a virtual device (which clearly does not apply here, for the obvious reasons), you would need to run your target OS on some extensible emulator like QEMU if you want to do something like that. Although this approach,indeed, gives you a chance to create a truly custom device that suits your particular needs and can be made as simple or complex as you want it to be, I don’t really think this is what they are looking for. After all, Peter made it clear that they are looking for some simplistic real-life device that is commercially available, so that even a FPGA-based custom device is not the optimal option.

Anton Bassov

Yeah, Peter mentioned those including MSIs. I could be totally wrong,
MS has now emulated USB !!

I was basically thinking to abstract those interface like : Mapping
Registers, Interrupts ( with timeouts to cover the DPCforIsr etc )
without using polling etc.

Perhaps, it way too complicated, but yes, once done, it could be
morphed to act as Network, Removable Media etc…

Let others give practical feed back :slight_smile:

-pro

I was basically thinking to abstract those interface like : Mapping Registers, Interrupts ( with timeouts to cover the DPCforIsr etc )
without using polling etc.

Well, the task in itself is not infeasible, but implementing something like that would require developing some “Virtual PCI/PCIe” bus, plus, apparently, making certain modifications to the existing system services (IoConnectInterrupt() is the very first thing that gets into my head) in order to integrate it with the existing architecture.

The very first real-life analogy that comes to my mind here is advising Peter to invest in few oil rigs, as well as in oil refinery, at the time when the only thing he asks us about is a direction to the nearest petrol station…

Let others give practical feed back :slight_smile:

In my opinion, Calvin’s suggestion seems to be the most reasonable one in so far. All you have to do is to get some dirt-cheap low-end network adapter. Once we are speaking about a simple class exercise, you don’t really need to think of it as of a real-life network interface that is utilised by the bound protocol drivers. Therefore, you don’t really need to expose it to NDIS.

Once you haven’t got any existing clients, apart from your test application, there are no compulsory interfaces that you are required to implement on the upper edge, which means it is up to you to decide upon the API that it exposes to its clients. Therefore, you don’t need to make use of all its potential capabilities. For the purpose of this exercise, you can think of it just as of a simple device that is capable of sending and receiving data, so that you can make it as simple as you wish.

To summarise, the key points here are wide availability, low cost, flexibility of choosing its programming model, availability of register-level documentation, as well of an open-source implementation that you may use as a potential reference…

Anton Bassov

On Sat, May 16, 2020 at 2:15 AM anton_bassov
wrote:

> OSR https://community.osr.com/
>
> anton_bassov commented on Need a Suggestion for a Simple Sample PCIe Device
>
> > I was basically thinking to abstract those interface like : Mapping
> Registers, Interrupts ( with timeouts to cover the DPCforIsr etc )
> >
> > without using polling etc.
>
> Well, the task in itself is not infeasible, but implementing something
> like that would require developing some “Virtual PCI/PCIe” bus, plus,
> apparently, making certain modifications to the existing system services
> (IoConnectInterrupt() is the very first thing that gets into my head) in
> order to integrate it with the existing architecture.
>
> The very first real-life analogy that comes to my mind here is advising
> Peter to invest in few oil rigs, as well as in oil refinery, at the time
> when the only thing he asks us about is a direction to the nearest petrol
> station…
> >>> oil now dirt cheap
> > Let others give practical feed back :slight_smile:
>
> In my opinion, Calvin’s suggestion seems to be the most reasonable one in
> so far. All you have to do is to get some dirt-cheap low-end network
> adapter. Once we are speaking about a simple class exercise, you don’t
> really need to think of it as of a real-life network interface that is
> utilised by the bound protocol drivers. Therefore, you don’t really need to
> expose it to NDIS.
>
>>> This is same path I took back around 2000 , had a ddk example. That was
pci days. Hope there is one pci-e based

I did not look this article before I proposed the idea –
http://htfsoftware.com/wp-content/uploads/2016/01/PciDevSim1.pdf

I know it easier said than done :slight_smile:

Calvin, along with all the veterans here are prolific windows kernel
engineer, so …

-pro

This thing has 3 LEDs, 4 GPIO lines, and a couple of high-speed connectors, onboard flash, and JTAG, $199. Xilinx supplies a driver that even does DMA, but you can always do your own driver.
https://picoevb.com/

I have to say Xilinx has grown smarter about their drivers. Their previous PCIe IP, from Northwest Logic, worked fine, but they supplied an enormously complicated driver suite that involved a surprisingly large set of interoperating drivers. I threw it out and wrote my own. Their newer PCIe IP block comes with a driver that is simple and flexible, so that I might actually just use theirs instead of writing one.

$200 seems like a lot for the stated goal. I’d look for something cheap and common, which has a Linux driver.
For example these serial cards for $24.
Then write an exemplary correct and didactic Windows driver :wink:
They are easy to use: send it some data and it will show on a serial terminal on the PC. Type something in the terminal and it’ll get an interrupt.
– pa

I did not look this article before I proposed the idea –
http://htfsoftware.com/wp-content/uploads/2016/01/PciDevSim1.pdf

This is a really interesting one…

Thanks a lot, Pro

Anton Bassov

This thing has 3 LEDs, 4 GPIO lines…

Oh, Mr. Roberts… that is extremely intriguing. Yes, I have to order one of those. I mean, just to fool with.

Thank you very much for that.

Xilinx has grown smarter about their drivers

In some ways. We have had multiple occasions to work with the Xilinx XDMA library and sample driver. It has many very serious architectural issues. Just sayin’…

Thank you again for the pointer that that little Pico thingy. That and an Express to M.2 adapter and I might be good to go.

Fascinating board. And I already have Vivado installed on my system at home…

ETA: I ordered one, and a pile of m.2 to PCIe adapters. BTW, they sell the Pico thingy on Amazon (though in my excitement I ordered it from the link Mr. Roberts posted).

BUt… friends… keep the device suggestions coming. Please.

Peter

1 Like

And I already have Vivado installed on my system at home…

Anyone who has complained about Visual Studio and the WDK tools will generally stop doing so after a little exposure to Vivado and the other Xilinx tools.

1 Like

LOL… too true.

The biggest problem I’m having solid ideas about the PicoEVB is that there are no good examples… even the GitHub sample uses the infamous Xilinx XDMA library. I know that driver very well. Ugh. And there’s absolutely nothing tutorial in terms of Vivado projects.

They provide the constraints file. But… I don’t know what if I have the Verilog chops to create my own “memory mapped DIO, with MSI on Change of State” application entirely from scratch, which appears to be what will be required. I guess I shall see.

The other thing that made me laugh (literally) about the picoEVB is that it apparently takes 30 minutes to program the bitstream into it.

I’m still waiting for the other, industrial, DIO and ADC card I linked above to arrive. That, sadly, doesn’t interrupt on COS on the DIOs. Sigh.

Peter

Sigh!

PicoEVB: The example is in VHDL, and is super-complicated relative to my abilities.

Peter

PicoEVB: The example is in VHDL, and is super-complicated relative to my abilities.

What about the following one

https://numato.com/product/galatea-pci-express-spartan-6-fpga-development-board

Although it is Spartan 6 -based (which means you would need XILINX ISE, which is much less user-friendly, compared to Vivado, ifyou decide to try this option)
here is a brief tutorial available. Judging from its screenshot, a sample design in Verilog is available. To be honest, just cannot be asked to experimentally establish it, although I’ve got ISE 14.7 installed, but I trust their words.

https://numato.com/kb/getting-started-with-pcie-on-galatea/

OTOH, it does not sound too Windows-friendly

Anton Bassov

What about the following one

Oh, that’s a good thought. But, you noted the issue: You have to use ISE.

Well, after days of searching (and a whole day spent in Vivado trying to create my own design targeted at the PicoEVB), I think I may have found a board: National Instruments, to the rescue!

The board is the PCIe-6509. It meets all the criteria: DIO, simple interface, MSI on line change of state.

The only downside is that it support 96 DIO lines, which is… you know… about 94 more than I need for this example. But I can hide some of that complexity for the purposes of the class, I think.

The register map is public, and their docs look to be sufficient to actually be able to use the device.

I was excited to see that they have a (now long deprecated) WDM driver that’s public for the device… until I realized that what their WDM driver does is map the device’s register space into user-space (via \device\PhysicalMemory, no less). Now I know why the WDM driver has been deprecated. But, no matter, there are user-mode examples that show how to program the registers to make the device do things.

We have NI folks who occasionally visit this Community. I’m kinda surprised nobody suggested their board.

So, it looks like I can skip the FPGA programming (leaving my skills at the “flip THIS switch, light THAT light!” level)… and concentrate on what I sort of know how to do.

Peter

I started on this.

https://www.intel.com/content/www/us/en/programmable/products/boards_and_kits/dev-kits/altera/kit-cyclone-iv-starter.html

That said I have a strong hardware background so I can open up the platform designer/QSys/Sopc builder tools from the relevant Intel FPGA (formally Altera) and figure out the BAR offsets for myself.

If you want something specific you can get it design. Avid Tech in Cleveland were taken over by Avnet.

https://www.avid-tech.com/services/turnkey-design-and-manufacturing/hardware-design/

They are brilliant went it comes to PCIe hardware design.

Thanks.

I’ve gone ahead with the NI PCIe-6509. Lovely device, actually. Very flexible. I’d love to write a real driver for it some day (one for actual production use that fully exploits all its features, as opposed to one that designed for noobs to learn from).

Peter