I'm writing a driver for a simple digital input device. The result of processing of input data has to be made persistent from within a driver, as data cannot be lost even in case of a power failure. The system maintains power for a short time only (10-20ms) after mains power failure using a power reserve.
As the system has flash memory storage device, I was thinking of using that physical device to store the critical data as it becomes available from the digital input device.
I would be quite happy to use a simple disk partition as a linear arrays of storage blocks, I don't need a file system.
The main problem is I don't even know where to start, as I couldn't find in WDF anything that talks specifically about storage devices, partitioning, file systems, etc. In particular:
how can find out the list of devices from within a WDF driver? The list of partitions?
how can I open the partition as an WDFIOTARGET, if that's possible? How do I know the name of the device to use?
how do I do seek, read and write on such a device?
As you can see from my questions, I'm totally clueless, I don't know where to start I've written a few WDF drivers that work just fine, but this area is totally new to me...
I know that you are looking for concrete information about how to access a disk device from your driver, but first it should be pointed out that there is no way to ever prevent all possible data loss, so you need to have some kind of strategy to handle that no matter how rarely it happens
also, 20 ms is within the scheduling time frame. It is very unlikely that you can achieve a high level of reliability within that timeframe. Especially if you have no way to know when the mains power has been lost before the hardware starts to shutdown. Most power supplies have some tolerance for lost input current. In a desktop, the power supply might have capacitors to bridge say 100 ms. Some server systems can bridge several seconds of lost input current. but there is rarely any way to know that this is happening. If you don't have a way to know, then the question becomes not one of not loosing any data, but of curtailing the max expected loss. note expected, as in you expect to loose data on every event, and the goal is to control how much
When you say 'the system', that probably implies that you have control over the hardware. If not control yourself, you can be certain of it - so this is some sort of embedded situation and not a mass market product. So the first step to answering your question about how to find the device to use is to know how it is connected to your system and exactly what type of device it is.
Then you will have a couple of choices, but it is hard to help without the basics
I forgot to mention that the system does have a power fail signal, after which a 20ms countdown begins.
This allows me to block any further requests that may imply more modifications to the persistent state. It also gives me some time to finish updating persistent structures so when the power is finally gone there are no dangling pointers, etc.
The original solution used battery-backed RAM for storage and it worked just fine. But that is no longer an option with the new hardware, so one option I'm being given is a flash device.
Which brings me back to my original problem: can I, and how, use a raw disk partition, to keep persistent data required by my driver?
Seriously, I have full info on various latencies involved from the hardware engineering department, so it looks it would all fit comfortably within 20ms target, not just a single block write, but multiple ones.
Additionally, the storage device itself has its own power-loss detection and local power-reserve that allows it to guarantee that any data in its cache will be written to the storage. At least, that's what they promise.
How your driver receives this signal? Thru some GPIO interface or ACPI?
We've done a similar thing on a Linux board with eMMC storage which had no other activity. Wrote ~ 1KB of data and dismounted the volume, in ~20 ms. Not sure if 10 ms is enough. On a bare metal system, maybe.
Often a Windows system has a bunch of various filesystem filters, useful or not. These will add to the latency.
Well, to answer your original question: in Windows, drivers can open and write files. So you can just do this. You can also use registry which is backed by disk files. The question is, how you propagate the power-off signal to Windows so that the disk caches are properly flushed and system shuts down properly.
Embedded-friendly versions (Windows IoT or Embedded Standard) have special means to mitigate filesystem corruption due to hard shut down, such as read-only disk filter etc. You may want to look into that.
I was hoping I could do that, but my question was HOW? Where are the descriptions of APIs? How do I open the device if I know the device name and partition I want to use? I know the kernel has its own namespaces and how it enumerates various devices, but these usually have nothing to do with device names we see on user level.
Registry is an interesting idea, but it's not suitable for my purpose.
As for flushing disk caches, this is done by hardware. The device I'm about to use has an additional power reserve source (I think just a large capacitor) that has enough energy to commit data from the device cache into the persistent memory.
Caching inside the disk is another matter. If there is a file system mounted, it will have a cache. Your IO can bypass it, but special restrictions apply. Similarly, if there is no file system mounted, special restrictions apply.
so, how is your storage connected to the system? can you be confident that it will always be connected this way? Does the user / operator have any choice in the storage location?
Does that storage need to mount a file system (i.e. so it can store other stuff besides your special data), or will you do raw disk IO?
Those answers will tell you what kind of name you need, and how you can know how to find / build it.
one way or another, you are going to want to make sector aligned writes with caching disabled and the force unit access flag set. Avoiding data corruption, and recovering from data loss are hard, especially if you need to have multiple disk writes 'in flight' concurrently
I just need a raw partition, no file systems involved. I'm also aware that many drivers in the driver stack may have some internal caches. However, I expect that one of the APIs I'm looking for will also have an equivalent of flush operation, that would force all of these to the hardware device.
The device is connected to the system via NVMe bus and since this is an embedded system, it's a fixed solution, you can't replace it with, say, USB. There is no user choice in this.
Anyway, this discussion is really diverging from the main question that I have and for which there is still no answer: where are the APIs, available on kernel level, I could use in order to store/retrieve data from my kernel driver? The link to MS documentation would be great.
I've written a few WDF drivers that work just fine, but this area is totally new to me...