If it is the case that fast_mutex is not able to do the latter example, then it is actually more like a spinlock, and not a mutex at all.
Well, to begin with, the scenario B (i.e the one of the nested acquisition of 2 different locks) is perfectly fine with ANY constructs, including spinlocks. The only condition here is that the target OS has to take the original IRQL in case of Windows ( or the original state of interrupt flag in case of Linux) into consideration when releasing a lock. As long as it gets done by the target OS, there is no problem here whatsoever.
However, the scenario A is a totally different story as far as a spinlock owner is concerned - if you try something like that you are just bound to deadlock because of the test-and-set. This is why spinlocks owners have to ensure that this unfortunate scenario cannot occur. In the Windows world they do so by means of elevating IRQL to DPC_LEVEL before attempting test-and-set. The OSes that allow the use of spinlocks in ISRs have to disable interrupts before attempting test-and-set…
Concerning the fast mutex, you can think of it just of a combination of a mutex and a spinlock.
First ExAcquireFastMutex() tries test-and-set, and if it fails, it goes blocking on the dispatcher object (probably, after having had polled the target flag for some reasonable number of iterations). This is why it is “fast” - if the contention is low and/or the mutex is released quickly neither an owner nor a contender needs to go to the system dispatcher upon acquisition/release, effectively reducing the acquisition effort to a simple test-and-set. In this sense, it is conceptually similar to the adaptive mutex on Illumos. The only difference is that it polls the state of the flag, rather than the one of the owner thread. However, if you look at the whole thing from the owner’s perspective, it is, for all practical purposes, just a spinlock that cannot be acquired recursively- a recursive acquisition attempt by the owner thread guarantees a deadlock because of the test-and-set. Therefore, in order to avoid this unfortunate scenario, ExAcquireFastMutex() elevates IRQL to APC_LEVEL before attempting test-and-set…
If that was the case, I would totally understand your reaction, that would be a terrible, terrible use.
As I told you already, what you do is equivalent, in Linux terms, to enabling interrupts while holding a spinlock. I really hope there is no need to explain to you the gravity of this mistake…
Yes, that is how you would do it, and perhaps any Windows dev, would do it that way.
…
Unfortunately, I know Unix inside-and-out but not Windows, so I ported it they way I know, the same way I ported it to OSX.
I hope you DO realise that ZFS is a world on its own, with no easy mapping of its operations to ANY major OS in existence, including even Solaris that it was developed on. In order to make it usable on the host OS ZFS POSIX layer has been designed. What it does is just interfacing ZFS DSL objects, and presenting them as files and directories to the target OS. It makes certain assumptions about the target OS, particularly about the way it handles the file system operations. Once ZFS had been originally designed for Solaris, the more similar to Solaris in this respect the target OS is, the less modifications porting ZFS to it requires.
OSX is more or less similar to FreeBSD by the virtue of being a BSD derivative. Therefore, ZFS POSIX layer naturally maps to OSX file system operations, and does not require the significant modifications to itself when it gets ported to OSX. However, Windows NT kernel is a totally different world, and Solaris-targeting ZFS POSIX layer is completely foreign to it. This is why you are more than likely to get into the trouble if you try to bluntly and dumbly adjust it to Windows. Therefore, assuming that you want it to work fine under Windows, it would be better to interface ZFS objects to Windows in a way that naturally maps to the Windows filesystem operations, rather than trying to emulate the ways that are totally unnatural and foreign for it…
Certainly, the above does not necessarily imply that your code is going to crash and burn straight away - as I can see, you have managed to get away with a deadly serious bug in so far. However, it does not necessarily imply that your code is perfectly fine either, does it…
Anton Bassov