Like IOCTL_DISK_IS_WRITABLE is there for finding whether disk is online or
not, do we have any IOCTL to know whether volume is offline or not. I am
trying to direct write on volume at the time of failover and it is failing
sometimes with status_device_busy in the dispatch routine for device IO
control (volume offline IOCTL).Can anyone explain the meaning of this
status. For that purpose i want to know whether volume is offline or disk is
offline or both are offline. I am doing this for cluster in win2k8
My 2 cents…
In Windows clustering solution, as soon as the service/app is failover-ed,
the respective disk will go to ‘Reserved’ state in disk management; it means
the application/driver cannot to any read/write IO to the disk. At the
timing of failover, I feel the status is expected one! I don’t know, whether
any IOCTL/API exist to find volume is offline or not.
Is your problem is, your driver couldn’t able to find when the ‘Failover
Cluster’ is starting it failover process?
Out of curiosty… Can you tell me, what is the operation you are trying to
do from your driver at the time of failover?
Regards,
T.V.Gokul.
On Fri, Dec 26, 2008 at 6:55 PM, Rohit wrote:
> Like IOCTL_DISK_IS_WRITABLE is there for finding whether disk is online or
> not, do we have any IOCTL to know whether volume is offline or not. I am
> trying to direct write on volume at the time of failover and it is failing
> sometimes with status_device_busy in the dispatch routine for device IO
> control (volume offline IOCTL).Can anyone explain the meaning of this
> status. For that purpose i want to know whether volume is offline or disk is
> offline or both are offline. I am doing this for cluster in win2k8
>
> — NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and
> other seminars visit: http://www.osr.com/seminars To unsubscribe, visit
> the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
At 16:30 28/12/2008, Gokul TV wrote:
My 2 cents…
In Windows clustering solution, as soon as the service/app is
failover-ed, the respective disk will go to ‘Reserved’ state in disk
management; it means the application/driver cannot to any read/write
IO to the disk. At the timing of failover, I feel the status is
expected one! I don’t know, whether any IOCTL/API exist to find
volume is offline or not.
Your understanding is out of date. Server 2008 switched to using
Persistent Reservations. Typically it uses a reservation type of
Write_Exclusive_Registrants_only. Which means that all other
registrants can still read the volume. In effect the volume is not offline.
Mark
Mark,
>Your understanding is out of date. Server 2008 switched to using
Persistent Reservations. Typically it uses a reservation type of
>Write_Exclusive_Registrants_only.
I’m afraid, you are talking about Cluster Shared Volume (new feature in
Win2k8 R2, Cluster failover) with SCSI-3 PR implementation. To supplement my
point, you can even do persistent reservations with Windows Server 2003 MSCS
cluster, if your underlying multipathing solution maps SCSI-2
RESERVE/RELEASE commands to SCSI-PR commands.
Ok, without digressing myself, I agree in Win2k8, ‘Failover Clustering’ has
been moved to native SCSI-3 PR support and they reserve with
‘WRITE-EXCLUSIVE-REGISTRANTS-ONLY’ reservation type; but it doesn’t mean
that, the other node (non-owning/challenging node) can read the same shared
volume or disk. Becuase the registration keys which is used to reserve a
disk, across the nodes will never be the same.
Coming to the OP observation, my understanding is, whenever the failover
starts the owning node, brings the volume to the OFFLINE state (and later to
RESERVED state), before releases the disk reservation (IIRC, it is achieved
by PR-CLEAR and un-register itself by PR-REGISTER with Registration keys
0x00…0). So that the taking over node can register (with PR-REGISTER + new
unique PR Registration keys) and RESERVE (with PR-RESERVE), to use the disk
further.
> Which means that all other registrants can still read the volume. In
effect the volume is not offline.
All registrants with **SAME** Registration Keys can read the volume; this is
used by the multipathing solution to register the multiple paths leading to
the same disk, with same registration keys - ultimately to achieve path
redundancy & load balancing for the disk, even with clustering solution.
Regards,
Gokul TV
Thanks Gokul & Mark for the reply.
In win 2k8 cluster i have observed that IOCTL IOCTL_VOLUME_OFFLINE is issued
by rhs .exe at the time of failover for all the volumes lying on cluster
disk. I am trying to write some data directly to volume in that IOCTL and
after i passed the IOCTL down. This is working fine for failover through
cluster admin gui. But when i shutdown actiive node, I observed that Volume
Offline IOCTL is still coming but some times i am not able to write data to
the underlying volume. This is happening rarely. I am getting this behaviour
once in a while.
Regards
Rohit
On Mon, Dec 29, 2008 at 9:42 AM, Gokul TV wrote:
> Mark,
>
> >>Your understanding is out of date. Server 2008 switched to using
> Persistent Reservations. Typically it uses a reservation type of
> >>Write_Exclusive_Registrants_only.
>
> I’m afraid, you are talking about Cluster Shared Volume (new feature in
> Win2k8 R2, Cluster failover) with SCSI-3 PR implementation. To supplement my
> point, you can even do persistent reservations with Windows Server 2003 MSCS
> cluster, if your underlying multipathing solution maps SCSI-2
> RESERVE/RELEASE commands to SCSI-PR commands.
>
> Ok, without digressing myself, I agree in Win2k8, ‘Failover Clustering’ has
> been moved to native SCSI-3 PR support and they reserve with
> ‘WRITE-EXCLUSIVE-REGISTRANTS-ONLY’ reservation type; but it doesn’t mean
> that, the other node (non-owning/challenging node) can read the same shared
> volume or disk. Becuase the registration keys which is used to reserve a
> disk, across the nodes will never be the same.
>
> Coming to the OP observation, my understanding is, whenever the failover
> starts the owning node, brings the volume to the OFFLINE state (and later to
> RESERVED state), before releases the disk reservation (IIRC, it is achieved
> by PR-CLEAR and un-register itself by PR-REGISTER with Registration keys
> 0x00…0). So that the taking over node can register (with PR-REGISTER + new
> unique PR Registration keys) and RESERVE (with PR-RESERVE), to use the disk
> further.
>
> >> Which means that all other registrants can still read the volume. In
> effect the volume is not offline.
>
> All registrants with SAME Registration Keys can read the volume; this
> is used by the multipathing solution to register the multiple paths leading
> to the same disk, with same registration keys - ultimately to achieve path
> redundancy & load balancing for the disk, even with clustering solution.
>
> Regards,
> Gokul TV
> — NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and
> other seminars visit: http://www.osr.com/seminars To unsubscribe, visit
> the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
–
Regards
Rohit Gauba
“A positive thought is the seed of a positive result”
Rohit,
Are you seeing any IOCTL_STORAGE_PERSISTENT_RESERVATIONS_OUT, before or
after volume offline IOCTL?
I think, before you write to the disk and sending down the volume offline
IOCTL, the ‘Failover Cluster’ (rhs.exe is part of Failover Cluster Solution
- for Resource Hosting Subsystem) issues PR clear commands to release the
reservation and the challenging node is taking over the disk by reserving
it. But I repeat, this is my guess :).
Are you saving any information in the disk for the challenging node, to use
after failover? In such case, there are other supported ways to acheive
this, like writing into the cluster configuration database (cluster registry
hive).
You can find more information in,
http://download.microsoft.com/download/5/b/9/5b97017b-e28a-4bae-ba48-174cf47d23cd/SER119_WH06.ppt
http://technet.microsoft.com/en-us/magazine/2008.07.failover.aspx
Regards,
T.V.Gokul.
Gokul,
Volume offline IOCTL is coming before persistent reserve out command and
both volume offline IOCTL and persistent reserve out command comes in the
same thread context.
Regards
Rohit Gauba
On Mon, Dec 29, 2008 at 11:58 AM, Gokul TV wrote:
> Rohit,
>
> Are you seeing any IOCTL_STORAGE_PERSISTENT_RESERVATIONS_OUT, before or
> after volume offline IOCTL?
>
> I think, before you write to the disk and sending down the volume offline
> IOCTL, the ‘Failover Cluster’ (rhs.exe is part of Failover Cluster Solution
> - for Resource Hosting Subsystem) issues PR clear commands to release the
> reservation and the challenging node is taking over the disk by reserving
> it. But I repeat, this is my guess :).
>
> Are you saving any information in the disk for the challenging node, to use
> after failover? In such case, there are other supported ways to acheive
> this, like writing into the cluster configuration database (cluster registry
> hive).
>
> You can find more information in,
>
> http://download.microsoft.com/download/5/b/9/5b97017b-e28a-4bae-ba48-174cf47d23cd/SER119_WH06.ppt
> http://technet.microsoft.com/en-us/magazine/2008.07.failover.aspx
>
>
> Regards,
> T.V.Gokul.
>
>
> — NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and
> other seminars visit: http://www.osr.com/seminars To unsubscribe, visit
> the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
–
Regards
Rohit Gauba
“A positive thought is the seed of a positive result”
Rohit,
Do you have the SRB_STATUS, while the STATUS_BUSY (which is NT_STATUS) is
returned? If you own miniport driver or multipathing solution in the stack,
you can easily find this SRB STATUS (and you cannot get this SRB_STATUS
above disk.sys in the stack).
BTW, I still dint get your intention, why you want to write the information
in the shared disk, during failover process? If I get the clear picture, we
can triage or find better alternatives.
Regards,
T.V.Gokul.