Using HPAs

OSR_Community_User · February 17, 2011, 1:10am

All,

Once a segment of the storage device is marked as HPA (Host protected
area, also known as Hidden protected area) can we write/read to/from
them from the running OS. I have heard that to access this area from a
running kernel we will need to resize it back to the native size. The
specific IOCTLS need to be sent to the storage CDO and it does it for
us. But what about SCSI/ATA commands, can they be used to write to these
areas without reverting the previous shrink operation?

Reference:

http://en.wikipedia.org/wiki/Host_protected_area

http://www.thinkwiki.org/wiki/Hidden_Protected_Area

Thanks,

AB

Gary_Little-3 · February 17, 2011, 8:02am

What is it you’re trying to do, and why do you want to do it?

Gary G. Little

----- Original Message -----
From: “Amitrajit Banerjee”
To: “Windows System Software Devs Interest List”
Sent: Thursday, February 17, 2011 12:09:28 AM
Subject: [ntdev] Using HPAs

All,

Once a segment of the storage device is marked as HPA (Host protected area, also known as Hidden protected area) can we write/read to/from them from the running OS. I have heard that to access this area from a running kernel we will need to resize it back to the native size. The specific IOCTLS need to be sent to the storage CDO and it does it for us. But what about SCSI/ATA commands, can they be used to write to these areas without reverting the previous shrink operation?

Reference:

http://en.wikipedia.org/wiki/Host_protected_area

http://www.thinkwiki.org/wiki/Hidden_Protected_Area

Thanks,

AB

—
NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

amitr0-2 · February 17, 2011, 8:44am

Gary,

We are investigating on whether this area can be used to have a restore
partition, where we back up restore data. But What I understand is that from
a running OS, anything outside the shrunk area can?t be accessed (I am still
flaky about the details, as this is a very primary level of investigation).

Essentially we need to access that area from a running OS to update any of
the restore files if a newer version is found and some other similar tasks.
We don?t want the user to have to reboot into recovery environment to get
this working.

Hence my doubts.

SO can this be done?

Regards

AB

PS: The list manager bounced my earlier reply attempts twice, so resending
in text format from a diff email ID…

On Thu, Feb 17, 2011 at 6:31 PM, Gary G. Little wrote:

> What is it you’re trying to do, and why do you want to do it?
>
> Gary G. Little
>
> ----- Original Message -----
> From: “Amitrajit Banerjee”
> To: “Windows System Software Devs Interest List”
> Sent: Thursday, February 17, 2011 12:09:28 AM
> Subject: [ntdev] Using HPAs
>
> All,
>
>
>
> Once a segment of the storage device is marked as HPA (Host protected area,
> also known as Hidden protected area) can we write/read to/from them from the
> running OS. I have heard that to access this area from a running kernel we
> will need to resize it back to the native size. The specific IOCTLS need to
> be sent to the storage CDO and it does it for us. But what about SCSI/ATA
> commands, can they be used to write to these areas without reverting the
> previous shrink operation?
>
>
>
> Reference:
>
> http://en.wikipedia.org/wiki/Host_protected_area
>
> http://www.thinkwiki.org/wiki/Hidden_Protected_Area
>
>
>
> Thanks,
>
>
>
>
>
> AB
>
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

–

- amitr0

Gary_Little-3 · February 17, 2011, 9:28am

The quick answer to your question is yes it can be done. I’ve done it, though I was writing diagnostics to test access to that area and not trying to claim large chunks of real estate for an unknown reason. However, it requires knowledge of vendor unique commands and structures that are used to access those areas, as well as knowledge of the format that is used. Note that the drive firmware uses those areas to hold diagnostic information so don’t assume that by your expanding them you will have an additional X gig of unused space for your restore “partition”. You could well find your restore area was claimed by Momma firmware to record why the disk went belly up at 3AM.

So … yes you can, but more than likely you will have to acquire proprietary information from each drive manufacturer, and even partner with them, to expand those areas to allocate space that will NOT be used to hold diagnostic data.

Gary G. Little

----- Original Message -----
From: “amitr0”
To: “Windows System Software Devs Interest List”
Sent: Thursday, February 17, 2011 7:43:31 AM
Subject: Re: [ntdev] Using HPAs

Gary,

We are investigating on whether this area can be used to have a restore partition, where we back up restore data. But What I understand is that from a running OS, anything outside the shrunk area can’t be accessed (I am still flaky about the details, as this is a very primary level of investigation).

Essentially we need to access that area from a running OS to update any of the restore files if a newer version is found and some other similar tasks. We don’t want the user to have to reboot into recovery environment to get this working.

Hence my doubts.

SO can this be done?

Regards

AB

PS: The list manager bounced my earlier reply attempts twice, so resending in text format from a diff email ID…

On Thu, Feb 17, 2011 at 6:31 PM, Gary G. Little < xxxxx@comcast.net > wrote:

What is it you’re trying to do, and why do you want to do it?

Gary G. Little

----- Original Message -----
From: “Amitrajit Banerjee” < xxxxx@symantec.com >
To: “Windows System Software Devs Interest List” < xxxxx@lists.osr.com >
Sent: Thursday, February 17, 2011 12:09:28 AM
Subject: [ntdev] Using HPAs

All,

Once a segment of the storage device is marked as HPA (Host protected area, also known as Hidden protected area) can we write/read to/from them from the running OS. I have heard that to access this area from a running kernel we will need to resize it back to the native size. The specific IOCTLS need to be sent to the storage CDO and it does it for us. But what about SCSI/ATA commands, can they be used to write to these areas without reverting the previous shrink operation?

Reference:

http://en.wikipedia.org/wiki/Host_protected_area

http://www.thinkwiki.org/wiki/Hidden_Protected_Area

Thanks,

AB

—
NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
—
NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

–

- amitr0
— NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and other seminars visit: http://www.osr.com/seminars To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

OSR_Community_User · February 17, 2011, 12:19pm

I think there’s a bit of confusion here. What Gary described is something vendor unique, and is never exposed through standard commands, AFAIK, with a caveat that I’m only familiar with Seagate’s implementation(s).

The HPA is not vendor specific, the opcode(s) are defined in the ATA (and maybe SCSI?) spec(s). Once it is set, commands requesting access to those LBAs are supposed to be returned with IDNF or ABRT set in the Error register. In order to expose them to update the contents, they must be reset. That’s the whole point of the exercise, to mark some LBAs unreachable unless it’s reset.

The OP will need to reset/update/”set to original” if (s)he wants to do what is described in the original posts.

Phil

Philip D. Barila (303) 776-1264

From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Gary G. Little
Sent: Thursday, February 17, 2011 7:27 AM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] Using HPAs

The quick answer to your question is yes it can be done. I’ve done it, though I was writing diagnostics to test access to that area and not trying to claim large chunks of real estate for an unknown reason. However, it requires knowledge of vendor unique commands and structures that are used to access those areas, as well as knowledge of the format that is used. Note that the drive firmware uses those areas to hold diagnostic information so don’t assume that by your expanding them you will have an additional X gig of unused space for your restore “partition”. You could well find your restore area was claimed by Momma firmware to record why the disk went belly up at 3AM.

So … yes you can, but more than likely you will have to acquire proprietary information from each drive manufacturer, and even partner with them, to expand those areas to allocate space that will NOT be used to hold diagnostic data.

Gary G. Little

----- Original Message -----
From: “amitr0”
To: “Windows System Software Devs Interest List”
Sent: Thursday, February 17, 2011 7:43:31 AM
Subject: Re: [ntdev] Using HPAs

Gary,

We are investigating on whether this area can be used to have a restore partition, where we back up restore data. But What I understand is that from a running OS, anything outside the shrunk area can’t be accessed (I am still flaky about the details, as this is a very primary level of investigation).

Essentially we need to access that area from a running OS to update any of the restore files if a newer version is found and some other similar tasks. We don’t want the user to have to reboot into recovery environment to get this working.

Hence my doubts.

SO can this be done?

Regards

AB

PS: The list manager bounced my earlier reply attempts twice, so resending in text format from a diff email ID…

On Thu, Feb 17, 2011 at 6:31 PM, Gary G. Little wrote:

What is it you’re trying to do, and why do you want to do it?

Gary G. Little

----- Original Message -----
From: “Amitrajit Banerjee”
To: “Windows System Software Devs Interest List”
Sent: Thursday, February 17, 2011 12:09:28 AM
Subject: [ntdev] Using HPAs

All,

Once a segment of the storage device is marked as HPA (Host protected area, also known as Hidden protected area) can we write/read to/from them from the running OS. I have heard that to access this area from a running kernel we will need to resize it back to the native size. The specific IOCTLS need to be sent to the storage CDO and it does it for us. But what about SCSI/ATA commands, can they be used to write to these areas without reverting the previous shrink operation?

Reference:

http://en.wikipedia.org/wiki/Host_protected_area

http://www.thinkwiki.org/wiki/Hidden_Protected_Area

Thanks,

AB

—
NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

—
NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

–

- amitr0
— NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and other seminars visit: http://www.osr.com/seminars To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

—
NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

amitr0-2 · February 17, 2011, 2:36pm

>The OP will need to reset/update/?set to original? if (s)he wants to do
what is described in the original posts.
Yes Phil, this is my understanding too after reading the material online.
These areas are not accessible through regular OS. But then again I read
that it is possible in linux and that all rootkit revealers and forensic
tools are based on linux to salvage this. I don’t understand why it cant be
done in wondows then (please excuse me, if I am wrong, my research on this
is still very nascent)

The only thing I can think of is the storport in windows actually cares abt
this and the linux storport (or equiv) doesnt…

On Thu, Feb 17, 2011 at 10:49 PM, Philip D Barila wrote:

> I think there?s a bit of confusion here. What Gary described is
> something vendor unique, and is never exposed through standard commands,
> AFAIK, with a caveat that I?m only familiar with Seagate?s
> implementation(s).
>
>
>
> The HPA is not vendor specific, the opcode(s) are defined in the ATA (and
> maybe SCSI?) spec(s). Once it is set, commands requesting access to those
> LBAs are supposed to be returned with IDNF or ABRT set in the Error
> register. In order to expose them to update the contents, they must be
> reset. That?s the whole point of the exercise, to mark some LBAs
> unreachable unless it?s reset.
>
>
>
> The OP will need to reset/update/?set to original? if (s)he wants to do
> what is described in the original posts.
>
>
>
> Phil
>
>
>
> Philip D. Barila (303) 776-1264
>
>
>
> From: xxxxx@lists.osr.com [mailto:
> xxxxx@lists.osr.com] *On Behalf Of *Gary G. Little
> Sent: Thursday, February 17, 2011 7:27 AM
>
> To: Windows System Software Devs Interest List
> Subject: Re: [ntdev] Using HPAs
>
>
>
> The quick answer to your question is yes it can be done. I’ve done it,
> though I was writing diagnostics to test access to that area and not trying
> to claim large chunks of real estate for an unknown reason. However, it
> requires knowledge of vendor unique commands and structures that are used to
> access those areas, as well as knowledge of the format that is used. Note
> that the drive firmware uses those areas to hold diagnostic information so
> don’t assume that by your expanding them you will have an additional X gig
> of unused space for your restore “partition”. You could well find your
> restore area was claimed by Momma firmware to record why the disk went belly
> up at 3AM.
>
> So … yes you can, but more than likely you will have to acquire
> proprietary information from each drive manufacturer, and even partner with
> them, to expand those areas to allocate space that will NOT be used to hold
> diagnostic data.
>
> Gary G. Little
>
> ----- Original Message -----
> From: “amitr0”
> To: “Windows System Software Devs Interest List”
> Sent: Thursday, February 17, 2011 7:43:31 AM
> Subject: Re: [ntdev] Using HPAs
>
> Gary,
>
>
>
> We are investigating on whether this area can be used to have a restore
> partition, where we back up restore data. But What I understand is that from
> a running OS, anything outside the shrunk area can?t be accessed (I am still
> flaky about the details, as this is a very primary level of investigation).
>
>
>
> Essentially we need to access that area from a running OS to update any of
> the restore files if a newer version is found and some other similar tasks.
> We don?t want the user to have to reboot into recovery environment to get
> this working.
>
>
>
> Hence my doubts.
>
>
>
> SO can this be done?
>
>
>
> Regards
>
>
>
> AB
>
>
>
> PS: The list manager bounced my earlier reply attempts twice, so resending
> in text format from a diff email ID…
>
>
>
>
>
> On Thu, Feb 17, 2011 at 6:31 PM, Gary G. Little
> wrote:
>
> What is it you’re trying to do, and why do you want to do it?
>
> Gary G. Little
>
> ----- Original Message -----
> From: “Amitrajit Banerjee”
> To: “Windows System Software Devs Interest List”
> Sent: Thursday, February 17, 2011 12:09:28 AM
> Subject: [ntdev] Using HPAs
>
> All,
>
>
>
> Once a segment of the storage device is marked as HPA (Host protected area,
> also known as Hidden protected area) can we write/read to/from them from the
> running OS. I have heard that to access this area from a running kernel we
> will need to resize it back to the native size. The specific IOCTLS need to
> be sent to the storage CDO and it does it for us. But what about SCSI/ATA
> commands, can they be used to write to these areas without reverting the
> previous shrink operation?
>
>
>
> Reference:
>
> http://en.wikipedia.org/wiki/Host_protected_area
>
> http://www.thinkwiki.org/wiki/Hidden_Protected_Area
>
>
>
> Thanks,
>
>
>
>
>
> AB
>
>
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
>
>
>
> –
>
> - amitr0
> — NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and
> other seminars visit: http://www.osr.com/seminars To unsubscribe, visit
> the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

–

- amitr0

OSR_Community_User · February 17, 2011, 2:47pm

I’m not sure what you mean by the statement: These are not accessible
through the regular OS. The reserved LBAs are not accessible through
standard commands. FULL STOP. That is, unless you reset the HPA using the
appropriate commands. You can do the entire “Read current HPA
setting”/“reset HPA”/“update your recovery data”/“Restore the HPA” from a
single user-mode application. You need to have appropriate privilege, of
course. Look up the various flavors of ATA/SCSI_PASS_THROUGH for a means of
executing the appropriate ATA/SCSI commands to perform the steps other than
“update your recovery data”. How you perform that step depends on how the
“recovery data” is formatted.

Not sure what you mean by “it is possible in Linux”, either. It’s just as
possible in Linux as in Windows.

Phil

Philip D. Barila (303) 776-1264

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of amitr0
Sent: Thursday, February 17, 2011 12:35 PM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] Using HPAs

The OP will need to reset/update/“set to original” if (s)he wants to do
what is described in the original posts.

Yes Phil, this is my understanding too after reading the material online.
These areas are not accessible through regular OS. But then again I read
that it is possible in linux and that all rootkit revealers and forensic
tools are based on linux to salvage this. I don’t understand why it cant be
done in wondows then (please excuse me, if I am wrong, my research on this
is still very nascent)

The only thing I can think of is the storport in windows actually cares abt
this and the linux storport (or equiv) doesnt…

On Thu, Feb 17, 2011 at 10:49 PM, Philip D Barila
wrote:

I think there’s a bit of confusion here. What Gary described is something
vendor unique, and is never exposed through standard commands, AFAIK, with a
caveat that I’m only familiar with Seagate’s implementation(s).

The HPA is not vendor specific, the opcode(s) are defined in the ATA (and
maybe SCSI?) spec(s). Once it is set, commands requesting access to those
LBAs are supposed to be returned with IDNF or ABRT set in the Error
register. In order to expose them to update the contents, they must be
reset. That’s the whole point of the exercise, to mark some LBAs
unreachable unless it’s reset.

The OP will need to reset/update/“set to original” if (s)he wants to do what
is described in the original posts.

Phil

Philip D. Barila (303) 776-1264

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Gary G. Little
Sent: Thursday, February 17, 2011 7:27 AM

To: Windows System Software Devs Interest List

Subject: Re: [ntdev] Using HPAs

The quick answer to your question is yes it can be done. I’ve done it,
though I was writing diagnostics to test access to that area and not trying
to claim large chunks of real estate for an unknown reason. However, it
requires knowledge of vendor unique commands and structures that are used to
access those areas, as well as knowledge of the format that is used. Note
that the drive firmware uses those areas to hold diagnostic information so
don’t assume that by your expanding them you will have an additional X gig
of unused space for your restore “partition”. You could well find your
restore area was claimed by Momma firmware to record why the disk went belly
up at 3AM.

So … yes you can, but more than likely you will have to acquire
proprietary information from each drive manufacturer, and even partner with
them, to expand those areas to allocate space that will NOT be used to hold
diagnostic data.

Gary G. Little

----- Original Message -----
From: “amitr0”
To: “Windows System Software Devs Interest List”
Sent: Thursday, February 17, 2011 7:43:31 AM
Subject: Re: [ntdev] Using HPAs

Gary,

We are investigating on whether this area can be used to have a restore
partition, where we back up restore data. But What I understand is that from
a running OS, anything outside the shrunk area can’t be accessed (I am still
flaky about the details, as this is a very primary level of investigation).

Essentially we need to access that area from a running OS to update any of
the restore files if a newer version is found and some other similar tasks.
We don’t want the user to have to reboot into recovery environment to get
this working.

Hence my doubts.

SO can this be done?

Regards

AB

PS: The list manager bounced my earlier reply attempts twice, so resending
in text format from a diff email ID…

On Thu, Feb 17, 2011 at 6:31 PM, Gary G. Little
wrote:

What is it you’re trying to do, and why do you want to do it?

Gary G. Little

----- Original Message -----
From: “Amitrajit Banerjee”
To: “Windows System Software Devs Interest List”
Sent: Thursday, February 17, 2011 12:09:28 AM
Subject: [ntdev] Using HPAs

All,

Once a segment of the storage device is marked as HPA (Host protected area,
also known as Hidden protected area) can we write/read to/from them from the
running OS. I have heard that to access this area from a running kernel we
will need to resize it back to the native size. The specific IOCTLS need to
be sent to the storage CDO and it does it for us. But what about SCSI/ATA
commands, can they be used to write to these areas without reverting the
previous shrink operation?

Reference:

http://en.wikipedia.org/wiki/Host_protected_area

http://www.thinkwiki.org/wiki/Hidden_Protected_Area

Thanks,

AB

—
NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

—
NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

–

- amitr0
— NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and
other seminars visit: http://www.osr.com/seminars To unsubscribe, visit the
List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

—
NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

—
NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

–

- amitr0
— NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and
other seminars visit: http://www.osr.com/seminars To unsubscribe, visit the
List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

Gary_Little-3 · February 17, 2011, 4:14pm

Hmmm … I must have snorted my ice cream when I wrote that last. I was indeed referring to the stuff that we were doing for Seagate, but agree that the HPA is not involved in that. Given a controller such as an LSI, you can even do this using IOCTL_SCSI_MINIPORT, but there you do indeed need info from the controller OEM to use that interface.

Gary G. Little

----- Original Message -----
From: “Philip D Barila”
To: “Windows System Software Devs Interest List”
Sent: Thursday, February 17, 2011 1:46:39 PM
Subject: RE: [ntdev] Using HPAs

I’m not sure what you mean by the statement: These are not accessible through the regular OS. The reserved LBAs are not accessible through standard commands. FULL STOP. That is, unless you reset the HPA using the appropriate commands . You can do the entire “Read current HPA setting”/”reset HPA”/”update your recovery data”/”Restore the HPA” from a single user-mode application. You need to have appropriate privilege, of course. Look up the various flavors of ATA/SCSI_PASS_THROUGH for a means of executing the appropriate ATA/SCSI commands to perform the steps other than “update your recovery data”. How you perform that step depends on how the “recovery data” is formatted.

Not sure what you mean by “it is possible in Linux”, either. It’s just as possible in Linux as in Windows.

Phil

Philip D. Barila (303) 776-1264

From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of amitr0
Sent: Thursday, February 17, 2011 12:35 PM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] Using HPAs

> The OP will need to reset/update/”set to original” if (s)he wants to do what is described in the original posts.

Yes Phil, this is my understanding too after reading the material online. These areas are not accessible through regular OS. But then again I read that it is possible in linux and that all rootkit revealers and forensic tools are based on linux to salvage this. I don’t understand why it cant be done in wondows then (please excuse me, if I am wrong, my research on this is still very nascent)

The only thing I can think of is the storport in windows actually cares abt this and the linux storport (or equiv) doesnt…

On Thu, Feb 17, 2011 at 10:49 PM, Philip D Barila < xxxxx@barila.net > wrote:

I think there’s a bit of confusion here. What Gary described is something vendor unique, and is never exposed through standard commands, AFAIK, with a caveat that I’m only familiar with Seagate’s implementation(s).

The HPA is not vendor specific, the opcode(s) are defined in the ATA (and maybe SCSI?) spec(s). Once it is set, commands requesting access to those LBAs are supposed to be returned with IDNF or ABRT set in the Error register. In order to expose them to update the contents, they must be reset. That’s the whole point of the exercise, to mark some LBAs unreachable unless it’s reset.

The OP will need to reset/update/”set to original” if (s)he wants to do what is described in the original posts.

Phil

Philip D. Barila (303) 776-1264

From: xxxxx@lists.osr.com [mailto: xxxxx@lists.osr.com] On Behalf Of Gary G. Little
Sent: Thursday, February 17, 2011 7:27 AM

To: Windows System Software Devs Interest List

Subject: Re: [ntdev] Using HPAs

The quick answer to your question is yes it can be done. I’ve done it, though I was writing diagnostics to test access to that area and not trying to claim large chunks of real estate for an unknown reason. However, it requires knowledge of vendor unique commands and structures that are used to access those areas, as well as knowledge of the format that is used. Note that the drive firmware uses those areas to hold diagnostic information so don’t assume that by your expanding them you will have an additional X gig of unused space for your restore “partition”. You could well find your restore area was claimed by Momma firmware to record why the disk went belly up at 3AM.

So … yes you can, but more than likely you will have to acquire proprietary information from each drive manufacturer, and even partner with them, to expand those areas to allocate space that will NOT be used to hold diagnostic data.

Gary G. Little

----- Original Message -----
From: “amitr0” < xxxxx@gmail.com >
To: “Windows System Software Devs Interest List” < xxxxx@lists.osr.com >
Sent: Thursday, February 17, 2011 7:43:31 AM
Subject: Re: [ntdev] Using HPAs

Gary,

We are investigating on whether this area can be used to have a restore partition, where we back up restore data. But What I understand is that from a running OS, anything outside the shrunk area can’t be accessed (I am still flaky about the details, as this is a very primary level of investigation).

Essentially we need to access that area from a running OS to update any of the restore files if a newer version is found and some other similar tasks. We don’t want the user to have to reboot into recovery environment to get this working.

Hence my doubts.

SO can this be done?

Regards

AB

PS: The list manager bounced my earlier reply attempts twice, so resending in text format from a diff email ID…

On Thu, Feb 17, 2011 at 6:31 PM, Gary G. Little < xxxxx@comcast.net > wrote:

What is it you’re trying to do, and why do you want to do it?

Gary G. Little

----- Original Message -----
From: “Amitrajit Banerjee” < xxxxx@symantec.com >
To: “Windows System Software Devs Interest List” < xxxxx@lists.osr.com >
Sent: Thursday, February 17, 2011 12:09:28 AM
Subject: [ntdev] Using HPAs

All,

Once a segment of the storage device is marked as HPA (Host protected area, also known as Hidden protected area) can we write/read to/from them from the running OS. I have heard that to access this area from a running kernel we will need to resize it back to the native size. The specific IOCTLS need to be sent to the storage CDO and it does it for us. But what about SCSI/ATA commands, can they be used to write to these areas without reverting the previous shrink operation?

Reference:

http://en.wikipedia.org/wiki/Host_protected_area

http://www.thinkwiki.org/wiki/Hidden_Protected_Area

Thanks,

AB

—
NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

—
NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

–

- amitr0
— NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and other seminars visit: http://www.osr.com/seminars To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

—
NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

—
NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

–

- amitr0
— NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and other seminars visit: http://www.osr.com/seminars To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
—
NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

amitr0-2 · February 18, 2011, 1:07am

Phil, Gary,

Thanks for the answers and resolving some of my doubts.

best regards

amit

On Fri, Feb 18, 2011 at 2:43 AM, Gary G. Little wrote:

> Hmmm … I must have snorted my ice cream when I wrote that last. I was
> indeed referring to the stuff that we were doing for Seagate, but agree that
> the HPA is not involved in that. Given a controller such as an LSI, you can
> even do this using IOCTL_SCSI_MINIPORT, but there you do indeed need info
> from the controller OEM to use that interface.
>
> Gary G. Little
>
> ----- Original Message -----
> From: “Philip D Barila”
> To: “Windows System Software Devs Interest List”
> Sent: Thursday, February 17, 2011 1:46:39 PM
> Subject: RE: [ntdev] Using HPAs
>
> I?m not sure what you mean by the statement: These are not accessible
> through the regular OS. The reserved LBAs are not accessible through
> standard commands. FULL STOP. That is, unless you reset the HPA using
> the appropriate commands. You can do the entire ?Read current HPA
> setting?/?reset HPA?/?update your recovery data?/?Restore the HPA? from a
> single user-mode application. You need to have appropriate privilege, of
> course. Look up the various flavors of ATA/SCSI_PASS_THROUGH for a means of
> executing the appropriate ATA/SCSI commands to perform the steps other than
> ?update your recovery data?. How you perform that step depends on how
> the ?recovery data? is formatted.
>
>
>
> Not sure what you mean by ?it is possible in Linux?, either. It?s just as
> possible in Linux as in Windows.
>
>
>
> Phil
>
>
>
> Philip D. Barila (303) 776-1264
>
>
>
> From: xxxxx@lists.osr.com [mailto:
> xxxxx@lists.osr.com] *On Behalf Of *amitr0
> Sent: Thursday, February 17, 2011 12:35 PM
> To: Windows System Software Devs Interest List
> Subject: Re: [ntdev] Using HPAs
>
>
>
> >The OP will need to reset/update/?set to original? if (s)he wants to do
> what is described in the original posts.
>
> Yes Phil, this is my understanding too after reading the material online.
> These areas are not accessible through regular OS. But then again I read
> that it is possible in linux and that all rootkit revealers and forensic
> tools are based on linux to salvage this. I don’t understand why it cant be
> done in wondows then (please excuse me, if I am wrong, my research on this
> is still very nascent)
>
>
>
> The only thing I can think of is the storport in windows actually cares abt
> this and the linux storport (or equiv) doesnt…
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Thu, Feb 17, 2011 at 10:49 PM, Philip D Barila
> wrote:
>
> I think there?s a bit of confusion here. What Gary described is something
> vendor unique, and is never exposed through standard commands, AFAIK, with a
> caveat that I?m only familiar with Seagate?s implementation(s).
>
>
>
> The HPA is not vendor specific, the opcode(s) are defined in the ATA (and
> maybe SCSI?) spec(s). Once it is set, commands requesting access to those
> LBAs are supposed to be returned with IDNF or ABRT set in the Error
> register. In order to expose them to update the contents, they must be
> reset. That?s the whole point of the exercise, to mark some LBAs
> unreachable unless it?s reset.
>
>
>
> The OP will need to reset/update/?set to original? if (s)he wants to do
> what is described in the original posts.
>
>
>
> Phil
>
>
>
> Philip D. Barila (303) 776-1264
>
>
>
> From: xxxxx@lists.osr.com [mailto:
> xxxxx@lists.osr.com] *On Behalf Of *Gary G. Little
> Sent: Thursday, February 17, 2011 7:27 AM
>
>
> To: Windows System Software Devs Interest List
>
> Subject: Re: [ntdev] Using HPAs
>
>
>
> The quick answer to your question is yes it can be done. I’ve done it,
> though I was writing diagnostics to test access to that area and not trying
> to claim large chunks of real estate for an unknown reason. However, it
> requires knowledge of vendor unique commands and structures that are used to
> access those areas, as well as knowledge of the format that is used. Note
> that the drive firmware uses those areas to hold diagnostic information so
> don’t assume that by your expanding them you will have an additional X gig
> of unused space for your restore “partition”. You could well find your
> restore area was claimed by Momma firmware to record why the disk went belly
> up at 3AM.
>
> So … yes you can, but more than likely you will have to acquire
> proprietary information from each drive manufacturer, and even partner with
> them, to expand those areas to allocate space that will NOT be used to hold
> diagnostic data.
>
> Gary G. Little
>
> ----- Original Message -----
> From: “amitr0”
> To: “Windows System Software Devs Interest List”
> Sent: Thursday, February 17, 2011 7:43:31 AM
> Subject: Re: [ntdev] Using HPAs
>
> Gary,
>
>
>
> We are investigating on whether this area can be used to have a restore
> partition, where we back up restore data. But What I understand is that from
> a running OS, anything outside the shrunk area can?t be accessed (I am still
> flaky about the details, as this is a very primary level of investigation).
>
>
>
> Essentially we need to access that area from a running OS to update any of
> the restore files if a newer version is found and some other similar tasks.
> We don?t want the user to have to reboot into recovery environment to get
> this working.
>
>
>
> Hence my doubts.
>
>
>
> SO can this be done?
>
>
>
> Regards
>
>
>
> AB
>
>
>
> PS: The list manager bounced my earlier reply attempts twice, so resending
> in text format from a diff email ID…
>
>
>
>
>
> On Thu, Feb 17, 2011 at 6:31 PM, Gary G. Little
> wrote:
>
> What is it you’re trying to do, and why do you want to do it?
>
> Gary G. Little
>
> ----- Original Message -----
> From: “Amitrajit Banerjee”
> To: “Windows System Software Devs Interest List”
> Sent: Thursday, February 17, 2011 12:09:28 AM
> Subject: [ntdev] Using HPAs
>
> All,
>
>
>
> Once a segment of the storage device is marked as HPA (Host protected area,
> also known as Hidden protected area) can we write/read to/from them from the
> running OS. I have heard that to access this area from a running kernel we
> will need to resize it back to the native size. The specific IOCTLS need to
> be sent to the storage CDO and it does it for us. But what about SCSI/ATA
> commands, can they be used to write to these areas without reverting the
> previous shrink operation?
>
>
>
> Reference:
>
> http://en.wikipedia.org/wiki/Host_protected_area
>
> http://www.thinkwiki.org/wiki/Hidden_Protected_Area
>
>
>
> Thanks,
>
>
>
>
>
> AB
>
>
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
>
>
>
> –
>
> - amitr0
> — NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and
> other seminars visit: http://www.osr.com/seminars To unsubscribe, visit
> the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
>
>
>
> –
>
> - amitr0
> — NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and
> other seminars visit: http://www.osr.com/seminars To unsubscribe, visit
> the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

–

- amitr0