520 Byte Sector Harddisk problem

NTDEV
21

It’s not nearly as cut-and-dried as you might think, really. The only way everything can be made to work is by having the adapter/miniport surface a LOGICAL sector size of 512 (and probably also a physical sector size of 512, incidentally), and just “handle it” internally… but that’s quite a big problem unless the drives themselves allow 512 byte aligned transfers. Otherwise, there’d be a lot of read-modify-write cycles needed (or lots of data recopying).

By your definition, there are a LOT – like I would guess the majority – of very stupid apps and drivers. Including, as Mr. Edwards pointed out, the BIOS.

That 512 number has been a constant for a LONG time. And there’s LOTs of software that uses the equivalent of

#define SECTOR_SIZE 512

Yeah… except for all those SCSI passthrough users and users of IOCTLs that contain information about sectors.

I’m not saying it’s impossible, I’m just saying that when you lift this rock, it’s pretty astonishing what you find crawling out.

Peter
OSR

22

> More so, I expect Linux to be the same, and, from the article provided by Anton, it is obvious that

the data sectors are still 512,

Linux kernel thinks in terms of 512-byte sectors - although blk_queue_hardsect_size() allows you to specify
sector size that your hardware supports so that all I/O requests are properly aligned at the beginning of a hardware sector and the length of each request is an integral number of sectors, you still have to translate all sector numbers accordingly. The immediate consequence is that sector size has to be a multiple of 512( it cannot exceed PAGE_CACHE_SIZE for understandable reasons )

Linux does not support 520-byte sectors unless the drive is formatted with DIF and hanging off a DIF-capable HBA…

Anton Bassov

23

In my opinion, extra 8 bytes on these disks are used for not only CRC
but also application specific tag(checksum).
Application specific tags are often useful in detecting lost write
issues where you a give a buffer to disk to write, disk acknowledges the
write but data never makes it to stable medium or the entire blk gets
overwritten(crc check will still pass when data is read!) erroneously.
The original writer (maybe FS!) reads the data but also gets back its
own tag as part of read. This allows for detection of lost write issues.

Full benefit can only be realized when entire stack(FS-LDM-DISK-HBA) has
the infrastructure for writing, reading and checking this 8 byte data.

Harish

-----Original Message-----
From: xxxxx@osr.com [mailto:xxxxx@osr.com]
Sent: Monday, August 09, 2010 9:13 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] 520 Byte Sector Harddisk problem

It’s not nearly as cut-and-dried as you might think, really. The only
way everything can be made to work is by having the adapter/miniport
surface a LOGICAL sector size of 512 (and probably also a physical
sector size of 512, incidentally), and just “handle it” internally…
but that’s quite a big problem unless the drives themselves allow 512
byte aligned transfers. Otherwise, there’d be a lot of
read-modify-write cycles needed (or lots of data recopying).

By your definition, there are a LOT – like I would guess the majority
– of very stupid apps and drivers. Including, as Mr. Edwards pointed
out, the BIOS.

That 512 number has been a constant for a LONG time. And there’s LOTs
of software that uses the equivalent of

#define SECTOR_SIZE 512

Yeah… except for all those SCSI passthrough users and users of IOCTLs
that contain information about sectors.

I’m not saying it’s impossible, I’m just saying that when you lift this
rock, it’s pretty astonishing what you find crawling out.

Peter
OSR

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

24

“Maxim S. Shatskih” wrote in message
news:xxxxx@ntdev…
> I think that the following is a major architectural thing in Windows,
> deeply hard-coded to the OS:
> a) sector size <= page size
> b) page size is an integral multiple of sector size

This indeed is major architectural thing, deeply hard-coded to the OS.
It comes from the DEC virtual memory architechure that influenced Windows
and various unixes:
A RAM page maps to a disk storage unit.

> More so, I expect Linux to be the same, and, from the article provided by
> Anton, it is obvious that the data sectors are still 512, and the
> remaining 8 bytes/sector are provided via some additional kinda
> “out-of-band” interface (for software RAID implementations above the disk
> stack I think).
>
> Now about 4KB sectors. Their only negative footprint would result from the
> alignment issues, but IIRC all of them support 512byte-aligned transfers
> anyway (but slower).

WD provides utility for their large sector disks that realigns stuff
properly.
Run it once, and the disk will be optimized.

– pa

> As about the rest of 4KB/s issue - even if the app uses sector sizes and
> sector numbers (and not byte offsets), it must be very stupid to
> hardcode sector size == 512. And, if there is no such hard-code, then the
> app is fine.
>
> Anyway Windows IO API is formulated in terms of byte offsets and not
> sector numbers.
>
> –
> Maxim S. Shatskih
> Windows DDK MVP
> xxxxx@storagecraft.com
> http://www.storagecraft.com
>
>

25

By the way, in cdrom.sys when you use non-power of 2 sector size, such as with IOCTL_CDROM_RAW_READ for audio CD (2352 bytes), you need to pass offsets in 2048 bytes per sector. But that’s just because cdrom.sys knows about raw audio block size.

26

> By the way, in cdrom.sys when you use non-power of 2 sector size, such as with

IOCTL_CDROM_RAW_READ for audio CD (2352 bytes), you need to pass offsets in 2048 bytes

Yes, and on underlying MMC level you must pass block numbers and not offsets.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com