Why can't we write less than the sector size when the WriteFile handle is PhysicalDrive?

I learned that when you try to write less than the sector size using WriteFile when the handle is PhysicalDrive, you’ll get the error ERROR_INVALID_PARAMETER, and based on other questions asked here its because when writing to disk the size parameter of the WriteFile has to be multiplicand of the sector size.

so basically i have three questions:

  1. why is it OK to write any number of bytes when the handle is a normal file handle? i want to know the detail, is it because a different driver gets the IRP requests when i open the handle to PhysicalDrive compared to a normal file and that driver behaves differently? and if so, what are the drivers responsible for each of them?

  2. isn’t this really redundant? meaning if i want to just write 4 byte in a sector, i have to first read the content of it and then modify the 4 byte and then write the entire sector back! isn’t there a faster way to do this?

  3. is there anyway to bypass this and somehow only write a small amount of bytes in a sector? (i don’t care what the handle is)

Disk drive hardware is only capable of reading/writing in sector sized units.

When you write to a file, data is not written directly to disk (unless you specify the appropriate option to force the system to do so). Rather, the file system works with the windows cache manager to cache the data you write. It saves that data only in memory in cache (if necessary, issuing a read of the data you’re updating), and writes it out “sometime later”. Your write is declared complete as soon as the data has been copied to the cache. Even after the data has been written to disk, the data remains in cache for “a while.”

Likewise with read operations. You ask to read 5 bytes from a file, the file system, cache manager, and memory manager work together to read in and cache some number of sectors of data from the file. The file system then returns you the 5 bytes you asked for.

Hope that helps,

Peter

1 Like

>> why is it OK … when the handle is a normal file handle?

Because the file system driver will do the read-modify-write for you.
No, the Cc does it for the file system, which in turn returns it to
the user. A file system which does not support Cc should not allow
reads/writes which are not sector aligned.
Lanman did - and broke a lot of code when it stopped doing so in Windows 10.

" because at least based on Windows kernel programming book the File system
drivers are below the File manager (where mini filters reside) !"

a) there is no such thing as this ‘file manager’.
b) writing/reading physicaldriveN is accessing the disk subsystem directly,
which is indeed below the filesystem and volume drivers, and yes of course
it bypasses the file system stack entirely, including any filesystem filter
drivers.
c) get a different book.

If you have admin mode enabling you to access the physical disks, you
certainly could write all sorts of bogus crap there. Good luck with your
endeavours.

Mark Roddy

Note: The email was trying to reply to an invalid Comment (297691).

Peter - this is a very patient explanation on your part

@MBond2 said:
Peter - this is a very patient explanation on your part

Thank you, Mr. Bond.

Patience does not come naturally to me, and is something on which I’ve been working. So, it’s nice to hear that feedback.

Peter

1 Like

@Mark_Roddy said:
" because at least based on Windows kernel programming book the File system
drivers are below the File manager (where mini filters reside) !"

a) there is no such thing as this ‘file manager’.
b) writing/reading physicaldriveN is accessing the disk subsystem directly,
which is indeed below the filesystem and volume drivers, and yes of course
it bypasses the file system stack entirely, including any filesystem filter
drivers.
c) get a different book.

If you have admin mode enabling you to access the physical disks, you
certainly could write all sorts of bogus crap there. Good luck with your
endeavours.

Mark Roddy

Note: The email was trying to reply to an invalid Comment (297691).

Can you please suggest another good book that explains these stuff the best ? some people have suggested other books to me, but they were from almost 15-20 years ago and i was not sure if the information inside them still holds in windows 7/10 or not

i want to understand exactly how all the file system drivers and miniport/port drivers work in detail and learn how to program at the lowest layers like miniport/port layer, so any book suggestion that does explain them is appreciated

@“Peter_Viscarola_(OSR)” said:
Disk drive hardware is only capable of reading/writing in sector sized units.

When you write to a file, data is not written directly to disk (unless you specify the appropriate option to force the system to do so). Rather, the file system works with the windows cache manager to cache the data you write. It saves that data only in memory in cache (if necessary, issuing a read of the data you’re updating), and writes it out “sometime later”. Your write is declared complete as soon as the data has been copied to the cache. Even after the data has been written to disk, the data remains in cache for “a while.”

Likewise with read operations. You ask to read 5 bytes from a file, the file system, cache manager, and memory manager work together to read in and cache some number of sectors of data from the file. The file system then returns you the 5 bytes you asked for.

Hope that helps,

Peter

Thank you for your detailed explanation.

There are lots of good resources on the web and in print. Here is some good lower-level information; not too technical but covers a lot of ground from the disk perspective. https://mapr.com/blog/an-introduction-to-disk-storage/

1 Like

without looking at anything you reference, the fundamentals of storage are much older than this book.