Some questions related to serial number of USB, storages and readers

NTDEV
1

I'm currently working (or you can say, playing) with methods to obtain serial number of disk drives and have observed some scenarios that would like to have explanation. I appreciate any knowledge about them. I'll split the concerns into cases for easy reading.

To getting started, I understand that an USB reports a device identification number in the form
USB\VID_<4 characters of vendor id>&PID_<4 characters of product id>\<identification number>
where the identification number can be the serial number of the USB, or generated by the system if there isn't one. The identification number is formed by some rules of Windows related to information reported by the bus.

  1. I have an USB (let's call this A) connected to the PC. Its device instance id is USB\VID_xxxx&PID_yyyy\6&123456&1. I beleive that 6&123456&1 is the identification number generated by the system, not the actual serial of A (since it has the 6& prefix and &1 postfix). At the beginning, this ID number persists when I plug A into different port of the PC. However, after some time i call CM_Setup_Devnode on the USB, this ID changes. Note that this change does not happens every time CM_Setup_Devnode is triggered, but it does occasionally

[?] My question: Why doesn't this ID number change on changing the port, but on CM_Setup_Devnode calls, and on random times?

  1. I have another USB (let's call this B) and B has a serial X. I can extract this serial from both an USB kernel lower filter driver and SetupDi APIs. However, from the filesystem point of view, I cannot retrieve this using IOCTL_STORAGE_QUERY_PROPERTY: it returns only garbage characters. I did a small check using smartctl on the disk drive mounted by B and smartctl also cannot get the serial number of it

[?] My question: Is it true that X is the serial number of the USB only, not the disk inside it? (therefore I cannot get the serial of the disk drive inside the USB)

  1. I have a Seagate hard disk drive dock (the Seagate dock) and a Samsung HDD. When the HDD is put into the dock, I saw the USB Mass Storage entry in Device Manager, which i think is of the dock. The dock reports its device identification number as USB\VID_mmmm&PID_nnnn\2JHKDGF. I thought the 2JHKDGF is the serial number of the dock, however the tag on the dock shows another serial number.

[?] My question: Why is the dock report another serial number than on the tag? Which one should i follow?

  1. I have another disk drive dock, a Unitek one, which use usbjmicron protocol to communicate between the disk and PC. I put the Samsung HDD into this Unitek dock and check the device identification number of the dock, which now reports the serial number of the HDD in a messy way (the characters of the serial is messed up in an undefined order - not big endian nor little endian). I put another Western Digital Purple HDD (WDP HDD) to the dock, and now the device identification number reports another serial number (which is not of the WPD HDD)

[?] My question: Why is the identification number of the dock inteferred with the serial number of the disks, in a messy way?

  1. In case (3), the identification number of the Seagate dock persists for all HDD and port connected. In case (4), the identification number of the Unitek dock changes when a new HDD is inserted

[?] My question: Why is that? Does it relate to the protocol used for communication?

  1. I have a card reader. When I insert my SD card to it and try to get the disk drive information, I can only get the information of the card reader

[?] My question: Why is that? Is it true that the SD card does not specify any disk information and the only thing it store is the data and therefore, it must be read by a reader? If it's correct, then is it true that I cannot distinguish different SD cards?

Final question: I want to identify and distinguish the USB drive, the HDDs, cards, dock and card reader. From the above scenarios, what conclusion bellow is good? What other conclusion should I make, or should I adjust any of them?

  • USB can be identify and distinguished by the device identification number reported by the USB. The serial number (if there is) is also the serial number of the drive inside it

  • Other kinds of disk can be identified by their model, product name, revision (if there is), and serial number

  • Some docks does not have reliable serial number

  • Card readers can be identified by the device identification number

  • SD cards cannot be distinguised

Again, I would like to send thanks for reading my questions and give answers/references to all my concerns.

2

6&123456&1

Officially, this hardware identifier is undocumented, and any reliance on it is unsupported. Unofficially, it has stayed the same since USB support was introduced. The "6" says you are plugged into the 6th hub layer. The middle number is randomized. The final "1" is your port number in the hub, which might not relate to physical port numbers in any identifiable way.

That identifier will definitely change when you unplug/replug. More specifically, a new identifier will be created. If someone has a file open on the device, the old device object cannot be deleted until all the handles are closed.

Yes, the serial number in the USB\xxx hardware identifier is the serial number from the USB Device Descriptor. Many disk drive manufacturers do not program serial numbers in their descriptors.

USB disk drives use a SCSI command set. The SCSI command set was established LONG before USB was ever thought of, and has its own serial number protocol. Further, the USB scheme has to work with ALL USB devices. At the time the USB storage device is enumerated, there might not even be a drive installed.

So, you need to use the disk drive abstractions to extract the disk drive serial number, not the USB abstractions.

1 Like
3

Hi Tim,

Unofficially, it has stayed the same since USB support was introduced. The "6" says you are plugged into the 6th hub layer. The middle number is randomized. The final "1" is your port number in the hub, which might not relate to physical port numbers in any identifiable way.
That identifier will definitely change when you unplug/replug.

Thanks for the explanation on the randomized identifier. So if I understand you right, if a USB doesn't have a serial number, that identifier with be randomized everytime I replug the USB. So, for the behavior of CM_Setup_Devnode on the USB, it is the same as a replug, so the identifier will likely be change. If there is a change, it is because its logical port number and logical hub number has changed (not the physical port). Am i correct?

You need to use the disk drive abstractions to extract the disk drive serial number, not the USB abstractions

I also notice this and tried to use IOCTL_STORAGE_QUERY_PROPERTY on \\??\\PhysicalDriveX to get the serial number of the disk drive. As I wrote in case (2), for some USB, I cannot get the serial number of the disk drive. More specifically, the USB B only returns ☺♥ for the serial field (i inserted all the blanks here for correctness). So, can i conclude that these smiles and hearts are the serial number of the disk drive, or it does not have any serial number at all? If it truly does not have any serial number, why does it report such garbage characters?

Update: I have a related 7th question. Pardon me to put it here since I suddenly recalled it but cannot edit my original post:

  1. The Samsung HDD has 4 partitions displayed on My Computer. In my minifilter, I receive InstanceSetupCallback for the different pair of (disk number:partition): (1;0), (1;1), (1;2), (1;3). For the first partition (1;0), I notice that it associates with a disk object and therefore, use DEVPKEY_Device_Parent to get the parent's device instance ID of the disk USB\xxx, which is the of the dock in this case. For the other partition, I can only get ROOT\volmgr\0000, which is the device instance ID of the Volume Manager.

[?] My question: I would like to know how the volumes of the same hard disk drive are organized and their relationships to the disk object, the USB stack and volume manager. Can I get the physical parent of the volume when I'm at partition x (x != 0) and how?

4

Hi, still would like to hear any knowledge from you all :smiley:

5

So, can i conclude that these smiles and hearts are the serial number of the disk drive, or it does not have any serial number at all?

Are you checking for an error return, or are you assuming that it succeeds?

6

Hi Tim,

I checked for a success return. Then, in the payload returned by IOCTL_STORAGE_QUERY_PROPERTY, i checked if SerialNumberOffset is nonzero before trying to get the serial, but the data is still unreasonable. What do you think of that?

7

Have you looked at the exact bytes returned? Are they consistent when you call for the same disk?

8

Hi there,

Yes, i've examined the bytes returned several times. It stays the same across checks

9

I think you are going to find great inconsistency for what you are attempting. If you look into the unRAID Linux distro they boot from a USB flash drive and license by the USB drive GUID. People have run into drives with no GUID. There are cheap drives that every one produced has the same GUID. There are counterfeit of major brands that do not have unique GUIDs. People are constantly asking what drive to buy because they try all the ones they have laying around and they either have invalid GUIDs or have been blacklisted as known duplicates.

10

Hi Beaver,

Yes, i understand your concern, and i know that it's impossible to make sure every devices is different. Therefore I just want to know the behavior of the minifilter on these devices to better understand them. Customers have to take responsibilities on using their devices for their purpose :smile:

So i guess I need to accept that the invalid characters are the "serial number" of the disk drive inside the usb (which is invalid) since the minifilter reports this every time and other tools also reports nothing. I have to take the USB's serial number (which is, again, might not be the actual serial number of the USB) in order to identify the device (to some extent)

How about the behaviors of the docks with external disk as i've described in case 3->6?

Thanks again for tackling with my questions :smile: My main purpose is to understanding correctly how they works - so, pardon me if some concerns are naive :sweat_smile:

11

Can you show the bytes that you get? A hex dump

12

You might want to look at what MSFT has done in this area, as they have a long history trying to get this right.

Device Unique Identifiers (DUIDs) for Storage Devices - Windows drivers | Microsoft Learn

You aren't going to be doing anything better than this.

1 Like
13

@MBond2 Hi Bond,

I attached here the full debug view for querying the USB information using IOCTL_STORAGE_QUERY_PROPERTY from my filesystem minifilter in kernel mode. Here we have:

  • In the Local pane: buffer STORAGE_DEVICE_DESCRIPTOR descriptor at address 0xffff9f87aef077c0, with the details:
    • VendorIdOffset, ProductIdOffset, ProductRevisionOffset and SerialNumberOffset are 0x48, 0x51, 0x62, and 0x67, respectively
    • cProductId, cRevision, cSerialNumber, cVendorId (extracted from the descriptor for easy reading) are DataTraveler 3.0, 0000, 0000000005○↑, and Kingston, respectively
  • In the Memory pane: The dump of the descriptor buffer (all the blue highlighted part). I red underlined the part related to the serial number for reference. The 2 final bytes of the serial (before the NULL) are 0x06 0x18, which in ASCII are ○↑ - i refer it as garbage characters.

Do you have any suggestion of what it is?

Screenshot_90
Screenshot_901919×1031 128 KB

@Mark_Roddy Hi Mark,

Thanks for the reference. I've also read about this DUIDs, but I still have some concerns about it:

Some devices don't provide enough information for the system to guarantee that the device's DUID will be sufficiently unique for all uses and all DUID consumers. If the operating system can retrieve unique IDs from the VPD of the device, it can create a DUID that is sufficiently unique for all DUID consumers. But if the system must create a DUID from the device layout signature alone, the DUID is sufficiently unique for some DUID consumers but not for others.

Does it mean if the storage does not provide information in page 0x83, there is still a chance that the DUIDs will collide across machines, isn't it? If it's not, then how the collision is avoided?

And, since this should not be compared byte-by-byte, how can i store this information globally (like into a database) so that I can reuse it across computers in a group?

14

If there is no non-volatile unique data provided by the device then it is basically impossible to guarantee that the device can be uniquely identified. The DUID approach just consolidates all the approaches to disk identification in one api, but it cannot resolve the ambiguity problem, nor can you.

You can of course allocate a small partition and write a guid into it. Obviously the partition can be deleted or modified.

1 Like
15

Hi Mark,

You can of course allocate a small partition and write a guid into it.

That's a nice idea that I've also thought about when brainstorming for solutions to tagging the devices. Of course, everyone can format the USB device so this GUID obviously cannot be sticked to the drive for the USB's whole life.

I still have some question (case 3->7) waiting for some hints from you all :smile: