how to find out Physical Sector of MFT file record of file?

Hi Folks,

I am investigating how to find out physical sector address of MFT entry of perticular file using programming constructs?

For example: I want to find out sector of file c:\test\test.txt, I know MFTStartLcn but using this information how to go to actual sector of the MFT record of test.txt file?

Any help is highly appreciated.

Cheers,
Amrat

xxxxx@gmail.com wrote:

I am investigating how to find out physical sector address of MFT entry of perticular file using programming constructs?

For example: I want to find out sector of file c:\test\test.txt, I know MFTStartLcn but using this information how to go to actual sector of the MFT record of test.txt file?

Have you done any reading at all on the layout of an NTFS file system
and the format of the MFT? Sector numbers aren’t interesting; you deal
with byte offsets. One of the MFT metafiles gives you the location of
the root directory. You read that to determine the location of the
“test” directory. You then read that to determine the location of
“test.txt”.

Why do you want to do this?


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

Hi Amrat,

The goal you are trying to achieve can be done in three ways:

  1. As Tim Roberts pointed out, you can read the INDEX attributes of every
    directory from the volume root. (Very Long way)
  2. After getting the MftLcn from bootsector, you can enumerate every file
    and search for File/Directory names,
    and test for ParentDirectory’s Segment Reference Number. This is a very
    long way too.
  3. Fastest method is getting the file’s cluster number via
    FSCTL_GET_RETRIEVAL_POINTERShttp:then
    you can easily read
    the contents of the file.

    Hope it helps,

    Emre TINAZTEPE

    On Tue, Jan 31, 2012 at 8:22 PM, Tim Roberts wrote:

    > xxxxx@gmail.com wrote:
    > > I am investigating how to find out physical sector address of MFT entry
    > of perticular file using programming constructs?
    > >
    > > For example: I want to find out sector of file c:\test\test.txt, I know
    > MFTStartLcn but using this information how to go to actual sector of the
    > MFT record of test.txt file?
    >
    > Have you done any reading at all on the layout of an NTFS file system
    > and the format of the MFT? Sector numbers aren’t interesting; you deal
    > with byte offsets. One of the MFT metafiles gives you the location of
    > the root directory. You read that to determine the location of the
    > “test” directory. You then read that to determine the location of
    > “test.txt”.
    >
    > Why do you want to do this?
    >
    > –
    > Tim Roberts, xxxxx@probo.com
    > Providenza & Boekelheide, Inc.
    >
    >
    > —
    > NTDEV is sponsored by OSR
    >
    > For our schedule of WDF, WDM, debugging and other seminars visit:
    > http://www.osr.com/seminars
    >
    > To unsubscribe, visit the List Server section of OSR Online at
    > http://www.osronline.com/page.cfm?name=ListServer
    ></http:>

Thanks guys - that helps.

Tim: I know it works on byte offset, may be I was not very clear on what I wanted.
I want to find out physical sector of MFT record(not file extent or data extent) of file inside MFT.

All I want to do is:

  1. Use VOLUME_LOGICAL_OFFSET and pass it to IOCTL with IOCTL_VOLUME_LOGICAL_TO_PHYSICAL control code and which would return the physical sector.

  2. I have to fill the VOLUME_LOGICAL_OFFSET.LogicalOffset.

How do I calculate logicaloffset(BYTE Offset) using ntfsVolData.MftStartLcn and MFTRecordNumber?
Are MFTRecrodNumbers(for files and directories) indexed in MFT layout incrementally after first 16 reserved Entries?

Emre: Thanks for inputs.

Please suggest, how to go after what I mentioned above.

Thanks,
Amrat

Hi Amrat,

In order to calculate the offset of a file from MftRecordNumber, you need
to get the Extents of $MFT file.
Since $MFT file’s $DATA attribute is layed out to disk in parts, you will
need to calculate on which offsets do they start and
size of them.
These fragmentations are called “Runs”.
To give an example :
My disk’s $MFT is composed of two Runs,

  1. LCN 0xC0000 - Lenght 0x4540 (Disk offset 0xC0000000)
  2. LCN 0x3A45B34 - Length 0x2DC80 => this LCN is relative to the first one
    which makes it @ LCN 0x3B05B34

At this point, we know that we have total (0x4540 + 0x2DC80) clusters which
totals to 0x321C0 Clusters.
Given the size of cluster is 4096 Bytes (from VolData) and size of
MftRecord is 1024 Bytes,
this means each clusters holds 4 MftRecords. Which means i have a total
0x321C0 * 4 = 0xC8700 Mft Records( 820992 decimal)

In order to programatically get to this point :

  1. Load MFT record FSCTL_GET_NTFS_FILE_RECORD or read size of MftRecord
    structure from the MftStartLcn (they point to same offset)
  2. FindDataAttribute of $MFT file
  3. Get run array of the file (LCN & Length)

In order to find information about MFT Parsing functions like FindAttribute
/ FindRun , you may have a look at Garry Nebbets doc
here
http://www.installsetupconfig.com/win32programming/1996%20AppE_apnilife.pdf

Emre TINAZTEPE

On Wed, Feb 1, 2012 at 8:13 AM, wrote:

> Thanks guys - that helps.
>
> Tim: I know it works on byte offset, may be I was not very clear on what I
> wanted.
> I want to find out physical sector of MFT record(not file extent or data
> extent) of file inside MFT.
>
> All I want to do is:
>
> 1. Use VOLUME_LOGICAL_OFFSET and pass it to IOCTL with
> IOCTL_VOLUME_LOGICAL_TO_PHYSICAL control code and which would return the
> physical sector.
>
> 2. I have to fill the VOLUME_LOGICAL_OFFSET.LogicalOffset.
>
> How do I calculate logicaloffset(BYTE Offset) using
> ntfsVolData.MftStartLcn and MFTRecordNumber?
> Are MFTRecrodNumbers(for files and directories) indexed in MFT layout
> incrementally after first 16 reserved Entries?
>
> Emre: Thanks for inputs.
>
> Please suggest, how to go after what I mentioned above.
>
> Thanks,
> Amrat
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

Hi Emre,

Thanks a lot for the insight, that really helps. I will try this out.

Also I know ntfsVolData.MftStartLcn (0xC00000) and MFTRecordNumber (say $191102), now based on this could I calculate something like below for getting the logical offset(Bytes) of this MFTRecord in $MFT.

LogicalOffset = ntfsVolData.MftStartLcn.QuadPart * 4096(cluster size) + (MFTRecordNumber * ntfsVolData.BytesPerFileRecordSegment);

then pass this LogicalOffset to IOCTL with
IOCTL_VOLUME_LOGICAL_TO_PHYSICAL control code and which would return the
physical sector MFTRecord of file?

Thanks,
Amrat

Hi Emre,

Also Could you give me pointers or give sample code to do what you mentioned below?

In order to programatically get to this point :

  1. Load MFT record FSCTL_GET_NTFS_FILE_RECORD or read size of MftRecord
    structure from the MftStartLcn (they point to same offset)
  2. FindDataAttribute of $MFT file
  3. Get run array of the file (LCN & Length)
    {Amart: I want to find out LBA or Physical sector or perticular MFT entry in $MFT}

Thanks in advance.

Cheers,
Amrat

Hi Amrat,

Also I know ntfsVolData.MftStartLcn (0xC00000) and MFTRecordNumber (say

$191102), now based on this could I calculate something like below for
getting the logical offset(Bytes) of this MFTRecord in $MFT.
LogicalOffset = ntfsVolData.MftStartLcn.
QuadPart * 4096(cluster size) + (MFTRecordNumber * ntfsVolData.
BytesPerFileRecordSegment);

The sample you have provided assumes that MFT is not fragmented. I mean it
assumes MFT Starts @ X and
goes Y clusters continuesly. But the real problem here is, it doesn’t in
real life. So, you need to deal
with 2 or more parts. Here is my pseudo algo for this :

DWORD RequestedIndex = 30450;
UINT64 VirtualOffset = RequestedIndex * BytesPerFileRecord; // This is the
offset if $MFT DATA was continuous.
UINT64 RealDiskOffset = 0;
INT64 CumulativeOffset = 0;
UINT64 CumulativeSize = 0;

while(!NoMoreRuns) { // Run means a block of MFT data

DWORD RunLen = GetRunLen();
INT64 RunLcn = GetRunLcn(); // This needs to be signed because it
is relative to (n-1)th run

CumulativeSize += RunLen * BytesPerCluster;
CumulativeOffset += RunLcn * BytesPerCluster; // This needs to be
signed because it is relative to (n-1)th run

if(VirtualOffset <= CumulativeSize) {
// Here is our requested index
RealDiskOffset = CumulativeOffset + (VirtualOffset -
(CumulativeSize - RunLen));
break;
}

}

You can have a look some examples at :
*Gary Nebbett - Native Api Reference Appendix E*
http://inform.pucp.edu.pe/~inf232/Ntfs/ntfs_doc_v0.5/help/glossary.html
(Look for Runs)
http://code.google.com/p/bingo-file-search/source/browse/trunk/core-test/core/NTFS.cpp?r=46

Emre TINAZTEPE

On Wed, Feb 1, 2012 at 12:48 PM, wrote:

> Hi Emre,
>
> Also Could you give me pointers or give sample code to do what you
> mentioned below?
>
> In order to programatically get to this point :
> 1. Load MFT record FSCTL_GET_NTFS_FILE_RECORD or read size of MftRecord
> structure from the MftStartLcn (they point to same offset)
> 2. FindDataAttribute of $MFT file
> 3. Get run array of the file (LCN & Length)
> {Amart: I want to find out LBA or Physical sector or perticular MFT entry
> in $MFT}
>
> Thanks in advance.
>
> Cheers,
> Amrat
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

Hi Emre,

Thanks a lot for the pointers.

How do I find out the NoMoreRuns for the $MFT, infact how do I find out runs of $MFT (how to traverse $DATA Attribute of $MFT)??
How do we implement GetRunLen() and GetRunLcn() ?

Do you have reference code for doing this complete task?

Thanks in advance.

Cheers,
Amrat

Hi,

You can use Gary Nebbet’s document, just look at Appendix E (NTFS On-Disk
Structure).

Emre TINAZTEPE

On Wed, Feb 1, 2012 at 6:05 PM, wrote:

> Hi Emre,
>
> Thanks a lot for the pointers.
>
> How do I find out the NoMoreRuns for the $MFT, infact how do I find out
> runs of $MFT (how to traverse $DATA Attribute of $MFT)??
> How do we implement GetRunLen() and GetRunLcn() ?
>
> Do you have reference code for doing this complete task?
>
> Thanks in advance.
>
> Cheers,
> Amrat
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

Shipping product code should not be relying on undocumented implementation details of filesystem formats customer-side. That is a great way to corrupt customer data.

  • S (Msft)

From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Emre Tinaztepe
Sent: Wednesday, February 01, 2012 8:19 AM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] how to find out Physical Sector of MFT file record of file?

Hi,

You can use Gary Nebbet’s document, just look at Appendix E (NTFS On-Disk Structure).

Emre TINAZTEPE
On Wed, Feb 1, 2012 at 6:05 PM, > wrote:
Hi Emre,

Thanks a lot for the pointers.

How do I find out the NoMoreRuns for the $MFT, infact how do I find out runs of $MFT (how to traverse $DATA Attribute of $MFT)??
How do we implement GetRunLen() and GetRunLcn() ?

Do you have reference code for doing this complete task?

Thanks in advance.

Cheers,
Amrat


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

— NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and other seminars visit: http://www.osr.com/seminars To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

>

Shipping product code should not be relying on undocumented implementation
details of filesystem formats customer-side. That is a great way to
corrupt customer data.

I totally agree, if this is for a commercial product, there are a lot to
research about NTFS…

On Wed, Feb 1, 2012 at 6:45 PM, Skywing wrote:

> Shipping product code should not be relying on undocumented
> implementation details of filesystem formats customer-side. That is a
> great way to corrupt customer data.
>
>
**
>
> - S (Msft)

>
> ****
>
> From: xxxxx@lists.osr.com [mailto:
> xxxxx@lists.osr.com] On Behalf Of Emre Tinaztepe
> Sent: Wednesday, February 01, 2012 8:19 AM
> To: Windows System Software Devs Interest List
> Subject: Re: [ntdev] how to find out Physical Sector of MFT file record
> of file?
>
>

>
> Hi,
>
> You can use Gary Nebbet’s document, just look at Appendix E (NTFS On-Disk
> Structure).
>
> Emre TINAZTEPE

>
> On Wed, Feb 1, 2012 at 6:05 PM, wrote:

>
> Hi Emre,
>
> Thanks a lot for the pointers.
>
> How do I find out the NoMoreRuns for the $MFT, infact how do I find out
> runs of $MFT (how to traverse $DATA Attribute of $MFT)??
> How do we implement GetRunLen() and GetRunLcn() ?
>
> Do you have reference code for doing this complete task?****
>
>
> Thanks in advance.
>
> Cheers,
> Amrat
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
>
> — NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and
> other seminars visit: http://www.osr.com/seminars To unsubscribe, visit
> the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer

>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

Thanks Emre for the inputs, that really helps.

Emre - I was wondering how to find out if MFTEntry is in particular LCN of $MFT? Given I know the MFTRecordNumber.

Thanks in advance.

Cheers,
Amrat

The sample i have provided in our last conversation exactly does that :slight_smile:

If you are still having difficulties about understanding “Run” concept,
take a look at
http://inform.pucp.edu.pe/~inf232/Ntfs/ntfs_doc_v0.5/concepts/data_runs.html

If you get the point of Run Concept, all others problems are related to
programming, nothing more…
Think about a Run as *some continuous clusters starting at some offset.*

*0*------------------------------*10*
------------------------------------------------*70*
---------------------------------*110*-----------------------------------------------
(Cluster #)
*[======RUN1=========]*=======SOME OTHER DATA========*
[========RUN2=========]*=======SOME OTHER DATA========

*Question : *
If you know RUN1 is 10 clusters starting @ offset 0, and Run2 is 40
Clusters starting at offset 70.
Given a cluster size of 4096 and MftRecord size of 1024, how would you
calculate the offset of
60th* *MFT Entry? Answer lies in the scheme:)

Emre TINAZTEPE

On Thu, Feb 2, 2012 at 4:10 PM, wrote:

> Thanks Emre for the inputs, that really helps.
>
> Emre - I was wondering how to find out if MFTEntry is in particular LCN of
> $MFT? Given I know the MFTRecordNumber.
>
>
> Thanks in advance.
>
> Cheers,
> Amrat
>
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

Hi Emre,

Thanks for information.
If I am not wrong, that should lie to RUN2 - start 70 and offset 2048(2k).

Also I can find out the runlist for $MFT but how to interpret the runlist programatically?
The pointers that you mentioned in the earlier replies talk about RunLength, RunCount, RunLCN. What is the difference in RunLength and RunCount?

Thanks in advance.

Cheers,
Amrat

> I am investigating how to find out physical sector address of MFT entry

Why do you need this? please re-consider.

If you really need this - then take the Linux NTFS source and write your own metadata parser.


Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

Hi Amrat,

RunCount shows the number of fragmentations a file has.
RunLength shows the size of a fragmentation while RunLCN pojnts where it
starts.

Think about a file which has 10 Clusters of data, splitted into 3 pieces
(RunCount)
Each piece starts at a cluster offset (RunLCN) and continues for multiple
clusters (RunLength)

If you really need this - then take the Linux NTFS source and write your

own metadata parser.

Maxim is right but if are not ahead of a problem which requires a full
blown NTFS parser, you will not need to delve into linux-ntfs.
And according to what you have written at the beginning, i guess Gary
Nebbetts Appendix E will be enough
for you. But if you need a more complex parser, you must grab the logic
behind NTFS.
Here is a general info about NTFS :
http://technet.microsoft.com/en-us/library/cc781134(WS.10).aspx

And more spesifically about Runs and fragment of a file, take a look at
Jeff Hughes’ article.
http://blogs.technet.com/b/askcore/archive/2009/10/16/the-four-stages-of-ntfs-file-growth.aspx

Also, buying some Forensic books would definitely help.

Emre TINAZTEPE

On Fri, Feb 3, 2012 at 8:37 AM, wrote:

> Hi Emre,
>
> Thanks for information.
> If I am not wrong, that should lie to RUN2 - start 70 and offset 2048(2k).
>
> Also I can find out the runlist for $MFT but how to interpret the runlist
> programatically?
> The pointers that you mentioned in the earlier replies talk about
> RunLength, RunCount, RunLCN. What is the difference in RunLength and
> RunCount?
>
> Thanks in advance.
>
> Cheers,
> Amrat
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>