Windows System Software -- Consulting, Training, Development -- Unique Expertise, Guaranteed Results

Home NTDEV
Before Posting...
Please check out the Community Guidelines in the Announcements and Administration Category.

More Info on Driver Writing and Debugging


The free OSR Learning Library has more than 50 articles on a wide variety of topics about writing and debugging device drivers and Minifilters. From introductory level to advanced. All the articles have been recently reviewed and updated, and are written using the clear and definitive style you've come to expect from OSR over the years.


Check out The OSR Learning Library at: https://www.osr.com/osr-learning-library/


how to find out Physical Sector of MFT file record of file?

amrat_sharmaamrat_sharma Member Posts: 22
Hi Folks,

I am investigating how to find out physical sector address of MFT entry of perticular file using programming constructs?

For example: I want to find out sector of file c:\test\test.txt, I know MFTStartLcn but using this information how to go to actual sector of the MFT record of test.txt file?

Any help is highly appreciated.

Cheers,
Amrat

Comments

  • Tim_RobertsTim_Roberts Member - All Emails Posts: 13,594
    [email protected] wrote:
    > I am investigating how to find out physical sector address of MFT entry of perticular file using programming constructs?
    >
    > For example: I want to find out sector of file c:\test\test.txt, I know MFTStartLcn but using this information how to go to actual sector of the MFT record of test.txt file?

    Have you done any reading at all on the layout of an NTFS file system
    and the format of the MFT? Sector numbers aren't interesting; you deal
    with byte offsets. One of the MFT metafiles gives you the location of
    the root directory. You read that to determine the location of the
    "test" directory. You then read that to determine the location of
    "test.txt".

    Why do you want to do this?

    --
    Tim Roberts, [email protected]
    Providenza & Boekelheide, Inc.

    Tim Roberts, [email protected]
    Providenza & Boekelheide, Inc.

  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    Hi Amrat,

    The goal you are trying to achieve can be done in three ways:
    1. As Tim Roberts pointed out, you can read the INDEX attributes of every
    directory from the volume root. (Very Long way)
    2. After getting the MftLcn from bootsector, you can enumerate every file
    and search for File/Directory names,
    and test for ParentDirectory's Segment Reference Number. This is a very
    long way too.
    3. Fastest method is getting the file's cluster number via
    FSCTL_GET_RETRIEVAL_POINTERSthen
    you can easily read
    the contents of the file.

    Hope it helps,

    Emre TINAZTEPE

    On Tue, Jan 31, 2012 at 8:22 PM, Tim Roberts wrote:

    > [email protected] wrote:
    > > I am investigating how to find out physical sector address of MFT entry
    > of perticular file using programming constructs?
    > >
    > > For example: I want to find out sector of file c:\test\test.txt, I know
    > MFTStartLcn but using this information how to go to actual sector of the
    > MFT record of test.txt file?
    >
    > Have you done any reading at all on the layout of an NTFS file system
    > and the format of the MFT? Sector numbers aren't interesting; you deal
    > with byte offsets. One of the MFT metafiles gives you the location of
    > the root directory. You read that to determine the location of the
    > "test" directory. You then read that to determine the location of
    > "test.txt".
    >
    > Why do you want to do this?
    >
    > --
    > Tim Roberts, [email protected]
    > Providenza & Boekelheide, Inc.
    >
    >
    > ---
    > NTDEV is sponsored by OSR
    >
    > For our schedule of WDF, WDM, debugging and other seminars visit:
    > http://www.osr.com/seminars
    >
    > To unsubscribe, visit the List Server section of OSR Online at
    > http://www.osronline.com/page.cfm?name=ListServer
    >
  • amrat_sharmaamrat_sharma Member Posts: 22
    Thanks guys - that helps.

    Tim: I know it works on byte offset, may be I was not very clear on what I wanted.
    I want to find out physical sector of MFT record(not file extent or data extent) of file inside MFT.

    All I want to do is:

    1. Use VOLUME_LOGICAL_OFFSET and pass it to IOCTL with IOCTL_VOLUME_LOGICAL_TO_PHYSICAL control code and which would return the physical sector.

    2. I have to fill the VOLUME_LOGICAL_OFFSET.LogicalOffset.

    How do I calculate logicaloffset(BYTE Offset) using ntfsVolData.MftStartLcn and MFTRecordNumber?
    Are MFTRecrodNumbers(for files and directories) indexed in MFT layout incrementally after first 16 reserved Entries?

    Emre: Thanks for inputs.

    Please suggest, how to go after what I mentioned above.

    Thanks,
    Amrat
  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    Hi Amrat,

    In order to calculate the offset of a file from MftRecordNumber, you need
    to get the Extents of $MFT file.
    Since $MFT file's $DATA attribute is layed out to disk in parts, you will
    need to calculate on which offsets do they start and
    size of them.
    These fragmentations are called "Runs".
    To give an example :
    My disk's $MFT is composed of two Runs,
    1. LCN 0xC0000 - Lenght 0x4540 (Disk offset 0xC0000000)
    2. LCN 0x3A45B34 - Length 0x2DC80 => this LCN is relative to the first one
    which makes it @ LCN 0x3B05B34

    At this point, we know that we have total (0x4540 + 0x2DC80) clusters which
    totals to 0x321C0 Clusters.
    Given the size of cluster is 4096 Bytes (from VolData) and size of
    MftRecord is 1024 Bytes,
    this means each clusters holds 4 MftRecords. Which means i have a total
    0x321C0 * 4 = 0xC8700 Mft Records( 820992 decimal)

    In order to programatically get to this point :
    1. Load MFT record FSCTL_GET_NTFS_FILE_RECORD or read size of MftRecord
    structure from the MftStartLcn (they point to same offset)
    2. FindDataAttribute of $MFT file
    3. Get run array of the file (LCN & Length)

    In order to find information about MFT Parsing functions like FindAttribute
    / FindRun , you may have a look at Garry Nebbets doc
    here
    http://www.installsetupconfig.com/win32programming/1996 AppE_apnilife.pdf

    Emre TINAZTEPE


    On Wed, Feb 1, 2012 at 8:13 AM, wrote:

    > Thanks guys - that helps.
    >
    > Tim: I know it works on byte offset, may be I was not very clear on what I
    > wanted.
    > I want to find out physical sector of MFT record(not file extent or data
    > extent) of file inside MFT.
    >
    > All I want to do is:
    >
    > 1. Use VOLUME_LOGICAL_OFFSET and pass it to IOCTL with
    > IOCTL_VOLUME_LOGICAL_TO_PHYSICAL control code and which would return the
    > physical sector.
    >
    > 2. I have to fill the VOLUME_LOGICAL_OFFSET.LogicalOffset.
    >
    > How do I calculate logicaloffset(BYTE Offset) using
    > ntfsVolData.MftStartLcn and MFTRecordNumber?
    > Are MFTRecrodNumbers(for files and directories) indexed in MFT layout
    > incrementally after first 16 reserved Entries?
    >
    > Emre: Thanks for inputs.
    >
    > Please suggest, how to go after what I mentioned above.
    >
    > Thanks,
    > Amrat
    >
    >
    > ---
    > NTDEV is sponsored by OSR
    >
    > For our schedule of WDF, WDM, debugging and other seminars visit:
    > http://www.osr.com/seminars
    >
    > To unsubscribe, visit the List Server section of OSR Online at
    > http://www.osronline.com/page.cfm?name=ListServer
    >
  • amrat_sharmaamrat_sharma Member Posts: 22
    Hi Emre,

    Thanks a lot for the insight, that really helps. I will try this out.

    Also I know ntfsVolData.MftStartLcn (0xC00000) and MFTRecordNumber (say $191102), now based on this could I calculate something like below for getting the logical offset(Bytes) of this MFTRecord in $MFT.

    LogicalOffset = ntfsVolData.MftStartLcn.QuadPart * 4096(cluster size) + (MFTRecordNumber * ntfsVolData.BytesPerFileRecordSegment);

    then pass this LogicalOffset to IOCTL with
    IOCTL_VOLUME_LOGICAL_TO_PHYSICAL control code and which would return the
    physical sector MFTRecord of file?


    Thanks,
    Amrat
  • amrat_sharmaamrat_sharma Member Posts: 22
    Hi Emre,

    Also Could you give me pointers or give sample code to do what you mentioned below?

    In order to programatically get to this point :
    1. Load MFT record FSCTL_GET_NTFS_FILE_RECORD or read size of MftRecord
    structure from the MftStartLcn (they point to same offset)
    2. FindDataAttribute of $MFT file
    3. Get run array of the file (LCN & Length)
    {Amart: I want to find out LBA or Physical sector or perticular MFT entry in $MFT}

    Thanks in advance.

    Cheers,
    Amrat
  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    Hi Amrat,

    Also I know ntfsVolData.MftStartLcn (0xC00000) and MFTRecordNumber (say
    > $191102), now based on this could I calculate something like below for
    > getting the logical offset(Bytes) of this MFTRecord in $MFT.
    > LogicalOffset = ntfsVolData.MftStartLcn.
    > QuadPart * 4096(cluster size) + (MFTRecordNumber * ntfsVolData.
    > BytesPerFileRecordSegment);
    >

    The sample you have provided assumes that MFT is not fragmented. I mean it
    assumes MFT Starts @ X and
    goes Y clusters continuesly. But the real problem here is, it doesn't in
    real life. So, you need to deal
    with 2 or more parts. Here is my pseudo algo for this :

    DWORD RequestedIndex = 30450;
    UINT64 VirtualOffset = RequestedIndex * BytesPerFileRecord; // This is the
    offset if $MFT DATA was continuous.
    UINT64 RealDiskOffset = 0;
    INT64 CumulativeOffset = 0;
    UINT64 CumulativeSize = 0;

    while(!NoMoreRuns) { // Run means a block of MFT data

    DWORD RunLen = GetRunLen();
    INT64 RunLcn = GetRunLcn(); // This needs to be signed because it
    is relative to (n-1)th run

    CumulativeSize += RunLen * BytesPerCluster;
    CumulativeOffset += RunLcn * BytesPerCluster; // This needs to be
    signed because it is relative to (n-1)th run

    if(VirtualOffset <= CumulativeSize) {
    // Here is our requested index
    RealDiskOffset = CumulativeOffset + (VirtualOffset -
    (CumulativeSize - RunLen));
    break;
    }

    }

    You can have a look some examples at :
    *Gary Nebbett - Native Api Reference Appendix E*
    http://inform.pucp.edu.pe/~inf232/Ntfs/ntfs_doc_v0.5/help/glossary.html
    (Look for Runs)
    http://code.google.com/p/bingo-file-search/source/browse/trunk/core-test/core/NTFS.cpp?r=46


    Emre TINAZTEPE

    On Wed, Feb 1, 2012 at 12:48 PM, <[email protected]> wrote:

    > Hi Emre,
    >
    > Also Could you give me pointers or give sample code to do what you
    > mentioned below?
    >
    > In order to programatically get to this point :
    > 1. Load MFT record FSCTL_GET_NTFS_FILE_RECORD or read size of MftRecord
    > structure from the MftStartLcn (they point to same offset)
    > 2. FindDataAttribute of $MFT file
    > 3. Get run array of the file (LCN & Length)
    > {Amart: I want to find out LBA or Physical sector or perticular MFT entry
    > in $MFT}
    >
    > Thanks in advance.
    >
    > Cheers,
    > Amrat
    >
    > ---
    > NTDEV is sponsored by OSR
    >
    > For our schedule of WDF, WDM, debugging and other seminars visit:
    > http://www.osr.com/seminars
    >
    > To unsubscribe, visit the List Server section of OSR Online at
    > http://www.osronline.com/page.cfm?name=ListServer
    >
  • amrat_sharmaamrat_sharma Member Posts: 22
    Hi Emre,

    Thanks a lot for the pointers.

    How do I find out the NoMoreRuns for the $MFT, infact how do I find out runs of $MFT (how to traverse $DATA Attribute of $MFT)??
    How do we implement GetRunLen() and GetRunLcn() ?

    Do you have reference code for doing this complete task?

    Thanks in advance.

    Cheers,
    Amrat
  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    Hi,

    You can use Gary Nebbet's document, just look at Appendix E (NTFS On-Disk
    Structure).

    Emre TINAZTEPE

    On Wed, Feb 1, 2012 at 6:05 PM, wrote:

    > Hi Emre,
    >
    > Thanks a lot for the pointers.
    >
    > How do I find out the NoMoreRuns for the $MFT, infact how do I find out
    > runs of $MFT (how to traverse $DATA Attribute of $MFT)??
    > How do we implement GetRunLen() and GetRunLcn() ?
    >
    > Do you have reference code for doing this complete task?
    >
    > Thanks in advance.
    >
    > Cheers,
    > Amrat
    >
    > ---
    > NTDEV is sponsored by OSR
    >
    > For our schedule of WDF, WDM, debugging and other seminars visit:
    > http://www.osr.com/seminars
    >
    > To unsubscribe, visit the List Server section of OSR Online at
    > http://www.osronline.com/page.cfm?name=ListServer
    >
  • Ken_JohnsonKen_Johnson Member - All Emails Posts: 1,559
    Shipping product code should not be relying on undocumented implementation details of filesystem formats customer-side. That is a great way to corrupt customer data.

    - S (Msft)

    From: [email protected] [mailto:[email protected]] On Behalf Of Emre Tinaztepe
    Sent: Wednesday, February 01, 2012 8:19 AM
    To: Windows System Software Devs Interest List
    Subject: Re: [ntdev] how to find out Physical Sector of MFT file record of file?

    Hi,

    You can use Gary Nebbet's document, just look at Appendix E (NTFS On-Disk Structure).

    Emre TINAZTEPE
    On Wed, Feb 1, 2012 at 6:05 PM, > wrote:
    Hi Emre,

    Thanks a lot for the pointers.

    How do I find out the NoMoreRuns for the $MFT, infact how do I find out runs of $MFT (how to traverse $DATA Attribute of $MFT)??
    How do we implement GetRunLen() and GetRunLcn() ?

    Do you have reference code for doing this complete task?

    Thanks in advance.

    Cheers,
    Amrat

    ---
    NTDEV is sponsored by OSR

    For our schedule of WDF, WDM, debugging and other seminars visit:
    http://www.osr.com/seminars

    To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

    --- NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and other seminars visit: http://www.osr.com/seminars To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    >
    > Shipping product code should not be relying on undocumented implementation
    > details of filesystem formats customer-side. That is a great way to
    > corrupt customer data.
    >

    I totally agree, if this is for a commercial product, there are a lot to
    research about NTFS...


    On Wed, Feb 1, 2012 at 6:45 PM, Skywing wrote:

    > Shipping product code should not be relying on undocumented
    > implementation details of filesystem formats customer-side. That is a
    > great way to corrupt customer data.****
    >
    > ** **
    >
    > - S (Msft)****
    >
    > ** **
    >
    > *From:* [email protected] [mailto:
    > [email protected]] *On Behalf Of *Emre Tinaztepe
    > *Sent:* Wednesday, February 01, 2012 8:19 AM
    > *To:* Windows System Software Devs Interest List
    > *Subject:* Re: [ntdev] how to find out Physical Sector of MFT file record
    > of file?****
    >
    > ** **
    >
    > Hi,
    >
    > You can use Gary Nebbet's document, just look at Appendix E (NTFS On-Disk
    > Structure).
    >
    > Emre TINAZTEPE****
    >
    > On Wed, Feb 1, 2012 at 6:05 PM, wrote:****
    >
    > Hi Emre,
    >
    > Thanks a lot for the pointers.
    >
    > How do I find out the NoMoreRuns for the $MFT, infact how do I find out
    > runs of $MFT (how to traverse $DATA Attribute of $MFT)??
    > How do we implement GetRunLen() and GetRunLcn() ?
    >
    > Do you have reference code for doing this complete task?****
    >
    >
    > Thanks in advance.
    >
    > Cheers,
    > Amrat
    >
    > ---
    > NTDEV is sponsored by OSR
    >
    > For our schedule of WDF, WDM, debugging and other seminars visit:
    > http://www.osr.com/seminars
    >
    > To unsubscribe, visit the List Server section of OSR Online at
    > http://www.osronline.com/page.cfm?name=ListServer****
    >
    >
    > --- NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and
    > other seminars visit: http://www.osr.com/seminars To unsubscribe, visit
    > the List Server section of OSR Online at
    > http://www.osronline.com/page.cfm?name=ListServer****
    >
    > ---
    > NTDEV is sponsored by OSR
    >
    > For our schedule of WDF, WDM, debugging and other seminars visit:
    > http://www.osr.com/seminars
    >
    > To unsubscribe, visit the List Server section of OSR Online at
    > http://www.osronline.com/page.cfm?name=ListServer
    >
  • amrat_sharmaamrat_sharma Member Posts: 22
    Thanks Emre for the inputs, that really helps.

    Emre - I was wondering how to find out if MFTEntry is in particular LCN of $MFT? Given I know the MFTRecordNumber.


    Thanks in advance.

    Cheers,
    Amrat
  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    The sample i have provided in our last conversation exactly does that :)

    If you are still having difficulties about understanding "Run" concept,
    take a look at
    http://inform.pucp.edu.pe/~inf232/Ntfs/ntfs_doc_v0.5/concepts/data_runs.html

    If you get the point of Run Concept, all others problems are related to
    programming, nothing more...
    Think about a Run as *some continuous clusters starting at some offset.*

    *0*------------------------------*10*
    ------------------------------------------------*70*
    ---------------------------------*110*-----------------------------------------------
    (Cluster #)
    *[======RUN1=========]*=======SOME OTHER DATA========*
    [========RUN2=========]*=======SOME OTHER DATA========

    *Question : *
    If you know RUN1 is 10 clusters starting @ offset 0, and Run2 is 40
    Clusters starting at offset 70.
    Given a cluster size of 4096 and MftRecord size of 1024, how would you
    calculate the offset of
    60th* *MFT Entry? Answer lies in the scheme:)

    Emre TINAZTEPE

    On Thu, Feb 2, 2012 at 4:10 PM, wrote:

    > Thanks Emre for the inputs, that really helps.
    >
    > Emre - I was wondering how to find out if MFTEntry is in particular LCN of
    > $MFT? Given I know the MFTRecordNumber.
    >
    >
    > Thanks in advance.
    >
    > Cheers,
    > Amrat
    >
    >
    >
    > ---
    > NTDEV is sponsored by OSR
    >
    > For our schedule of WDF, WDM, debugging and other seminars visit:
    > http://www.osr.com/seminars
    >
    > To unsubscribe, visit the List Server section of OSR Online at
    > http://www.osronline.com/page.cfm?name=ListServer
    >
  • amrat_sharmaamrat_sharma Member Posts: 22
    Hi Emre,

    Thanks for information.
    If I am not wrong, that should lie to RUN2 - start 70 and offset 2048(2k).

    Also I can find out the runlist for $MFT but how to interpret the runlist programatically?
    The pointers that you mentioned in the earlier replies talk about RunLength, RunCount, RunLCN. What is the difference in RunLength and RunCount?

    Thanks in advance.

    Cheers,
    Amrat
  • Maxim_S._ShatskihMaxim_S._Shatskih Member Posts: 10,396
    > I am investigating how to find out physical sector address of MFT entry

    Why do you need this? please re-consider.

    If you _really_ need this - then take the Linux NTFS source and write your own metadata parser.

    --
    Maxim S. Shatskih
    Windows DDK MVP
    [email protected]
    http://www.storagecraft.com
  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    Hi Amrat,

    RunCount shows the number of fragmentations a file has.
    RunLength shows the size of a fragmentation while RunLCN pojnts where it
    starts.

    Think about a file which has 10 Clusters of data, splitted into 3 pieces
    (RunCount)
    Each piece starts at a cluster offset (RunLCN) and continues for multiple
    clusters (RunLength)

    If you _really_ need this - then take the Linux NTFS source and write your
    > own metadata parser.
    >

    Maxim is right but if are not ahead of a problem which requires a full
    blown NTFS parser, you will not need to delve into linux-ntfs.
    And according to what you have written at the beginning, i guess Gary
    Nebbetts Appendix E will be enough
    for you. But if you need a more complex parser, you must grab the logic
    behind NTFS.
    Here is a general info about NTFS :
    http://technet.microsoft.com/en-us/library/cc781134(WS.10).aspx

    And more spesifically about Runs and fragment of a file, take a look at
    Jeff Hughes' article.
    http://blogs.technet.com/b/askcore/archive/2009/10/16/the-four-stages-of-ntfs-file-growth.aspx

    Also, buying some Forensic books would definitely help.

    Emre TINAZTEPE

    On Fri, Feb 3, 2012 at 8:37 AM, wrote:

    > Hi Emre,
    >
    > Thanks for information.
    > If I am not wrong, that should lie to RUN2 - start 70 and offset 2048(2k).
    >
    > Also I can find out the runlist for $MFT but how to interpret the runlist
    > programatically?
    > The pointers that you mentioned in the earlier replies talk about
    > RunLength, RunCount, RunLCN. What is the difference in RunLength and
    > RunCount?
    >
    > Thanks in advance.
    >
    > Cheers,
    > Amrat
    >
    > ---
    > NTDEV is sponsored by OSR
    >
    > For our schedule of WDF, WDM, debugging and other seminars visit:
    > http://www.osr.com/seminars
    >
    > To unsubscribe, visit the List Server section of OSR Online at
    > http://www.osronline.com/page.cfm?name=ListServer
    >
Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Upcoming OSR Seminars
OSR has suspended in-person seminars due to the Covid-19 outbreak. But, don't miss your training! Attend via the internet instead!
Internals & Software Drivers 30 Nov 2020 LIVE ONLINE
Writing WDF Drivers 7 Dec 2020 LIVE ONLINE
Developing Minifilters Early 2021 LIVE ONLINE