Alternate Data Streams

Alternate Data Streamsand the NTFS file system

Nephi Johnson, BYU, CS345 Summer 2009

Overview• The general word for an Alternate Data Stream is a file system fork.

• File system forks are found in Macintosh (data, resource, and named forks), Windows (alternate data streams), and Novell environments (multiple data

streams).

• Using file system forks is a way to store additional variable-length information with a file in addition to the normal data stream.

• Windows ADS is Microsoft’s way of implementing Macintosh’s resource fork.


Alternate Data Streams

• Alternate Data Streams are used to store additional information with a file, such as file access/modification times.

• Some applications legitimately use them to store metadata about a file

• Malware and viruses (not to mention hackers) use them to hide files and executables

• Anti-virus software doesn’t always look in alternate data streams

• An ADS is invisible to the user without the use of special tools (**note – in Windows Vista, the dir /R command will display

alternate data streams)

“This feature permits related data to be managed as a single unit... For example, a graphics program can store a thumbnail image of a bitmap in a named data stream

within the NTFS file containing the image.”--Microsoft

NTFS Overview• In order to understand how alternate data streams work, you must

understand the NTFS file system.

• NTFS is often compared with the FAT file system. Instead of a FAT table, NTFS has a Master File Table (MFT). It also has a copy of the MFT (akin to FAT2) that is used for backups and recovery.

• A MFT is similar to a FAT, but instead of the directories storing info about the files and the files containing only data, the directories only store attributes of the directory and the files contain all the info about themselves.

• This is the basic layout of an NTFS partition

NTFS Overview - MFT• When a volume is formatted with NTFS, an MFT file is created with the

first 16 entries reserved for use as metadata for the file system (shown on next slide).

• Windows reserves 12.5% of available disk space (the “MFT Zone”) for future growth by the MFT. This is never used by the user unless everything else has already been used.

• Entry sizes in the MFT are determined by the cluster size. Entries have this general format

NTFS Overview – MFT Metadata


System File File Name

# Purpose of the File

Master file table

$Mft 0Contains one base file record for each file and folder on an NTFS volume. If the allocation information for a file or folder is too large to fit within a single record, other file records are allocated as well.

Master file table 2

$MftMirr 1A duplicate image of the first four records of the MFT. This file guarantees access to the MFT in case of a single-sector failure.

Log file $LogFile 2Contains a list of transaction steps used for NTFS recoverability. Log file size depends on volume size and can be as large as 4 MB. It is used by Windows 2000 to restore consistency to NTFS after a system failure. For more information about the log file, see NTFS Recoverability earlier in this chapter.

Volume $Volume 3 Contains information about the volume, such as the volume label and the volume version.

Attribute definitions

$AttrDef 4 A table of attribute names, numbers, and descriptions.

Root file name index

$ 5 The root folder.

Cluster bitmap $Bitmap 6 A representation of the volume showing which clusters are in use.

Boot sector $Boot 7Includes the BPB used to mount the volume and additional bootstrap loader code used if the volume is bootable.

Bad cluster file $BadClus 8 Contains bad clusters for the volume.

Security file $Secure 9 Contains unique security descriptors for all files within a volume.

Upcase table $Upcase 10 Converts lowercase characters to matching Unicode uppercase characters.

NTFS extension file

$Extend 11 Used for various optional extensions such as quotas, reparse point data, and object identifiers.

12-15

Reserved for future use.

NTFS Overview – Files and Dirs• Each entry in the MFT describes a file or dir and is a collection of

attributes (yes, even the file data) [see next slide]

• If the total size of a file’s attributes (remember, attributes also include the file data) is smaller than the record size in the MFT (1 KB), the entire file will be stored in the MFT.

• If an attribute’s value(s) are small enough to fit inside the MFT entry, then that attribute is called resident. (filenames, timestamps are always resident)

• Otherwise, some attributes are made non-resident and a pointer to a new data run or extent is stored in the Attribute List attribute. The actual values of the non-resident attributes are stored in the extent.

• An extent is a contiguous “run” of clusters used to

store an attribute’s data.


NTFS Overview – File/Dir Attr.Attribute Type Description

Standard Information Information such as access mode (read-only, read/write, and so forth) timestamp, and link count.

Attribute List Locations of all attribute records that do not fit in the MFT record.

File NameA repeatable attribute for both long and short file names. The long name of the file can be up to 255 Unicode characters. The short name is the 8.3, case-insensitive name for the file. Additional names, or hard links, required by POSIX can be included as additional file name attributes.

DataFile data. NTFS supports multiple data attributes per file. Each file typically has one unnamed data attribute. A file can also have one or more named data attributes.

Object ID A volume-unique file identifier. Used by the distributed link tracking service. Not all files have object identifiers.

Logged Tool StreamSimilar to a data stream, but operations are logged to the NTFS log file just like NTFS metadata changes. This attribute is used by EFS.

Reparse PointUsed for mounted drives. This is also used by Installable File System (IFS) filter drivers to mark certain files as special to that driver.

Index Root Used to implement folders and other indexes.

Index Allocation Used to implement the B-tree structure for large folders and other large indexes.

Bitmap Used to implement the B-tree structure for large folders and other large indexes.

Volume Information Used only in the $Volume system file. Contains the volume version.

NTFS Overview – Files and Dirs 2• Each entry in the MFT can contain one unnamed $DATA attribute and

multiple named $DATA attributes (yes, this includes directories!)

• Each of these data attributes are commonly called data streams

• Any stream that is not the default data attribute (unnamed) is called an Alternate Data Stream.

• The only way to completely delete an ADS is to delete the file or directory itself. However, an ADS can be overwritten to make the old data inaccessible through normal means.

Alternate Data Streams in action!


But here’s some pictures just in case

(Notice, no change in file sizeand no indication of the alternatestream)

Questions• Files and Directories in NTFS ...

a. are completely different from each other

b. are collections of attributes

c. work exactly like they do in FAT

d. don’t exist

• Entries in the MFT can have multiple data streams b/c...a. they can have multiple file names

b. sectors are larger than most clusters

c. they can have multiple data attributes

d. of non-resident attributes

Answers• Files and Directories in NTFS ...

a. are completely different from each other

b. are collections of attributes

c. work exactly like they do in FAT

d. don’t exist

• Entries in the MFT can have multiple data streams b/c...a. they can have multiple file names

b. sectors are larger than most clusters

c. they can have multiple data attributes

d. of non-resident attributes

Reference[1] - Discovering alternate data streams using .NET

http://msdn.microsoft.com/en-us/magazine/cc163677.aspx[2] - Clusters and Extents

http://msdn.microsoft.com/en-us/library/aa363841(VS.85).aspx[3] - Comparisons between FAT, HPFS, and NTFS

http://support.microsoft.com/kb/100108[4] - White paper on hiding data in NTFS (a very interesting read)

http://www.forensicfocus.com/downloads/ntfs-hidden-data-analysis.pdf[5] - An excellent NTFS overview and reference

http://technet.microsoft.com/en-us/library/cc781134(WS.10).aspx[6] - An old (2001) but good walkthrough of NTFS

http://www.pcguide.com/ref/hdd/file/ntfs/arch.htm[7] - Explains the MFT and files with slightly more detail than [5]

http://technet.microsoft.com/en-us/library/cc938949.aspx[8] - Wikipedia entry on File System Forks

http://en.wikipedia.org/wiki/Alternate_data_streams[9] - Talk of ADS from a security perspective

http://www.forensicfocus.com/dissecting-ntfs-hidden-streams[10] An excellent white paper on Alternate Data Streams and how to enumerate them

http://www.sans.org/reading_room/whitepapers/honors/alternate_data_streams_out_of_the_shadows_and_into_the_light_1503?show=1503.php&cat=honors

Alternate Data Streams

Technology

Transcript of Alternate Data Streams