fileXray

1

FILEXRAY PLUS

fileXrayUSER GUIDE AND REFERENCE

2

© 2010 iohead LLC. All Rights Reserved.

The content of this document is furnished for information use only, is subject to change without no-tice, and should not be construed as a commitment by iohead LLC. iohead LLC assumes no responsi-bility or liability for any errors or inaccuracies that may appear in the information contained in this document.

Apple, the Apple logo, Mac, and Mac OS are trademarks of Apple Inc., registered in the U.S. and other countries. Other product names mentioned herein may be trademarks of Apple Inc. or of other compa-nies.

End User License Agreement for the fileXray Software

PLEASE READ THIS SOFTWARE END USER LICENSE AGREEMENT (“LICENSE”) CAREFULLY BEFORE PURCHASING OR USING THE fileXray SOFTWARE (DEFINED BELOW). BY USING THE fileXray SOFTWARE, YOU ARE AGREEING TO BE BOUND BY THE TERMS OF THIS LICENSE. IF YOU DO NOT AGREE TO THE TERMS OF THIS LICENSE, DO NOT USE THE fileXray SOFTWARE.

IMPORTANT NOTE: This software may be used to reproduce, modify, publish and distribute materials. It is licensed to you only for reproduction, modification, publication and distribution of non-copyrighted materials, materials in which you own the copyright, or materials you are authorized or legally permitted to reproduce, modify, publish and distribute. If you are uncertain about your right to copy, modify, publish and distribute any material you should contact your legal advisor.

1. General. The software, including any and all of its versions and variations, tools, utilities, sample or example code, docu-mentation, and other materials accompanying this License, whether on disk, print or electronic documentation, in read only memory, or any other media (collectively, the “fileXray Software”) are licensed, not sold, to you by iohead LLC (“iohead”) and/or iohead’s licensors. The rights granted herein are limited to iohead’s and/or iohead’s licensors’ respective intellec-tual property rights in the fileXray Software and do not include any other patents or intellectual property rights. You own the media on which the fileXray Software is recorded but iohead and/or iohead’s licensors retain ownership of their respec-tive portions of the fileXray Software itself. The terms of this License will govern any software upgrades provided by iohead that replace and/or supplement the original fileXray Software, unless such upgrade is accompanied by a separate license in which case the terms of that license will govern.

2. License Types. The fileXray Software is made available to you under the terms of one of two license types depending on which license type you purchased: the Professional-Use License or the Personal-Use License.

3. Permitted License Uses and Restrictions.

A. fileXray Software with the Professional-Use License. If you purchased the Professional-Use License and if you use the fileXray Software, you are subject to all the terms of this License except the terms in Section 3.B. The Professional-Use License is a single-user single-installation license. It is valid for a single user using the fileXray Software on a single physi-cal computer. It permits the use of the fileXray Software by businesses and organizations as long as the single-user single-installation terms are adhered to. You may use the fileXray Software only in compliance with the permitted use as defined in this Section 3.A.

B. fileXray Software with the Personal-Use License. If you purchased the Personal-Use License and if you use the fileXray Software, you are subject to all the terms of this License except the terms in Section 3.A. The Personal-Use License is a single-user multiple-installation license for individuals. It permits the fileXray Software to be used by a single user on any number of that user’s personal machines in the same household. It does not permit use by businesses or organizations. You may use the fileXray Software only in compliance with the permitted use as defined in this Section 3.B.

C. Other Restrictions. Except as and only to the extent expressly permitted by this License or to the extent that the fol-lowing restrictions are prohibited by applicable law, you may not copy, decompile, reverse engineer, disassemble, attempt to derive the source code of the fileXray Software, modify, decrypt, create derivative works of, incorporate into or compile in combination with your own programs, sublicense or otherwise redistribute the fileXray Software.

4. Transfer. You may not rent, lease, lend, redistribute or sublicense the fileXray Software. NFR (Not for Resale) and Evalua-tion Copies: Notwithstanding other sections of this License, fileXray Software labeled as Not for Resale or Evaluation by iohead, or otherwise provided to you by iohead on a promotional basis may only be used for demonstration, testing and evaluation purposes and may not be resold or transferred.

5. Termination. This License is effective until terminated. Your rights under this License will terminate automatically without notice from iohead if you fail to comply with any term(s) of this License. In some cases, the specific license you receive may have an expiration date. If so, your rights under this License will terminate automatically on the expiration date without notice from iohead.

6. Disclaimer of Warranties. YOU EXPRESSLY ACKNOWLEDGE AND AGREE THAT USE OF THE fileXray SOFTWARE IS AT YOUR SOLE RISK AND THAT THE ENTIRE RISK AS TO SATISFACTORY QUALITY, PERFORMANCE, ACCURACY AND EFFORT IS WITH YOU. THE fileXray SOFTWARE IS PROVIDED “AS IS,” WITH ALL FAULTS AND WITHOUT WARRANTY OF ANY KIND, AND IOHEAD LLC AND IOHEAD LLC’S LICENSORS (COLLECTIVELY REFERRED TO AS “IOHEAD” FOR THE PURPOSES OF SECTIONS 6 AND 7) HEREBY DISCLAIM ALL WARRANTIES AND CONDITIONS WITH RESPECT TO THE fileXray SOFTWARE, EITHER EXPRESS, IMPLIED OR STATUTORY, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES AND/OR CONDITIONS OF MERCHANTABILITY, OF SATISFACTORY QUALITY, OF FITNESS FOR A PARTICULAR PURPOSE, OF ACCURACY, OF QUIET ENJOYMENT, AND NON-INFRINGEMENT OF THIRD PARTY RIGHTS. IOHEAD DOES NOT WARRANT AGAINST INTERFERENCE WITH YOUR ENJOYMENT OF THE fileXray SOFTWARE, THAT THE FUNCTIONS CONTAINED IN THE fileXray SOFTWARE WILL MEET YOUR REQUIREMENTS, THAT THE OPERATION OF THE fileXray SOFTWARE WILL BE UNINTERRUPTED OR ERROR-FREE, OR THAT DEFECTS IN THE fileXray SOFTWARE WILL BE CORRECTED. YOU FURTHER ACKNOWLEDGE THAT THE fileXray SOFTWARE IS NOT INTENDED OR SUITABLE FOR USE IN SITUATIONS OR ENVIRONMENTS WHERE THE FAILURE OF, OR ERRORS OR INACCURACIES IN THE CON-TENT, DATA OR INFORMATION PROVIDED BY, THE fileXray SOFTWARE COULD LEAD TO DEATH, PERSONAL INJURY, OR SE-VERE PHYSICAL OR ENVIRONMENTAL DAMAGE, INCLUDING WITHOUT LIMITATION THE OPERATION OF NUCLEAR FACILITIES, AIRCRAFT NAVIGATION OR COMMUNICATION SYSTEMS, AIR TRAFFIC CONTROL, LIFE SUPPORT OR WEAPONS SYSTEMS. NO ORAL OR WRITTEN INFORMATION OR ADVICE GIVEN BY IOHEAD OR AN IOHEAD AUTHORIZED REPRESENTATIVE SHALL CRE-

3

ATE A WARRANTY. SHOULD THE fileXray SOFTWARE PROVE DEFECTIVE, YOU ASSUME THE ENTIRE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION.

7. Limitation of Liability. TO THE EXTENT NOT PROHIBITED BY LAW, IN NO EVENT SHALL IOHEAD BE LIABLE FOR PERSONAL INJURY, OR ANY INCIDENTAL, SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES WHATSOEVER, INCLUDING, WITHOUT LIMI-TATION, DAMAGES FOR LOSS OF PROFITS, CORRUPTION OR LOSS OF DATA, BUSINESS INTERRUPTION OR ANY OTHER COM-MERCIAL DAMAGES OR LOSSES, ARISING OUT OF OR RELATED TO YOUR USE OR INABILITY TO USE THE fileXray SOFTWARE, HOWEVER CAUSED, REGARDLESS OF THE THEORY OF LIABILITY (CONTRACT, TORT OR OTHERWISE) AND EVEN IF IOHEAD HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

8. Export Control. You may not use or otherwise export or reexport the fileXray Software except as authorized by United States law and the laws of the jurisdiction(s) in which the fileXray Software was obtained. In particular, but without limita-tion, the fileXray Software may not be exported or re-exported (a) into any U.S. embargoed countries or (b) to anyone on the U.S. Treasury Department’s list of Specially Designated Nationals or the U.S. Department of Commerce Denied Person’s List or Entity List. By using the fileXray Software, you represent and warrant that you are not located in any such country or on any such list. You also agree that you will not use the fileXray Software for any purposes prohibited by United States law, including, without limitation, the development, design, manufacture or production of nuclear, chemical or biological weap-ons.

9. Third Party Acknowledgements. Portions of the fileXray Software may utilize the following copyrighted material, the use of which is hereby acknowledged.

A. Apple Inc. (partition parsing)Copyright 1996,1997,1998 by Apple Computer, Inc.All Rights Reserved.

Permission to use, copy, modify, and distribute this software and its documentation for any purpose and without fee is hereby granted, provided that the above copyright notice appears in all copies and that both the copyright notice and this permission notice appear in supporting documentation.

APPLE COMPUTER DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. IN NO EVENT SHALL APPLE COMPUTER BE LIABLE FOR ANY SPECIAL, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN ACTION OF CONTRACT, NEGLIGENCE, OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

B. The NetBSD Foundation Inc. (number formatting)Copyright (c) 1997, 1998, 1999, 2002 The NetBSD Foundation, Inc.All Rights Reserved.

This code is derived from software contributed to The NetBSD Foundation by Jason R. Thorpe of the Numerical Aerospace Simulation Facility, NASA Ames Research Center, by Luke Mewburn and by Tomas Svensson.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following dis-claimer.2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following dis-claimer in the documentation and/or other materials provided with the distribution.3. All advertising materials mentioning features or use of this software must display the following acknowledgement: This product includes software developed by the NetBSD Foundation, Inc. and its contributors.4. Neither the name of The NetBSD Foundation nor the names of its contributors may be used to endorse or promote prod-ucts derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IM-PLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOW-EVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLI-GENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

C. Christos Zoulas (magic library)Copyright (c) 2008 Christos ZoulasAll Rights Reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following dis-claimer.

4

2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following dis-claimer in the documentation and/or other materials provided with the distribution.THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IM-PLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOW-EVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLI-GENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

D. The OpenSSL Project (standard functions)Copyright (c) 1998-2003 The OpenSSL Project.All Rights Reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following dis-claimer.2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following dis-claimer in the documentation and/or other materials provided with the distribution.3. All advertising materials mentioning features or use of this software must display the following acknowledgment: “This product includes software developed by the OpenSSL Project for use in the OpenSSL Toolkit. (http://www.openssl.org/)”4. The names “OpenSSL Toolkit” and “OpenSSL Project” must not be used to endorse or promote products derived from this software without prior written permission. For written permission, please contact [email protected]. Products derived from this software may not be called “OpenSSL” nor may “OpenSSL” appear in their names without prior written permission of the OpenSSL Project.6. Redistributions of any form whatsoever must retain the following acknowledgment: “This product includes software de-veloped by the OpenSSL Project for use in the OpenSSL Toolkit (http://www.openssl.org/)”

THIS SOFTWARE IS PROVIDED BY THE OpenSSL PROJECT “AS IS” AND ANY EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE OpenSSL PROJECT OR ITS CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHER-WISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

E. Csaba Henk (ublio library)Copyright 2006 Csaba Henk <[email protected]>All Rights Reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.THIS SOFTWARE IS PROVIDED BY THE AUTHOR “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSE-QUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CON-TRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

10. Additional Bundled Software. The fileXray Software includes a copy of the NuFS Software, which is also an iohead prod-uct. By using the fileXray Software, you are also agreeing to be bound by the terms of NuFS Software End User License Agreement.

11. Online Version of this License. An online version of this License is available at: http://iohead.com/legal/fileXray/ An online version of the NuFS Software End User License agreement is available at: http://iohead.com/legal/NuFS/

12. fileXray Software End User License Agreement Version: 1

If you agree to the terms of this license, you may use the above version number to run the fileXray Software installer in non-interactive mode.

5

http://www.openssl.org


mailto:[email protected]






http://iohead.com/legal/fileXray/

http://iohead.com/legal/fileXray/

http://iohead.com/legal/NuFS/

http://iohead.com/legal/NuFS/

6

This page intentionally left blank.

User Guide 11

0. Minimum System Requirements 13

1. Installing and Getting Started 14

2. Introduction to fileXray 18

3. Basic Usage 22

4. Analyzing Volume Structure 26

5. Analyzing File System Objects 28

6. Analyzing Volume Usage 29

7. Monitoring File System Changes 30

8. Retrieving Volume Content 31

9. Altering Volume Behavior 32

10. Extending fileXray: Developing and Using Filters 33

11. Advanced Forensics and Content Recovery 34

12. References 35

Reference 37

-‐-‐allocation 39

-‐-‐badblock 41

-‐-‐block BLOCK 42

-‐-‐btree BTREE_NAME 44

-‐-‐checksum COMPONENTS 48

-‐-‐cnid CNID 50

-‐-‐device DEVICE 52

-‐-‐disallow_mounting SECONDS 56

-‐-‐diskusage 59

-‐-‐enable_xattr_extents 61

-‐-‐exhaustive 63

-‐-‐filter FILTER 65

bmactime 66

compressed 68

creatorcode 68

device 69

7

dirhardlink 69

empty 69

emptyforks 69

fifo 70

hardlink 70

immutable 70

lsR 71

macho 71

name 72

namei 72

nameprefix 72

nameprefixi 72

namesuffix 73

namesuffixi 73

nodename 73

null 74

socket 74

subname 74

subnamei 74

sxid 74

symlink 75

typecode 75

xattrname 76

xattr 76

Implementing Your Own Filters 77

-‐-‐filter_args STRING 81

-‐-‐force 82

-‐-‐fragmentation FORK_TYPE 83

-‐-‐freespace 85

-‐-‐fsspec PARENT_CNID:NAME 86

-‐-‐hotfiles 88

-‐-‐iec 89

introspect 90

Introspecting Files 91

Introspecting Folders 93

8

Introspecting File Hard Links 94

Introspecting Directory Hard Links 98

Introspecting Symbolic Links 102

-‐-‐journal 104

-‐-‐journal_names 107

-‐-‐list 110

-‐-‐list_records RECORD_TYPE 112

-‐-‐monitor 114

-‐-‐monitor_exclude VOLUME 116

-‐-‐monitor_include VOLUME 117

-‐-‐monitor_logfile LOGFILE 118

-‐-‐mount_data 119

-‐-‐next_allocation BLOCK 123

-‐-‐node NODE 126

-‐-‐noidmap 127

-‐-‐output OUTPUT_FILE 129

-‐-‐partition START[,SIZE] 130

-‐-‐path PATH 131

Dot-Dot 131

Link Resolution 132

Terminal Slash 132

Long Node Names and Name Mangling 132

-‐-‐read COMPONENT 134

-‐-‐readonly 135

-‐-‐scavenge 136

-‐-‐summary FORK_TYPE 144

-‐-‐top N 146

-‐-‐trawl QUERY 147

-‐-‐uncompress 151

-‐-‐undelete_cookie COOKIE 153

-‐-‐usedspace 155

-‐-‐userfs_mount MOUNT_POINT 156

9

-‐-‐userfs_type USERFS_TYPE 157

Arbitrary File System 157

Free Space File System 160

Structure File System 161

Used Space File System 162

Scavenger File System 163

-‐-‐version 167

-‐-‐volume VOLUME_PATH 168

-‐-‐volume_header 169

-‐-‐who_owns_byte BYTE 172

-‐-‐who_owns_block BLOCK 173

10

User Guide

11

12


0. Minimum System Requirements

• Mac OS X 10.5 Leopard

13

1. Installing and Getting Started

The fileXray Software (fileXray) ships as a single Universal Binary called fileXray-installer (the installer). Both fileXray and the installer are command-line programs. When you purchased fileXray, you would have re-ceived a personalized URL to download the installer. Please save this URL as you can use it to download or re-download the latest version of fileXray that your license is eligible for. The latest version number string of fileXray and the SHA-1 checksum of the fileXray-installer binary can be retrieved from:

http://iohead.com/fileXray/latest.txt

During fileXray installation, the installer copies the fileXray program ex-ecutable to a folder you specify and populates the /Library/Application Support/fileXray/ folder. fileXray installation creates no other files or fold-ers. fileXray also does not create any preference or configuration files at in-stallation time or at any other time.

1.1 First Installation

To install, reinstall, or upgrade fileXray, run the fileXray-installer pro-gram on the command line—typically, in a Terminal application window. The first time you install fileXray on a computer, the installer will ask you to pro-vide the following information.

• The path to the license data file. This is the ASCII file you would have re-ceived as an attachment to the email containing the installer download URL. You can either save the attachment, or save the cut-and-pasted ASCII content (license blob) to a file.

• The email address associated with the license. This is the email address you used to purchase fileXray.

• Confirmation that you have read and accepted the fileXray End User License Agreement (EULA).

• The path to a folder in which to copy the fileXray executable. You will need superuser privileges to install fileXray. Therefore, run the installer using the sudo command. By default, if an fileXray executable already exists in the specified folder, it will not be overwritten. See Section 1.3 on how to explicitly instruct the installer to overwrite any existing fileXray executable.

The following page shows an example of installing fileXray.

14



If fileXray is installed on a computer, you can use the --version option of fileXray to view version and license information.

1.2 Upgrading or Reinstalling

If you previously installed fileXray on a computer and valid license data is already present, the installer will normally ask only one question: the path to a folder in which to copy the fileXray executable. This would typically be the case if you are upgrading to a new version or reinstalling for some reason.

However, if a new version of fileXray is accompanied by an updated End User License Agreement (EULA), you may be asked to provide certain information again even if valid license data is present on that computer. Specifically, you may be asked to provide the email address associated with the license along with confirmation that you have read and accepted the updated EULA.

1.3 Non-Interactive or Scripted Installation

The installation described in Section 1.1 is interactive in that you are required to provide certain information to the installer by typing responses to prompts displayed on the screen. The same information can be provided to the installer as command-line arguments. This allows for non-interactive installation or up-grade of the fileXray software, say, by a script in a managed environment. You can use the --help option of the installer to display the options that the installer accepts.

15

$ chmod +x ./fileXray-installer$ sudo ./fileXray-installerPlease type the path to the license file : /tmp/fileXray-license.txt

Please type the email address for this license : [email protected]

You must read and accept the End User License Agreement (EULA) to continue. The EULA is available at http://filexray.com/eula.txtHave you read the EULA and do you accept it? (yes/no) yes

Where do you wish to place fileXray?Please type the full path to an existing folder: /usr/local/bin

fileXray installed as /usr/local/bin/fileXray. Enjoy.$

$ fileXray --versionfileXray plus 1.0.0Copyright (c) 2010 iohead LLC. All Rights Reserved.

Licensed to: John Doe <[email protected]>Product : fileXray plus (licensed for professional use)Purchased : Monday Nov 2, 2009

The --accept option is used to specify to the installer that the installing authority1 has read and accepted the End User License Agreement (EULA). The argument to --accept is the version number of the EULA, which can be found at the end of the EULA document. The most recent version of the EULA will apply to the latest version of fileXray.

Note that for a fully non-interactive installation, all required arguments must be specified, otherwise the installer will prompt for the missing information, rendering the installation interactive.

For a first installation, the required arguments are --accept, --email, --li-cense, and --installfolder.

For an upgrade or a reinstallation, assuming that valid license data from an earlier installation is present, the required arguments are --installfolder and perhaps --overwrite. In the case that the new version comes with a newer version of the EULA, the --accept argument will also be required.

The following is an example of a fully non-interactive installation.

16

1 The installing authority is a user or system administrator who has the legal authority to ac-cept the fileXray EULA and subsequently install fileXray (either interactively or non-interactively, say, through a script) in conformance with a valid instance of the fileXray li-cense.

$ fileXray-installer --helpfileXray-installer 1.0.0Copyright (c) 2010 iohead LLC. All Rights Reserved.This is the fileXray installer. Usage:

fileXray-installer [ARGUMENTS]

This program accepts the following options:

-a EULA, --accept=EULA End User License Agreement version that you have read and do accept -e EMAIL, --email=EMAIL email address associated with license -h, --help print this help message and exit -l PATH, --license=PATH path to license data file -i PATH, --installfolder=PATH install fileXray in this folder -o, --overwrite overwrite existing fileXray binary if necessary -v, --version display version information and exit

Please refer to the fileXray User Guide for more details.

$ sudo ./fileXray-installer --accept=<version> [email protected] \ --license=/tmp/fileXray-license.txt --installfolder=/usr/local/bin \ --overwritefileXray installed as /usr/local/bin/fileXray. Enjoy.$

1.4 Uninstalling

fileXray installation creates the following on-disk entities:

• The fileXray executable, which is created in a preexisting user-specified folder

• The /Library/Application Support/fileXray/ folder and its contents

fileXray can be completely uninstalled by deleting the fileXray executable and the /Library/Application Support/fileXray/ folder. There are no preference files, cache files, or other files to uninstall.

1.5 Running fileXray

As a command-line program, fileXray is meant to be run from a command shell such as that provided by the Terminal application in Mac OS X. The fileXray Universal Binary contains executables for 3 architectures:

• 64-bit Intel (preferred if you have a 64-bit Intel processor)• 32-bit Intel• 32-bit PowerPC (requires at least a G4 processor)

fileXray will not run under Rosetta.

1.6 System Resources and fileXray Performance

In general, fileXray has no specific requirements in terms of processor speed and memory. That said, fileXray operations that perform volume-wide analy-ses, searching, or scavenging are likely to benefit from more system resources. As a rule of thumb, the time consumed by such operations will depend on fac-tors such as the following.

• Size and population of the volume on which the operation is being per-formed.

• Speed of the disk drive or other storage medium on which the volume re-sides along with the speed of the connecting bus.

• Amount of RAM available on the system.• Processor speed.

17

2. Introduction to fileXray

Files, and consequently file systems, underlie most of the interactions users have with the operating system. The HFS+ file system in Mac OS X supports a rich and interesting set of features. It is eminently useful and rewarding to learn what these features are, how they work, and how you can use them to your advantage. Understanding advanced file system techniques will help you in gaining a fine-grained control of a system, whether it be for administration, analysis, debugging, investigation1, optimization, or just plain fun.

fileXray puts a rich repertoire of such techniques at your fingertips.

2.1 HFS+

HFS+, also referred to as “HFS Plus” or the “Mac OS X Extended” volume for-mat, is the default and the only practically viable volume format for Mac OS X installations. Historically, it was possible to install Mac OS X on a UFS volume, albeit at the cost of compatibility issues. However, since Mac OS X 10.5 (Leop-ard), HFS+ is the only volume format that a Mac OS X installation can reside on. Moreover, almost all2 of Apple’s devices that use file system storage use HFS+. In other words, HFS+ is ubiquitous when dealing with Apple’s comput-ers and devices.

HFS+ was introduced with Mac OS 8.1 in early 1998. It has evolved greatly since its inception. Especially with the advent and subsequent evolution of Mac OS X, a large number of features have been retrofitted into HFS+. Conse-quently, the HFS+ implementation in Mac OS X has grown large3 and complex over the years.

Just as Mac OS X brings together the philosophies of the classic Macintosh world and the Unix world, HFS+ is an amalgamation of file system ideas and concepts from these worlds. Moreover, Mac OS X has multiple programming interfaces for accessing files: some Unix-based and some derived from classic Macintosh interfaces. (This is not necessarily a good thing: the differences and similarities in these various interfaces is often a source of confusion among de-velopers.) Along similar lines, Mac OS X provides both graphical applications and command-line programs for working with files and file systems. In general, there are multiple layers of software between an HFS+ volume and an applica-

18

1 A major portion of digital forensics boils down to analyzing and dissecting a file system vol-ume.

2 iPods meant for use with Windows and certain iPod models in general use the FAT32 volume format.

3 The HFS+ implementation in Mac OS X 10.6 (Snow Leopard) is nearly 60,000 lines of kernel code.

tion that’s accessing files residing on that volume. The following illustration shows a high-level overview.

For information on an HFS+ volume to be accessible by standard applications, the volume must be “mounted” by the operating system’s HFS+ implementa-tion. Each of the software layers in the above illustration can potentially “mas-sage” the information as it passes up from raw storage to an application—ex-tended attributes and metadata can be hidden, metadata and even data can be dynamically transformed, and so on. This means that any introspection or analysis you perform on an HFS+ volume through a program is subject to the cumulative terms and conditions of the software layers used by the program. In particular, the operating system’s HFS+ implementation and the various pro-gramming interfaces are neither meant for nor optimized for investigative use. In fact, several types of information necessary for or beneficial to comprehen-sively analyzing an HFS+ volume is simply not available through any standard programming interfaces. This is where fileXray comes in.

19

!"#$%#$&#'

()*+,-+.+/0-1234'#3#&5)567&

8&79#:;0-+')<#$

=->+06'#+?@2"+AB+C6D$)$<E

B)$D7&+06'#+()&)F#$

G-06'#/)&9'# G-06'#()&)F#$ G-06'#H$)44#$B7*7)

B)$D7&

B)$D7&?44'6*)567&

B7*7)?44'6*)567&

/<D$69?44'6*)567&

G-!IC

!"#$

%&

/0-1;7'J3#

2.2 fileXray

fileXray is a powerful, versatile, extensible, and safe command-line tool for analyzing, dissecting, monitoring, scavenging, and performing forensic investi-gation on HFS+ file system volumes. fileXray even has features that are not specific to HFS+ but are generally useful to file system forensics and debug-ging.

At a very high level, fileXray’s functionality can be classified under the fol-lowing two categories.

1. Things that simply cannot be done by standard programs. Most of what fileXray can do belongs to this category.2. Things that can partially or perhaps even entirely be done by standard programs, but fileXray can do them better and/or considerably faster. Some of fileXray’s features fall under this category.

Examples of features in the first category include displaying internal details of data structures associated with an HFS+ volume as a whole, displaying inter-nal details of individual file system objects, calculating various types of volume statistics, mapping arbitrary disk blocks to files, analyzing internal and exter-nal volume journals, analyzing volume fragmentation, and scavenging for de-leted content.

Examples of features in the second category include monitoring live file system activity, listing directory contents, and rapidly searching for file system objects by name, type, or other attributes.

A key architectural aspect of fileXray is that it does not rely on the operating system’s HFS+ implementation—it directly accesses an HFS+ volume and proc-esses all of the volume’s contents itself. The following is a conceptual view.

20

!"#$%#$&#'

()*+$#),"+-.+/0#+,#123#+-$+.2'#+4&,#$'52&6+/0#+789:+1-'4;#

789:<-'4;#

!"#$%&'(

=4"/-;789:

>;?'#;#&/)/2-&

@@@

A$-3#""2&6+B-,4'#"

That is, fileXray essentially contains a custom implementation of the HFS+ file system. It reads from the raw storage underlying an HFS+ volume and processes all HFS+ data structures itself. This has several implications, for ex-ample:

• fileXray can perform introspections and analyses that would other-wise not be possible since the relevant information required is not ac-cessible through any standard programming interface.

• fileXray can operate on an HFS+ volume even if it is offline (that is, not mounted). This may be highly desirable in certain use cases.

• fileXray can optimize its HFS+ implementation for its own use cases.

In general, fileXray cannot be used to modify an HFS+ volume. In fact, fileXray guarantees safety by always accessing the raw storage in read-only mode. However, a few fileXray operations can indirectly modify the state of an HFS+ volume when explicitly instructed to do so. The documentation will call out these options and the modifications they can cause to occur.

The remainder of this User Guide briefly outlines fileXray features. A detailed discussion of the features along with usage examples follows in the fileXray Reference.

21

3. Basic Usage

Most fileXray operations are performed on an HFS+ volume, which can be either online (mounted) or offline1 (unmounted). A few operations, however, are meant only for mounted volumes. There also are operations that are not HFS+ specific and do not require an HFS+ volume.

fileXray supports all flavors of HFS+: Journaled or Non-Journaled, Case-Sensitive or Case-Preserving-Case-Insensitive, including HFS+ volumes em-bedded in legacy HFS “wrappers.” The legacy HFS format itself is not sup-ported, however.

3.1 Specifying a Volume

An HFS+ volume can be specified to fileXray in several ways.

• Explicitly by specifying the block or character device through the --de-vice option. The volume may be either mounted or unmounted. For ex-ample:

• Explicitly by specifying a raw file system dump file or a disk image through the --device option. The volume may be either mounted or unmounted. For example:

• Explicitly by specifying a mount point or a path residing within the vol-ume through the --volume option. The volume must be mounted. For example:

22

1 fileXray accesses the raw storage underlying an HFS+ volume and itself interprets the file system structure and contents, without relying on the operating system’s HFS+ implementa-tion. This is what allows fileXray to handle offline volumes.

$ fileXray --device /dev/disk0s2 [other options…]…

$ fileXray --device /dev/rdisk0s3 [other options…]…

$ fileXray --device disk0s4 [other options…]…

$ fileXray --device /tmp/Backup.dmg [other options…]…

$ fileXray --device /tmp/rootfs-raw.dd [other options…]…

• Implicitly by specifying a non-volume file system object such as a file or folder, from which fileXray will derive the volume. The volume must be mounted. See Section 3.2 for examples of specifying file system objects.

• Implicitly by leaving out a volume specification altogether, in which case fileXray will use the root volume as long as any other specified options do not contradict the root volume’s usage. For example:

If you use the --device option to specify a device or a file containing a device image, fileXray will automatically use a volume if one of the following is true:

• The specified device contains an HFS+ volume “directly” on it. This means either the volume is an HFS+ formatted slice (such as /dev/disk0s2) of a “whole” device (such as /dev/disk0), or the device is a whole device without any partitioning and it simply contains an HFS+ volume. The latter case is uncommon but feasible and valid.

• The device is a whole device with valid partitioning on it and it contains exactly one HFS+ partition.

fileXray can detect and parse GUID Partition Table (GPT), Apple Partition Map (APM), and Master Boot Record (MBR) partitioning schemes. If the speci-fied --device argument is such that multiple HFS+ partitions are detected, fileXray will list all partitions that it found and will prompt you to choose one through the --partition option. For example:

23

$ fileXray --device /tmp/test.dmg [other options…]No flavor of HFS+ found.

However, a GUID Partition Table was detected on the given image file with thefollowing 2 HFS+ partition table entries:

starts size in mnemonicat byte bytes

20480 536870912 Backup HD (HFS+, 537 MB)536891392 536829952 Pictures HD (HFS+, 537 MB)

Use --partition (-e) to specify a particular partition table entry.

$ fileXray --device /tmp/test.dmg --partition 20480,536870912 [other options…]…

$ fileXray --volume "/Volumes/Macintosh HD" [other options…]…

$ fileXray --volume "/Volumes/Macintosh HD/mach_kernel" [other options…]…

# Display the root volume’s Volume Header$ sudo fileXray --volume_header…

3.2 Specifying Folders1 and Files

Numerous types of file system objects can exist on an HFS+ volume: folders, regular files, hard links, directory hard links, symbolic links, device files, named pipes, Unix Domain socket files, and so on. From an implementation detail standpoint, however, HFS+ has only two types of fundamental objects in the file system hierarchy: folders and files. All user-visible objects are imple-mented using these two types under the hood. Several fileXray operations are targeted for folders and files. You can specify a file system object to fileXray in several ways:

• Explicitly by specifying the object’s Catalog Node ID (CNID2) through the --cnid option. For example:

• Explicitly by specifying the object’s parent’s CNID along with the object’s name through the --fsspec option. For example:

• Explicitly by specifying the absolute path to the object through the --path option. The specified path may contain directory hard links, file hard links, symbolic links, double dots, mangled node names, and so on. How fileXray will resolve path components is described in the discus-sion of the --path option. For example:

• Implicitly by specifying either an absolute or relative path to the object. Doing so requires the target volume to be mounted. (In contrast, the three aforementioned ways of specifying an object work on either mounted or unmounted volumes.) For example:

24

1 We use the terms “folders” and “directories” interchangeably in this document.

2 The CNID is a 32-bit unsigned integer assigned to each file and folder on an HFS+ volume. Except in the case of file and directory hard links, an object’s CNID is its inode number as ex-ported by the BSD layer.

$ fileXray --cnid 12345 [other options…]…

$ fileXray --fsspec 2:mach_kernel [other options…]…

$ fileXray --path /Users/johndoe/Pictures/Test.jpg [other options…]…

$ fileXray [other options…] ../Test.jpg…

3.3 Switching Information Display or Processing Modes

Certain fileXray options can be used to change the behavior of other options. For example:

• The --exhaustive option can be additionally specified to either enable a verbose information display mode or enable more exhaustive information processing—depending on the operation it’s used in conjunction with.

• By default, fileXray uses SI multiples (powers of 10) when displaying byte values. The --iec option can be additionally used to switch to pow-ers of 2.

• By default, fileXray maps owner and group identifiers (both numerical identifiers and UUIDs) to names. The name resolution is done in the con-text of the machine fileXray is running on. If this behavior is undesir-able, the name resolution can be disabled by additionally specifying the --noidmap option.

3.4 General Purpose Features

As noted earlier, certain fileXray features are not specific to HFS+. These fea-tures include:

• Temporarily disabling automatic volume mounting by Disk Arbitration. (See --disallow_mounting.) This is useful while performing forensic or security analyses.

• Attempting to force a mounted volume of any type into read-only mode. (See --readonly.)

• The Arbitrary File System, which allows arbitrary byte ranges on a device or a file to be read through a synthetic file system wherein the file name encodes the starting offset and number of bytes to read. For example, reading a virtual file named 512,4096.txt in the Arbitrary File System will read 4096 bytes from offset 512 of the device or file in question. (See Arbitrary File System.)

• Live monitoring of file system modification activity on any volume type. (See --monitor.)

25

4. Analyzing Volume Structure

fileXray provides the means to perform a complete analysis of an HFS+ vol-ume’s internal data structures.

The --volume_header option displays the Volume Header, the Alternate Vol-ume Header, and if applicable, the HFS Master Directory Block. Besides dis-playing the contents of these data structures, --volume_header annotates the output with additional information computed dynamically. The output of --volume_header can be thought of as a high-level summary of the volume’s structure.

For each mounted HFS+ volume, the Mac OS X kernel maintains various run-time data structures in memory. The contents of several of these data struc-tures are useful in debugging and analysis. The --mount_data option retrieves this information from kernel memory and displays it.

HFS+ employs several B-Trees in its implementation: the Catalog B-Tree, the Extents Overflow B-Tree, the Attributes B-Tree, and the Hot File Clustering B-Tree. The --btree option can be used to introspect each of these B-Trees. In conjunction with the --node and --list_records options, --btree can also be used to analyze individual nodes and records in a given B-Tree. These op-tions can just as well be used to dump the contents of an entire B-Tree.

The --allocation option displays the state—in-use or free—of each allocation block1 on the volume. When used with the --exhaustive option, --alloca-tion ascribes each block to its “owner”—that is, it tags each in-use block with the type of the file system entity that’s using it. The --who_owns_block and --who_owns_byte options can identify which specific file or file system entity, if any, a given block or byte on the volume belongs to.

The --journal option displays the state and structure of the journal on a journaled2 HFS+ volume. In conjunction with the --exhaustive option, --journal analyzes each transaction found in the journal and also determines the ownership of each block recorded within a transaction. fileXray supports both internal (locally resident) and external3 (resident on another device) jour-nals.

26

1 Space on an HFS+ volume is allocated to files in units called allocation blocks. The size of an allocation block for a given volume is fixed at volume creation time.

2 Metadata journaling is enabled by default on modern-day HFS+ volumes.

3 By default, HFS+ uses an internal journal on Mac OS X client and typical server installations. The Mac OS X diskutil command can be used to move a volume’s journal to an external de-vice.

fileXray also includes built-in synthetic file systems to export certain types of volume information as a “file system,” allowing arbitrary programs to conven-iently access and operate on that information.

For example, the Structure File System essentially turns an HFS+ volume “in-side out”: given an HFS+ volume, it “mounts” the volume such that the resul-tant file system exposes key HFS+ data structures as files that can be normally read. For example, the Catalog-B Tree and the journal show up as files that you can analyze with tools such as hex editors, strings, and grep.

As another example, the Free Space File System makes a volume’s free extents visible as files that can be examined with standard tools. The Used Space File System does a similar thing for a volume’s used extents.

27

5. Analyzing File System Objects

When targeted at a file system object, say, a folder, regular file, hard link, di-rectory hard link, etc., fileXray displays all the metadata associated with that object along with various types of implementation details and dynamically computed auxiliary information. For example, if a file or directory hard link is examined, fileXray will enumerate1 all sibling file or directory hard links.

Object introspection also lists and shows the details of each extended attrib-ute2, if any, associated with the object. If an extended attribute happens to be the one that’s used to implement Access Control Lists (ACLs) on Mac OS X, fileXray will process the ACL and display the results. Similarly, if an ex-tended attribute is the one that’s used to implement transparently compressed3 files, fileXray will process that attribute and display details of the compres-sion scheme as applied to that object.

The introspect section of the Reference provides several examples of using fileXray to analyze file system objects.

28

1 This feature is applicable to hard links and directory hard links that were created on Mac OS X 10.5 (Leopard) or newer.

2 fileXray supports both inline and extent-based extended attributes.

3 Transparently compressed files, also known as “HFS-Compressed Files,” were introduced in Mac OS X 10.6 (Snow Leopard).

6. Analyzing Volume Usage

fileXray provides several ways to quantify and qualify volume usage.

The --allocation option provides a dump of the Allocation File, providing a quick look at each allocation block’s in-use/free status. When used with the --exhaustive option, --allocation annotates each allocation block’s status with the type of the file system entity that owns the block. The --who_owns_block option identifies the exact owner of a block—that is, given a block number, it tells you whether the block is free and if not, which system or user file it is allocated to. The --who_owns_byte option answers a similar question for any given byte on a volume.

The --freespace and --usedspace options display lists of free and used ex-tents, respectively. The lists can be readily sorted to determine the largest and smallest free and used space chunks on a volume. In particular, the list dis-played by --freespace represents how discontiguous (fragmented) the avail-able space on a volume is. fileXray can also expose a volume’s free and used spaces as synthetic file systems courtesy of the Free Space File System and the Used Space File System, respectively. In particular, the Free Space File Sys-tem’s contents are virtual files each representing a contiguous chunk of free space on the volume. This allows for convenient analysis and searching of a volume’s free blocks using programs of your choosing.

The --summary options computes a rich summary consisting of the numbers of various types of file system objects the file has and several statistics on the data and resource forks residing on a volume. When combined with the --top option, --summary can be used to display the largest data or resource forks on a volume.

The --fragmentation option can be used to quantify how fragmented a vol-ume is. When used with the --exhaustive option, it lists relevant details of all file forks that have non-zero fragmentation. When used with the --top option, it can be used to display the most fragmented forks on a volume.

The --hotfiles option can be used to list the “hottest” files on a volume, as tracked and determined by the Hot File Clustering scheme that’s built into the Mac OS X HFS+ implementation. Hot files are tracked in a B-Tree, which can be introspected through options such as --btree, --node, and --list_re-cords.

The --diskusage option can be used to compute a given file system object’s on-disk usage, which includes not only the data and resource forks (for files), but also the cumulative disk space used by the names and values of any ex-tended attributes the object might have.

29

7. Monitoring File System Changes

fileXray includes a built-in file system activity monitor that uses Mac OS X’s fsevents1 file system event notification mechanism. The monitor is not limited to HFS+ volumes: it can be used to monitor modifications to any type of vol-ume. It can be invoked through the --monitor option. While the monitor is running, it displays file system event notifications effectively in real time as they arrive from the kernel. A typical use of such functionality is to track which file system objects are being added, removed, or modified by some activity such as the launch of a program or during the installation of some software.

The --monitor_exclude and --monitor_include options can be used to tell the monitor to exclude or include, respectively, one or more volumes from being monitored.

Moreover, the monitor’s output can be logged to a file specified by the --moni-tor_logfile option in such a way that the writes to the log do not generate their own log messages.

30

1 The fsevents mechanism is also used by the Spotlight feature of Mac OS X.

8. Retrieving Volume Content

fileXray provides several ways to read content—both metadata and da-ta—from an HFS+ volume.

The --block option can be used to retrieve individual allocation blocks from a volume.

The --read option allows retrieving the contents of data forks, resource forks, and extended attributes. If a file is transparently compressed, it can have com-pressed content in either its resource fork or an extended attribute. fileXray can be optionally told through the --uncompress option to uncompress such content in flight.

Moreover, all of fileXray’s built-in file systems provide seamless ways of re-trieving various types of volume content.

31

9. Altering Volume Behavior

As a matter of policy, fileXray always opens a volume it is targeted at in read-only mode. Therefore, fileXray is incapable of directly1 modifying a volume. There are some fileXray options that can alter the state of a mounted volume, or intentionally prevent any volume from being mounted at all.

The --disallow_mounting option can be used to temporarily disable auto-matic mounting by Disk Arbitration of all and any volumes that are discovered as their storage devices are attached to the system. This feature, which is par-ticularly desirable and useful while performing file system forensics, is not lim-ited to HFS+ volumes.

The --enable_xattr_extents enables extent-based extended attributes on a volume provided the operating system’s HFS+ implementation supports them. By default, HFS+ stores extended attributes inline in the Attributes B-Tree. Given the default file system initialization parameters, this limits extended at-tribute size to 3802 bytes. Extent-based extended attributes are stored in allo-cation blocks that are referred to by “real” extents as in the case of data and resource forks. Therefore, the size of such attributes can be larger.

The --next_allocation option can be used to suggest to the system’s HFS+ implementation to start its search for a free allocation block at a given block.

32

1 You can, however, tell fileXray to save retrieved volume content and save it to a file specified by the --output option. In this case, technically (if one must split hairs) fileXray is modifying some volume, perhaps even the volume it is being targeted at if the argument to --output re-sides on that volume.

10. Extending fileXray: Developing and Using Fil-ters

fileXray can be extended by third parties through the filter mechanism that’s built into it. (See Implementing Your Own Filters for details.) A filter is a module that receives callbacks from fileXray as it walks the file system hierarchy on an HFS+ volume. Each callback invocation provides a filter a chance to exam-ine the details of a single file system object. The filter is free to apply any crite-ria to “match” or “reject” the passed object. Typically, a filter would print some information about objects it matches.

fileXray comes with a large number—over two dozen—of built-in filters. Most filters search for file system objects with some given properties, say, objects that are hard links or objects that have the setuid or setgid bits set. There also exist name-matching filters that look for exact, prefix, suffix, or substring name matches either case-sensitively or case-sensitively depending on the specific filter in question. Some filters do rather complex tasks: for example, the macho filter looks for Mach-O files and computes per-architecture object file sizes. The --filter section of the Reference describes each built-in filter.

One aspect of built-in fileXray filters is that they are considerably faster than their usual high-level counterparts. Consider the sxid filter, which searches for setuid and setgid file system objects on a volume.

The sxid filter took less than 15 seconds to scan the entire volume, which in this example is rather heavily populated at about 1.5 million files and folders. One can perform a similar search using the standard find command on the same volume as shown below.

Note that we told find to examine only files and limit itself to the root volume. find took about 286 seconds.1 We see that the sxid filter is nearly 20x faster than find.

By default, a filter scans the entire given HFS+ volume. Optionally, it can be told to limit itself to a given folder and its subfolders.

33

1 The absolute time taken would depend on multiple factors including the underlying hard-ware. However, we are only interested in relative times here.

$ time sudo fileXray --filter builtin:sxid > /dev/null0.80s user 1.09s system 12% cpu 14.894 total

$ time sudo find / -xdev -type f $ -perm -4000 -o -perm -2000 $ > /dev/null4.09s user 63.42s system 23% cpu 4:46.04 total

11. Advanced Forensics and Content Recovery

fileXray has an extensive feature set geared for file system forensics.

To begin with, the --disallow_mounting option provides a convenient solu-tion to an often cited problem: that of preventing volumes on external devices to be automatically mounted when the devices are connected to the computer. The --disallow_mounting option lets you temporarily disable such automatic mounting without having to remove or rename any configuration files and without having to stop any system daemons such as diskarbitrationd.

The --journal_names option dissects the volume’s journal and harvests file system object names. When displaying the harvested names, it annotates the output with the type of file system activity that’s likely to have occurred involv-ing each name—for example, if an object with that name was deleted, moved, renamed, and so on. When run with the --exhaustive option, --jour-nal_names “diffs” the journal and volume copies of the blocks recorded in the journal and indicates which parts of the metadata, if any, have changed.

The --trawl option scans the volume looking for blocks that match “magic” patterns (signatures). This option uses the same “magic” mechanism that un-derlies the file command. The set of signatures is arbitrarily extensible by the user.

The --scavenge options scans the volume looking for deleted files and folders. The result of the scavenge operation is a list of potentially recoverable files. It shows you a list of such files along with their metadata details, including which of the deleted blocks are likely to have been overwritten. It also allows you to “undelete” such scavenged files. The Scavenger File System provides a synthetic file system view of the results of scavenging a volume.

The Free Space File System provides a convenient way to identify and search through the free extents of a volume. The analog for used extents is the Used Space File System.

The Arbitrary File System provides a novel way of accessing arbitrary byte ranges on a given storage device.

fileXray filters can be used to search a volume for objects with specific at-tributes. In particular, the bmactime family of filters can be used to search for objects one or more of whose timestamps fall within a given range. The results of the bmactime filter can provide a “timeline” view of past file system activity.

The --checksum option can be used to compute hashes of one or more on-disk components of file system objects.

34

12. References

This User Guide has briefly outlined fileXray features. Detailed discussion of each feature, descriptions of the various ways in which the features can be used, and usage examples are deferred to the fileXray Reference, which is contained in the latter part of this document.

To be able to harness the full functionality of fileXray, it will help greatly to have a solid understanding of HFS+ architecture and how HFS+ works. The best software tool that can help gain such understanding is fileXray itself, but it will also help to have documentation at hand. The following references are strongly recommended.

• Technical Note TN1150: HFS Plus Volume Format, which describes1 the on-disk format for an HFS Plus Volume and is available at:

http://developer.apple.com/mac/library/technotes/tn/tn1150.html

• Chapter 12 from the book Mac OS X Internals: A Systems Approach. The chapter, titled The HFS Plus File System, describes both the structure and operation of HFS+ in detail. TN1150 and this chapter complement each other rather well.

http://www.amazon.com/gp/product/0321278542/

35

1 At the time of this writing, TN1150 is not up-to-date with newer features such as directory hard links and HFS-compressed files.





36


Reference

37

38


--allocation

Display the contents of the HFS+ Allocation File.

By default, fileXray will print a series of lines, each of which begins with an allocation block number in hexadecimal. The remainder of the line consists of a series of byte values in hexadecimal. Each bit in a byte represents the state of a single allocation block: a used block is represented by a bit whose value is 1 and a free allocation block is represented by a bit whose value is 0. The alloca-tion block number at the beginning of the line is the first one represented by the bytes on that line. The following is an example.

In the above example, allocation blocks 0 through 255, which are represented by the first two lines of fileXray output, are marked as used in the Allocation File. This is represented by all bits being 1 in the 32 bytes representing the blocks. Allocation blocks 6144 through 6150 are in use, but block 6151 is free, as represented by the 0xfe byte value on the line that begins with block 6144 (0x1800). Blocks 6152 onwards are free, as represented by the 0x00 byte val-ues.

To view a more detailed and visual representation of the Allocation File, you can additionally specify the --exhaustive option, which, for each allocation block on the volume, shows one of the following characters.

--allocation

39

-a

# Block is reserved or used by the Volume Header or the Alternate Volume Header.

_ Block is unused.

A Block is used by the Allocation File.

C Block is used by the Catalog B-Tree.

E Block is used by the Extents Overflow B-Tree.

X Block is used by the Attributes B-Tree.

F Block is otherwise used, for example, by a file.

! An error occurred in determining the block's ownership.

$ fileXray --device /dev/disk0s2 --allocation 00000000 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff 0x000080 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff … 0x001800 fe 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 …

Allocation Blocks120 through 127




The following is an example of the exhaustive view of the Allocation File.

In this example, allocation block 0 is reserved. (Part of it is in use by the Vol-ume Header, whose size at 512 bytes is smaller than the size of the allocation block: 4096 bytes. Since HFS+ does not allocate partial blocks, the entire allo-cation block is marked as used.) Subsequent allocation blocks are used by vol-ume data structures and files, as indicated by the A, F, E, and X characters. Next, there is a chunk of free space beginning at block number 0x67c0 and ending at block number 0x67f2. Block numbers 0x67f3 onwards belong to the Catalog B-Tree, as indicated by the C characters.

Note that as in the non-exhaustive output mode, the output in exhaustive mode also consists of lines beginning with allocation block numbers. However, each line shows the state of up to 64 allocation blocks. Given the typical alloca-tion block size of 4096 bytes, every 1 MiB of volume space will need 4 lines of output.

fileXray provides several other means, including synthetic file systems, to visualize and introspect the free and used portions of an HFS+ volume.

See Also

•--exhaustive

•--freespace

•--usedspace

•--userfs_type freespace

•--userfs_type structure

•--userfs_type usedspace

•--who_owns_block BLOCK

•--who_owns_byte BYTE

--allocation

40

$ fileXray --device /dev/disk0s2 --allocation --exhaustive 00000000 #AAAAAAAAFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF 0x000040 FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF… 0x001000 EEEEEEEEEXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX… 0x0067c0 ___________________________________________________CCCCCCCCCCCCC…

Volume Header Allocation File File

Extents Overflow B-Tree Free Block Attributes B-Tree Catalog B-Tree

--badblock

List the extent(s) of the HFS+ Bad Allocation Block File, if one is present on the volume.

The Bad Allocation Block File is neither a regular file nor a special file. It’s not a regular file in that there’s no record of it in the HFS+ Catalog. It’s not a special file in that there’s no reference to it in the HFS+ Volume Header. However, there is a Catalog Node ID (CNID) reserved for the Bad Allocation Block File: 5. HFS+ can mark certain areas on the disk as “bad” (or unusable for some reason) by assigning the corresponding allocation blocks to this CNID. If so, one or more extents belonging to the “file” with CNID 5 will be recorded in the HFS+ Extents Overflow File.

The Bad Allocation Block File is not used in the later versions of Mac OS X. One of its historical uses was in “wrapper” volumes, wherein an HFS Standard volume contained an HFS+ volume embedded in it. In those cases, the entire space corresponding to the embedded HFS+ volume was marked as belonging to the Bad Allocation File within the HFS Standard wrapper volume.

--badblock

41

-K

$ fileXray --device /dev/disk0s2 --badblockNo bad block extent found.

--block BLOCK

Specify an allocation block number or an operation-specific block or stride size when used with another option.

It can be used along with the following options.

† Although listed here for the sake of completeness, modifying the stride size for any operation involving journal dissection should not be necessary and is not advised.

When used with --output, --block instructs fileXray to read the contents of the allocation block number BLOCK and either save them to a file or dump them to the standard output depending on the argument to --output.

Note that the volume begins at allocation block number 0. The following exam-ple shows how to read and save that allocation block to a file.

The HFS+ Volume Header resides at an offset of 1024 bytes from the start of the volume. In this example, the volume’s allocation block size is the default 4096 bytes. Therefore, the block we copied contains the Volume Header. We can use the hexdump program to examine the first few bytes of the volume header.

The following shows the case when the destination is the standard output.

--block BLOCK

42

-B BLOCK

--output Specifies allocation block number to read.

--trawl Specifies stride size for the trawl operation.

--journal_names Specifies stride size for journal dissection†.

--scavenge Specifies stride size for journal dissection†.

--userfs_type scavenger Specifies stride size for journal dissection†.

$ fileXray --device /dev/disk0s2 --block 0 --output /dev/stdout | \ hexdump -s 1024 -n 12 -xc0000400 2b48 0400 0080 0020 4648 4a53 0000400 H + \0 004 200 \0 \0 H F S J 000040c

$ fileXray --device /dev/disk0s2 --block 0 --output /tmp/block0$ ls -las /tmp/block08 -rw------- 1 user wheel 4096 Nov 2 12:00 /tmp/block0$ fileXray -s 1024 -n 12 -xc /tmp/block00000400 2b48 0400 0080 0020 4648 4a53 0000400 H + \0 004 200 \0 \0 H F S J 000040c

HFS+ volume signature Last mounted version

When used with the --trawl option, --block specifies the block size the trawling operation should use for its “stride.” By default, fileXray trawls the volume one allocation block at a time. The --block option allows you to specify a smaller custom smaller size for this operation, which is useful if you want to match content that’s not at the beginning of a file. The stride size specified must be no larger than the volume’s allocation block size.

In the following example, we are looking for the remnants of a deleted PDF file on a volume. The PDF file in question was—for the sake of this example—em-bedded at an offset of 512 bytes within another file. The volume’s allocation block size is the default of 4096 bytes. Therefore, the first trawl operation that uses the default stride size does not find the PDF file’s remnants since it at-tempts its pattern matching every 4096 bytes and therefore “steps over” the content we are looking for. The second trawl operation, where we use --block to specify a custom stride size of 512 bytes, does find the remnants. See the section on --trawl for more details.

See Also

•--journal_names

•--output OUTPUT_FILE

•--read COMPONENT

•--scavenge

•--trawl QUERY

•--undelete_cookie COOKIE

•--userfs_type arbitrary

•--userfs_type scavenger

--block BLOCK

43

$ fileXray --device /dev/disk0s2 --trawl /usr/share/file/magic/pdf$ fileXray --device /dev/disk0s2 --trawl /usr/share/file/magic/pdf --block 5120x1003200 PDF document, version 1.6$

--btree BTREE_NAME

Specify an HFS+ B-Tree.

BTREE_NAME must be one of the following.

When used by itself, this option displays the contents of the given B-Tree’s header node. The following is an example.

Note that the information displayed includes the header node’s node descriptor, the B-Tree header record, and the B-Tree user data record. The map record in

--btree BTREE_NAME

44

-b BTREE_NAME

attributes Attributes B-Tree.

catalog Catalog B-Tree.

extents Extents Overflow B-Tree.

hotfiles Hot File Clustering B-Tree.

$ fileXray --device /dev/disk0s2 --btree hotfiles# HFS+ Hot File Clustering (HFC) B-Tree# B-Tree Node Descriptor fLink = 0 bLink = 0 kind = 1 kBTHeaderNode (Header) height = 0 numRecords = 3 reserved = 0# B-Tree Header Record treeDepth = 2 rootNode = 3 leafRecords = 7280 firstLeafNode = 40 lastLeafNode = 1 nodeSize = 4096 bytes maxKeyLength = 10 bytes totalNodes = 48 freeNodes = 7 reserved1 = 0 clumpSize = 65536 (ignored) btreeType = 128 (kUserBTreeType) keyCompareType = 0 (unspecified/default) attributes = 00000000000000000000000000000010 . kBTBigKeys (keyLength is UInt16)# B-Tree User Data Record magic = 0xff28ff26 version = 1 duration = 216000 seconds timebase = Mon Nov 2 12:00:00 2009 timeleft = 125746 seconds threshold = 24 maxfileblks = 2560 blocks maxfilecnt = 1000 tag = CLUSTERED HOT FILES B-TREE

the header node is not displayed; it can be seen by additionally using the --node option to view all details of the header node, as described next.

The --btree option can be combined with the --node option to display infor-mation about a specific node in the given HFS+ B-Tree. The --node option re-quires a node number as an argument. (The number of the first node in an HFS+ B-Tree is 0.) The type and amount of information displayed will depend on the given node’s role in the B-Tree. In the following example, we inspect node number 3 in the volume’s Catalog B-Tree. The node’s on-disk location is also displayed, along with the node’s size, the number of free bytes in it, and the number of records contained in it.

If --node is given 0 as an argument (that is, the header node of an HFS+ B-Tree), then the information displayed will include the B-Tree map record.

You can add the --exhaustive option which will also dump all free bytes—if there are any—within the given node. In the following example, most of the bytes in the node are free, beginning at byte offset 0x68 within the node. The device offset shown is the corresponding byte offset on the device that the vol-ume resides on.

Furthermore, the --list_records option can be used to list records either in an entire given B-Tree, or if the --node option is specified, in a specific node of

--btree BTREE_NAME

45

$ fileXray --device /dev/disk0s2 --btree catalog --node 3# Node 3 (begins at 512-byte sector # 0x33fb0, 3984/4096 bytes free)# B-Tree Node Descriptor fLink = 0 bLink = 0 kind = 0 kBTIndexNode (Index) height = 2 numRecords = 3 reserved = 0

$ fileXray --device /dev/disk0s2 --btree catalog --node 3 --exhaustive# Node 3 (begins at 512-byte sector # 0x33fb0, 3984/4096 bytes free)# B-Tree Node Descriptor fLink = 0 bLink = 0 kind = 0 kBTIndexNode (Index) height = 2 numRecords = 3 reserved = 0 free space in node = 3984 bytes at offset 0x68 (device offset 0x67f6068) 0x000060 00 00 00 … 00 00 00 0x000070 00 00 00 00 00 00 00 00 00 00 00 … 00 00 00 0x000080 00 00 00 00 00 00 00 00 00 00 00 … 00 00 00… 0x000fe0 00 00 00 00 00 00 00 00 00 00 00 … 00 00 00

that B-Tree. The --list_records option requires as an argument the type of records to list. The type depends on the B-Tree in question, although the type “any” causes all record types to be listed in any HFS+ B-Tree. See the --list_records page for details.

The following is an example of listing all leaf records in the Catalog B-Tree. Note that on a real-life volume with a large number of files, this can generate rather large amounts of output.

The following is an example of listing records in a specific Catalog B-Tree node. First, we examine the Catalog B-Tree metadata, which tells us that the tree’s first leaf node is 40. Next, we examine node 40, which shows that the node has 16 records. Finally, we list those records.

--btree BTREE_NAME

46

$ fileXray --device /dev/disk0s2 --btree catalog --list_records any… # lists all records in the Catalog B-Tree; massive amount of output

$ fileXray --device /dev/disk0s3 --btree catalog… leafRecords = 7280 firstLeafNode = 40 lastLeafNode = 1…

$ fileXray --device /dev/disk0s3 --btree catalog --node 40# Node 40 (begins at 512-byte sector # 0x4158, 1522/4096 bytes free)# B-Tree Node Descriptor fLink = 6 bLink = 0 kind = -1 kBTLeafNode (Leaf) height = 1 numRecords = 16 reserved = 0

$ fileXray --device /dev/disk0s3 --btree catalog --node 40 --list_records any…# Record 12 in Leaf Node 40# Key keyLength = 6 parentID = 16 nodeName.length = 0 nodeName.unicode = # Identity path = untitled:/.journal# Catalog File Thread Record node = 173 parentID = 2 nodeName = .journal…

See Also

•--exhaustive

•--node NODE

•--list_records RECORD_TYPE

--btree BTREE_NAME

47

--checksum COMPONENTS

Compute a SHA-1 message digest (“checksum”) of one or more on-disk compo-nents of the given file system object.

The target file system object must be specified through other options such as --cnid, --fsspec, or --path.

The COMPONENTS argument to --checksum is a comma-separated list contain-ing one or more of the following component specifiers.

By default, a data or resource fork’s logical byte size will be used for the hash computation. In particular, if that size is not a multiple of the volume’s alloca-tion block size, the trailing physical bytes that are beyond the fork’s logical size will not be included in the computation. If you wish to change this behavior and have the hash cover entire allocation blocks regardless of the logical fork sizes, you can additionally specify the --exhaustive option.

If catalog_metadata is specified as one of the components, fileXray will in-clude the file or folder’s last access time (atime) in the checksum computation. To exclude the last access time, you can specify noatime as a “virtual” compo-nent. If noatime is specified, fileXray will use a zero value for the last access time.

The following is an example of computing a checksum across a file’s data fork contents and the values of its extended attributes. We can see the checksums changing as data is appended to the file and as an extended attribute is set. We also see the checksum reverting to an earlier value as the extended attribute is removed.


48

-C COMPONENTS

data Include the data fork contents.

resource Include the resource fork contents.

extents_data Include the data fork extents.

extents_resource Include the resource fork extents.

catalog_thread Include the Catalog Thread Record.

catalog_metadata Include the Catalog File or Folder Record.

noatime Exclude last access time wherever applicable.

xattr_names Include the names of all extended attributes.

xattr_data Include the contents of all extended attributes.

everything Include all possible components.

See Also

•--exhaustive

•--cnid CNID

•--fsspec PARENT_CNID:NAME

•--path PATH


49

# Create an empty file and compute checksum.$ touch /tmp/testfile$ sudo fileXray --checksum data,xattr_data /tmp/testfileda39a3ee5e6b4b0d3255bfef95601890afd80709

# Append some data to the file and compute checksum.$ echo hello >> /tmp/testfile$ sudo fileXray --checksum data,xattr_data /tmp/testfilef572d396fae9206628714fb2ce00f72e94f2258f

# Set an extended attribute and compute checksum.$ xattr -w some_key some_value /tmp/testfile$ sudo fileXray --checksum data,xattr_data /tmp/testfile 53aa3cc9891f5765fb1eeaaf0678388e112b00d3

# Modify the extended attribute’s value and compute checksum.$ xattr -w some_key some_other_value /tmp/testfile$ sudo fileXray --checksum data,xattr_data /tmp/testfile 5732206bbc1a9cbea6a100a97a50d73d55b9073e

# Remove the extended attribute and compute checksum.$ xattr -d some_key /tmp/testfile$ sudo fileXray --checksum data,xattr_data /tmp/testfilef572d396fae9206628714fb2ce00f72e94f2258f

--cnid CNID

Specify a file system object—file or folder—of interest through its Catalog Node ID (CNID).

This option can be used with other options when fileXray requires a file sys-tem object to operate upon. The --cnid option is one of the several ways in which a file system object can be specified to fileXray. It is useful when you wish to map a CNID to the corresponding file system object.

Let us consider an example. On HFS+, the first fifteen (1 through 15) CNIDs are reserved for use by the file system itself. For instance, CNID 2 is always the folder ID of the root folder, CNID 3 is used for the Extents Overflow File, CNID 4 is used for the Catalog B-Tree, and so on. CNID 16 is the first CNID that is available for “users.” However, in the case of journaled HFS+ volumes, the journal is implemented as a “regular” file on the volume, even though it is made invisible to users by the kernel. When a new HFS+ volume is created, the journal is the first file created under normal circumstances. Therefore, on typi-cal HFS+ volumes, CNID 16 belongs to the journal. We can use the --cnid op-tion to inspect the details of the file system object whose CNID is 16. In the ex-ample on the next page, the volume in question is a root volume. We see that CNID 16 indeed corresponds to the volume’s journal.

Note that it is possible for the journal file to not have CNID 16. In such a case, it is still possible to directly inspect the file using the --fsspec option, which allows specifying a target file system object through its parent’s CNID and the object’s name.

See Also

•--checksum COMPONENTS


•--diskusage

•--list

•--scavenge

•--path PATH

•--read COMPONENT

--cnid CNID

50

-c CNID

--cnid CNID

51

$ sudo fileXray --cnid 16 path = MacHD:/.journal# Catalog File Thread Record# Record 8 in node 27553 beginning at 512-byte sector 0x3a7c48 parentID = 2 nodeName = .journal# Catalog File Record# Record 6 in node 56930 beginning at 512-byte sector 0x41a858 type = file file ID = 16 flags = 0000000000000010 . File has a thread record in the catalog. reserved1 = 0 createDate = Mon Nov 2 18:57:00 2009 contentModDate = Mon Nov 2 18:57:00 2009 attributeModDate = 0 accessDate = 0 backupDate = 0 # BSD Info ownerID = 0 (root) groupID = 0 (wheel) adminFlags = 00000000 ownerFlags = 00000000 fileMode = ---------- linkCount = 0 textEncoding = 0 reserved2 = 0 # Finder Info fdType = 0x6a726e6c (jrnl) fdCreator = 0x6866732b (hfs+) fdFlags = 0101000000000000 . kNameLocked . kIsInvisible fdLocation = (v = 0, h = 0) opaque = 0 # Data Fork logicalSize = 25165824 bytes (25 MB) totalBlocks = 6144 fork temperature = no record in Hot File B-Tree clumpSize = 0 extents = startBlock blockCount % of file

0x747 0x1800 100.00 %

6144 allocation blocks in 1 extents total. 6144.00 allocation blocks per extent on an average. # Resource Fork logicalSize = 0 bytes

--device DEVICE

Specify an HFS+ volume through its underlying device.

DEVICE is normally a block device (for example /dev/disk0s2) or a character device (for example /dev/rdisk0s2). In some cases, such as that of the root volume, the corresponding devices cannot be read by unprivileged processes. Therefore, you will need superuser access so that fileXray can open the de-vice in question for reading. Note that fileXray never opens a device for writing under any circumstances.

The --device option is one of the ways in which an HFS+ volume can be ex-plicitly specified to fileXray, the other being the --volume option. If no HFS+ volume is explicitly specified through these options, fileXray will attempt to determine the device to use from other specified information, such as the path to a file or a folder in case of mounted volumes. If the combination of user-provided arguments is such that no volume is specified directly or indirectly, fileXray will use the root volume provided the other arguments are valid.

Often, disks and disk images contain more than one volume, which leads to the notion of a “whole” device (for example /dev/disk0) and “slices” (for exam-ple /dev/disk0s1 and /dev/disk0s2). For the purpose of this discussion, a “slice” is synonymous with a “partition.” Although it is possible for an HFS+ file system to reside on a whole device, typical HFS+ volumes will reside on slices—specifically, the beginning of an HFS+ volume usually coincides with the begin-ning of a slice.

If the DEVICE argument is such that it does contain an HFS+ volume on it (that is, DEVICE is either a slice or it’s a whole device containing no partitioning and an HFS+ volume), fileXray will simply use that volume.

If, however, DEVICE is a whole device with valid partitioning, fileXray will parse the partition table and attempt to detect all HFS+ volumes that may exist on the slices of the given device. If one or more HFS+ volumes are found,

--device DEVICE

52

-d DEVICE

$ fileXray --device /dev/disk0s2 --volume_header# HFS+ Volume Volume size = 250 GB (233 GiB)

# Volume Header signature = 0x482b (H+) version = 0x4 lastMountedVersion = 0x4846534a (HFSJ) attributes = 10000000000000000010000000000000 . kHFSVolumeJournaled (volume has a journal) journalInfoBlock = 0x746… # Volume Header SHA-1 ed54ee3088851cee7113f55f6d1f839003540627

fileXray will display information on them and prompt the user to specify a particular HFS+ volume using the --partition option. fileXray supports the following partitioning schemes.

• GUID Partition Table (GPT)

• Apple Partition Map (APM)

• Master Boot Record (MBR)

In the following example, the device in question has three HFS+ partitions, or more specifically, partitions with flavors of HFS+, since one of them contains an HFSX (case-sensitive HFS+) volume. Using the --device option with the whole disk (/dev/disk1) as an argument causes fileXray to enumerate the HFS+ partitions it finds on the device. In particular, fileXray displays the starting byte offset and the size of each partition. Depending on the partition-ing scheme type, fileXray will also display some identifying information in the “mnemonic” column.

You can then specify the volume of interest using the --partition option.

--device DEVICE

53

$ fileXray --device /dev/disk1 --volume_headerNo flavor of HFS+ found.

However, a GUID Partition Table was detected on the given device with thefollowing 3 HFS+ partition table entries:


20480 214749184 HFSPlusJournaledHD (HFS+, 215 MB)214769664 214749184 HFSXJournaledHD (HFSX, 215 MB)858783744 214937600 HFSPlusHD (HFS+, 215 MB)


$ fileXray --device /dev/disk1 --partition 214769664,214749184 --volume_header# HFS+ Volume Volume size = 215 MB (205 MiB)

# Volume Header signature = 0x4858 (HX) version = 0x5 lastMountedVersion = 0x4846534a (HFSJ) attributes = 10000000000000000010000000000000 . kHFSVolumeJournaled (volume has a journal) journalInfoBlock = 0x3… # Volume Header SHA-1 200d216cedfe9fa7a52c96c8015d9c5b819b679b

Moreover, as a special case, fileXray allows DEVICE to be a regular file in-stead of a block or a character device. This is useful when you have a raw dump of a device that you want to inspect with fileXray without even at-taching any (virtual) devices. As with a “real” device, a file can contain data for either a single partition or a whole device.

It is important to note that a device dump must be “raw”—that is, it must not require any additional transformations such as decompression or decryption. In other words, for a disk image file to be used directly by fileXray, the image must not be compressed, encrypted, or sparse. fileXray will reject such an image.

If you do have such an image that is compressed, encrypted, or sparse, either convert it using the Mac OS X hdiutil command-line program, or simply at-tach it (optionally without mounting it) using hdiutil and use fileXray on the resultant block device instead of the image file.

For example, to convert a compressed disk image to an uncompressed disk im-age, you can use hdiutil as follows.

To attach a disk image without mounting any constituent volumes, you can use hdiutil as follows.

--device DEVICE

54

$ fileXray --device /private/tmp/evidence.dmg --volume_header…

$ fileXray --device "Mac OS X Install DVD.toast" --volume_headerNo flavor of HFS+ found.

However, an Apple Partition Map was detected on the given image file with thefollowing HFS+ partition table entry:


535601152 3310940160 Mac_OS_X (HFS+, 3.3 GB)

Use --partition (-e) to specify this partition table entry.

$ fileXray --device /private/tmp/compressed.dmg --volume_headerNo flavor of HFS+ found.This looks like a compressed disk image.Before using fileXray, use hdiutil to attach it.

$ hdiutil convert /path/to/input.dmg -format UDTO -o /path/to/output.dmg…

In the above example, the disk image contains three HFS+ volumes. The attach operation results in block and character devices being assigned to the disk im-age (the whole device /dev/disk1) and also to each of the constituent volumes (/dev/disk1s1 through /dev/disk1s4). Now you can use any these devices as an argument to the --device option of fileXray.

You can eject an attached disk image as follows.

See Also

•--disallow_mounting SECONDS

•--partition START[,SIZE]

•--volume VOLUME_PATH

•hdiutil(1)

--device DEVICE

55

$ hdiutil attach -nomount /path/to/diskimage.dmg/dev/disk1 GUID_partition_scheme /dev/disk1s1 Microsoft Basic Data /dev/disk1s2 Apple_HFS /dev/disk1s3 Apple_HFS /dev/disk1s4 Apple_HFS

$ hdiutil eject disk1“disk1” unmounted.“disk1” ejected.

--disallow_mounting SECONDS

Temporarily disallow automatic mounting of any and all volumes (including non-HFS+ ones) for the specified period of time.

By default, the Disk Arbitration mechanism in Mac OS X probes newly discov-ered storage devices for mountable volumes. Mounting an HFS+ volume in read-write mode, which is the default, will modify the volume in question be-cause both low-level and high-level file system activity can occur at mount time. For example, timestamps and counters can get updated, the journal can get replayed, file system objects can get created or deleted, and so on. This is highly undesirable if you wish to attach and access the storage device for re-covery or forensic purposes or otherwise wish to keep it unmodified.

With fileXray, you can not only fully analyze an unmounted (offline) volume, you can use the --disallow_mounting option to prevent the volume of inter-est from being automatically mounted as you attach the corresponding storage device to the computer. When this option is used, fileXray will block for SEC-ONDS seconds, during which time any new volumes that appear will not be al-lowed to automatically mount. However, the devices underlying these volumes will be allowed to attach, which in turn means that you can use fileXray on the devices. As a device attaches, fileXray will print the corresponding block device name(s) and if possible, the corresponding file system type(s) and vol-ume name(s).

In the following example, we use fileXray to disallow automatic mounting for 60 seconds. While mounting is disabled, we attach an external disk drive con-taining a GUID Partition Table with four volumes on it. We see that fileXray prints information about each volume as that volume’s mounting is attempted by the system and disallowed by fileXray. In more concrete terms, the vol-umes are now attached but not mounted. We can now use fileXray on the device names corresponding to HFS+ volumes, say, disk1s2.

Note that this method disables both automatic mounting by the Disk Arbitra-tion mechanism and user-initiated mounting through the Mac OS X Disk Util-ity application. However, you can still mount volumes manually if you use the mount() system call: either directly using the mount command-line program or using a custom program that invokes the mount() system call. This way, you


56

-T SECONDS

$ fileXray --disallow_mounting 60Disallowing mounting through Disk Arbitration for 60 seconds.…# Now attach an external device…disk1s2 hfs Untitled 2disk1s4 hfs Untitled 4disk1s3 hfs Untitled 3disk1s1 msdos UNTITLED 1

have flexibility in controlling which volumes can get mounted and which can-not.

A noteworthy point is regarding the use of this method with disk images. If you use --disallow_mounting and then attempt to mount a volume residing on a disk image (say, by double-clicking the disk image or by using the open command-line program), fileXray will indeed disallow mounting, but the end result will be undesirable: the image will be detached after the mount attempt. Therefore, the block device name(s) displayed by fileXray will be stale and therefore useless. This is because the operating system’s default mode of op-eration when dealing with disk images is to fail the attach attempt if no file sys-tems mount. You can handle this case in several ways.

The simplest approach is to use the hdiutil command-line program with the -nomount option to attach the disk image without mounting any volumes. fileXray may or may not be running with the --disallow_mounting option.

Alternatively, you can use the Disk Utility application while fileXray is run-ning with the --disallow_mounting option. Drag and drop the disk image in question to the Disk Utility sidebar, select the image, and click on the “Open” button in the Toolbar. fileXray will disallow the mounting of any volumes but they will remain attached.


57

$ hdiutil attach -nomount /path/to/diskimage.dmg/dev/disk1 GUID_partition_scheme /dev/disk1s1 Microsoft Basic Data /dev/disk1s2 Apple_HFS /dev/disk1s3 Apple_HFS /dev/disk1s4 Apple_HFS

See Also

•--device DEVICE

•--partition START[,SIZE]


•hdiutil(1)


58

--diskusage

Calculate disk usage of a file system object.

The target file system object must have been specified using other options such as --cnid, --fsspec, or --path.

If the target object is a file, its disk usage is the sum of the sizes of the file’s data fork, resource fork, and the cumulative disk space used by the names and values of all its extended attributes.

The following example shows examining the Safari application executable on Mac OS X 10.6, which uses transparent file system compression for several system components. As the output of the ls command shows, the “advertised” file size is the uncompressed size. The on-disk size shown by fileXray is con-siderably smaller.

The following example shows examining the HFS+ journal’s disk usage using the --fsspec option to specify the journal file by its parent folder’s Catalog Node ID (CNID) and its name. (If the journal is local to an HFS+ volume, it re-sides in the root folder, which has the CNID 2.)

The following example shows examining the Catalog B-Tree’s disk usage using the --cnid option to specify the Catalog B-Tree’s CNID, which is always 4.

If the target file system object is a folder, its disk usage is the cumulative disk usage of the folder’s contents, computed recursively. Additionally, if a folder has any extended attributes, the cumulative disk space used by the names and values of the extended attributes is included in the folder’s disk usage. The fol-lowing is an example.

--diskusage

59

-D

$ ls -l /Applications/Safari.app/Contents/MacOS/Safari-rwxr-xr-x 1 root wheel 5096208 Jul 27 23:53 /Applications/…/Safari

$ sudo fileXray --diskusage /Applications/Safari.app/Contents/MacOS/Safari1988513 logical bytes (2.0 MB) in 486 allocation blocks (2.0 MB) for the \ resource fork.17 logical bytes in 1 extended attribute name.16 logical bytes in 1 extended attribute value.

$ sudo fileXray --diskusage --fsspec 2:.journal25165824 logical bytes (25 MB) in 6144 allocation blocks (25 MB) for the \ data fork.

$ sudo fileXray --diskusage --cnid 4814743552 logical bytes (815 MB) in 198912 allocation blocks (815 MB) for \ the data fork.

Some fileXray operations, such as --diskusage, may seem analogous to higher-level operations that can be performed using standard programs or command-line tools. For example, the du command-line program can be used to display disk usage statistics of files and folders.

However, there are differences. For example, the kernel hides certain extended attributes by filtering them out within the in-kernel HFS+ implementation. Since fileXray itself parses raw HFS+ data structures, it is not subject to any such filtering and has a complete view of on-disk information. Therefore, in the --diskusage case, fileXray can report disk space consumed by all extended attributes whereas a higher-level program cannot.

See Also

•--cnid CNID


•--path PATH

--diskusage

60

$ sudo fileXray --diskusage /bin3610554 logical bytes (3.6 MB) in 894 allocation blocks (3.7 MB) for the \ resource fork.493 logical bytes in 29 extended attribute names.464 logical bytes in 29 extended attribute values.

--enable_xattr_extents

Enable extent-based extended attributes on a given mounted HFS+ volume. This operation requires superuser privileges.

A mounted volume must also be specified using the --volume option, which requires either the mount point of the volume or the path to a file system object on the volume as an argument. When used with this option, --volume has a special case: if you wish to change the next allocation block on the root volume, you must explicitly use “/” as the argument to --volume. Since --en-able_xattr_extents alters fundamental volume behavior, its use on the root volume requires explicit specification.

To use this option successfully, the HFS+ implementation in the operating sys-tem version you are running must support extent-based extended attributes. At the time of this writing, although Mac OS X Leopard (10.5.x) and Mac OS X Snow Leopard (10.6.x) have experimental implementations of extent-based ex-tended attributes, by default, only inline extended attributes are allowed.

An inline extended attribute is one that fits within a single Attributes B-Tree node while maintaining any structural overheads and other requirements for B-Tree nodes. The default node size for the Attributes B-Tree is 8192 bytes, as can be seen using the --btree option. As can be shown through some calcu-lations, this leads to a maximum inline attribute size of 3802 bytes. In other words, on an HFS+ volume that was created using default settings, the maxi-mum size for an extended attribute’s value is 3802 bytes. We can verify this on a test volume as follows.


61

-A

$ cd /private/tmp$ hdiutil create -megabytes 32 -fs HFSJ -volname test hfsj.dmg...............................................................................created: /private/tmp/hfsj.dmg$ hdiutil attach hfsj.dmg/dev/disk1 GUID_partition_scheme /dev/disk1s1 Apple_HFS /Volumes/test

$ cd /Volumes/test$ touch file$ xattr -w testkey `perl -e 'print "A" x 3802'` file$ fileXray file…# Attribute Data Record (Inline)# Record 0 in node 1 beginning at 512-byte sector 0x620 recordType = 0x10 reserved[0] = 0 reserved[1] = 0 attrSize = 3802 bytes attrData = 41 41 41 41 41 41 41 41 41 41 41 41 41 41 41 41 A A A A A A A A A A A A A A A A …$ xattr -w testkey `perl -e 'print "A" x 3803'` filexattr: [Errno 7] Argument list too long: 'file'

We create and attach a new disk image with an HFS+ volume, create a file on it, set an extended attribute whose value is 3802 bytes in size, and examine the file using fileXray. Next, we see that our attempt to set the attribute’s value to be a byte larger (3803 bytes) fails. We can now use the --enable_xattr_ex-tents option on this volume to enable extent-based extended attributes, and retry the failed operation.

See Also


•--mount_data

•--path PATH



62

$ sudo fileXray --volume /Volumes/test --enable_xattr_extents$ xattr -w testkey `perl -e 'print "A" x 3803'` file$ fileXray file…# Attributes

# Attribute Key keyLength = 26 pad = 0 fileID = 24 startBlock = 0 attrNameLen = 7 attrName = testkey# Attribute Fork Data Record (Extent-Based)# Record 1 in node 1 beginning at 512-byte sector 0x620 recordType = 0x20 reserved = 0 logicalSize = 3803 bytes clumpSize = 0 bytes totalBlocks = 1 extents = startBlock blockCount % of attribute

0x1022 0x1 100.00 %$

--exhaustive

A meta option that can be used along with several other options to enable “ex-haustive” or “detailed” behavior.

What constitutes detailed behavior differs from option to option. When used with an option that does not have such behavior, --exhaustive is a no-op and is silently discarded by fileXray.

The following table outlines how --exhaustive modifies the behavior of other fileXray options.

--exhaustive

63

-E

--allocation Enable a more detailed and more visual view of the Alloca-tion File by displaying an identifying character for each allocation block on the volume.

--btree BTREE_NAME Dump all free bytes, if any, within a given node in the given B-Tree.

--checksum COMPONENTS Checksum entire allocation blocks allocated to a given fork-based component even if the logical size of the fork is not a multiple of the allocation block size.

--fragmentation FORK_TYPE Display details of each fragmented fork on the volume.

--journal In addition to displaying summary information on the journal, analyze transactions in the journal buffer and display the resultant detailed information.

--journal_names When listing object names found in the journal, also show differences between the volume and journal copies of me-tadata whenever applicable and possible.

--monitor Run the file system modification monitor in verbose mode.

--read COMPONENT Copy entire allocation blocks assigned to a given fork-based component even if the logical size of the fork is not a multiple of the allocation block size.

--scavenge When scavenging for deleted objects on a volume and list-ing the results, enable detailed output mode, showing more metadata.

--scavenge When scavenging for a specific deleted object in order to recover its content, recover all allocation blocks, including known clobbered ones, as is—do not substitute blocks that are known to have been reused with zero-filled blocks.

--userfs_type scavenger When serving scavenged blocks for data or resource forks, always serve the current on disk content even if a block is clobbered—do not substitute blocks that are known to have been reused with zero-filled blocks.

--volume_header In addition to displaying the Volume Header, also display the Alternate Volume Header.

See Also

•--allocation

•--btree BTREE_NAME

•--checksum COMPONENTS

•--fragmentation FORK_TYPE

•--journal

•--journal_names

•--monitor

•--read COMPONENT

•--scavenge


•--volume_header

--exhaustive

64

--filter FILTER

Load and run the fileXray “filter” implemented in the dynamic library whose path is FILTER. An fileXray filter can optionally take a string argument specified through the --filter_args option.

As a special case, FILTER can have the literal prefix builtin: followed by a built-in filter’s name. fileXray includes the following built-in filters. Those with highlighted names require filter arguments.

--filter FILTER

65

-x FILTER

bmactime List objects with timestamps within a given range.

compressed List HFS+ compressed files.

creatorcode List files that have the given creator code.

device List block or character special files. (That is, “device” files.)

dirhardlink List directory hard links.

empty List files that have no extended attributes and whose data and resource forks are both empty.

emptyforks List file whose data and resource forks are both empty.

fifo List named pipes. (That is, “fifos.”)

hardlink List file hard links.

immutable List immutable file system objects.

lsR List all paths.

macho List Mach-O files along with their per architecture sizes.

name List objects whose name matches a given name (case sensitive).

namei List objects whose name matches a given name (case insensitive).

nameprefix List objects whose name have a given prefix (case sensitive).

nameprefixi List objects whose name have a given prefix (case insensitive).

namesuffix List objects whose name have a given suffix (case sensitive).

namesuffixi List objects whose name have a given suffix (case insensitive).

nodename List the parent node IDs and node names of all objects.

null Do nothing; useful for establishing baselines in benchmarks.

socket List Unix Domain socket files.

subname List objects whose name contains a given string (case sensitive).

subnamei List objects whose name contains a given string (case insensitive).

sxid List setuid and setgid files and folders.

symlink List symbolic links.

typecode List files that have the given file type code.

xattrname List all unique extended attribute names (keys) in use.

xattr List objects that have a given extended attribute.

Let us now look at examples of using each of these built-in filters and if appli-cable, the arguments they accept. After discussing the built-in filters, we will look at how to implement your own filters.

bmactime

The bmactime filter is a family of filters—a meta filter if you will—whose names all end with the suffix “time”. The prefix can be a permutation of one or more of the characters ‘b’, ‘m’, ‘a’, and ‘c’. For example, “atime”, “btime” “ctime”, “mtime”, “bmactime”, and “cmtime” are all valid filter names for the bmactime meta filter. The prefix characters represent HFS+ timestamps as follows.

The bmactime filter requires as an argument a time range consisting of a be-ginning time and an ending time. The beginning and ending times can be speci-fied either as seconds or as date strings.

The first format (time as seconds) is T1,T2 such that:

• T1 and T2 represent time in seconds since 00:00:00 January 1, 1970, Coordinated Universal Time (UTC), without including leap seconds.

• T1 is less than or equal to T2.• At most one of T1 or T2 can be omitted, although the comma is still

required. A missing T1 value is the same as specifying 0 for T1. A missing T2 value is the same as specifying the current time for T2.

• As a special case, T1 and T2 can be negative. In this case, they mean the last T1 or T2 seconds, respectively.

The second format (time as a date string) is S1,S2 such that:

• S1 and S2 are date strings formatted as “%b %d %H:%M:%S %Z %Y”, where the conversion specifications are the same as used by strfti-me(3). An example of this format is “Nov 2 12:00:00 PDT 2009”.

• The date represented by S1 is earlier than or the same as that repre-sented by S2.

• At most one of S1 or S2 can be omitted, although the comma is still required. A missing S1 value is the same as specifying 00:00:00 January 1, 1970 UTC for S1. A missing S2 value is the same as specifying the current local time for S2.

--filter FILTER

66

b Time of creation (birthtime).

m Time of last content (data) modification.

a Time of last access.

c Time of last attribute (metadata) modification.

• When specified on the fileXray command line, the eventual string S1,S2 must be appropriately quoted since the constituent date strings contain whitespace.

As noted earlier, the conversion specifications (%b, %d, and so on) have the same meaning as they have in the strftime(3) C library function. The fol-lowing is a quick reference.

Note that for a file system object to be matched by this filter, all specified times-tamps must match. For example, if bmctime is the specific filter used, then fileXray will look for file system objects whose creation times (b), attribute modification times (m), and content modification times (c) all fall within the given time range.

In the following example, we look for file system objects that were modified within the last 60 seconds. We pipe the output through the sort command-line program to get a timeline view.

Note the timestamp identifiers column in the output. This column will contain identifiers for the timestamps that the current output line applies to. A dot rep-resents a timestamp that does not apply to the current line. For example, if the specified filter is bmactime, then we are looking for four timestamps (b, m, a, and c). Now, for a given file system object, all of these timestamps could have the same values, or all of them could be different, or some could be the same,

--filter FILTER

67

%b National representation of the abbreviated month name (for example, Nov)

%d Day of month as a decimal number (01-31)

%H Hour (24-hour clock) as a decimal number (00-23)

%M Minute as a decimal number (00-59)

%S Second as a decimal number (00-60)

%Z Time zone name (for example, PDT)

%Y Year with century as a decimal number (for example, 2009)

$ sudo fileXray --filter builtin:mctime --filter_args -60, | sort -n1256751021 Mon Nov 2 10:30:21 2009 1529932 -rw------- .m.c MacHD:/.Spotlight-V100/1256751021 Mon Nov 2 10:30:21 2009 205344 -rw-r--r-- .m.c MacHD:/private/var/log/1256751021 Mon Nov 2 10:30:21 2009 205858 -rw------- .m.c MacHD:/private/var/db/s1256751021 Mon Nov 2 10:30:21 2009 302227 drwx------ .m.c MacHD:/private/var/db/s1256751021 Mon Nov 2 10:30:21 2009 3089114 -r--r----- .m.c MacHD:/private/var/audi1256751021 Mon Nov 2 10:30:21 2009 3169431 -rw-r----- .m.c MacHD:/private/var/log/1256751021 Mon Nov 2 10:30:21 2009 3169532 -rw-r--r-- .m.c MacHD:/private/var/log/

Time (secondssince epoch)

Catalog Node ID Timestamp identifiers: timestampsthis line of output applies to

etc. The output contains one line per file for each timestamp value that’s differ-ent, whereas equal timestamps are coalesced into a single line. In the above example, we are looking for two timestamps (m and c), which happen to be equal for each matching file. Therefore, there is only one line of output per matching file. If a matching file had m and c timestamps that were different, it would have two lines of output: one for m and another for c.

compressed

The compressed filter lists all HFS+ compressed files on the volume. Beginning with version 10.6 (Snow Leopard), Mac OS X supports transparent file com-pression at the file system level. Several system components are compressed by default. Depending on the file’s size and potentially other factors, the com-pressed data resides either in the file’s resource fork or inline within an ex-tended attribute that’s set on the file. The following is an example of using the compressed filter.

creatorcode

The creatorcode filter lists all files with the given creator code. The following is an example of using this filter to look for files with the creator code TVOD, which is traditionally assigned to QuickTime movies.

--filter FILTER

68

$ sudo fileXray --filter builtin:compressed…resource 15031 76064 MacHD:/usr/lib/libpam.1.dylibresource 43316 115968 MacHD:/usr/lib/libpam.2.dylibresource 353810 805072 MacHD:/usr/lib/libcrypto.0.9.dylibresource 91235 219120 MacHD:/usr/lib/libssl.0.9.dylibresource 1425730 3363872 MacHD:/usr/lib/libcrypto.0.9.7.dylibresource 289740 699088 MacHD:/usr/lib/libssl.0.9.7.dylibinline 2058 5679 MacHD:/System/Library/OpenSSL/misc/CA.plinline 1546 3758 MacHD:/System/Library/OpenSSL/misc/CA.shinline 110 119 MacHD:/System/Library/OpenSSL/misc/c_hashinline 120 152 MacHD:/System/Library/OpenSSL/misc/c_infoinline 104 112 MacHD:/System/Library/OpenSSL/misc/c_issuerinline 102 110 MacHD:/System/Library/OpenSSL/misc/c_name…

Location ofcompresseddata

Compressedsize

Uncompressedsize

$ sudo fileXray --filter builtin:creatorcode --filter_args TVOD…TVOD WAVE 607613 MacHD:/Applications/Microsoft Office 2008/…/Sounds/Tick TockTVOD WAVE 607614 MacHD:/Applications/Microsoft Office 2008/…/Sounds/Typewriter…

Creator Code Type Code Catalog Node ID

device

The device filter lists all block and character special files on the volume. Typical HFS+ volumes, including root volumes, will typically have no such files be-cause devices on Mac OS X reside in the device file system (devfs), which is a pseudo file system mounted on /dev at system startup time.

dirhardlink

The dirhardlink filter lists all directory hard links on the volume. For each directory hard link with a link count of N, there are N + 1 lines in the output, listed in the order the corresponding records exist in the Catalog B-Tree. There is one line that displays the link target and the number of links that point to it. The remaining N lines display the paths for each of the links.

Directory hard link targets reside in a hidden system folder that is normally in-visible to user applications: /.HFS+ Private Directory Data\x000d/. Its Catalog Node ID can be seen in the output of the --volume_header option.

emptyemptyforks

The empty filter lists files whose data and resource forks are both empty (zero logical size) and additionally there are no extended attributes associated with the files. The emptyforks filter lists files whose data and resource forks are both empty, although there still could be content in extended attributes.

--filter FILTER

69

$ sudo fileXray --filter builtin:empty…MacHD:/private/etc/hosts.equivMacHD:/System/.localizedMacHD:/System/Library/.localizedMacHD:/Applications/.localizedMacHD:/Library/Preferences/DirectoryService/.DSIsRunningMacHD:/private/var/run/.DSRunningSP1MacHD:/private/var/run/automount.initializedMacHD:/private/var/run/com.apple.loginwindow.didRunThisBoot…

$ sudo fileXray --filter builtin:dirhardlink…2 links -> dir_3169701MacHD:/private/tmp/test/dir1 -> dir_3169701MacHD:/private/tmp/test/subdir/dir2 -> dir_3169701…

A directory hard link

Target of a directoryhard link

fifo

The fifo filter lists all named pipes (fifos) on the volume.

hardlink

The hardlink filter lists all hard links on the volume. The output format is along the same lines as that of the dirhardlink filter.

File hard link targets reside in a hidden system folder that is normally invisible to user applications: /\x0000\x0000\x0000\x0000HFS+ Private Data/. Its Catalog Node ID can be seen in the output of the --volume_header option.

immutable

The immutable filter lists all immutable files and folders on the volume. Im-mutability can be set at the system level, at the user level, or both. The filter’s output indicates this by using the s (system) and u (user) prefixes.

--filter FILTER

70

$ sudo fileXray --filter builtin:fifo prw--w--w- MacHD:/private/var/spool/postfix/public/pickup prw--w--w- MacHD:/private/var/spool/postfix/public/qmgr$

$ sudo fileXray --filter builtin:hardlink…MacHD:/private/etc/find.codes => iNode117493MacHD:/private/etc/fstab.hd => iNode24625MacHD:/private/etc/gettytab => iNode24626…6 links => iNode1873987…MacHD:/private/etc/hostconfig => iNode24806MacHD:/private/etc/kern_loader.conf => iNode24629MacHD:/private/etc/krb5.keytab => iNode205629MacHD:/private/etc/master.passwd => iNode24630…

A directory hard link

Target of a directoryhard link

$ sudo fileXray --filter builtin:immutable u 205318 MacHD:/System/Library/CoreServices/boot.efi u 18 MacHD:/\x0000\x0000\x0000\x0000HFS+ Private Data/ u 19 MacHD:/.HFS+ Private Directory Data\x000d/$

Immutabilitylevel

Catalog Node ID

Note that the implementation of file and directory hard links on HFS+ employs immutable files. The immutable filter does not list such hard link implementa-tion files on purpose. (The dirhardlink and hardlink filters, respectively, can be used to list directory and file hard links.)

lsR

The lsR filter lists all paths on the volume along with the Catalog Node ID of each file or folder. The filter is named after the command “ls -R”, which, if executed at the root level of a volume, would recursively list all subdirectories encountered.

macho

The macho filter lists all Mach-O files on the volume. It differs from most fileXray filters in that it examines file content along with file metadata. After identifying a file as a Mach-O file, the filter further examines it to check if it is a multi-architecture (“fat”) file. If so, the filter calculates the logical size of each architecture-specific thin subfile contained within the fat file. The macho filter recognizes the following architectures: i386, x86_64, ppc, ppc64, and arm. Any other architecture found in a Mach-O file is categorized as “other.” You can op-tionally specify a comma-separated list of architectures as the filter argument, in which case it will only show files containing at least one of the specified ar-chitectures. The following is an example of using the macho filter.

If the on-disk Mach-O file (fat or otherwise) is HFS+ compressed, the macho fil-ter will indicate compression by displaying the character z before the file’s path. However, the per architecture sizes computed by this filter are the actual uncompressed sizes.

--filter FILTER

71

$ sudo fileXray --filter builtin:lsR…3699 MacHD:/System/Library/Extensions/AppleIRController.kext/3867 MacHD:/System/Library/Extensions/AppleKeyswitch.kext/3897 MacHD:/System/Library/Extensions/AppleLPC.kext/3912 MacHD:/System/Library/Extensions/AppleLSIFusionMPT.kext/15974 MacHD:/System/Library/Extensions/AppleMatch.kext/… Catalog Node ID

$ sudo fileXray --filter builtin:machoi386 x86_64 ppc ppc64 arm other path …592 504 668 0 584 0 MacHD:/usr/lib/bundle1.o888 936 972 0 1072 0 MacHD:/usr/lib/crt1.10.5.o688 816 696 0 880 0 MacHD:/usr/lib/crt1.10.6.o2112 1840 3480 0 1620 0 MacHD:/usr/lib/crt1.o…95136 95312 93968 0 0 0 z MacHD:/usr/sbin/netstat… File is compressed

namenamei

The name filter lists all file system objects whose names exactly match the name specified by the filter argument. The namei filter is the case-insensitive variant of the name filter. Note that the HFS+ volume in question can be either case-sensitive or case-insensitive—these filters perform their own matching and work the same on either type of volumes, as do the other built-in name-matching filters.

nameprefixnameprefixi

The nameprefix filter lists all file system objects whose names contain the prefix specified by the filter argument. The nameprefixi filter is the case-insensitive variant of the nameprefix filter.

--filter FILTER

72

$ sudo fileXray --filter builtin:name --filter_args SystemVersion.plistMacHD:/System/Library/CoreServices/SystemVersion.plistMacHD:/Developer/SDKs/MacOSX10.4u.sdk/System/Library/CoreServices/SystemVersion.plistMacHD:/Developer/SDKs/MacOSX10.5.sdk/System/Library/CoreServices/SystemVersion.plistMacHD:/Developer/SDKs/MacOSX10.6.sdk/System/Library/CoreServices/SystemVersion.plistMacHD:/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS3.1.sdk/\ System/Library/CoreServices/SystemVersion.plistMacHD:/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/\ iPhoneSimulator3.1.sdk/System/Library/CoreServices/SystemVersion.plistMacHD:/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS2.2.1.sdk/\ System/Library/CoreServices/SystemVersion.plistMacHD:/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS3.0.sdk/\ System/Library/CoreServices/SystemVersion.plistMacHD:/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/\ iPhoneSimulator3.0.sdk/System/Library/CoreServices/SystemVersion.plist

$ sudo fileXray --filter builtin:nameprefix --filter_args Adobe…MacHD:/Library/Fonts/AdobeFangsongStd-Regular.otfMacHD:/Library/Fonts/AdobeHeitiStd-Regular.otfMacHD:/Library/Fonts/AdobeKaitiStd-Regular.otfMacHD:/Library/Fonts/AdobeMingStd-Light.otfMacHD:/Library/Fonts/AdobeMyungjoStd-Medium.otfMacHD:/Library/Fonts/AdobeSongStd-Light.otf…MacHD:/Library/Application Support/Adobe/TypeSupport/CMaps/Adobe-CNS1-0MacHD:/Library/Application Support/Adobe/TypeSupport/CMaps/Adobe-CNS1-1MacHD:/Library/Application Support/Adobe/TypeSupport/CMaps/Adobe-CNS1-2MacHD:/Library/Application Support/Adobe/TypeSupport/CMaps/Adobe-CNS1-3MacHD:/Library/Application Support/Adobe/TypeSupport/CMaps/Adobe-CNS1-4MacHD:/Library/Application Support/Adobe/TypeSupport/CMaps/Adobe-CNS1-5…

namesuffixnamesuffixi

The namesuffix filter lists all file system objects whose names contain the suffix specified by the filter argument. The namesuffixi filter is the case-insensitive variant of the namesuffix filter. In the following example, we look for names that have the .app suffix—that is, typical application bundles.

nodename

The nodename filter lists the parent Catalog Node IDs and node names of all files and folders on the volume. The output will have as many lines as the sum of the volume’s file and folder counts.

You can use programs such as awk, sort, and uniq on the output of this filter to identify the most used object names, the longest object names, the folders with the most children, and other interesting data.

--filter FILTER

73

$ sudo fileXray --filter builtin:namesuffix --filter_args .app…MacHD:/System/Library/CoreServices/Dock.app/MacHD:/System/Library/CoreServices/Expansion Slot Utility.app/MacHD:/System/Library/CoreServices/Memory Slot Utility.app/MacHD:/System/Library/CoreServices/Finder.app/MacHD:/System/Library/ScriptingAdditions/FontSyncScripting.app/MacHD:/Applications/Font Book.app/MacHD:/System/Library/CoreServices/Front Row.app/MacHD:/System/Library/CoreServices/HelpViewer.app/MacHD:/System/Library/CoreServices/Bluetooth Setup Assistant.app/MacHD:/System/Library/CoreServices/BluetoothAudioAgent.app/MacHD:/System/Library/CoreServices/OBEXAgent.app/MacHD:/Applications/Image Capture.app/MacHD:/System/Library/Image Capture/Support/Application/AutoImporter.app/MacHD:/System/Library/Services/ImageCaptureService.app/…

$ sudo fileXray --filter builtin:nodename… 179816:YahooLicense.rtf 175774:InfoPlist.strings 3183590:AboutBox.rtf 3183590:AppleTVPrefs.nib/ 3187109:objects.xib 3183590:AppleTVSetup.nib/ 3187111:objects.xib 3183590:AppleTVStream.nib/ 3187113:objects.xib 3183590:AppleTVStreamingPrefs.nib/…

A file name

A folder name (with a trailing slash)

Parent folder’s Catalog Node ID

Separator (colon)

null

Like most other fileXray filters, the null filter walks the entire file system hierarchy, examining each file and folder. However, unlike other filters, the null filter does not do anything with the information it examines. It can be used as a baseline for the time it takes for fileXray to examine the file system hierarchy on a given system. The time taken will depend on several factors such as the number of files and folders on a volume, how fragmented the vol-ume is, and most importantly, how much I/O bandwidth the system has.

socket

The socket filter lists all Unix Domain sockets on the volume.

subnamesubnamei

The subname filter lists all file system objects whose names contain the string specified by the filter argument. The subnamei filter is the case-insensitive variant of the subname filter. These filters have the most permissive matching among all built-in name matching filters.

sxid

The sxid filter lists all setuid and setgid file system objects on the volume. A setuid or setgid object is one that has its set-user-ID-on-execution or set-group-ID-on-execution bits, respectively, set.

Note that both files and folders can have these bits set. Consider the setuid bit. In the case of an executable file, the setuid bit changes how the file executes: if otherwise permitted, the file will execute with the effective user ID set to the

--filter FILTER

74

$ sudo fileXray --filter builtin:socket… srw-rw-rw- MacHD:/private/var/run/SCHelper srw-rw-rw- MacHD:/private/var/run/syslog srwxrwxrwx MacHD:/private/var/run/usbmuxd srwxr-xr-x MacHD:/private/var/run/vmnat.95 srw------- MacHD:/private/var/run/vpncontrol.sock srw-rw-rw- MacHD:/private/var/run/asl_input srw-rw-rw- MacHD:/private/var/run/com.apple.ActivityMonitor.socket…

$ sudo fileXray --filter builtin:subnamei --filter_args malaysiaMacHD:/Library/Application Support/Apple/iChat Icons/Flags/Malaysia.gifMacHD:/Library/Dictionaries/New Oxford… y.dictionary/Contents/Images/MALAYSIA.png$

user ID of the file’s owner. In the case of a folder, this bit affects ownership of file system objects created within the folder: if otherwise permitted, newly cre-ated objects will be owned by the directory owner and not by the user ID of the creating process.

symlink

The symlink filter lists all symbolic links (along with their link targets) on the volume.

typecode

The typecode filter lists all files with the given file type code. The following is an example of using this filter to look for files with the type code FFIL, which is traditionally assigned to a category of font files.

--filter FILTER

75

$ sudo fileXray --filter builtin:symlink…MacHD:/usr/libexec/oah/Shims/CoreFoundation.framework/Versions/Current -> AMacHD:/usr/libexec/oah/Shims/IOKit.framework/IOKit -> Versions/Current/IOKitMacHD:/usr/libexec/oah/Shims/IOKit.framework/Resources -> Versions/Current/ResourcesMacHD:/usr/libexec/oah/Shims/IOKit.framework/Versions/Current -> A …

$ sudo fileXray --filter builtin:sxid…-r-sr-xr-x root(0) wheel(0) MacHD:/usr/libexec/authopen -r-sr-xr-x root(0) wheel(0) MacHD:/usr/libexec/chkpasswd -r-sr-xr-x root(0) wheel(0) MacHD:/usr/bin/top -r-sr-xr-x root(0) wheel(0) MacHD:/usr/sbin/traceroute -r-sr-xr-x root(0) wheel(0) MacHD:/usr/sbin/traceroute6 -r-sr-xr-x root(0) wheel(0) MacHD:/usr/lib/sa/sadc -r-xr-sr-x root(0) tty(4) MacHD:/usr/bin/wall -r-xr-sr-x root(0) tty(4) MacHD:/usr/bin/write -rwxr-sr-x root(0) procmod(9) MacHD:/usr/bin/shark -r-sr-xr-x root(0) wheel(0) MacHD:/usr/bin/quota -r-sr-xr-x root(0) wheel(0) MacHD:/usr/sbin/scselect …

Group ID

Group name (as resolvedon the current system)

User ID

User name (as resolvedon the current system)

$ sudo fileXray --filter builtin:typecode --filter_args FFIL…DMOV FFIL 609039 MacHD:/Library/Fonts/Microsoft/MistralDMOV FFIL 609040 MacHD:/Library/Fonts/Microsoft/Modern No. 20DMOV FFIL 609041 MacHD:/Library/Fonts/Microsoft/Monotype Corsiva…

Creator Code Type Code Catalog Node ID

xattrname

The xattrname filter lists the names of all unique extended attribute keys used on the volume. For each unique extended attribute name found, the filter also lists the number of file system objects for which that attribute is set.

xattr

The xattr filter lists all file system objects that have the given extended attrib-ute name set. The attribute name is specified as the filter argument. For exam-ple, you can look for objects with attributes such as com.apple.quarantine (quarantined files and folders), com.apple.decmpfs (transparently compressed files), and com.apple.system.Security (objects with Access Control Lists). The following example shows using the xattr filter to list quarantined files.

--filter FILTER

76

$ sudo fileXray --filter builtin:xattrnames117467 com.apple.quarantine132809 com.apple.decmpfs29473 com.apple.system.Security115 com.apple.TextEncoding5 com.apple.xcode.PlistType93 com.apple.XcodeGenerated720 com.vmware.backupReenabled50 com.apple.metadata:kMDItemWhereFroms15 com.apple.diskimages.recentcksum4 com.macromates.caret2 com.apple.metadata:kMDItemIsScreenCapture2 com.apple.metadata:kMDItemScreenCaptureType5 AppCrashCount5 AppDuration5 SubmissionSkipped5 SystemCrashCount5 UserDuration5 bug_type5 displayName5 name5 os_version5 version46 com.apple.AddressBook.ImageTransform.ABClipRect_146 com.apple.AddressBook.ImageTransform.ABClipRect_1_id1 com.apple.diskimages.fsck1 xcodebuild____/Developer____1 com.apple.system.hfs.firstlink185 com.apple.metadata:kMDItemAttributeChangeDate8 com.apple.metadata:com_apple_backup_excludeItem8 ca.moelppd.iskimages.fsck10 NSStoreType10 NSStoreUUID

Implementing Your Own Filters

As noted at the beginning of this section, fileXray can load third-party filters. A third-party filter is a dynamic library that implements up to three functions, one of which is mandatory (fileXray_filter_callback) and two are optional (fileXray_filter_init and fileXray_filter_fini). When a filter library’s path is provided to fileXray through the --filter option, fileXray looks up the library and dynamically loads it.

The annotated header file (fileXray_filter.h) on the following page de-scribes these functions, the arguments they receive, and how they are invoked by fileXray.

--filter FILTER

77

$ sudo fileXray --filter builtin:xattr --filter_args com.apple.quarantine…MacHD:/Users/johndoe/Library/Safari/WebpageIcons.dbMacHD:/Users/johndoe/Library/Preferences/com.apple.Safari.RSS.plistMacHD:/Users/johndoe/Library/Caches/MetadataMacHD:/Users/johndoe/Library/Caches/Metadata/SafariMacHD:/Users/johndoe/Library/Caches/Metadata/Safari/BookmarksMacHD:/Users/johndoe/Library/Caches/com.apple.Safari/Webpage PreviewsMacHD:/Users/johndoe/Library/Caches/Metadata/Safari/History…

--filter FILTER

78

#ifndef __FILEXRAY_FILTER_H__#define __FILEXRAY_FILTER_H__

/* * A "path computer" function is passed as an argument to your filter's callback * function. The purpose of this argument is to allow your callback function to * compute the path to the file or folder record in question IF you need to do * so--typically because the record in question matches some criteria and you * want to print its path. * * Currently, the purpose of the fobject argument is private. When you call the * path computer function, you MUST set this argument to NULL. */typedef const char* (*fx_path_computer_t)(const void* fobject);

/* * A "record swapper" function is passed as an argument to your filter's * callback function. The record swapper allows your callback to endian-swap * (if necessary) the entire file system object record. HFS+ is big endian on * disk. Therefore, on little-endian machines, you will need to swap record * fields. Note that the record swapper swaps the entire file or folder record. * If you are only interested in one or few fields, this may be too expensive. * In that case, you should not call the swapper and should swap fields of * interest yourself. */typedef void (*fx_record_swapper_t)(void* record);

/* * Your filter's "init" function is called once before any records are processed * by fileXray. If any filter argument string was passed on the fileXray command * line, it will be forwarded to the init function as filter_args. If no filter * argument string was provided, filter_args will be NULL. The init function is * optional for your filter to implement. If you don't need to do any initial * setup in your filter or if your filter does not need any arguments, you can * choose not to implement this function. * * The required name for your filter's init function is: * * filexray_filter_init * * Your init function must return a 0 on success. If it returns any other value, * fileXray will terminate further processing. */typedef int (*fx_init_t)(const char* filter_args);

/* * fileXray will call your filter's callback function once for each file or * folder record in the HFS+ catalog. The purpose of the computePath and * swapRecord arguments is described above. The info argument is a pointer to * to either an HFS+ folder record or an HFS+ file record: that is, * struct HFSPlusCatalogFolder and struct HFSPlusCatalogFile, respectively, * both of which are defined in <hfs/hfs_format.h>. * * The required name for your filter's callback function is: * * filexray_filter_callback * * Your callback function must return a 0 value if further record processing * is to continue. If you wish to terminate processing, perhaps because you * found what you were looking for, or you ran into some error, you can return * a non-zero value to cause record processing to terminate. */typedef int (*fx_callback_t)(void* info, const fx_path_computer_t computePath, const fx_record_swapper_t swapRecord);

/* * Your filter's "fini" function is called once after fileXray is done with * record processing, because either all records have been processed, or your * callback returned a non-zero value causing fileXray to stop processing. * * The required name for your filter's callback function is: * * filexray_filter_fini */typedef void (*fx_fini_t)(void);

#endif /* __FILEXRAY_FILTER_H__ */

Let us use the fileXray_filter.h header file to implement an example filter that takes a number as its argument and prints the paths to all files and fold-ers whose Catalog Node IDs are numerically lower than that number. Let us call it the “low inode” filter. The following is the entire source code for this filter.

--filter FILTER

79

/* filexray_filter_lowinode.c */

#include <stdio.h>#include <stdlib.h>#include <limits.h>#include <sys/types.h>#include <stdint.h>#include <hfs/hfs_format.h>#include <libkern/OSByteOrder.h>#include "filexray_filter.h"

static uint32_t inodeUpperLimit;

intfilexray_filter_init(const char* filter_args){ if (!filter_args) { fprintf(stderr, "This filter requires an argument: the inode upper limit.\n"); return -1; }

unsigned long val = strtoul(filter_args, NULL, 0); if ((val == ULONG_MAX) || (val > 0xFFFFFFFFUL)) { fprintf(stderr, "Invalid value %s for the inode upper limit.\n", filter_args); return -1; }

inodeUpperLimit = (uint32_t)val;

return 0;}

intfilexray_filter_callback(void* info, fx_path_computer_t computePath, fx_record_swapper_t swapRecord){ u_int32_t inodeNumber;

int16_t recordType = OSSwapBigToHostInt16(*(int16_t*)info);

if (recordType == kHFSPlusFolderRecord) {

inodeNumber = OSSwapBigToHostInt32(((struct HFSPlusCatalogFolder*)info)->folderID);

} else if (recordType == kHFSPlusFileRecord) {

inodeNumber = OSSwapBigToHostInt32(((struct HFSPlusCatalogFile*)info)->fileID);

} else { return 0; }

if (inodeNumber < inodeUpperLimit) { fprintf(stdout, "%-12u %s\n", inodeNumber, computePath(NULL)); }

return 0;}

voidfilexray_filter_fini(void){ /* Nothing to do here. It would also be fine leave this unimplemented. */}

We can compile and try the low inode filter as follows.

By default, filters are volume wide. That is, a filter potentially processes all file system objects on the given volume, although a filter callback function can choose to terminate processing based on arbitrary criteria. You can usually run a filter in directory wide mode by specifying a directory either implicitly or through the --cnid, --fsspec, and --path options. In directory wide mode, a filter will limit its operation to the specified directory, including (recursively) any subdirectories it might have. The following is an example.

See Also

•--filter_args STRING

--filter FILTER

80

$ gcc -arch x86_64 -arch i386 -arch ppc -Wall -dynamiclib -o lowinode.dylib \ fileXray_filter_lowinode.c$ fileXray --volume /Volumes/PreciousHD --filter lowinode.dylib \ --filter_args 1002 PreciousHD:22 PreciousHD:/.fseventsd19 PreciousHD:/.HFS+ Private Directory Data\x000d16 PreciousHD:/.journal17 PreciousHD:/.journal_info_block20 PreciousHD:/.Trashes18 PreciousHD:/\x0000\x0000\x0000\x0000HFS+ Private Data21 PreciousHD:/.Trashes/50123 PreciousHD:/.fseventsd/fseventsd-uuid

$ sudo fileXray --filter builtin:sxid /bin-r-sr-xr-x root(0) wheel(0) MacHD:/bin/ps -r-sr-xr-x root(0) wheel(0) MacHD:/bin/rcp

--filter_args STRING

Pass STRING as an argument string to an fileXray filter, which in turn must have been specified through the --filter option.

The contents of STRING are parsed only by the filter. If STRING contains whitespace or any other characters that may be specially interpreted by the command shell, you must appropriately quote STRING.

See Also

•--filter FILTER

--filter_args STRING

81

-X STRING

--force

A meta option that can be used along with other options. It enables fileXray to continue certain operations even when an apparent failure occurs.

Currently, the only operation affected by --force is mounting a device or disk image using the Arbitrary File System, which is one of the virtual file systems built into fileXray. Normally, fileXray expects the device or disk image to contain an HFS+ volume. However, given the usefulness and generic nature of the Arbitrary File System, you may want to use it on arbitrary devices and files. In such a case, you can additionally specify --force to make fileXray accept targets that are not HFS+ volumes: for example, volumes of a type other than HFS+ or even regular files.

See Also

•--userfs_mount MOUNT_POINT


--force

82

--fragmentation FORK_TYPE

Display summary information on the fragmentation (if any) of the specified types of forks on the volume. FORK_TYPE must be one of data, resource, or any.

For the purpose of this option, a fragmented fork is one whose content is not fully contiguous on disk. In other words, a fragmented fork has at least two ex-tents. Another measure of “high” fragmentation is whether a fork has more than eight extents, since information about the first eight extents can be stored “inline” in the Catalog B-Tree record of the file in question.

If the --exhaustive option is additionally specified, a line of detailed informa-tion on each fragmented fork of the matching type will also be displayed. Each line contains a comma-separated list whose format is shown in the following example.

The numerically annotated fields in this example have the following meanings.

1. The Catalog Node ID (CNID) of the file.2. The type of the fork: data or resource.3. A colon-separated list of the number of allocation blocks in each of the

fork’s extents. This list as as many items as the number of fragments (extents) the fork has.

4. The fork’s logical size in bytes.5. The total number of allocation blocks the fork has.6. The number of fragments (extents) the fork has.7. The average number of allocation blocks per extent the fork has.8. The computed path to the file.


83

$ sudo fileXray --fragmentation anyOut of 1271933 non-zero data forks total, 1270765 (99.908 %) have no fragmentation.98 data forks have more than 8 fragments.Out of 72077 non-zero resource forks total, 72072 (99.993 %) have no fragmentation.No resource fork has more than 8 fragments.

$ sudo fileXray --fragmentation data --exhaustive…792328,data,2:2:2:1,24580,7,4,1.75,MacHD:/Applications/.DS_Store2593929,data,1:5:4,40144,10,3,3.33,MacHD:/private/var/log/apache2/access_log3379626,data,1:40,167276,41,2,20.50,MacHD:/private/var/log/asl/2009.11.14.asl3073012,data,3:2:1:6:51:5:31,405490,99,7,14.14,MacHD:/private/var/log/kernel.log…

➊ ➋ ➌ ➍ ➎➏ ➑➐

You can additionally specify the --top option to list details of the volume’s top N most fragmented forks. The number N is specified as an argument to the --top option.

The following example (output cropped to fit on this page) shows using --top to list the top ten most fragmented data forks on the volume. Note that it is possible that the argument N passed to --top is larger than the number of fragmented forks that exist on the volume. In such a case, all fragmented forks of the given type will be listed. In all cases, the forks are listed with the most fragmented fork at the bottom of the screen. This way, if the screen scrolls, you’ll still see the most fragmented forks without having to scroll up.

See Also

•--exhaustive

•--hotfiles

•--summary FORK_TYPE

•--top N


84

$ sudo fileXray --fragmentation data --top 10# Summary of fork fragmentation on the volumeOut of 1271933 non-zero data forks total, 1270765 (99.908 %) have no fragmentation.98 data forks have more than 8 fragments

# Top 10 forks with the most extents on the volumerank extents blk/extents cnid path10 177 4443.12 684630 MacHD:/Users/johndoe/Documents/Virtual Mac9 193 52.71 1529932 MacHD:/.Spotlight-V100/Store-V1/Stores/E9A8 210 2478.32 651492 MacHD:/Users/johndoe/Documents/Virtual Mac7 213 2049.05 687557 MacHD:/Users/johndoe/Documents/Virtual Mac6 234 2218.67 686111 MacHD:/Users/johndoe/Documents/Virtual Mac5 275 10485.76 650957 MacHD:/Users/johndoe/Documents/Virtual Mac4 299 1752.51 677798 MacHD:/Users/johndoe/Documents/Virtual Mac3 340 113.79 3000266 MacHD:/Users/johndoe/Library/Caches/com.ap2 355 3094.31 2614539 MacHD:/private/tmp/log.tar.bz21 392 1337.02 674452 MacHD:/Users/johndoe/Documents/Virtual Mac

--freespace

Display all free extents on the volume.

For each free extent, fileXray displays the following information: the number of allocation blocks in that extent, the starting and ending block numbers in hexadecimal, and the amount of associated free space.

You can pipe the output through “sort -n” to view a list of contiguous free space chunks sorted by chunk size. In the following example, we see that the largest contiguous chunk on the volume has 13,525,354 allocation blocks. The chunk starts at allocation block number 0x2d3e1b9 and ends at allocation block number 0x3a24322. Given the allocation block size of 4096 bytes, this chunk amounts to 55 GB.

Another way to visualize free extents on a volume is to use the built-in freespace virtual file system, which can be accessed through the --userf-s_type option.

See Also

•--usedspace




--freespace

85

-f FORK_TYPE

$ sudo fileXray --freespace | sort -n# Allocation block size = 4096 bytes# Allocation blocks free = 21273297 (0x1449ad1)# Allocation blocks total = 60965668 (0x3a24324)# Free Contiguous Starting @ Ending @ Free Space 1 0xb6dd8 0xb6dd8 4.1 KB 1 0xb868e 0xb868e 4.1 KB … 338640 0x14d77 0x67846 1.4 GB 716772 0xd8228 0x18720b 2.9 GB 1631113 0x2a88bd6 0x2c16f5e 6.7 GB 13525354 0x2d3e1b9 0x3a24322 55 GB

--fsspec PARENT_CNID:NAME

Specify a file system object—file or folder—of interest through a tuple consist-ing of the object’s parent folder’s Catalog Node ID (PARENT_CNID) and the ob-ject’s name (NAME). NAME must be provided as a UTF-8 string. This option can be used with other options when an operation requires a file system object to operate upon. It is one of the several ways in which a file system object can be specified to fileXray.

There are system files and folders that HFS+ intentionally makes it “hard” (by incorporating “hard to type” characters in the name) or “impossible” (by explic-itly filtering them in the kernel) to access. fileXray provides several ways to examine such file system objects. The --fsspec option is also useful in such cases.

For example, on a journaled volume with a locally resident journal (the de-fault), the journal resides in the root folder as a file named .journal. The file cannot be accessed from user space because a Catalog B-Tree lookup function in the kernel explicitly return an ENOENT error when a user program attempts to access this file. Given the file’s name and the fact that the root folder’s Cata-log Node ID is always 2, we can use the --fsspec option to examine the file.


86

-F PARENT_CNID:NAME

$ ls -las /.journalls: /.journal: No such file or directory

$ sudo fileXray --fsspec 2:.journal path = MacHD:/.journal# Catalog File Thread Record# Record 8 in node 27553 beginning at 512-byte sector 0x3a7c48 parentID = 2 nodeName = .journal# Catalog File Record# Record 6 in node 56930 beginning at 512-byte sector 0x41a858 type = file file ID = 16… # Finder Info fdType = 0x6a726e6c (jrnl) fdCreator = 0x6866732b (hfs+) fdFlags = 0101000000000000 . kNameLocked . kIsInvisible fdLocation = (v = 0, h = 0) opaque = 0 # Data Fork logicalSize = 25165824 bytes (25 MB) totalBlocks = 6144 fork temperature = no record in Hot File B-Tree clumpSize = 0 extents = startBlock blockCount % of file

0x747 0x1800 100.00 %

6144 allocation blocks in 1 extents total.…

See Also

•--cnid CNID

•--diskusage

•--list

•--path PATH

•--read COMPONENT


87

--hotfiles

Display the “hottest” files on the volume if the volume uses Hot File Clustering (HFC).

Hot File Clustering is an HFS+ optimization that aims to improve the perform-ance of small, frequently accessed files on a volume. It is implemented as a multi-staged clustering scheme that tracks “hot” files (except journal and quota files) and dynamically relocates them to a designated “hot space” on the vol-ume. Mac OS X uses Hot File Clustering only on boot volumes. Under certain circumstances, such as too little space being free on a boot volume, Mac OS X can choose to disable Hot File Clustering.

The --hotfiles option requires the --top option to specify a number N, which will cause fileXray to display the N hottest files.

The following example (screenshot cropped to fit on this page) shows using --top to list the top ten hottest on the volume.

Hot File Clustering uses a B-Tree to keep track of hot files. You can analyze the B-Tree using the --btree option.

Live Hot File Clustering parameters—if any—for a mounted HFS+ volume can be seen in the output of the --mount_data option. In particular, you can de-termine from that output if Hot File Clustering is enabled or not for the volume.

See Also



•--mount_data

•--top N

--hotfiles

88

-H

$ sudo fileXray --hotfiles --top 10# Top 10 Hottest Files on the Volumerank temperature cnid path1 370044 1847972 MacHD:/Deve…/SDKs/MacOSX10.5.sdk/usr/include/op…2 37567 1847262 MacHD:/Deve…/SDKs/MacOSX10.5.sdk/usr/include/i3…3 33320 1848473 MacHD:/Deve…/SDKs/MacOSX10.5.sdk/usr/include/sy…4 26466 1848481 MacHD:/Deve…/SDKs/MacOSX10.5.sdk/usr/include/sy…5 26093 1848570 MacHD:/Deve…/SDKs/MacOSX10.5.sdk/usr/include/sy…6 25840 1847599 MacHD:/Deve…/SDKs/MacOSX10.5.sdk/usr/include/ma…7 25776 1847608 MacHD:/Deve…/SDKs/MacOSX10.5.sdk/usr/include/ma…8 24577 1704924 MacHD:/darwin/xnu-1456.1.26/security/conf/versi…9 24577 1704913 MacHD:/darwin/xnu-1456.1.26/security/conf/kerne…10 24577 1704909 MacHD:/darwin/xnu-1456.1.26/security/conf/files…

A total of 2674 Hot Files are being tracked.

--iec

A global option that tells fileXray to use powers of 2 and IEC 60027-2 pre-fixes when displaying byte values.

By default, fileXray uses SI multiples (powers of 10) when displaying byte values. For example, 1 MB would mean 1,000,000 bytes rather than 1,048,576 (that is, 1024 × 1024) bytes. When the --iec option is specified, fileXray will use powers of 2 instead and switch to using IEC 60027-2 prefixes. For exam-ple, 1,048,576 bytes will be displayed as 1 MiB.

The following example shows the effect --iec has on the output of the --list option when the latter is used to list the contents of the /bin folder.

--iec

89

-I

$ sudo fileXray --list --path /bin…25537 -r-xr-xr-x @---z root wheel 0 26 KB date24600 -r-xr-xr-x @---z root wheel 0 17 KB dd24601 -r-xr-xr-x @---z root wheel 0 13 KB df25516 -r-xr-xr-x @---z root wheel 0 7.1 KB domainname25538 -r-xr-xr-x @---z root wheel 0 7.7 KB echo25686 -r-xr-xr-x @---z root wheel 0 46 KB ed25539 -r-xr-xr-x @---z root wheel 0 16 KB expr25540 -r-xr-xr-x @---z root wheel 0 7.3 KB hostname25541 -r-xr-xr-x @---z root wheel 0 9.5 KB kill…

$ sudo fileXray --list --path /bin --iec…25537 -r-xr-xr-x @---z root wheel 0 25 KiB date24600 -r-xr-xr-x @---z root wheel 0 17 KiB dd24601 -r-xr-xr-x @---z root wheel 0 13 KiB df25516 -r-xr-xr-x @---z root wheel 0 6.9 KiB domainname25538 -r-xr-xr-x @---z root wheel 0 7.5 KiB echo25686 -r-xr-xr-x @---z root wheel 0 45 KiB ed25539 -r-xr-xr-x @---z root wheel 0 16 KiB expr25540 -r-xr-xr-x @---z root wheel 0 7.1 KiB hostname25541 -r-xr-xr-x @---z root wheel 0 9.3 KiB kill…

introspect

There is no actual fileXray command-line option called introspect. If a file or folder is specified to fileXray but no operation to perform is specified, fileXray will display metadata details of the file or folder, allowing you to “in-trospect” individual file system objects. This section shows some examples to describe the format of the introspection output produced by fileXray.

introspect

90

$ sudo fileXray /tmp/testfile# Catalog File Thread Record# Record 78 in node 45652 beginning at 512-byte sector 0x3ee778 parentID = 51 nodeName = testfile# Catalog File Record# Record 31 in node 42648 beginning at 512-byte sector 0x3e2bb8 type = file file ID = 3294534 flags = 0000000000001111 . Any forks may only be opened for reading. . File has a thread record in the catalog. . File has extended attributes. . File has security data (ACLs). reserved1 = 0 createDate = Mon Nov 9 16:31:42 2009 contentModDate = Mon Nov 9 16:31:42 2009 attributeModDate = Mon Nov 9 18:28:10 2009 accessDate = Mon Nov 9 18:28:10 2009 backupDate = 0 # BSD Info ownerID = 501 (johndoe) groupID = 0 (wheel) adminFlags = 00000000 ownerFlags = 00100010 . UF_IMMUTABLE (file may not be changed) . UF_COMPRESSED (file is hfs-compressed) fileMode = -rw-r--r-- linkCount = 1 textEncoding = 0 reserved2 = 0 # Finder Info fdType = 0 fdCreator = 0 fdFlags = 0100100000001100 . kIsStationery . kIsInvisible . kColor (Color label = red) fdLocation = (v = 0, h = 0) opaque = 0

continued…

➊

➋

➌

➍

➎

➏

Introspecting Files

The previous page shows an excerpt from the detailed output on a regular file. The numerically annotated parts of this excerpt mean the following.

1. Information on the Catalog B-Tree node containing this file’s thread record: the record number in the containing B-Tree node, the node number, and the raw sector offset of the node on the device underly-ing the volume.

2. Information on the Catalog B-Tree node containing this file’s file re-cord: the record number in the containing B-Tree node, the node number, and the raw sector offset of the node on the device underly-ing the volume.

3. Descriptions of the flag bits that are set on the file. (This file is append-only, has extended attributes, and an access control list.)

4. Descriptions of the BSD-level system and user flags that are set on the file. (This file has user-level immutability set. It is also an HFS+ compressed file.)

5. The Finder color label, which is part of the Finder flags. (This file has a red label.)

6. Other Finder flags.

The next details that fileXray displays about a file are descriptions of its data and resource forks. For each fork, fileXray will display all of the fork’s ex-tents, along with fragmentation statistics.

Next, fileXray will display all extended attributes—if any—that the file has. This includes any “special” extended attributes that are normally hidden or en-tirely filtered by the kernel. For example, HFS+ implements access control lists (ACLs) via extended attributes. If the file in question has an ACL, fileXray will display the raw contents of the corresponding extended attribute. Moreover,

introspect

91

… # Data Fork logicalSize = 0 bytes # Resource Fork logicalSize = 7537374 bytes (7.5 MB) totalBlocks = 1841 fork temperature = no record in Hot File B-Tree clumpSize = 137 extents = startBlock blockCount % of file

0x2e5f02b 0x731 100.00 %

1841 allocation blocks in 1 extents total. 1841.00 allocation blocks per extent on an average.

continued…

fileXray will parse the ACL itself and display details of the resultant security information. In the current example, the file does have an ACL, as shown be-low.

The implementation of HFS+ compressed files also employs extended attrib-utes. If the file in question is HFS+ compressed, fileXray will display the as-sociated raw data stored in the relevant extended attribute. It will also parse the compression data and display the resultant details such as the type of compression scheme used, the location of compressed content, the file’s un-

introspect

92

…# Attributes

# Attribute Key keyLength = 62 pad = 0 fileID = 3294534 startBlock = 0 attrNameLen = 25 attrName = com.apple.system.Security# Attribute Data Record (Inline)# Record 7 in node 14003 beginning at 512-byte sector 0x4a568 recordType = 0x10 reserved[0] = 0 reserved[1] = 0 attrSize = 68 bytes attrData = 01 2c c1 6d 00 00 00 00 00 00 00 00 00 00 00 00 , m 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00 d1 d4 2b f9 + 89 3d 4e 9a 95 67 46 12 79 f2 d1 52 00 00 00 01 = N g F y R 00 00 00 04 # File Security Information fsec_magic = 0x12cc16d fsec_owner = 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 fsec_group = 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 # ACL Record acl_entrycount = 1 acl_flags = 0 (hi = 0, lo = 0) # ACL Entry 0 ace_applicable = d1 d4 2b f9 89 3d 4e 9a 95 67 46 12 79 f2 d1 52 user = johndoe uid = 501 group = D1D42BF9-893D-4E9A-9567-461279F2D152 ace_flags = 00000000000000000000000000000001 (0x000001) . KAUTH_ACE_PERMIT ace_rights = 00000000000000000000000000000100 (0x000004) . KAUTH_VNODE_WRITE_DATA/KAUTH_VNODE_ADD_FILE

continued…

compressed size, and so on. In the current example, the file is HFS+ com-pressed, as shown below.

Introspecting Folders

As in the case of files, introspecting a folder with fileXray will result in a vari-ety of information being displayed, much of which is similar between files and folders. The following is an example.

introspect

93

…# Attribute Key keyLength = 46 pad = 0 fileID = 3294534 startBlock = 0 attrNameLen = 17 attrName = com.apple.decmpfs# Attribute Data Record (Inline)# Record 11 in node 1209 beginning at 512-byte sector 0x185c8 recordType = 0x10 reserved[0] = 0 reserved[1] = 0 attrSize = 16 bytes attrData = 66 70 6d 63 04 00 00 00 60 ea 1c 01 00 00 00 00 f p m c ` compression magic = cmpf compression type = 4 (resource fork has compressed data) uncompressed size = 18672224 bytes

compressed chunks = 285 total details of chunks = byteOffset byteCount 0x8ec 0x767f… 0x721747 0xea61

$ sudo fileXray /Library path = MacHD:/Library# Catalog Folder Thread Record…# Catalog Folder Record# Record 18 in node 56930 beginning at 512-byte sector 0x41a858 type = folder folder ID = 244 flags = 0000000000001100 . Folder has extended attributes. . Folder has security data (ACLs). valence = 56…# Attributes…# Record 2 in node 4 beginning at 512-byte sector 0x13a78… attrName = com.apple.system.Security…

Introspecting File Hard Links

File hard links on HFS+ are conceptually similar to those on typical Unix sys-tems: they represent multiple directory entries referring to common file con-tent. Implementation-wise, HFS+ hard links use a special hard-link file for each directory entry. The common file content is stored in another special file: the indirect-node file. All indirect-node files are stored in a special directory called the private metadata folder. The latter, which resides in the root folder of a vol-ume, is normally invisible to the user. It has a name that begins with four NUL characters and is therefore “hard” to type.

Let us look at an example using fileXray.

We begin by creating a file called, say, file1. Before we create a hard link to this file, we examine its details so that we can determine which file properties change after link creation.

Let us now create a hard link to file1.

Let us see what, if anything, has changed about file1 now that we have cre-ated a hard link to it.

introspect

94

$ mkdir /tmp/test ; cd /tmp/test$ echo "This is file1" > file1$ sudo fileXray file1 path = MacHD:/private/tmp/test/file1…# Catalog File Record# Record 94 in node 45652 beginning at 512-byte sector 0x3ee778 type = file file ID = 3298018 flags = 0000000000000010… # BSD Info ownerID = 501 (johndoe) groupID = 0 (wheel) adminFlags = 00000000 ownerFlags = 00000000 fileMode = -rw-r--r-- linkCount = 1 textEncoding = 0 reserved2 = 0 # Finder Info fdType = 0 fdCreator = 0 fdFlags = 0000000000000000…

$ ln file1 file2

We see that a lot has changed—the on-disk nature of file1 has transformed quite a bit. The original file content has “moved” to an indirect-node file. What was file1 before in the Catalog B-Tree has been replaced with a new directory entry altogether: one that has a new Catalog Node ID (CNID) within the file sys-tem. This new directory entry is also a file, but with several special properties. Its type and creator (as stored in the Finder Info metadata) are hlnk and hfs+, respectively. It is marked immutable. Since the content is in the indirect-node file, this special entry’s data and resource forks are both empty. Moreover, the on-disk owner and group ID fields have been repurposed to act as the previous and next links, respectively, in what’s called a hard link chain. The creator and type codes are set to special values: hfs+ and hlnk, respectively.

Hard link chains is a feature that was introduced in Mac OS X 10.5 (Leopard). It enables the file system to keep track of hard links via doubly linked lists of CNIDs connecting hard links together. In the current example, we see that the previous link ID is 3298170. Let us examine that CNID with fileXray.

introspect

95

$ sudo fileXray file1 path = MacHD:/private/tmp/test/file1…# Catalog File Record# Record 88 in node 45652 beginning at 512-byte sector 0x3ee778 type = file (hard link) indirect node CNID = 3298018 indirect node file = MacBookHD:/\x0000\x0000\x0000\x0000HFS+ Private Data/iNode3298018 file ID = 3298169 flags = 0000000000100010 . File has a thread record in the catalog. . File has hardlink chain.… # BSD Info ownerID = 3298170 (previous link ID) groupID = 0 (next link ID) adminFlags = 00000000 ownerFlags = 00000010 . UF_IMMUTABLE (file may not be changed) fileMode = -r--r--r-- iNodeNum = 3298018 (link reference number) textEncoding = 0 reserved2 = 0 # Finder Info fdType = 0x686c6e6b (hlnk) fdCreator = 0x6866732b (hfs+) fdFlags = 0000000100000000 . kHasBeenInited… # Data Fork logicalSize = 0 bytes # Resource Fork logicalSize = 0 bytes…

We see that CNID 3298170 corresponds to the other reference (hard link) we just created: file2. The properties of this reference are similar to those of the other reference file1. They do differ in their CNIDs. (Indeed, they are imple-mented as two separate on-disk files.) They also differ in their previous and next link IDs—as we can see, file1 and file2 are connected in a hard link chain.

The file’s content is in an indirect-node file, which is now the on-disk object with the original CNID (3298018). Let us examine this CNID. As shown on the following page, the indirect-node file acts as a container for several of the origi-nal file’s properties. In particular, it has the original file’s content, owner ID, and group ID. A reserved field (reserved1) contains the CNID of the head of the hard link chain.

introspect

96

$ sudo fileXray -c 3298170 path = MacHD:/private/tmp/test/file2…# Catalog File Record# Record 87 in node 45652 beginning at 512-byte sector 0x3ee778 type = file (hard link) indirect node CNID = 3298018 indirect node file = MacBookHD:/\x0000\x0000\x0000\x0000HFS+ Private Data/iNode3298018 file ID = 3298170 flags = 0000000000100010 . File has a thread record in the catalog. . File has hardlink chain.

… # BSD Info ownerID = 0 (previous link ID) groupID = 3298169 (next link ID) adminFlags = 00000000 ownerFlags = 00000010 . UF_IMMUTABLE (file may not be changed) fileMode = -r--r--r-- iNodeNum = 3298018 (link reference number) textEncoding = 0 reserved2 = 0 # Finder Info fdType = 0x686c6e6b (hlnk) fdCreator = 0x6866732b (hfs+) fdFlags = 0000000100000000 . kHasBeenInited… # Data Fork logicalSize = 0 bytes # Resource Fork logicalSize = 0 bytes…

Note that if you now delete one of the hard link files, say, file2, things will not revert back to how they were to begin with. You will now have file1 as the only hard-link file along with the indirect-node file still holding the content.

When displaying information on a hard link, fileXray also displays paths to any sibling hard links. For example, if a file has N hard links, then using fileXray to introspect any one of those hard links will also enumerate the paths to the remaining N − 1 hard links.

introspect

97

$ sudo fileXray -c 3298018 path = MacBookHD:/\x0000\x0000\x0000\x0000HFS+ Private Data/iNode3298018… # Catalog File Record# Record 10 in node 45723 beginning at 512-byte sector 0x3eebe8 type = file file ID = 3298018 flags = 0000000000100010 . File has a thread record in the catalog. . File has hardlink chain. reserved1 = 3298170 (first link ID)… # BSD Info ownerID = 501 (johndoe) groupID = 0 (wheel) adminFlags = 00000000 ownerFlags = 00000000 fileMode = -rw-r--r-- linkCount = 2… # Data Fork logicalSize = 14 bytes totalBlocks = 1 fork temperature = no record in Hot File B-Tree clumpSize = 0 extents = startBlock blockCount % of file

0x2ea6c2a 0x1 100.00 %


$ sudo fileXray file1…# Sibling Hard Links hard link = MacHD:/private/tmp/test/file2

Introspecting Directory Hard Links

Beginning with Mac OS X 10.5 (Leopard), HFS+ also supports directory hard links, although it is not straightforward for a user to create a directory hard link. This is because directory hard links are not really meant for third-party use—they are an implementation detail needed to make the Time Machine fea-ture of Mac OS X work.

At the time of this writing, the following conditions must be met for a directory hard link’s creation to be allowed. In the following list, “source” refers to the ex-isting directory that will be pointed at by the new directory hard link “destina-tion” that’s being created.

• The volume must be journaled.• The parent directories of the source and destination directories must

be different.• The source’s parent must not be the root directory.• The destination must not be in the root directory.• The destination must not be a descendent of the source.• The destination must not have any ancestor that’s a directory hard

link.

If these conditions are met, you can create a directory hard link on an HFS+ volume under Mac OS X 10.5 and above. The following C program, which uses the link() system call, can be used to experiment.

Let us create and examine a directory hard link. In the /tmp/test testing di-rectory we used earlier for file hard links, we will create a directory dir1 and a subdirectory subdir. In subdir, we will create a hard link dir2 to dir1. (Re-call that dir1 and dir2 are not allowed to have the same parent.)

introspect

98

/* dirlink.c */

#include <stdio.h>#include <unistd.h>

intmain(int argc, char** argv){ int ret = -1; if (argc == 3) { ret = link(argv[1], argv[2]); if (ret) { perror("link"); } } return ret;}

Before we create the directory hard link, let us examine the current on-disk details of the dir1 directory.

Let us now create the link and confirm that our expectations of directory hard link semantics are met.

Everything looks in order. Let us now use fileXray to see what actually hap-pened inside the file system. We looked at dir1’s on-disk details earlier. We can now see what has changed after we created a directory hard link to dir1. The next page shows the result.

We see that dir1’s transformation is even more drastic than what we had ob-served in the case of file hard links. After we created a directory hard link to dir1, it is not longer a directory on disk. In fact, the “real” directory—the link target—has moved to a special folder called .HFS Private Directory Da-ta\xd, just as the link target of a file hard link had moved to a (different) spe-cial folder. The link target’s name within the special folder is dir_3298482, where the number represents the original CNID of dir1. However, dir1 has not

introspect

99

$ gcc -Wall -o /tmp/dirhardlink dirhardlink.c$ cd /tmp/test$ mkdir dir1 subdir

$ sudo fileXray dir1 path = MacHD:/private/tmp/test/dir1…# Catalog Folder Record# Record 30 in node 63018 beginning at 512-byte sector 0x4324d8 type = folder folder ID = 3298482 flags = 0000000000000010… # BSD Info ownerID = 501 (johndoe) groupID = 0 (wheel) adminFlags = 00000000 ownerFlags = 00000000 fileMode = drwxr-xr-x linkCount = 1 textEncoding = 0 reserved2 = 0…

$ cd /tmp/test$ /tmp/dirhardlink dir1 subdir/dir2$ ls -lasdi dir1 subdir/dir23298482 0 drwxr-xr-x 2 johndoe wheel 68 Nov 10 10:47 dir13298482 0 drwxr-xr-x 2 johndoe wheel 68 Nov 10 10:47 subdir/dir2$ echo Hello > dir1/file$ cat subdir/dir2/fileHello

been replaced by another directory that “points to” the link target—it has been replaced by a file, or specifically, an alias file. The immutable alias file has fdrp and MACS as its type and creator codes, respectively. It also has a re-source fork. Moreover, we see that as in the case of file hard links, there exists a hard link chain.

introspect

100

$ sudo fileXray dir1 path = MacHD:/private/tmp/test/dir1…# Catalog File Record# Record 28 in node 63018 beginning at 512-byte sector 0x4324d8 type = file (alias, directory hard link) indirect node CNID = 3298482 indirect folder = MacHD:/.HFS+ Private Directory Data\x000d/dir_3298482 file ID = 3298603 flags = 0000000000100010 . File has a thread record in the catalog. . File has hardlink chain.

… # BSD Info ownerID = 3298604 (previous link ID) groupID = 0 (next link ID) adminFlags = 00000000 ownerFlags = 00000010 . UF_IMMUTABLE (file may not be changed) fileMode = -r--r--r-- iNodeNum = 3298482 (link reference number) textEncoding = 0 reserved2 = 0 # Finder Info fdType = 0x66647270 (fdrp) fdCreator = 0x4d414353 (MACS) fdFlags = 1000000000000000 . kIsAlias… # Data Fork logicalSize = 0 bytes # Resource Fork logicalSize = 464 bytes totalBlocks = 1 fork temperature = no record in Hot File B-Tree clumpSize = 0 extents = startBlock blockCount % of file

0x2ed51bb 0x1 100.00 %

rsrc contents = (up to 464 bytes) 00 00 01 00 00 00 01 9e 00 00 00 9e 00 00 00 32 00 00 00 00 00 00 00 00 … 00 00 00 00 00 00 00 1c 00 32 00 00 61 6c 69 73 00 00 00 0a 00 00 ff ff 2 a l i s 00 00 00 00 00 00 00 00

Let us also examine the link target. Its path would be “hard” to type because of the \xd character in the special folder’s name. We can use the link target’s CNID instead, which would be the original CNID of dir1. (It also shows up as the link reference number in dir1’s details.)

We see mostly what we would expect given our earlier observations on the im-plementation details of file hard links. Note that the link target folder has an extended attribute named com.apple.system.hfs.firstlink. The attribute’s value is an encoding of the CNID of the head of the directory hard link chain.

introspect

101

$ sudo fileXray -c 3298482 path = MacHD:/.HFS+ Private Directory Data\x000d/dir_3298482…# Catalog Folder Record# Record 10 in node 45511 beginning at 512-byte sector 0x3edea8 type = folder folder ID = 3298482 flags = 0000000000100100 . Folder has extended attributes. . Folder has hardlink chain.

… # BSD Info ownerID = 501 (johndoe) groupID = 0 (wheel) adminFlags = 00000000 ownerFlags = 00000000 fileMode = drwxr-xr-x linkCount = 2…# Attributes

# Attribute Key keyLength = 72 pad = 0 fileID = 3298482 startBlock = 0 attrNameLen = 30 attrName = com.apple.system.hfs.firstlink# Attribute Data Record (Inline)# Record 6 in node 15032 beginning at 512-byte sector 0x4e5b8 recordType = 0x10 reserved[0] = 0 reserved[1] = 0 attrSize = 8 bytes attrData = 33 32 39 38 36 30 34 00 3 2 9 8 6 0 4

Introspecting Symbolic Links

Symbolic links on HFS+ are implemented as regular file’s whose data fork con-tent is the UTF-8-encoded pathname—either relative or absolute, as set at link creation time—to the link’s target. The file system also sets the creator and type codes of symbolic links to special values: rhap and slnk, respectively. The following is an example of examining a symbolic link with fileXray.

introspect

102

$ sudo fileXray /etc/localtime path = MacHD:/private/etc/localtime# Catalog File Thread Record# Record 12 in node 9529 beginning at 512-byte sector 0x3615c8 parentID = 246 nodeName = localtime# Catalog File Record# Record 18 in node 9 beginning at 512-byte sector 0x33c2c8 type = file (symbolic link) linkTarget = /usr/share/zoneinfo/America/Los_Angeles file ID = 205988 flags = 0000000000000010 . File has a thread record in the catalog. reserved1 = 0 createDate = Wed Aug 26 19:14:48 2009 contentModDate = Wed Aug 26 19:14:48 2009 attributeModDate = Wed Aug 26 19:14:48 2009 accessDate = Wed Aug 26 19:14:48 2009 backupDate = 0 # BSD Info ownerID = 0 (root) groupID = 0 (wheel) adminFlags = 00000000 ownerFlags = 00000000 fileMode = lrwxr-xr-x linkCount = 1 textEncoding = 0 reserved2 = 0 # Finder Info fdType = 0x736c6e6b (slnk) fdCreator = 0x72686170 (rhap) fdFlags = 0000000000000000 fdLocation = (v = 0, h = 0) opaque = 0 # Data Fork logicalSize = 39 bytes totalBlocks = 1 fork temperature = no record in Hot File B-Tree clumpSize = 0 extents = startBlock blockCount % of file

0x3863b5 0x1 100.00 %


Note that the data fork, which contains only the UTF-8 encoded pathname to the link target, has the expected size. We can verify its contents by using the --read option to dump the data fork to the standard output.

See Also

•--cnid CNID


•--path PATH

introspect

103

$ sudo fileXray --read data --output /dev/stdout /etc/localtime/usr/share/zoneinfo/America/Los_Angeles

--journal

Display summary information on a journaled volume’s journal. If the --ex-haustive option is additionally specified, also display detailed journal infor-mation.

The summary information includes the contents of the Journal Info Block and the Journal Header.

In the following example, the --journal option is used to display information about an external journal—that is, the journal resides not within the HFS+ volume in question but on a separate device “external” to the volume. fileXray can work with both external and internal journals. Of course, in the case of an external journal, the journal device must be available (attached) for fileXray to access it.

In the following example, the journal resides externally.

The detailed information displayed when --exhaustive is specified includes the Journal Buffer and the transactions therein. The following page shows an annotated excerpt from the output of --journal run in exhaustive mode. The excerpt shows details of a single transaction. fileXray analyzes each block and computes several types of information on the block’s contents.

--journal

104

-j

$ sudo fileXray --device /dev/disk0s2 --journal# HFS+ Journal# Journal Info Block flags = 00000000000000000000000000000000 . Journal resides externally. device_signature = 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 offset = 40960 bytes size = 8388608 bytes (8.4 MB) ext_jnl_uuid = 9C3B4ABE-15ED-4BF4-828E-7509FA07AE75 (/dev/disk1s1) machine_serial_num = W12345AB6CD reserved = 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 # Journal Header magic = 0x4a4e4c78 (JNLx) endian = 0x12345678 start = 6358528 bytes end = 7860224 bytes size = 8388608 bytes blhdr_size = 32768 bytes checksum = 0x8a572bf jhdr_size = 512 bytes

The column labeled as “Block Owner” identifies the file system object that owns the block in question. The identifier is one of the following characters.

--journal

105

A Allocation File.

C Catalog B-Tree.

E Extents B-Tree.

V Volume Header.

X Extended Attributes B-Tree.

$ sudo fileXray --device /dev/disk0s2 --journal --exhaustive# HFS+ Journal# Journal Buffer…# begin transaction # Block List Header max_blocks = 2047 num_blocks = 14 bytes_used = 127488 checksum = 0x14c4f323 flags = 0x3 . check checksums . first header binfo[0].sequence_num = 0xb41509 block_info[ 1] { bnum 2c28 A , bsize 4096 bytes, bp 9f3e6db9 } block_info[ 2] { bnum 48b828 Cl, bsize 8192 bytes, bp 97486d27 } vabn 91705 , fabn 29ebe (btree node 85855) block_info[ 3] { bnum 3ee778 Cl, bsize 8192 bytes, bp a14d9276 } vabn 7dcef , fabn 164a8 (btree node 45652) block_info[ 4] { bnum 33c238 Ch, bsize 8192 bytes, bp 6fb1adf } vabn 67847 , fabn 0 (btree node 0) block_info[ 5] { bnum 3a7c48 Cl, bsize 8192 bytes, bp 6d9bea0a } vabn 74f89 , fabn d742 (btree node 27553) block_info[ 6] { bnum 373e28 Cl, bsize 8192 bytes, bp 85c76441 } vabn 6e7c5 , fabn 6f7e (btree node 14271) block_info[ 7] { bnum 4a628 Xl, bsize 8192 bytes, bp 31f2adb1 } vabn 94c5 , fabn 385c3 (btree node 115425) block_info[ 8] { bnum 13a38 Xh, bsize 8192 bytes, bp 924486d2 } vabn 2747 , fabn 31845 (btree node 101410) block_info[ 9] { bnum 2 V , bsize 512 bytes, bp c2d81893 } block_info[ 10] { bnum 489f28 Cl, bsize 8192 bytes, bp 1d6aed5e } vabn 913e5 , fabn 29b9e (btree node 85455) block_info[ 11] { bnum 3ef118 Cl, bsize 8192 bytes, bp 72474d1a } vabn 7de23 , fabn 165dc (btree node 45806) block_info[ 12] { bnum 363d08 Cl, bsize 8192 bytes, bp 36a0159 } vabn 6c7a1 , fabn 4f5a (btree node 10157) block_info[ 13] { bnum 361538 Cl, bsize 8192 bytes, bp 776e4038 } vabn 6c2a7 , fabn 4a60 (btree node 9520)# end transaction…Summary: 376 blocks using 3311616 bytes in 17 block lists.

Number of valid blocks

Bytes used in this transaction buffer

File AllocationBlock Number

B-Tree NodeNumber

VolumeAllocation

BlockNumber

BlockOwner

NodeType

If the block owner is a B-Tree, the adjacent column labeled as “Node Type” identifies the type of the B-Tree node the block belongs to. The identifier is one of the following characters.

In the case of a B-Tree, the line of output following each block_info line will show the corresponding volume-level allocation block number, the file-level al-location block number, and the B-Tree node number.

Caveats

For a real-life volume, the output generated in exhaustive mode can be quite massive.

Moreover, even though you can use this option on a live, mounted volume, but realize that the journal on a mounted volume—especially the root volume—is highly volatile and is quite likely to change even as fileXray dissects it. Moreover, when this option is used on a mounted volume with ongoing file sys-tem activity, it is possible for fileXray to run into temporal inconsistencies, and in some cases, even terminate ungracefully.

See Also

•--journal_names

•--scavenge


•--volume_header

--journal

106

f Free node.

h Header node.

i Index node.

l Leaf node.

m Map node.

--journal_names

Display file system object names that can be gleaned from dissecting a jour-naled HFS+ volume’s journal.

Specifically, fileXray will read and analyze the journal, looking for file system object names and associated metadata contained within the raw blocks found in the journal. Once done, fileXray will display all types of Catalog records it can piece together.

The information displayed by this option provides an idea of “recent” file sys-tem activity that has occurred on the volume. In most non-contrived cases, the definition of “recent” is the last time the journaled volume experienced file sys-tem activity when mounted in read-write mode. The amount of historical activ-ity that fileXray can glean also depends upon the size of the journal.

As is the case with all fileXray options, you can use this option on a live, mounted volume, but realize that the journal on a mounted volume—especially the root volume—is highly volatile and is quite likely to change even as fileXray dissects it. Moreover, when this option is used on a mounted volume with ongoing file system activity, it is possible for fileXray to run into tempo-ral inconsistencies, and in some cases, even terminate ungracefully.

The following is an example of the output displayed by --journal_names.

Let us examine the various parts of this output. In general, each line of output will contain information on one of four types of Catalog records fileXray found. The type is designated by each line’s first character, which can be one of the following.

--journal_names

107

-J

F The found record is a Catalog Folder Record.

f The found record is a Catalog File Record.

T The found record is a Catalog Folder Thread Record.

t The found record is a Catalog File Thread Record.

$ fileXray --device /dev/disk1s2 --journal_names…T ---- exists 2/20 .Trashes :TestHD:/.Trashes/F ---- exists 20/21 501 :TestHD:/.Trashes/501/F --c- recreated 2/24 Pictures :TestHD:/Pictures/f -pcn deleted 24/27 Background1.jpg :f -pcn deleted 24/28 Background2.jpg :F ---n renamed 2/215 Documents :TestHD:/Saved Documents/f -p-n moved,renamed 2/214 notes.pdf :TestHD:/Saved Documents/Ne…

Recordtype tag

Differencetags

File systemactivity tag

ParentCNID

ObjectCNID

Objectname

Literalcolon

Computed path ofobject on volume

An important aspect to realize about the journal’s operation is that a raw jour-nal block can contain several Catalog records that are there not because the corresponding file system objects were modified, deleted, etc., but simply be-cause the file system object resides in the same HFS+ block as one or more other file system objects that were modified or deleted. When in doubt about the reason for a name’s existence in the journal, the on-volume timestamps should also be consulted. fileXray will make a “best effort” attempt at identi-fying the type of activity that could have caused the record to be present in the journal. fileXray will tag each line of output with up to four character tags indicating some key “differences” it found between the journal and volume cop-ies of a file system object’s metadata.

Based on these differences, fileXray will also ascribe a high-level file system activity tag to each line of output. The tag would be one of the following.

The remainder of each output line includes the parent CNID, the object CNID, and the object name as it was found in the journal. Whenever possible, fileXray also compute the path—as it exists, or as it was likely to have ex-isted in the case of deleted objects—to the file system object in question. When an object’s parent folder cannot be found, typically because it has been deleted, fileXray may not compute the path. Often, the node name corresponding to the deleted parent will also exist in the output of --journal_names.

--journal_names

108

t Type (file or folder) differs.

p Parent Catalog Node ID (CNID) differs.

c Object ID (file or folder CNID) differs.

n Name differs.

exists The object exists both on the volume and in the journal. It is likely to be present in the journal because it happened to reside in the same file system block as another object that was modi-fied or deleted.

deleted The object has been deleted from the volume.

metamorphed The object’s type has changed—that is, previously, a file or a folder existed with this name, but now there is a folder or a file, respectively, with this name.

moved The object’s parent folder has changed.

moved,renamed The object’s parent folder and its name have both changed.

recreated The object exists on the volume but with a CNID that is differ-ent from what was found in the journal.

renamed The object’s name has changed.

When --journal_names is used along with the --exhaustive option, fileXray will perform additional analysis on the records found in the journal. In particular, it will examine the various pieces of metadata in any Catalog file and folder records found in the journal and compare them with their counter-parts—if they can be located—on the volume. In other words, fileXray will create a “diff” between the journal’s view and the volume’s view of the file and folder metadata that appears in journal transactions. The following excerpt is an example.

In this excerpt, we see two folder records. The folder named 501 has had its attribute modification date updated on the volume. The root folder has had 3 file system objects added to it. Its timestamps have also been updated. Note that for a real-life volume, the output generated by --journal_names in ex-haustive mode can be quite massive. Moreover, the caveat about using this op-tion on a mounted volume applies even more strongly to the exhaustive mode.

The --scavenge option performs a related operation, although it focusses on deleted objects and gathers more details about such objects. It can be used to identify certain deleted objects and recover blocks, if possible, from such ob-jects. The built-in Scavenger File System provides a virtual file system view of scavenged content.

See Also

•--exhaustive

•--journal

•--scavenge



--journal_names

109

$ fileXray --device /dev/disk1s2 --journal_names --exhaustive…F ---- exists 20/21 501 :TestHD:/.Trashes/501/< attributeModDate = Mon Nov 9 01:05:02 2009> attributeModDate = Sun Nov 8 20:47:33 2009

F ---- exists 1/2 TestHD :TestHD:/< valence = 9> valence = 6< contentModDate = Mon Nov 9 01:02:14 2009> contentModDate = Sun Nov 8 20:50:09 2009< attributeModDate = Mon Nov 9 01:02:14 2009> attributeModDate = Sun Nov 8 20:50:09 2009< accessDate = Mon Nov 9 01:02:10 2009> accessDate = 0

--list

List a given folder’s contents. The folder must be specified using another option such as --cnid, --fsspec, or --path.

The following is an example of listing the contents of the root folder on a stan-dard boot volume.

The output format is somewhat similar to, but not the same as that of the long output format of the ls command. Some noteworthy points are as follows.

• All files and folders are shown, including system objects that are fil-tered out by the kernel.

• For files, both the data and resource fork sizes are shown.• For hard links, both the link and the link target are shown.• The third column (labeled “@+SUZ”) has zero or more character tags

with the following meanings.

--list

110

-l

$ sudo fileXray --device /dev/disk0s2 --path / --listCNID mode @+SUZ user group data rsrc name

205063 -rw-rw-r-- ---- root admin 15 KB 0 .DS_Store80372 ---------- ---- root admin 0 0 .file205463 drwx------ ---- root admin .fseventsd/19 dr-xr-xr-t ---u root wheel .HFS+ Private Directory Data\x000d/205320 -rw------- ---- root wheel 262 KB 0 .hotfiles.btree16 ---------- ---- root wheel 25 MB 0 .journal17 ---------- ---- root wheel 4.1 KB 0 .journal_info_block205682 drwx------ ---- root admin .Spotlight-V100/20 d-wx-wx-wt ---- root _unknown .Trashes/25955 drwxr-xr-x ---- root wheel .vol/144 drwxrwxr-x -+-- root admin Applications/23912 drwxr-xr-x ---- root wheel bin/80384 drwxrwxr-t ---- root admin cores/24623 dr-xr-xr-x ---- root wheel dev/206903 drwxrwxr-x ---- root admin Developer/24624 lrwxr-xr-x ---- root wheel 11 0 etc -> private/etc205567 dr-xr-xr-x ---- root admin home/244 drwxrwxr-t -+-- root admin Library/26022 -rw-r--r-- ---- root wheel 19 MB 0 mach_kernel205566 dr-xr-xr-x ---- root admin net/80382 drwxr-xr-x ---- root wheel Network/50 drwxr-xr-x ---- root wheel private/4349 drwxr-xr-x ---- root wheel sbin/68 drwxr-xr-x ---- root wheel System/24664 lrwxr-xr-x ---- root wheel 11 0 tmp -> private/tmp37116 drwxr-xr-x ---- root admin Users/139 drwxr-xr-x ---- root wheel usr/24665 lrwxr-xr-x ---- root wheel 11 0 var -> private/var25956 drwxrwxrwt -+-- root admin Volumes/18 d--------- ---u root wheel \x0000\x0000\x0000\x0000HFS+ Pri-vate Data/

See Also

•--cnid CNID


•--path PATH

--list

111

@ File or folder has one or more extended attributes. (Although Access Control Lists are implemented as extended attributes, this tag does not take Access Control Lists into account.)

+ File or folder has an Access Control List. That is, the object has an extended attrib-ute named com.apple.system.Security.

S File or folder has system-level immutability set.

U File or folder has user-level immutability set.

Z File is HFS+ compressed.

--list_records RECORD_TYPE

Specify a B-Tree record type to be listed from the leaf nodes of an HFS+ B-Tree.

The valid values for RECORD_TYPE depend on the B-Tree in question, which in turn must be specified using the --btree option. RECORD_TYPE can be one of the following.

The record type none is a special case. If it is specified, fileXray will display summary information on each node in the given B-Tree without displaying any records. The following is an example.

Additionally, the --node option can be used along with this option to specify a particular node in an HFS+ B-Tree. In this case, RECORD_TYPE must be any. fileXray will then list all records residing in that particular node. The exam-ple on the following page shows listing the records in node number 1 of the Catalog B-Tree of a test volume.

See Also


•--node NODE


112

-L RECORD_TYPE

any Matches any record type in any B-Tree.

extent Matches Extent records in the Extents B-Tree.

file Matches File records in the Catalog B-Tree.

filethread Matches File Thread records in the Catalog B-Tree.

folder Matches Folder records in the Catalog B-Tree.

folderthread Matches Folder Thread records in the Catalog B-Tree.

hfcfile Matches Hot File records in the Hot Files B-Tree.

hfcthread Matches Hot File Thread records in the Hot Files B-Tree.

none Matches no records.

$ sudo fileXray --device /dev/disk0s2 --btree catalog --list_records none0 : Header, 3 records, height=0, bLink=0, fLink=568321 : Leaf, 22 records, height=1, bLink=56503, fLink=102252 : Leaf, 20 records, height=1, bLink=8495, fLink=9833 : Leaf, 8 records, height=1, bLink=56620, fLink=455414 : Leaf, 72 records, height=1, bLink=120, fLink=1323…226 : Index, 129 records, height=2, bLink=1229, fLink=1222227 : Index, 122 records, height=3, bLink=74259, fLink=0…


113

$ sudo fileXray --volume /Volumes/test --btree catalog --node 1 \ --list_records any…# Record 9 in Leaf Node 1# Key keyLength = 48 parentID = 2 nodeName.length = 21 nodeName.unicode = \x00\x00\x00\x00HFS+ Private Data# Identity path = test:# Catalog Folder Record type = folder folder ID = 18 flags = 0000000000010000 . Folder maintains a separate subfolder count. valence = 0 createDate = Mon Nov 2 20:40:05 2009 contentModDate = Mon Nov 2 20:40:05 2009 attributeModDate = Mon Nov 2 20:40:05 2009 accessDate = Mon Nov 2 20:40:05 2009 backupDate = 0 # BSD Info ownerID = 0 (root) groupID = 0 (wheel) adminFlags = 00000000 ownerFlags = 00000010 . UF_IMMUTABLE (file may not be changed) fileMode = d--------- linkCount = 1 textEncoding = 0 folderCount = 0 # Finder Info frRect = (top = 0, left = 0), (bottom = 0, right = 0) frFlags = 0101000000000000 . kNameLocked . kIsInvisible frLocation = (v = 16384, h = 16384) opaque = 0 # Opaque Finder Info scrollPosition = (v = 0, h = 0) reserved1 = 0 Opaque Finder Flags = 0000000000000000 reserved2 = 0 putAwayFolderID = 0

…

--monitor

Run the built-in file system modification monitor.

When you use Mac OS X, you frequently access files and folders. Several of these accesses result in the file system being modified. Developers, power us-ers, and even “regular” users who are curious would be interested in observing and understanding file system changes. The motivation might be for security reasons, for analyzing software, for troubleshooting, or just out of curiosity.

fileXray has a built-in monitor that uses the operating system’s fsevents mechanism to report file system modifications in real time. This feature is available for use on all types of volumes—it is not limited to HFS+ volumes.

While running, the monitor will block, waiting for file system modifications to occur. As the monitor gets notified by the kernel of file system activity, it dis-plays a log of the activity on the standard output. To terminate the monitoring, type control-C in the Terminal window that the monitor is running in.

The following shows an excerpt from the monitor’s output, which has been re-formatted to fit this page.

--monitor

114

$ sudo fileXray --monitor…888271320881501 Terminal/56698 create 3195304 -rw------- johndoe staff \ /Users/singh/Library/Preferences/com.apple.Terminal.plist.Sh1NlZN

888271321793660 Terminal/56698 write 3195304 -rw------- johndoe staff \ /Users/singh/Library/Preferences/com.apple.Terminal.plist.Sh1NlZN

888271321842828 Terminal/56698 chown 3195304 -rw------- johndoe staff \ /Users/singh/Library/Preferences/com.apple.Terminal.plist.Sh1NlZN

888271321865607 Terminal/56698 chown 3195304 -rw------- johndoe staff \ /Users/singh/Library/Preferences/com.apple.Terminal.plist.Sh1NlZN

888271322060324 Terminal/56698 rename 3188794 -rw------- johndoe staff \ /Users/singh/Library/Preferences/com.apple.Terminal.plist.Sh1NlZN -> \ /Users/singh/Library/Preferences/com.apple.Terminal.plist

888352076501675 syslogd/13 write 3190703 -rw-r----- root admin \ /private/var/log/system.log…

Catalog Node ID

File system operation

Process ID of the processthat caused the activity

Monotonicallyincreasing timestamp

Name of the process(if available) that causedthe activity

You can additionally specify the --exhaustive option, which will cause the monitor to produce more verbose output. The verbose output format is the same as that of the freely available fslogger1 program. The following is an ex-ample of the monitor running in verbose mode.

Note that although the monitor built into fileXray is similar to fslogger, it has more features such as:

• The ability to optionally log to a file specified by the --monitor_log-file option.

• The ability to exclude or include one or more specific volumes using the --monitor_exclude and --monitor_include options.

• The ability to choose between succinct and verbose output formats using the --exhaustive option. Only the succinct output can be logged to a file.

• Better buffering.

See Also

•--exhaustive

•--monitor_exclude VOLUME

•--monitor_include VOLUME

•--monitor_logfile LOGFILE

--monitor

115

1 http://osxbook.com/software/fslogger/

$ sudo fileXray --monitor --exhaustive…=> received 783 bytes# Event type = FSE_CREATE_FILE pid = 8367 (PubSubAgent) # Details # type len data FSE_ARG_STRING 62 string = /…/PubSub/Database/Database.sqlite3-journal FSE_ARG_DEV 4 dev = 0xe000002 (major 14, minor 2) FSE_ARG_INO 4 ino = 1125942859711111 FSE_ARG_MODE 4 mode = -rw-r--r-- (0x0081a4, vnode type VREG) FSE_ARG_UID 4 uid = 501 (johndoe) FSE_ARG_GID 4 gid = 20 (staff) FSE_ARG_INT64 8 tstamp = 890271144247776 FSE_ARG_DONE (0xb33f)…

http://osxbook.com/software/fslogger/

http://osxbook.com/software/fslogger/

--monitor_exclude VOLUME

Exclude (that is, do not monitor) the mounted volume specified by VOLUME, which can either be the mount point of the volume in question or the path to a file or folder within it. This option is used along with the --monitor option.

The --monitor_exclude option can be specified multiple times—once for each volume that you wish to exclude. All other volumes will be monitored. Volumes that are mounted after the monitor starts running will also be monitored.

Note that the --monitor_exclude and --monitor_include options are mutu-ally exclusive.

See Also

•--monitor



--monitor_exclude VOLUME

116

$ sudo fileXray --monitor --monitor_exclude /Volumes/Backup \ --monitor_exclude /Volumes/Music…

--monitor_include VOLUME

Include (that is, do monitor) the mounted volume specified by VOLUME, which can either be the mount point of the volume in question or the path to a file or folder within it. This option is used along with the --monitor option.

The --monitor_include option can be specified multiple times—once for each volume that you wish to include. No other volumes will be monitored. Volumes that are mounted after the monitor starts running will also not be monitored.

Note that the --monitor_exclude and --monitor_include options are mutu-ally exclusive.

See Also

•--monitor



--monitor_include VOLUME

117

$ sudo fileXray --monitor --monitor_include /…

--monitor_logfile LOGFILE

Specify the path (LOGFILE) to a file into which the built-in file system modifica-tion monitor will log its output.

The monitor filters out modifications to the log file itself, which prevents an in-finite stream of output from being generated. The monitor must be run in suc-cinct mode for it to log its output to a file—that is, the --exhaustive option must not be specified.

The path specified by LOGFILE must satisfy the following conditions.

• The file must not already exist. fileXray will not clobber an existing file.

• The path must not resolve to a file-backed disk image. fileXray will not log to a file that resides on such a disk image.

See Also

•--monitor



--monitor_logfile LOGFILE

118

--mount_data

Display a mounted HFS+ volume’s kernel-resident mount data.

The mount data contains several pieces of information useful for analysis and debugging. For example:

• The next unused Catalog Node ID (CNID), which gives an indication of the CNID the kernel is likely to use for the next newly created file sys-tem object on the volume.

• The allocation block number where the kernel will start its search for a free block the next time it needs needs a block.

• Information on the states of journaling and Hot File Clustering.• The maximum size for extended attributes that are stored inline.• Limits that trigger low disk space notifications.

The following pages show an example of viewing the root volume’s mount data.

--mount_data

119

-m

$ sudo fileXray --mount_data# Identification volume name = MacHD (volfs_id=234881026) block device number = { major=14, minor=2 }

# Volume Flags flags bitmap (32 bits) = 00000000000000000000000010001100 + HFS_WRITEABLE_MEDIA + HFS_CLEANED_ORPHANS + HFS_METADATA_ZONE

# Physical Description logical block size of disk = 512 logical block count on disk = 0x1d121920 alternate volume header location = 0x1d12191e physical block size of disk = 512 logical blocks per physical block = 1

# Access to VFS and Devices struct mount pointer = 0x6f47bd8 block device mounted vnode = 0x7003f38 vnode for Extents file = 0x7003ea4 vnode for Catalog file = 0x7003e10 vnode for Allocation file = 0x7003d7c vnode for Attributes file = 0x7003ce8 vnode for Startup file = 0 vnode for Attributes Data file = 0 cnode for Extents file = 0x700cef0 cnode for Extents file = 0x700cef0 cnode for Catalog file = 0x700ce14 cnode for Allocation file = 0x700cd38 cnode for Attributes file = 0x700cc5c cnode for Startup file = 0 raw device mounted = 0xe000002 size of a buffer cache buffer for I/O = 4096

continued…

--mount_data

120

…# Journaling journal for this volume = 0x700af00 vnode for journal device = 0x7003f38 start block of journal = 0x747 journal size = 25165824 journal file ID = 16 journal info block file ID = 17

# Notification Variables notification conditions bits = 00000000000000000000000000000000 freespace danger limit = 32000 freespace warning limit = 64000 freespace desired limit = 96000

# Metadata Zone metadata zone start block = 0x1 metadata zone end block = 0xd7fff hotfile start block = 0xb69c3 hotfile end block = 0xd7fff minimum allocation start = 0xd8000 freed block count = 0x451a8 hotfile free blocks = 0x1ff75 hotfile maximum blocks = 0x2163d overflow maximum blocks = 0 catalog maximum blocks = 0x2c7b

# Hot File Clustering clustering stage = HFC_RECORDING recording period start time = 2009 Oct 24 23:46:50 recording period stop time = 2009 Nov 1 10:46:50 opaque recording data = 0x42ac4004 maximum files to track = 1000 vnode for Hot Files B-Tree = 0

# Sparse device root vnode for backing fs = 0 (not applicable) sparse disk image band size = 0 (not applicable) backing fs last statfs = 0 (not applicable) backing fs max blocks = 0 (not applicable)

# Syncing/Freezing/Downgrading sync scheduled = 0 sync incomplete = 0 last sync request time = 0 last sync time = 1256963668634704 active threads = 0 maximum pending I/O = 0 freezing process = 0 downgrading process = 0

continued…

--mount_data

121

…# Default Values for HFS Standard / Non-Init Access default owner = { uid=99, gid=99 } directory protection bits mask = 755 file protection bits mask = 755 default encoding for non-HFS+ volumes = 0

# Persistent Fields (On-Disk, Dynamic) file system last modification time = 2009 Nov 14 21:34:28 number of files in file system = 1421932 number of directories in file system = 267512 free allocation blocks = 0x139f3f1 start block for next allocation search = 0x2e56ca9 sparse allocation = 0xb7293 next unused catalog node ID = 3196458 file system write count = 121772643 encodings in use = 0000000000000000000000000000000000000010000000000000000011001111

# Persistent Fields (On-Disk, Static) volume signature in vcb (runtime) = 0x482b volume flags in vcb (runtime) = 0xff00 volume attributes in vcb (runtime) = 0x80002000 journal info block in vcb (runtime) = 0x746 file system creation time = 2009 Nov 2 18:56:59 file system last backup time = allocation block size = 4096 total allocation blocks = 0x3a24324 do not allocate this block or beyond = 0x3a24324 clump size = 65536 finder info = 0x88000000 0x4d711100 0x00000000 0x00000000 0x00000000 0x88000000 0x90f28d01 0x0ead72cc VBMSt (HFS-only) = 0 AlBlSt (HFS-only) = 0

# Miscellaneous disk block where HFS+ starts = 0 volume bitmap I/O size = 4096 free block reserve = 64000 blocks on loan for delayed allocations = 0 cache of largest known free extents = 10 extents

# Times last mounted time = 2009 Nov 10 14:57:49 last mounted modification time = 2009 Nov 10 14:56:26 last modification time = 2009 Nov 14 21:34:28 file system creation time = 2009 Nov 2 18:56:59 file system creaation time (local) = 2009 Nov 2 11:56:59 metadata create time = 2009 Nov 2 18:56:59 file system last backup time =

continued…

Retrieving the mount data from the kernel requires superuser privileges. Therefore, to use this option for any mounted HFS+ volume, fileXray must be run with superuser privileges.

See Also

•--enable_xattr_extents

•--next_allocation BLOCK


--mount_data

122

…# Resize Variables resize files moved = 0 resize total files = 0

# Other maximum inline attribute size = 3802

--next_allocation BLOCK

Given a mounted HFS+ volume, suggest to the kernel to begin its next search for a free allocation block at allocation block number BLOCK.

A mounted volume must also be specified using the --volume option, which requires either the mount point of the volume or the path to a file system object on the volume as an argument. When used with this option, --volume has a special case: if you wish to change the next allocation block on the root volume, you must explicitly use “/” as the argument to --volume. Since --next_allo-cation alters block allocation, its use on the root volume requires explicit specification.

One of the interesting uses of the --next_allocation option is to intention-ally create a fragmented file. You can do this, for example, by writing to a file one block at a time. After every write, you can use fileXray to determine the block number occupied by the file’s end. Then you can use --next_alloca-tion to skip the next free block (if there’s one). Repeat this process for as many fragments as you need for the file. The following is a quick-and-dirty Perl script (mkfragmented.pl) that does this.


123

-N BLOCK

#! /usr/bin/perl -w# mkfragmented.pl: Create a file with N fragments (extents) on a given volume. my $FOUR_KB = "4" x 4096;my $FILEXRAY = "fileXray"; sub usage() { die "usage: $0 <volume> <nfragments>\n"; } (-x $FILEXRAY) or die "$0: missing fileXray.\n";($#ARGV == 1) or usage();

my $volume = $ARGV[0];my @sb = stat($volume);((-d $volume) && @sb) or usage();

my $needextents = int($ARGV[1]);

my $file = "$volume/fragmented.$$";(! -e $file) or die "$0: file $file already exists\n";`/bin/echo -n $FOUR_KB > "$file"`; # create a file(-e "$file") or die "$0: failed to create file ($file)\n"; WHILE_LOOP:while (1) { `/bin/sync`; my @out = `$FILEXRAY "$file" | grep -B 2 'allocation blocks'`; $out[0] =~ /^\s+([^\s]+)\s+([^\s]+)..*$/; my $lastStartBlock = $1; # starting block of the file's last extent my $lastBlockCount = $2; # number of blocks in the last extent $out[2] =~ /[\s*\d+] allocation blocks in (\d+) extents total.*/; my $curextents = int($1); # number of extents the file currently has if ($curextents >= $needextents) { # do we already have the needed extents? print "\ncreated $file with $curextents extents\n"; last WHILE_LOOP; }

# set volume's next allocation pointer to the block right after our file my $conflict = sprintf("0x%x", hex($lastStartBlock) + hex($lastBlockCount)); `sudo $FILEXRAY --next_allocation $conflict --volume $volume`; print "start=$lastStartBlock count=$lastBlockCount extents=$curextents ". "conflict=$conflict\n"; `echo hello > "$volume/dummy.txt"`; # create dummy file to consume space `/bin/echo -n $FOUR_KB >> "$file"`; # extend our file to cause discontiguity `rm "$volume/dummy.txt"`; # remove the dummy file} # WHILE_LOOP exit(0);

The following is an example of using the mkfragmented.pl script to create a file with 9 fragments. Since we are merely experimenting, we will use a test volume (say, as opposed to the root volume) for this. Note that the script as-sumes that fileXray is in your shell’s PATH.

This is also a good opportunity to witness the automatic on-the-fly defragmen-tation feature of HFS+. Assuming the necessary conditions for the automatic defragmentation mechanism are met, we can trigger the relocation and defrag-mentation of this file by simply reading it. The following example shows the mechanism at work.


124

$ sudo ./mkfrag.pl /Volumes/test 9start=0x229 count=0x1 extents=1 conflict=0x22astart=0x22b count=0x1 extents=2 conflict=0x22cstart=0x22d count=0x1 extents=3 conflict=0x22estart=0x22f count=0x1 extents=4 conflict=0x230start=0x231 count=0x1 extents=5 conflict=0x232start=0x233 count=0x1 extents=6 conflict=0x234start=0x235 count=0x1 extents=7 conflict=0x236start=0x237 count=0x1 extents=8 conflict=0x238

created /Volumes/test/fragmented.13171 with 9 extents

$ fileXray /Volumes/test/fragmented.13171… extents = startBlock blockCount % of file

0x229 0x1 11.11 % 0x22b 0x1 11.11 % 0x22d 0x1 11.11 % 0x22f 0x1 11.11 % 0x231 0x1 11.11 % 0x233 0x1 11.11 % 0x235 0x1 11.11 % 0x237 0x1 11.11 %

0x239 0x1 11.11 %

9 allocation blocks in 9 extents total. 1.00 allocation blocks per extent on an average.…

$ cat /Volumes/test/fragmented.13171 > /dev/null

$ fileXray /Volumes/test/fragmented.13171… extents = startBlock blockCount % of file

0x1238 0x9 100.00 %

9 allocation blocks in 1 extents total. 9.00 allocation blocks per extent on an average.…

Caveats

The --next_allocation option works by setting the value of the roving next-allocation pointer in the kernel through a system call. The pointer is maintained by the kernel as a hint for itself—the kernel is likely to use the hint as a start-ing point while searching for a free allocation block in most cases, but in some cases it might not. Therefore, the success of the --next_allocation operation ultimately depends upon whether the kernel honors the hint or not.

See Also

•--mount_data



125

--node NODE

Specify a node in an HFS+ B-Tree. NODE is a node number. The number of the first node in an HFS+ B-Tree is 0.

This option is used along with the --btree option to examine a specific node in an HFS+ B-Tree. Additionally, the --exhaustive and --list_records options allow you to introspect a node more deeply by displaying its constituent re-cords, free space, and other bookkeeping information.

The --btree and --list_records sections provide examples that involve us-ing the --node option.

See Also


•--list_records RECORD_TYPE

--node NODE

126

-n NODE

--noidmap

A global option that tells fileXray not to map user and group identifiers to user and group names, respectively. Moreover, user and group UUIDs will also not be mapped to their respective user and group identifiers, respectively, and will be displayed as is.

By default, fileXray displays names corresponding to these identifiers. Al-though some name mappings may not be machine-specific (for example, an ac-cess control entry uses a well-known UUID to represent “everyone”), most mappings are in the context of the current environment—that is, the user and group names displayed will be resolved on the machine fileXray is running on. Depending on the circumstances, this may or may not be desirable. The --noidmap option can be used to disable the aforementioned mapping, in which case fileXray will display numerical1 user and group identifiers, raw UUIDs, and names for well-known UUIDs. Consider the following example.

--noidmap

127

1 Even when the mapping is enabled, fileXray displays numerical identifiers in addition to resolved names.

$ touch /tmp/testfile.txt$ chmod +a 'johndoe allow write' /tmp/testfile.txt$ chmod +a 'everyone allow read' /tmp/testfile.txt

$ sudo fileXray /tmp/testfile.txt… # File Security Information fsec_magic = 0x12cc16d fsec_owner = 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 fsec_group = 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 # ACL Record acl_entrycount = 2 acl_flags = 0 (hi = 0, lo = 0) # ACL Entry 0 ace_applicable = ab cd ef ab cd ef ab cd ef ab cd ef 0 0 0 c user = $everyone group = $everyone ace_flags = 00000000000000000000000000000001 (0x000001) . KAUTH_ACE_PERMIT ace_rights = 00000000000000000000000000000010 (0x000002) . KAUTH_VNODE_READ_DATA/KAUTH_VNODE_LIST_DIRECTORY # ACL Entry 1 ace_applicable = d1 d4 2b f9 89 3d 4e 9a 95 67 46 12 79 f2 d1 52 user = johndoe uid = 501 group = $user gid = $user ace_flags = 00000000000000000000000000000001 (0x000001) . KAUTH_ACE_PERMIT ace_rights = 00000000000000000000000000000100 (0x000004) . KAUTH_VNODE_WRITE_DATA/KAUTH_VNODE_ADD_FILE…

In this example, we create a file and set two access control entries on it: one applies to a specific username and the other applies to “everyone.” We see that fileXray maps the well-known UUIDs to “everyone” and displays it as $eve-ryone. fileXray will display special or well-known mappings with a $ prefix.

It maps the user UUID in the second access control entry to a username and user ID, both of which are valid in the context of the current environment. (Note that the user, or in other cases a group, could be hosted on a Primary Domain Controller or an Active Directory Server.) Specifying the --noidmap option in this case will cause the user UUID to not be mapped as shown below.

--noidmap

128

$ sudo fileXray /tmp/testfile.txt… # File Security Information fsec_magic = 0x12cc16d fsec_owner = 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 fsec_group = 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 # ACL Record acl_entrycount = 2 acl_flags = 0 (hi = 0, lo = 0) # ACL Entry 0 ace_applicable = ab cd ef ab cd ef ab cd ef ab cd ef 0 0 0 c user = $everyone group = $everyone ace_flags = 00000000000000000000000000000001 (0x000001) . KAUTH_ACE_PERMIT ace_rights = 00000000000000000000000000000010 (0x000002) . KAUTH_VNODE_READ_DATA/KAUTH_VNODE_LIST_DIRECTORY # ACL Entry 1 ace_applicable = d1 d4 2b f9 89 3d 4e 9a 95 67 46 12 79 f2 d1 52 user = D1D42BF9-893D-4E9A-9567-461279F2D152 group = $user gid = $user ace_flags = 00000000000000000000000000000001 (0x000001) . KAUTH_ACE_PERMIT ace_rights = 00000000000000000000000000000100 (0x000004) . KAUTH_VNODE_WRITE_DATA/KAUTH_VNODE_ADD_FILE…

--output OUTPUT_FILE

Specify an output file into which data is to be saved.

The --output option is used along with other options that read data from a given volume and save it. This includes the following options.

Note that as a matter of policy, fileXray will not overwrite an existing file. Therefore, OUTPUT_FILE must not already exist. There is a special case, how-ever. If OUTPUT_FILE is one of the strings -, /dev/stdout, or /dev/fd/1, the retrieved data is sent to the standard output instead of being saved to a file.

See Also

•--block BLOCK

•--read COMPONENT

•--scavenge

•--uncompress


--output OUTPUT_FILE

129

-o OUTPUT_FILE

--block Read the given allocation block.

--read Read the given component from the given file system object.

--scavenge Read scavenged content from a deleted or renamed file.

--partition START[,SIZE]

Specify the start offset in bytes (START) and optionally the size in bytes (SIZE) of the target HFS+ volume. The start offset is relative to the beginning of the device fileXray is targeted at. The device is specified through the --device option.

The --partition option is used when fileXray is targeted at a block or char-acter device or a disk image containing two or more HFS+ partitions. If fileXray detects exactly one HFS+ partition on the device or disk image, it will use that partition automatically without further prompting.

Without the --partition option, when fileXray is targeted at a device or im-age containing two or more HFS+ partitions, it will detect the partition table type if the table is one of the three types used on Mac OS X:

• GUID Partition Table (GPT)

• Apple Partition Map (APM)

• Master Boot Record (MBR)

fileXray will then attempt to list all HFS+ partition table entries found within. You can then choose the desired entry for fileXray to operate upon and spec-ify it using the --partition option.

See Also

•--device DEVICE

--partition START[,SIZE]

130

-e START[,SIZE]

$ sudo fileXray --device /dev/disk0 --volume_headerNo flavor of HFS+ found.

However, a GUID Partition Table was detected on the given device with thefollowing HFS+ partition table entries:


209735680 249715376128 MacHD (HFS+, 250 GB)249925111808 16756736 BootHD (HFS+, 17 MB)


$ sudo fileXray --device /dev/disk0 --volume_header \ --partition 209735680,249715376128…

--path PATH

Specify a file system object through its absolute POSIX path (PATH), which must be provided as a UTF-8 string. The --path option is the canonical way to specify a file system object to fileXray.

It is necessary to use this option when you wish to examine file system objects by path on offline (that is, not mounted) volumes. On mounted volumes, it suf-fices to simply point fileXray to the file system object of interest, which im-plicitly specifies both the path of the object of interest and the volume it resides on. For example:

The explicit way to specify the same file to fileXray would be to use the --path option with the file’s absolute path as the argument as follows.

The explicit --path form is “purer” because in this mode, fileXray itself parses and looks up each path component on the volume—whether the volume is mounted or not. Besides being purer, this form is also more sophisticated and gives you more control over how any symbolic links, hard links, and direc-tory hard links in the given path are resolved.

There are some important points to note about how fileXray processes the argument to --path.

Dot-Dot

If a “..” appears in the path, fileXray will ensure that the component that it is “going back” from is indeed a folder.

--path PATH

131

-p PATH

$ cd$ pwd/Users/johndoe$ sudo fileXray Library/Safari/Bookmarks.plist…

$ sudo fileXray --path /Users/johndoe/Library/Safari/Bookmarks.plist…

$ sudo fileXray --path /usr/bin/.. path = MacHD:/usr# Catalog Folder Thread Record# Record 19 in node 8731 beginning at 512-byte sector 0x35e3e8 parentID = 2 nodeName = usr…

$ sudo fileXray --path /usr/bin/nasm/..Invalid path: nasm is not a folder.Path /usr/bin/nasm/.. not found on volume.

Link Resolution

If any intermediate (non-terminal) components of the path happen to be either symbolic or hard links, fileXray will follow them. However, fileXray will not resolve the terminal component if it is a symbolic or hard link. This is because the goal is to examine the true on-disk entity for the given path. Moreover, the details shown by fileXray do include the full path to the link’s target. There-fore, if you wish to obtain further detail on the link target, you can run fileXray directly on it.

Terminal Slash

If the specified path has a terminal slash, fileXray will verify that the file system object is a folder. If the object happens to be a directory hard link or a symbolic link, fileXray will resolve it in this case. The following examples il-lustrate this rule.

Long Node Names and Name Mangling

HFS+ supports file system object names up to 255 “characters” in length. More specifically, a file or folder name can be up to 255 Unicode characters, each of which consumes 16 bits on disk. The Unicode string representing a name is stored in fully decomposed form on disk, with composing characters stored in canonical order. When an HFS+ node name is transferred between the kernel and user spaces, it is encoded as ASCII-compatible UTF-8 bytes. Now, the Unix-style file system layers in Mac OS X—both within the BSD portion of the

--path PATH

132

# /somesymlink points to a file.$ fileXray --path /somesymlink… # fileXray will show details of the symbolic link file and not the link target

# /somesymlink points to a file.$ fileXray --path /somesymlink/… # fileXray will complain that the link target is not a directory

# /somesymlink points to a directory.$ fileXray --path /somesymlink… # fileXray will show details of the symbolic link file and not the link target

# /somesymlink points to a directory.$ fileXray --path /somesymlink/… # fileXray will show details of the link target (directory the link points to)

# /somesymlink points to a non-existent target (that is, a broken symbolic link)$ fileXray --path /somesymlink… # fileXray will show details of the symbolic link file

# /somesymlink points to a non-existent target (that is, a broken symbolic link)$ fileXray --path /somesymlink/… # fileXray will complain that /somesymlink/ was not found on the volume

kernel and the user-space C library—place a limit of 255 bytes for file and folder names. In other words, the UTF-8 representation must also fit in 255 bytes. This easily leads to a problem because there are Unicode characters that require multiple bytes when encoded as UTF-8. Consider a valid HFS+ name that consists of the Unicode character ‰ (U+2030) repeated 255 times. The UTF-8 representation requires 765 bytes. Mac OS X handles the problem by “mangling” or shortening such names and encoding the Catalog Node ID (CNID) within the shortened name. fileXray performs the same shortening/encoding within itself when dealing with such names.

In the following example, the 765 byte UTF-8 string is shortened to a 255 byte UTF-8 string. The shortened version incorporates the hexadecimal representa-tion of the file’s CNID.

See Also

•--cnid CNID


--path PATH

133

$ cd /tmp$ touch `perl -e 'print "‰"x255;'`$ ls -li3288816 -rw-r--r-- 1 singh staff 0 Nov 10 01:33 ‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰\‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰#322EF0

$ sudo fileXray ‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰\‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰‰#322EF0…

--read COMPONENT

Read and save all bytes of the given component of a given file system object.

COMPONENT is one of the following.

The file system object of interest must be specified through other options such as --cnid, --fsspec, and --path. The data read needs to be saved to a desti-nation file, which must be specified through the --output option.

By default, the --read option will copy only the logical bytes of a data or re-source fork. That is, if the fork’s size is not a multiple of the volume’s allocation block size, the last allocation block used by the file system object will be par-tially copied. To copy the last allocation block in its entirety, you can addition-ally specify the --exhaustive option.

If the target object is a file-system-level compressed file, it can have com-pressed data stored in its resource fork or in the com.apple.decmpfs extended attribute. By default, the --read option will copy data as is—that is, for com-pressed components, it will copy compressed data. You can tell fileXray to transparently uncompress compressed components by additionally specifying the --uncompress option.

See Also

•--cnid CNID

•--exhaustive


•--output OUTPUT_FILE

•--path PATH

•--uncompress

--read COMPONENT

134

-r COMPONENT

data Contents of the data fork.

resource Contents of the resource fork.

xattr:NAME Contents (value) of the extended attribute whose name is NAME.

--readonly

Given a volume that’s currently mounted in read-write mode, attempt to forci-bly remount it in read-only mode.

Unless you know exactly what you are doing, it is not advisable to use this op-tion on the root volume or on a volume that has files open for writing.

See Also

•--device DEVICE

•--volume VOLUME_NAME

--readonly

135

-R

$ sudo fileXray --volume /Volumes/TestHD --readonly/Volumes/TestHD converted from read-write to read-only.

--scavenge

Analyze a journaled HFS+ volume and scavenge for deleted files and folders. Optionally, examine a specific deleted file in detail and if possible, recover it.

The analysis performed by --scavenge can potentially be useful for recovering one or more blocks of recently deleted files. However, such recovery is never guaranteed on HFS+ volumes. In particular, recovery, whether whole or par-tial, will depend on numerous factors specific to the volume in question and how the volume was used after the files of interest were deleted.

By itself, the --scavenge option will display a “succinct” list of any file, folder, and thread records of deleted objects that fileXray finds on the volume. Each line of the output contains comma-separated values whose meanings are la-beled in the following example.

In general, each line of output will contain information on one of four types of Catalog records fileXray found. The type is designated by each line’s first comma-separated value: a single character whose value can be one of the fol-lowing.

The next comma-separated value is a pair of Catalog Node IDs (CNIDs): the parent folder’s CNID and the object’s own CNID. The two CNIDs are separated by a literal slash.

The next three comma-separated values apply only to file records—no values are shown for other record types.

--scavenge

136

-k

F The found record is a Catalog Folder Record.

f The found record is a Catalog File Record.

T The found record is a Catalog Folder Thread Record.

t The found record is a Catalog File Thread Record.

$ fileXray --device /dev/disk1s2 --scavenge…F,2/24,,,,Pictures:TestHD:/Pictures/T,2/24,,,,Pictures:TestHD:/Pictures/…f,24/159,0x52c70,18/18/18,5/5/5,Background.png:[CNID=24]/Background.pngf,24/147,0x51da2,2957/2957/2957,11/11/11,Mouse.tif:[CNID=24]/Mouse.tiff,24/148,0x51eb4,146/146/146,0/0/0,Water.jpg:[CNID=24]/Water.jpg…

Recordtype tag

ObjectCNID

Undeletecookie

Resource forkscavengability

Objectname

Literalcolon

Computed path ofobject on volume

Data forkscavengability

ParentCNID

The third comma-separated value is an “undelete cookie.” fileXray will often find multiple remnant records for a single deleted object. This is a side effect of how HFS+ is implemented and how the journaling mechanism works. For all file records, an opaque value—the undelete cookie—will be displayed for each record. The undelete cookie, which uniquely identifies an instance of a deleted file, is required if you wish to use --scavenge to further examine that specific deleted instance or to recover it. Note that the undelete cookie is a volatile en-tity. If the volume undergoes file system activity, a previously displayed cookie may no longer be valid.

The fifth and sixth comma-separated values are triplets of block number values in the case of file records. The individual blocks numbers are separated by lit-eral slashes. The triplets apply to the data fork and the resource fork, respec-tively. The first value in the triplet is the total number of allocation blocks the fork had—that is, the logical size of the fork. The second value is the number of allocation blocks that can be determined from the file record itself without go-ing to the Overflow Extents B-Tree. (Since the --scavenge option does not cur-rently scavenge overflow extents, this second value is the maximum number of blocks that could be recovered from a deleted file.) The third value, which is less than or equal to the second value, represents the number of fork blocks that are currently known to be free and not be in use by another file. Therefore, the best case for recovery is when all three values in a triplet are identical. However, it is important to realize that even if a file’s erstwhile blocks are cur-rently free, they could have been reused and freed any number of times be-tween now and the time the file was deleted.

The sixth comma-separated value contains the name and possibly the path of the file system object. The name, which will always be displayed, is followed by a literal colon. (An on-disk HFS+ node name cannot contain a colon.) The re-mainder of the line after the colon is the full path—as much as fileXray can compute—of the file system object. (It may not be possible to compute the path in several cases.) In some cases, as in the example being discussed here, a path component, such as a parent folder of a deleted object, may also have been deleted. Such objects are represented as [CNID=<cnid>] in the path. The deleted parent may or may not be present in the --scavenge output depending on the journal’s state. In the above example, the parent folder of the three de-leted files is represented as [CNID=24]. We see that there is a deleted folder with CNID 24: /Pictures.

Additionally, if the --exhaustive option is specified, the output is detailed in-stead of succinct. For each deleted object found, fileXray will display any me-tadata it can gather. In the case of deleted files, fileXray will also display the locations of the data fork and resource fork blocks that belonged to the file be-fore it was deleted. The following is an example of exhaustive output.

--scavenge

137

Note that on an active volume—one that is mounted, and especially the root volume—the journal is very volatile can wrap around rather quickly, overwrit-ing data that this option operates upon. Therefore, on a mounted volume, the information displayed by any option that processes the journal is volatile too, both with and without the --exhaustive option.

From amongst the deleted file instances that are listed by --scavenge, you can introspect a particular instance by specifying the file’s CNID through the

--scavenge

138

$ fileXray --device /dev/disk1s2 --scavenge --exhaustive…# Scavenging a Catalog File Record. path (synthesized) = [CNID=24]/Water name = Water.jpg parentID = 24 fileID = 148 undelete cookie = 0x51eb4 scavengability (data)= 146/146/146 scavengability (rsrc)= 0/0/0 # Catalog File Record Details type = file file ID = 148 flags = 0000000000000010 . File has a thread record in the catalog. reserved1 = 0 createDate = Sun Nov 8 20:53:22 2009 contentModDate = Sun Nov 8 20:53:22 2009 attributeModDate = Sun Nov 8 20:53:22 2009 accessDate = Sun Nov 8 20:53:22 2009 backupDate = 0 # BSD Info ownerID = 501 (johndoe) groupID = 20 (staff) adminFlags = 00000000 ownerFlags = 00000000 fileMode = -rw-r--r-- linkCount = 1 textEncoding = 0 reserved2 = 0 # Finder Info fdType = 0x4a504547 (JPEG) fdCreator = 0x3842494d (8BIM) fdFlags = 0000000001000000 . kIsShared fdLocation = (v = 0, h = 0) opaque = 0 # Data Fork logicalSize = 597757 bytes totalBlocks = 146 extents = startBlock blockCount % of file 0x1525b 0x92 100.00 % # Resource Fork logicalSize = 0 bytes…

--cnid option and the associated undelete cookie through the --undele-te_cookie option.

In the above example, we see that none of the deleted file’s data fork blocks are currently in use by other files. This is highly desirable if we wish to recover the deleted content. However, as noted earlier, the blocks could have been reused and freed—perhaps even multiple times. Therefore, although desirable, this still does not guarantee us that we will, in fact, be able to recover the deleted content.

--scavenge

139

$ fileXray --device /dev/disk1s2 --scavenge --cnid 148 --undelete_cookie 0x51eb4 path (synthesized) = [CNID=24]/Water.jpg name = Water.jpg parentID = 24 fileID = 148 undelete cookie = 0x51eb4 # Catalog File Record Details type = file file ID = 148 flags = 0000000000000010 . File has a thread record in the catalog. reserved1 = 0 createDate = Sun Nov 8 20:53:22 2009 contentModDate = Sun Nov 8 20:53:22 2009 attributeModDate = Sun Nov 8 20:53:22 2009 accessDate = Sun Nov 8 20:53:22 2009 backupDate = 0 # BSD Info ownerID = 501 (johndoe) groupID = 20 (staff) adminFlags = 00000000 ownerFlags = 00000000 fileMode = -rw-r--r-- linkCount = 1 textEncoding = 0 reserved2 = 0 # Finder Info fdType = 0x4a504547 (JPEG) fdCreator = 0x3842494d (8BIM) fdFlags = 0000000001000000 . kIsShared fdLocation = (v = 0, h = 0) opaque = 0 # Data Fork logicalSize = 597757 bytes totalBlocks = 146 extents = startBlock blockCount % of file 0x1525b 0x92 100.00 %

*** None of the deleted file's data fork blocks are CURRENTLY in use by other files, although they could have been reused and freed in the past.

# Resource Fork logicalSize = 0 bytes

Moreover, depending on a multitude of factors such as the size of the volume, the amount of free space it has, the amount and the type of file system activity that occurred on the volume after the file in question was deleted, and so on, we may find that some or perhaps even all of the original blocks are currently in use by other files, and therefore, almost certainly “lost.” The following exam-ple shows a case where almost all of a deleted file’s blocks have been clobbered. Note that fileXray identifies the clobbered blocks.

If --cnid is specified to introspect a deleted file corresponding to a particular CNID but no undelete cookie is specified through --undelete_cookie, fileXray will itself choose one of the deleted instances. Although fileXray will attempt to heuristically choose an “appropriate” instance, the choice may not be the most suitable under complex circumstances.

Finally, in addition to --cnid and --undelete_cookie, the --output option can be specified to actually perform an “undelete” operation—that is, fileXray will retrieve the deleted file’s blocks (both data and resource forks) and save them to the specified output file. Note that as a matter of policy, fileXray will not overwrite an existing file with the output file. Therefore, the output file specified through --output must not already exist.

--scavenge

140

$ sudo fileXray --scavenge --cnid 3292831 --undelete_cookie 0x1ed0fe8 path (synthesized) = MacBookHD:/Users/singh/Library/Safari/History.plist name = History.plist parentID = 301355 fileID = 3292831 undelete cookie = 0x1ed0fe8 # Catalog File Record Details type = file file ID = 3292831 # Data Fork logicalSize = 1056710 bytes totalBlocks = 258 extents = startBlock blockCount % of file 0x2e32d7e 0x102 100.00 % known reused blocks = 0x2e32d7e 0x2e32d7f 0x2e32d80 0x2e32d81 0x2e32d82 0x2e32d83 0x2e32d84 0x2e32d85 0x2e32d86 0x2e32d87 0x2e32d88 0x2e32d89 0x2e32d8a 0x2e32d8b 0x2e32d8c 0x2e32d8d… 0x2e32e76 0x2e32e77 0x2e32e78 0x2e32e79

*** AT LEAST 252 of the deleted file's 258 data fork blocks have been reused since the file was deleted.


By default, fileXray deals with known clobbered blocks by substituting them with zero-filled blocks. That is, if a block that once belonged to the deleted file is known to currently be in use, fileXray will replace that block’s contents with zeros in the output file. You can override this behavior using the --ex-haustive option, in which case fileXray will recover all blocks, including clobbered ones, as is.

Let us walk through an example.

We will use two volumes for this example: a non-root volume mounted at /Volumes/PreciousHD and the root volume. It is not recommended to recover from and to the same volume because we would want to perturb the “from” volume as little as possible—ideally not perturb it at all. Consider the following sequence of operations.

We create a new file on the PreciousHD volume and then “accidentally” delete it. We immediately use fileXray to examine the volume using --scavenge. In this case, the recently deleted file is locatable—both a file record and a file thread record are listed. We also see that all of the file’s 4559 data blocks are addressable. Moreover, none of those blocks are currently allocated to other files.

Let us now examine the deleted record in more detail. To do so, we will add the --cnid and --undelete_cookie options to the previous command line we used.

--scavenge

141

# Create some content, say, by copying a file.$ cp /mach_kernel /Volumes/PreciousHD/mach_kernel$ ls -lh /Volumes/PreciousHD/mach_kernel-rw-r--r--@ 1 johndoe wheel 18M Nov 2 10:48 /Volumes/PreciousHD/mach_kernel$ shasum /Volumes/PreciousHD/mach_kernelc8a709cf54f5847974c09a376f592950193f0540 /Volumes/PreciousHD/mach_kernel

# Now delete the file.$ rm /Volumes/PreciousHD/mach_kernel

# Use the --scavenge option and see if our deleted file shows up in its output.$ fileXray --volume /Volumes/PreciousHD --scavenge…f,2/217,0xe0164,4559/4559/4559,0/0/0,mach_kernel:TestHD:/mach_kernelt,2/217,,,,mach_kernel:TestHD:/mach_kernel…

$

The situation looks promising. Let us now add the --output option to recover the deleted blocks from PreciousHD and copy them to the root volume.

--scavenge

142

# Examine the killed record for the file of interest in more detail.$ fileXray --volume /Volumes/PreciousHD --scavenge --cnid 217 \ --undelete_cookie 0xe0164 path (synthesized) = TestHD:/mach_kernel name = mach_kernel parentID = 2 fileID = 217 undelete cookie = 0xe0164 # Catalog File Record Details type = file file ID = 217 flags = 0000000000000010 . File has a thread record in the catalog. reserved1 = 0 createDate = Mon Nov 9 14:51:49 2009 contentModDate = Mon Nov 9 14:51:50 2009 attributeModDate = Mon Nov 9 14:51:50 2009 accessDate = Mon Nov 9 14:51:49 2009 backupDate = 0 # BSD Info ownerID = 501 (johndoe) groupID = 20 (staff) adminFlags = 00000000 ownerFlags = 00000000 fileMode = -rw-r--r-- linkCount = 1 textEncoding = 0 reserved2 = 0 # Finder Info fdType = 0 fdCreator = 0 fdFlags = 0100000000000000 . kIsInvisible fdLocation = (v = 0, h = 0) opaque = 0 # Data Fork logicalSize = 18672224 bytes totalBlocks = 4559 extents = startBlock blockCount % of file 0x18764 0x11cf 100.00 %



# Recover the deleted blocks.$ fileXray --volume /Volumes/PreciousHD --scavenge --cnid 217 \ --undelete_cookie 0xe0164 --output /tmp/mach_kernel18672224 bytes copied from the data fork.

# Compare the recovered content with the original content we started with.$ shasum /mach_kernel /tmp/mach_kernelc8a709cf54f5847974c09a376f592950193f0540 /mach_kernelc8a709cf54f5847974c09a376f592950193f0540 /tmp/mach_kernel

Caveats

In this example, circumstances worked in our favor and we were able to do a perfect recovery. In general, for a perfect recovery to occur, at least the follow-ing conditions must be met.

• The deleted record must still exist in a journal transaction for fileXray to identify the deleted file and locate its erstwhile extents. Of course, the volume must be journaled to begin with.

• The erstwhile extents must be at most 8 in number. If a file is severely fragmented and has more than 8 extents (fragments), the 9th and sub-sequent extents will not be located by this process. A future version of fileXray may add support for searching for more extents.

• The erstwhile extents must both be currently free and must not have been overwritten at some time in the past after the file in question was deleted. In fact, if you save the recovered file to the same volume as you are recovering from, it is possible for the lost data to be clob-bered as we are trying to recover it. Therefore, it is not advised to re-cover from a volume to the same volume.

The volume from which you are attempting to recover should be kept as dor-mant as possible in terms of file system activity. Ideally, no file system activity should occur on the volume after the file in question is deleted.

See Also

•--cnid CNID

•--exhaustive

•--journal_names

•--trawl QUERY


--scavenge

143

--summary FORK_TYPE

Calculate usage statistics about the volume and display a summary.

FORK_TYPE must be one of the following.

The FORK_TYPE specification comes into play when the --top option is used to specify the number of the largest fork sizes that the --summary computation will remember. If the --top option is not specified, FORK_TYPE has no user-visible effect.

If a number N is specified through the --top option, then the summary infor-mation displayed will include a list of the top N largest fork sizes found on the volume. In other words, you can use the --summary and --top options to-gether to display the N largest files (by data fork size, resource fork size, or combined fork sizes) on the volume.

The following excerpt shows the use of the --summary option to identify the largest data and resource forks on a volume.

The example on the following page shows the entire output of --summary on a root volume. We have also specified --top to list the top 10 largest files on the volume. The paths in the illustration have been shortened to fit the page.

See Also


•--top N

--summary FORK_TYPE

144

-s FORK_TYPE

data Summary will have information on the largest data forks.

resource Summary will have information on the largest resource forks.

any Summary will have information on the largest (data + resource) forks.

$ sudo fileXray --summary resource --top 1…# The largest resource fork on the Volumerank size cnid path1 17 MB 1699734 MacHD:/System/Library/Frameworks/AppKit.framework/\ Versions/C/AppKit/..namedfork/rsrc

$ sudo fileXray --summary data --top 1…# The largest data fork on the Volumerank size cnid path1 12 GB 650957 MacHD:/Users/johndoe/Documents/\ Virtual Machines.localized/\ Mac OS X 10.4.vmwarevm/Mac OS X 10.4-flat.dmg

--summary FORK_TYPE

145

$ sudo fileXray --summary any --top 10# Volume Summary Information folders = 270627 files (non-folder objects) = 1427405 directory hard links = 0 hard links = 5750 symbolic links = 20711 folder aliases = 4 file aliases = 0 invisible files = 417 files with both forks empty = 115816 truly empty files (no xattrs even)= 8967 block/character special files = 0 fifo files = 2 socket files = 49 # Data Forks non-zero data forks = 1286025 fragmented data forks = 1231 data forks with > 8 fragments = 97 allocation blocks used = 42283229 allocated storage = 173192105984 bytes (173 GB) actual usage = 169696268962 bytes (170 GB) total extent records = 1286943 total extent descriptors = 1295701 overflow extent records = 918 overflow extent descriptors = 6970 # Resource Forks non-zero resource forks = 72718 fragmented resource forks = 6 resource forks with > 8 fragments = 0 allocation blocks used = 475404 allocated storage = 1947254784 bytes (1.9 GB) actual usage = 1727969304 bytes (1.7 GB) total extent records = 72718 total extent descriptors = 72725 overflow extent records = 0 overflow extent descriptors = 0

47154 files have content in both their data and resource forks.

# The 10 largest files (sum of data and resource forks) on the Volumerank size cnid path10 2.1 GB 1743913 MacHD:/…Mac OS X 10.6 Server-s004.vmdk9 2.1 GB 674452 MacHD:/…Virtual Disk-s002.vmdk8 2.1 GB 1743911 MacHD:/…Mac OS X 10.6 Server-s002.vmdk7 2.1 GB 674019 MacHD:/…Mac OS X 10.5-s008.vmdk6 3.2 GB 684630 MacHD:/OPENSTEP-flat.vmdk5 4.3 GB 205545 MacHD:/private/var/vm/sleepimage4 4.5 GB 2614539 MacHD:/private/tmp/2009-11-02-www.tar.bz23 4.6 GB 2616395 MacHD:/private/tmp/2009-11-14-www.tar.bz22 8.6 GB 702033 MacHD:/…Windows 2000 Professional-flat.vmdk1 12 GB 650957 MacHD:/…Mac OS X 10.4-flat.dmg

--top N

When used with another fileXray option, specifies the number N to be used for the top N files to be displayed in the context-specific categories.

The behavior when used with specific options is as follows.

See Also


•--hotfiles

•--summary FORK_TYPE

--top N

146

-t N

--fragmentation Shows the top N most fragmented files by virtue of the fragmentation of their data forks and/or resource forks. Whether only forks of one type are considered depends on the argument to --fragmentation.

--hotfiles Shows the top N “hottest” files on the volume provided the volume uses Hot File Clustering.

--summary Shows the top N largest files by virtue of the sizes of their data forks, resource forks, or the sums of both forks depending on the argument to --summary.

--trawl QUERY

Trawl the volume looking for blocks that match “magic” patterns (signatures) contained in the query file QUERY.

This option uses the same “magic” mechanism that underlies the file com-mand. (See magic(5) for an introduction. The /usr/share/file/magic/ di-rectory on Mac OS X contains numerous magic pattern files.)

By default, when this option is specified, fileXray will scan the free extents of the given volume one allocation block at a time. It will use the magic pattern(s) from the QUERY file to match against each block. If the pattern matches a block, fileXray will print the block’s byte offset on the volume along with a description of the specific pattern it matched.

This way, you can trawl the volume looking for, say, PDF documents or JPEG images. You can use any of the pattern files found in /usr/share/file/magic/, which cumulatively contain patterns to identify a very large number of file types. You can also concatenate two or more pattern files to provide a larger pattern set. Moreover, you can create your own patterns as long as they use the format described in magic(5). In the following example, the match indi-cates that byte offset 0x3ad000 on the volume marks the beginning of a PDF document.

The most convenient way to make use of trawling results is through the built-in Arbitrary File System, which allows you to access arbitrary byte ranges on the volume as on-the-fly files.

You can adjust the trawling behavior in a few ways.

By default, fileXray will trawl only the free blocks on the volume. That is, the --trawl operation searches free space, some or all of which may have been oc-cupied by files now deleted, for patterns. To make --trawl search the entire volume, including currently used blocks, additionally specify the --exhaus-tive option.

By default, fileXray will attempt to match each pattern contained in the QUERY file against some appropriate number of bytes at the beginning of each free allocation block—or each allocation block if --exhaustive is specified. This is usually sufficient if you are looking for “normal” (for example, not transparently compressed) files because a normal file’s beginning will coincide with its first allocation block’s beginning. If, however, you wish to look for con-

--trawl QUERY

147

-q QUERY

# Trawl the free extents on PreciousHD looking for PDF documents.$ fileXray --volume /Volumes/PreciousHD --trawl /usr/share/file/magic/pdf…0x3ad000 PDF document, version 1.6…

tent within a file, you can use the --block option to specify a smaller “stride” size, which will cause fileXray to attempt pattern matching against smaller blocks. Consider a contrived example. Suppose you are looking for a PDF file embedded at a 512-byte offset within some container document. That is, the PDF file starts at an offset of 512 bytes within the on-disk file. In this case, --trawl would not find such a PDF because by default, it would attempt pat-tern matching with a stride size that’s the same as the volume’s allocation block size, which in turn is 4096 bytes by default. We can use the --block op-tion to specify a stride size of 512 bytes, which would cause --trawl to exam-ine the volume in units of 512 bytes1.

Let us consider another example. Suppose we wish to look for pictures in some common image file formats—GIF, JPEG, PNG, TIFF, etc.—within the free blocks of a volume. The standard pattern file /usr/share/file/magic/images con-tains several predefined patterns to suit our need. We can combine that pattern file with another standard file /usr/share/file/magic/jpeg to get a bigger pattern set. The following example shows how to use --trawl to perform the appropriate search.

We see several matches in the output. (Note that it is entirely possible to get false positives if some random content simply happens to match a pattern.) As noted earlier, the easiest and nicest way of accessing these results is to use the Arbitrary File System. Once mounted, the Arbitrary File System allows you to create virtual file names on the fly wherein the file’s byte offset and size are en-coded in the name itself. Moreover, these virtual file names can have arbitrary file extensions. For example, when a file called 0x1000,65536.gif is accessed in an Arbitrary File System mount, the file’s content will start at byte offset

--trawl QUERY

148

1 This means the trawling operation would examine 8 times more data than the default case. Naturally, the operation would be slower.

$ fileXray --device /dev/disk0s2 --trawl /usr/share/file/magic/pdf$ fileXray --device /dev/disk0s2 --trawl /usr/share/file/magic/pdf --block 5120x1003200 PDF document, version 1.6$

# Create a reasonably complete pattern set for what we need.$ cat /usr/share/file/magic/images /usr/share/file/magic/jpeg > /tmp/mypatterns

$ fileXray --volume /Volumes/PreciousHD --trawl /tmp/mypatterns…0x27d000 PCX ver. 2.5 image data0x3ad000 JPEG image data, JFIF standard 1.020x4ad000 GIF image data, version 89a, 1800 x 18000x5a8000 TIFF image data, big-endian0x841000 PNG image, 1024 x 768, 8-bit/color RGBA, non-interlaced0x102f000 Targa image data - RGB0x1917000 PCX ver. 2.5 image data…

0x1000 on the volume, the content’s size will be 65536 bytes, and the .gif extension will allow it to be conveniently opened in an image viewer.

Caveats

• It is important to realize that a match does not mean that an entire document has been located. After all, a file is not guaranteed to be contiguous on disk. In fact, besides being discontiguous, it could even have its blocks scattered over the disk in a manner such that its latter blocks are at lower byte offsets on the volume than its earlier blocks. That said, on a typical real-life HFS+ volume that has not been filled up or nearly filled up to its capacity in the recent past, HFS+ does an admirable job of keeping most files contiguous on disk. Therefore, the --trawl option can be quite effective in most cases.

• When trawling free extents of a volume, there is another potential is-sue besides block contiguity: depending on how long ago the file in question was deleted and how the volume has been used since, it is possible for some number of the file’s blocks to have been reallocated and overwritten—perhaps even multiple times.

• This operation neither determines nor approximates any size for the matched content. In many contexts, this may not be a problem since overestimating the file size “works” for many file formats. If a precise size is required in some circumstances, the Arbitrary File System makes it easy to experiment.

• The --trawl operation may take a “long” time to finish. Depending on the size of a volume, the type and speed of the underlying storage de-vice, the fragmentation of the free space being searched, and the number and types of patterns being matched, the time taken by --trawl may vary. In the worst cases, --trawl may consume up to a few minutes per Gigabyte of storage trawled.

--trawl QUERY

149

# Mount the volume using the Arbitrary File System.$ mkdir /tmp/arbitrary$ fileXray --volume /Volumes/PreciousHD --userfs_type arbitrary \ --userfs_mount /tmp/arbitrary

# Now access trawling results “directly” using the Arbitrary File System.

# Byte offset 0x3ad000 is supposed to be the beginning of a JPEG file. Let# us use some reasonably large size, say, 1 MB. We can automagically open# the JPEG “file” as follows.$ open /tmp/arbitrary/0x3ad000,1048576.jpg…

# Similarly, we can open other files. $ open /tmp/arbitrary/0x841000,1048576.png…

See Also

•--block BLOCK

•--scavenge


--trawl QUERY

150

--uncompress

Used along with the --read option to tell fileXray to transparently uncom-press the output when reading a component that is compressed on disk.

If the specified component is not actually compressed on disk, this option is a no-op.

Beginning with version 10.6 (Snow Leopard), Mac OS X supports transparent file compression at the file system level. Several system components are com-pressed by default. Depending on the file’s size and potentially other factors, the compressed data resides either in the resource fork or inline within an ex-tended attribute. In the case of such a compressed file, the --read option cop-ies the resource fork or extended attribute as is, that is, compressed. Addition-ally specifying --uncompress causes fileXray to uncompress the resource fork or extended attribute and save the resultant content to the destination file.

The following is an example of a transparently compressed file whose com-pressed content is stored inline in an extended attribute.

--uncompress

151

-z

$ ls -las /etc/smb.conf.template0 -rw-r--r-- 1 root wheel 2955 May 21 22:06 /etc/smb.conf.template

$ sudo fileXray /etc/smb.conf.template… ownerFlags = 00100000 . UF_COMPRESSED (file is hfs-compressed)… # Data Fork logicalSize = 0 bytes # Resource Fork logicalSize = 0 bytes…# Attribute Key keyLength = 46 pad = 0 fileID = 97954 startBlock = 0 attrNameLen = 17 attrName = com.apple.decmpfs# Attribute Data Record (Inline)# Record 0 in node 5882 beginning at 512-byte sector 0x2a9d8 recordType = 0x10 reserved[0] = 0 reserved[1] = 0 attrSize = 1354 bytes attrData = 66 70 6d 63 03 00 00 00 8b 0b 00 00 00 00 00 00 f p m c … A k o i

compression magic = cmpf compression type = 3 (xattr has compressed data) uncompressed size = 2955 bytes

We see that even though the operating system reports the file as being 2955 bytes in size, the file actually has zero-sized data and resource forks on disk. The file’s real content, which is compressed, is stored inline within an extended attributed named com.apple.decmpfs. fileXray displays the details of the extended attribute. Using the --read option, we can retrieve the content of this extended attribute and save it to a file.

We see that the retrieved content’s size does not match the file’s advertised size. We can now tell fileXray to also uncompress the content as it retrieves it.

Let us now look at a transparently compressed file whose compressed content is stored in its resource fork. The following page shows the example of /bin/zsh. We see that even though the file has an advertised data fork size of 1.6 MB, the on-disk size of its data fork is zero. Instead, the file has compressed content in its resource fork, which is 803 KB in size. In this case too, there is an extended attribute named com.apple.decmpfs, although it’s used only for compression bookkeeping and not for any compressed data.

Again, we can use --uncompressed along with --read to retrieve the com-pressed content—this time from the resource fork—and save it uncompressed to a file.

See Also

•--read COMPONENT

--uncompress

152

$ sudo fileXray /etc/smb.conf.template --read xattr:com.apple.decmpfs \ --output /tmp/smb.conf.template.z$ ls -asl /tmp/smb.conf.template.z8 -rw------- 1 johndoe staff 1354 Nov 2 17:52 smb.conf.template.z

$ sudo fileXray /etc/smb.conf.template --read xattr:com.apple.decmpfs \ --uncompress --output /tmp/smb.conf.template$ ls -asl /tmp/smb.conf.template8 -rw------- 1 johndoe staff 2955 Nov 2 17:56 smb.conf.template

$ shasum /etc/smb.conf.template /tmp/smb.conf.template16b8c3748fe56c9a9a3417f8eab3f1583b139cc8 /etc/smb.conf.template16b8c3748fe56c9a9a3417f8eab3f1583b139cc8 /tmp/smb.conf.template

--undelete_cookie COOKIE

Specify a “cookie” value for use by the --scavenge option.

The --scavenge option, which can be used to search for deleted file system objects on a volume, will often find multiple remnants for a single deleted ob-ject. This is a side effect of how HFS+ is implemented and how the journaling mechanism works. When such objects are listed by --scavenge (in either suc-cinct mode or exhaustive mode of output), an opaque value, which fileXray refers to as an “undelete cookie,” will also be displayed alongside each object. The following excerpts give examples of these output formats.

Subsequently, you can use the --scavenge option to further examine a given deleted file in more detail by specifying the object’s Catalog Node ID (CNID). Moreover, you can use --scavenge to “undelete” a deleted file if its blocks are recoverable. Both of these operations (further examination and block recovery) require the target file system object to be specified through the --cnid option. Given that multiple instances of a deleted CNID can be found, fileXray must somehow choose an instance. This is where the undelete cookie comes in— each instance will have a unique undelete cookie.

Depending on the file system activity that occurred, the various instances may or may not differ in ways that are relevant to scavenging—some instances may be more recoverable than the others. If you wish to have fine control over the scavenging operation, you can use --undelete_cookie to address a particular instance of a deleted file system object.


153

$ fileXray --device /dev/disk1s2 --scavenge…f,24/159,0x52c70,18/18/18,5/5/5,Background.png:[CNID=24]/Background.pngf,24/147,0x51da2,2957/2957/2957,11/11/11,Mouse.tif:[CNID=24]/Mouse.tiff,24/148,0x51eb4,146/146/146,0/0/0,Water.jpg:[CNID=24]/Water.jpg…

$ fileXray --device /dev/disk1s2 --scavenge --exhaustive…# Scavenging a Catalog File Record. path (synthesized) = [CNID=24]/Water name = Water.jpg parentID = 24 fileID = 148 undelete cookie = 0x51eb4 scavengability (data)= 146/146/146 scavengability (rsrc)= 0/0/0 # Catalog File Record Details type = file file ID = 148

-u COOKIE

UndeleteCookies

If no instance is explicitly specified through --undelete_cookie, fileXray will automatically choose an instance. fileXray’s choice may not always be a good one.

The following is an example of using the --undelete_cookie option while re-covering a deleted file.

See Also

•--scavenge


154

# List scavengable objects.$ fileXray --device /dev/disk1s2 --scavenge…f,24/148,0x51eb4,146/146/146,0/0/0,Water.jpg:[CNID=24]/Water.jpg…

# Examine one of the objects.$ fileXray --device /dev/disk1s2 --scavenge --cnid 148 --undelete_cookie 0x51eb4… # Data Fork logicalSize = 597757 bytes totalBlocks = 146 extents = startBlock blockCount % of file 0x1525b 0x92 100.00 %



# Looks promising. Maybe we will be in luck. Undelete.$ fileXray --device /dev/disk1s2 --scavenge --cnid 148 \ --undelete_cookie 0x51eb4 --output /tmp/Water.jpg597757 bytes copied from the data fork.

--usedspace

Display all used extents on the volume.

For each used extent, fileXray displays the following information: the number of allocation blocks in that extent, the starting and ending block numbers in hexadecimal, and the amount of associated used space.

You can pipe the output through “sort -n” to view a list of contiguous used space chunks sorted by chunk size. In the following example, we see that the largest contiguous chunk on the volume has 1,550,870 allocation blocks. The chunk starts at allocation block number 0x85518a and ends at allocation block number 0x9cfb9f. Given the allocation block size of 4096 bytes, this chunk amounts to 6.4 GB.

Another way to visualize used extents on a volume is to use the built-in used-space virtual file system, which can be accessed through the --userfs_type option.

See Also

•--freespace




--usedspace

155

-1

$ sudo fileXray --usedspace | sort -n# Allocation block size = 4096 bytes# Allocation blocks total = 60965668 (0x3a24324)# Allocation blocks used = 42582081 (0x289c041)# Used Contiguous Starting @ Ending @ Used Space 1 0xd8227 0xd8227 4.1 KB 1 0x18a213 0x18a213 4.1 KB … 980864 0x1bf375a 0x1ce2ed9 4.0 GB 1037387 0x10b4d5b 0x11b21a5 4.2 GB 1048576 0x283455 0x383454 4.3 GB 1550870 0x85518a 0x9cfb9f 6.4 GB

--userfs_mount MOUNT_POINT

Specify the mount point for a built-in synthetic file system.

This option is used along with the --userfs_type option to make certain types of volume information accessible as user-space virtual file systems. The volume of interest must also be specified through other valid options.

The --userfs_type section of this document provides more details and exam-ples.

See Also

•--userfs_type USERFS_TYPE

--userfs_mount MOUNT_POINT

156

-M MOUNT_POINT

--userfs_type USERFS_TYPE

Specify the built-in synthetic file system to use.

This option is used along with the --userfs_mount option to make certain types of volume information accessible as user-space virtual file systems. The volume of interest must also be specified through other valid options.

fileXray includes the following built-in read-only virtual file systems.

Arbitrary File System

The Arbitrary File System contains no visible files by default. However, when you attempt to access a file whose name encodes a starting offset and a size, the corresponding content will be transparently made available through that file. The specific naming format is as follows.

[-]START_BYTE,SIZE_IN_BYTES[.extension]

For example, if you attempt to open a file called 0x5000,4096.txt, you will “see” a file whose contents come from the HFS+ volume’s on-disk byte range that starts at byte offset 0x5000 and is 4096 bytes in size. Optionally, START_BYTE can be negative, in which case the starting offset is relative to the end of the volume. For example, -0x5000,4096.txt would read 4096 bytes starting at an offset that’s 0x5000 bytes before the end of the volume. You can also specify multiple extents using the colon character as the separator. For example: 0x5000,4096:0x9000,4096:0xc000,4096.txt.

The Arbitrary File System is particularly useful for accessing the results of the --trawl operation.

Consider the following example. We can use the Arbitrary File System to mount an HFS+ volume on, say, /tmp/arbitrary. We know that the Volume Header, which is 512 bytes in size, resides at an offset of 1024 bytes from the beginning of the volume. The Alternate Volume Header, which is a reasonably-in-sync copy of the Volume Header, resides at an offset of 1024 bytes from the end of the volume. Given that the Arbitrary File System lets us access arbitrary byte ranges with either positive or negative offsets, we can read a few bytes from


157

-U USERFS_TYPE

arbitrary Arbitrary byte ranges on the can be accessed through files.

freespace Free space (extents) on the volume can be accessed through files.

scavenger Scavengable deleted content can be accessed through files.

structure Volume’s internal data structures can be accessed through files.

usedspace Used space (extents) on the volume can be accessed through files.

these two data structures and see if we get expected values for the volume sig-nature and the last mounted version signature.

Next, we can create a file with some content, look up the newly created file’s ex-tents, and try to access the content through the Arbitrary File System. The fol-lowing example illustrates this.


158

# Create a mount point.$ mkdir /tmp/arbitrary

# Use the Arbitrary File System to mount the root volume.$ sudo fileXray --userfs_type arbitrary --userfs_mount /tmp/arbitrary$ ls -las /tmp/arbitrarytotal 00 drwxr-xr-x 2 root wheel 0 Nov 2 17:35 .0 drwxrwxrwt 33 root wheel 1122 Nov 2 18:22 ..$

# Look at the first 12 bytes of the Volume Header, which is 512 bytes in size# and resides at an offset of 1024 bytes from the beginning of the volume.$ hexdump -n 12 -xc /tmp/arbitrary/1024,5120000000 2b48 0400 0080 0020 4648 4a53 0000000 H + \0 004 200 \0 \0 H F S J 000000c

# Look at the first 12 bytes of the Alternate Volume Header, which resides# at an offset of 1024 bytes from the end of the volume.$ hexdump -n 12 -xc /tmp/arbitrary/-1024,5120000000 2b48 0400 0080 0020 4648 4a53 0000000 H + \0 004 200 \0 \0 H F S J 000000c HFS+ Volume

SignatureLast Mounted

Version

# Create a file with some content.$ echo "This is some text." > /tmp/file.txt$ ls -l /tmp/file.txt-rw-r--r-- 1 johndoe wheel 19 Nov 2 02:05 /tmp/file.txt

# Determine the newly created file’s extents.$ sudo fileXray /tmp/file.txt… extents = startBlock blockCount % of file

0x2e9ccbc 0x1 100.00 %

1 allocation blocks in 1 extents total.…

# Allocation block size for this volume is 4096 bytes. Therefore, block# number 0x2e9ccbc amounts to byte offset 0x2e9ccbc000. We can use the# Arbitrary File System to read 19 bytes from this offset.$ cat /tmp/arbitrary/0x2e9ccbc000,19.txtThis is some text.

Given the general utility of the Arbitrary File System, fileXray allows you to mount non-HFS+ entities such as other types of volumes and even regular files. To do so, you must additionally specify the --force option. This mecha-nism provides a convenient way to access, introspect, and analyze arbitrary parts of volumes and files.

The following example shows mounting a Mach-O executable using the Arbi-trary File System. Once mounted, the well defined constituents of the file can be “directly” accessed by addressing them through file names consisting of the constituents’ offsets and sizes.

Note that the Arbitrary File System does not cache on-disk content. Therefore, reading its virtual files will retrieve the “latest” data.


159

# Create a mount point.$ mkdir /tmp/arbitrary

# Use the Arbitrary File System to mount a file instead of an HFS+ volume.# The --force option will enable such mounting despite the warning.$ sudo fileXray --userfs_type arbitrary --userfs_mount /tmp/arbitrary \ --device /mach_kernel --forceNo flavor of HFS+ found.$

$ ls -l /mach_kernel-rw-r--r--@ 1 root wheel 18672224 Jul 31 22:49 /mach_kernel$ df -h /tmp/arbitraryFilesystem Size Used Avail Capacity Mounted onfileXray@fuse0 18Mi 18Mi 0Bi 100% /private/tmp/arbitrary$

# Look for something interesting in the file. For example, the __cstring# section in the text segment of this Mach-O file.$ otool -l /mach_kernel… sectname __cstring segname __TEXT addr 0x005832c8 size 0x00057def offset 3683016…

# Access the __cstring section “directly” as an on-the-fly file through# the Arbitrary File System.$ hexedit /tmp/arbitrary/3683016,0x57def00000000 68 28 25 73 25 64 29 20 69 66 6E 65 74 5F 64 65 h(%s%d) ifnet_de00000010 74 61 63 68 5F 70 72 6F 74 6F 63 6F 6C 20 66 61 tach_protocol fa00000020 69 6C 65 64 2C 20 25 64 0A 00 25 73 25 64 3A 20 iled, %d..%s%d:00000030 25 73 20 77 61 6B 65 75 70 0A 00 00 00 69 66 5F %s wakeup....if_…

Free Space File System

The Free Space File System contains files representing free extents on a given volume. The idea is to isolate free space in easy-to-read contiguous chunks, which makes searching through free space much more convenient and faster in most cases.

The top-level freespace directory in the file system contains one or more vir-tual subdirectories whose names are of the format X_Y. X is simply a mono-tonically increasing decimal number starting at 0. Y represents a block number in hexadecimal, which is the starting block number of the first extent within the directory.

Inside each such directory named X_Y, there are at most 1024 virtual files —a new directory is created after the previous one is populated with 1024 files. Each file represents a free extent—that is, a range of contiguous free blocks. Each file’s name is of the form U_V. U is the extent’s starting block number and V is the number of blocks in the extent. Both U and V are represented in hexa-decimal. As noted earlier, the value of U for the first extent contained within the X_Y directory is the same as the value of Y.

Reading from such a file will return data from the on-disk blocks the file repre-sents. Note that the free extents are calculated at the time the Free Space File System is mounted. If the volume in question is mounted and the set of free ex-tent changes while the Free Space File System is mounted, the directory tree representing free extents will not update.


160

# Create a mount point.$ mkdir /tmp/freespace

# Use the Free Space File System to mount the root volume.$ sudo fileXray --userfs_type freespace --userfs_mount /tmp/freespace$ ls -las /tmp/freespace/freespace/total 00 drwxr-xr-x 38 root wheel 0 Nov 2 20:58 .0 drwxr-xr-x 3 root wheel 0 Nov 2 20:58 ..0 dr-xr-xr-x 1026 root wheel 0 Nov 2 20:58 00000000_00014d870 dr-xr-xr-x 1026 root wheel 0 Nov 2 20:58 00000001_003e47720 dr-xr-xr-x 1026 root wheel 0 Nov 2 20:58 00000002_004e8ae80 dr-xr-xr-x 1026 root wheel 0 Nov 2 20:58 00000003_005507830 dr-xr-xr-x 1026 root wheel 0 Nov 2 20:58 00000004_005b80230 dr-xr-xr-x 1026 root wheel 0 Nov 2 20:58 00000005_00bda9bd…0 dr-xr-xr-x 1026 root wheel 0 Nov 2 20:58 00000034_02d663100 dr-xr-xr-x 389 root wheel 0 Nov 2 20:58 00000035_02ec061a

Starting block numberof the first extent withinthis directory

Monotonically increasingnumber (index)

However, the Free Space File System does not cache on-disk content. There-fore, reading its virtual files will in fact retrieve the “latest” data. The following shows the last few contents of the last X_Y directory.

Structure File System

The Structure File System exposes the low-level structure of an HFS+ volume through a set of contiguous virtual files. Depending on the structure of the spe-cific volume mounted through this file system, fileXray will make available the following “files,” not all of which may be present on all volumes.

• The Volume Header.• The Alternate Volume Header.• The Journal Info Block.• The Journal File.• The Allocation File.• The Attributes B-Tree.• The Catalog B-Tree.• The Extents Overflow B-Tree.• The Hot Files B-Tree.• The Startup File.

Some of these entities, such as the Volume Headers, will have constant sizes, whereas the sizes of others will vary from volume to volume. Although the vir-tual files are conveniently contiguous, which is one of the primary benefits of this file system, the corresponding on-disk content for several of the entities is not contiguous.

Note that the Structure File System does not cache on-disk content. Therefore, reading its virtual files will retrieve the “latest” data for the files’ on-disk ex-tents. However, these on-disk extents are established at the time the Structure File System is mounted. Therefore, if the volume in question is live (mounted) and the extents change while the Structure File System is mounted, the changes will not be reflected in the latter.


161

$ ls -asl /tmp/freespace/freespace/00000035_02ec061a… 848 -rw-r--r-- 1 root wheel 424K Nov 2 21:18 02f0db4e-02f0dbb7 88 -rw-r--r-- 1 root wheel 44K Nov 2 21:18 02f0dc69-02f0dc73 2272 -rw-r--r-- 1 root wheel 1.1M Nov 2 21:18 02f0dc77-02f0dd92 114176 -rw-r--r-- 1 root wheel 56M Nov 2 21:18 02f0dd94-02f11553 530184 -rw-r--r-- 1 root wheel 259M Nov 2 21:18 02f11556-02f21836 16608 -rw-r--r-- 1 root wheel 8.1M Nov 2 21:18 02f21840-02f2205b92345904 -rw-r--r-- 1 root wheel 44G Nov 2 21:18 02f2205d-03a24322$

The following example shows the contents of a volume mounted through the Structure File System.

The Structure File System can be put to several uses. It allows you to visualize the storage space consumed purely by a volume’s “metadata.” It also allows you to conveniently introspect and search specific well-defined components of an HFS+ volume. For example, using a version of the strings command with support for 16-bit big endian encoding (the GNU strings program from the binutils package has such support), you can search for HFS+ node and at-tribute names within the Catalog and Attributes files, respectively.

Used Space File System

The Used Space File System contains files representing used extents on a given volume. Its purpose and contents are similar to that of the Free Space File Sys-tem, except that we are dealing with used extents instead of free extents.


162

# Create a mount point.$ mkdir /tmp/structure

# Use the Structure File System to mount the root volume.$ sudo fileXray --userfs_type structure --userfs_mount /tmp/structure$ ls -las /tmp/structuretotal 2274368 0 drwxr-xr-x 11 root wheel 0 Nov 2 21:52 . 0 drwxrwxrwt 34 root wheel 1156 Nov 2 21:52 .. 512 -rw-r--r-- 1 root wheel 262144 Nov 2 21:52 .hotfiles.btree 49152 -rw-r--r-- 1 root wheel 25165824 Nov 2 21:52 .journal 8 -rw-r--r-- 1 root wheel 180 Nov 2 21:52 .journal_info_block 14888 -rw-r--r-- 1 root wheel 7622656 Nov 2 21:52 Allocation 8 -rw-r--r-- 1 root wheel 512 Nov 2 21:52 AlternateVolumeHeader 602112 -rw-r--r-- 1 root wheel 308281344 Nov 2 21:52 Attributes1591296 -rw-r--r-- 1 root wheel 814743552 Nov 2 21:52 Catalog 16384 -rw-r--r-- 1 root wheel 8388608 Nov 2 21:52 ExtentsOverflow 8 -rw-r--r-- 1 root wheel 512 Nov 2 21:52 VolumeHeader$

$ /usr/local/bin/strings --versionGNU strings (GNU Binutils) 2.19.1…

$ /usr/local/bin/strings --encoding=b /tmp/structure/Attributes…com.apple.quarantinecom.apple.decmpfs…

Scavenger File System

The Scavenger File System provides a semi-hierarchical view of files that fileXray is able to scavenge from a journaled volume. The Scavenger File System performs a scavenging operation similar to the --scavenge option. It attempts to reconstruct the names and contents—both data and resource forks—of deleted files. It only selects at least one of whose forks is non-zero. That is, a file with both its data and resource forks empty is not considered. The file system also reconstructs one level of hierarchy: the names (when avail-able) and Catalog Node IDs (CNIDs) of the parent folders of the deleted files.

While scavenging, fileXray will usually find multiple instances of the meta-data remnants of files and folders. Depending on the file system activity that occurred on the volume, these remnants may or may not differ in ways that are relevant to scavenging. For example, remnants could differ only in one times-tamp, or they may differ in file size or extents. The --scavenge option displays all remnants it finds—it associates an “undelete cookie” with each instance to differentiate between the remnants and to allow you to select a particular in-stance to analyze or “undelete.” The Scavenger File System also shows all file remnants by placing distinct remnants in parent folders that are named after the respective files’ undelete cookies. The file system layout is as follows.

The Scavenger File System’s root level contains only synthetic subfolders whose names adhere to one of the following formats. (The italicized part of the format is literal, whereas the bold part is a parameter.)

The parentID_PCNID,NAME and parentID_PCNID folders each contain one or more COOKIE_CNID subfolders—one subfolder for each instance of a scavenged file. Each COOKIE_CNID subfolder a synthetic scavenged file instance that re-tains its original name. Reading such a file will return scavenged content from the appropriate file fork—data or resource. (The resource fork is contained in the com.apple.ResourceFork extended attribute of the scavenged file.) For


163

parentID_PCNID,PNAME PCNID will be replaced by the decimal CNID of the parent folder of a scavenged file. PNAME will be replaced by the parent folder’s name.

parentID_PCNID PCNID will be replaced by the decimal CNID of the parent folder of a scavenged file. This format is used when the parent folder’s name could not be scavenged.

COOKIE_CNID This format is used when the scavenged file’s parent folder is the root folder. COOKIE will be replaced by the hexadeci-mal undelete cookie of a scavenged file instance. CNID will be replaced by the file’s decimal CNID. If no scavenged file resided (before it was deleted) in the volume’s root folder, there will be no entries with this format at the root level of the Scavenger File System.

each block of a fork, the Scavenger File System will serve the actual on-disk block only if that block is currently free in the volume. If the block is currently allocated, some or all of its original content may have been clobbered. By de-fault, the Scavenger File System will substitute a clobbered block with a zero-filled block. However, depending on the file type and your scavenging goal, even partial information might be useful. To override the zero-filled substitution be-havior, you can additionally specify the --exhaustive option, in which case the Scavenger File System will serve on-disk blocks without checking if they are free or not.

Let us look at an example of using the Scavenger File System through a some-what contrived experiment. We begin by creating an HFS+ volume and copying some pictures to it. Then we delete the newly created folder by “mistake.” In the following example, we unmount the volume as well. In real life, the unmount-ing may or may not occur—there are pros and cons both ways. Keeping it mounted is fine if the volume is really quiescent. Unmounting it means there will be no further file system activity, but the very act of unmounting causes more file system activity—in particular, it flushes the journal. The Scavenger File System’s abilities are most appropriately thought of as “best effort,”, and so are the --scavenge operation’s. Let us see what can be done.

To get an idea of what is scavengable through fileXray, we can use the --scavenge option to display a list of what fileXray finds on the unmounted volume. Note that we use the --device option with the disk image path speci-fied as the “device.”


164

# Create an HFS+ disk image.$ hdiutil create -megabytes 1024 -fs HFSJ -volname TestHD /tmp/test.dmg............................................................................created: /tmp/test.dmg

# Attach the disk image and mount the HFS+ volume.$ open /tmp/test.dmg

# Create a subfolder on the volume and some pictures to it.$ mkdir /Volumes/TestHD/Pictures$ cp -v ~/Pictures/* /Volumes/TestHD/Pictures//Users/johndoe/Pictures/apple.tif -> /Volumes/TestHD/Pictures/apple.tif…/Users/johndoe/Pictures/fileXray.tif -> /Volumes/TestHD/Pictures/fileXray.tif…/Users/johndoe/Pictures/vista.jpg -> /Volumes/TestHD/Pictures/vista.jpg

# Delete the newly created Pictures folder on the test volume.$ rm -rf /Volumes/TestHD/Pictures

# Eject the volume.$ hdiutil eject /Volumes/TestHD

# Simulate panic, realizing that we deleted the wrong folder.$#@*!^!@!!!!***XXXOOO!!

Since content of interest does show up in the output of --scavenge, let us use the Scavenger File System on this volume. Note that two of the files shown in the excerpt have content in both forks. Moreover, the blocks belonging to all these files are currently free.

Note that the time taken to “prepare” the Scavenger File System will be longer for larger volumes.


165

# Use fileXray to display a list of scavengable content.$ fileXray --device /tmp/test.dmg --scavenge…f,24/84,0x34f30,2957/2957/2957,11/11/11,apple.tif:[CNID=24]/apple.tif…f,24/172,0x35a48,942/942/942,19/19/19,fileXray.tif:[CNID=24]/fileXray.tif…f,24/374,0x66656,144/144/144,0/0/0,vista.jpg:[CNID=24]/vista.jpg…

# Mount the disk image using the Scavenger File System.$ mkdir /tmp/scavenger$ fileXray --device /tmp/test.dmg --userfs_type scavenger \ --userfs_mount /tmp/scavengerPreparing the Scavenger File System...$

Note that the Scavenger File System snapshots the scavengable metadata as it populates itself. Once mounted, the Scavenger File System view will not change even if the underlying volume is normally mounted and file system activity is occurring.

Caveats

Although the Scavenger File System can be used on a live, mounted volume, it is not advisable to do so, especially if that volume is the root volume. The usual volatility of a root volume does not lend itself to scavenging very well. Therefore, it is best to use this file system (or in general, perform any type of scavenging operation) on an unmounted or at least quiescent non-root volume. Moreover, do not copy from the Scavenger File System to the volume that is currently be-ing scavenged.

See Also

•--force

•--freespace

•--journal_names

•--scavenge

•--trawl QUERY

•--usedspace



166

--version

Print software version and license information. If applicable, fileXray will also display the type of the product, such as “personal” or “pro.”

--version

167

$ fileXray --versionfileXray plus 1.0.0Copyright (c) 2010 iohead LLC. All Rights Reserved.

Licensed to: John Doe <[email protected]>Product : fileXray plus (licensed for professional use)Purchased : Monday Nov 2, 2009

--volume VOLUME_PATH

Specify an HFS+ volume through a path (VOLUME_PATH) that’s either the vol-ume’s mount point or is a file system object within a mounted volume.

The --volume option is one of the ways you can explicitly specify a volume for fileXray to operate upon. For this option to be used, the volume must be mounted.

Another way to explicitly specify a volume is through the --device option, which is more versatile in that it accepts both mounted volumes and offline de-vices. It also allows disk images to be specified.

If no volume is explicitly specified through the --device or --volume options, fileXray will attempt to determine the volume from a file system object path if one is specified. If the combination of arguments is such that no file system ob-ject path is specified, fileXray will use the root volume.

See Also

•--device DEVICE

--volume VOLUME_PATH

168

-V VOLUME_PATH

$ sudo fileXray --volume /Volumes/BackupHD [other options…]# Use the volume that’s mounted on /Volumes/BackupHD…

$ sudo fileXray --volume /some/path/to/melody.mp3 [other options…]# Use the volume on which the given melody.mp3 file resides …

$ sudo fileXray --volume / [other options…]# Use the root volume…

--volume_header

Display an HFS+ volume’s Volume Header. If the HFS+ volume is embedded within a legacy HFS wrapper, also display the HFS Master Directory Block.

If the --exhaustive option is additionally specified, --volume_header will also display the Alternate Volume Header, which is a copy of the volume header stored starting at 1024 bytes before the end of the volume.

In addition to displaying the contents of the Volume Header, the --volu-me_header option displays several pieces of secondary information that fileXray computes after analyzing the Volume Header. In other words, this option displays more information than is contained as is within the Volume Header—some of the information is derived by fileXray from the Volume Header.

The following is an example of examining a root volume’s Volume Header.

--volume_header

169

-v

$ sudo fileXray --volume_header# HFS+ Volume Volume size = 250 GB (233 GiB)

# Volume Header signature = 0x482b (H+) version = 0x4 lastMountedVersion = 0x4846534a (HFSJ) attributes = 10000000000000000010000000000000 . kHFSVolumeJournaled (volume has a journal) journalInfoBlock = 0x746 createDate = Mon Nov 2 18:56:59 2009 modifyDate = Mon Nov 14 17:41:10 2009 backupDate = 0 checkedDate = Mon Nov 2 18:56:59 2009 fileCount = 1422188 folderCount = 267513 /* not including the root folder */ blockSize = 4096 totalBlocks = 60965668 freeBlocks = 20533351 nextAllocation = 48966050 rsrcClumpSize = 65536 dataClumpSize = 65536 nextCatalogID = 3200347 writeCount = 122156696 encodingsBitmap = 00000000000000000000000000000000 00000010000000000000000011001111 . MacRoman . MacJapanese . MacChineseTrad . MacKorean . MacGreek . MacCyrillic . MacChineseSimpcontinued…

--volume_header

170

… # Finder Info # Bootable system blessed folder ID finderInfo[0] = 0x88 (MacHD:/System/Library/CoreServices) # Parent folder ID of the startup application finderInfo[1] = 0x11714d (MacHD:/System/Library/CoreServices/boot.efi) # Open folder ID finderInfo[2] = 0 # Mac OS 9 blessed folder ID finderInfo[3] = 0 # Reserved finderInfo[4] = 0 # Mac OS X blessed folder ID finderInfo[5] = 0x88 (MacHD:/System/Library/CoreServices) # VSDB volume identifier (64-bit) finderInfo[6] = 0x18df290 finderInfo[7] = 0xcc72ad0e # File System Boot UUID UUID = C91FC7AA-AC77-3D04-AB95-A55111B435EE

# Allocation Bitmap File (CNID 6) logicalSize = 7622656 bytes (7.6 MB) totalBlocks = 1861 clumpSize = 7622656 bytes extents = startBlock blockCount % of file

0x1 0x745 100.00 %


# Extents Overflow File (CNID 3) logicalSize = 8388608 bytes (8.4 MB) totalBlocks = 2048 clumpSize = 8388608 bytes extents = startBlock blockCount % of file

0x1f47 0x800 100.00 %


# Catalog File (CNID 4) logicalSize = 814743552 bytes (815 MB) totalBlocks = 198912 clumpSize = 116391936 bytes extents = startBlock blockCount % of file

0x67847 0x30900 100.00 %


continued…

Note that fileXray also displays the Catalog Node IDs of two special folders used internally by the file system.

The Private Metadata Folder is used to house two types of objects: the files that hold the content of file hard links (that is, the link targets of file hard links) and files that were unlinked while they were still open or otherwise busy. When an open file is unlinked, it is renamed and moved to the Private Metadata Folder, where it lives at least until it is closed.

The Directory Metadata Folder is used to house the directories that are the link targets of directory hard links.

See Also

•--exhaustive

•--mount_data

•--userfs_type structure

--volume_header

171

… # Startup File (CNID 7) logicalSize = 0 bytes

# Attributes File (CNID 8) logicalSize = 308281344 bytes (308 MB) totalBlocks = 75264 clumpSize = 154140672 bytes extents = startBlock blockCount % of file

0x2747 0x9300 50.00 % 0xba57 0x9300 50.00 %


# Volume Header SHA-1 cf66ad9b7a973b89c195cad9a5a71dccfc642c35

# Auxiliary System CNIDs HFS+ Private Metadata Folder = 18 HFS+ Directory Metadata Folder = 19

--who_owns_byte BYTE

Determine which file system object, if any, “owns” the byte at offset BYTE on the given HFS+ volume. Note that byte offset 0 represents the beginning of the volume.

The following examples show how to use --who_owns_byte. Note that in the case of /mach_kernel, we look at the file’s extents to know where the file be-gins on the volume. Since its first block is number 0x1b71e6 and the allocation block size for this volume is 4096 bytes, the file’s first byte is byte number 0x1b71e6000 (that is, 0x1b71e6 × 4096) on the volume.

See Also

•--who_owns_block BLOCK

--who_owns_byte BYTE

172

-w BYTE

$ sudo fileXray --who_owns_byte 1023Reserved

$ sudo fileXray --who_owns_byte 1024Volume Header

$ sudo fileXray /mach_kernel… extents = startBlock blockCount % of file

0x1b71e6 0x11cf 100.00 %…

$ sudo fileXray --who_owns_byte 0x1b71e6001 # allocation block size is 4096 bytesMacHD:/mach_kernel

--who_owns_block BLOCK

Determine which file system object, if any, “owns” the allocation block number BLOCK on the given HFS+ volume. Note that block number 0 represents the be-ginning of the volume.

The volume’s allocation block size can be seen in the Volume Header, which can be displayed using the --volume_header option.

The following examples show how to use --who_owns_byte.

See Also

•--volume_header

•--who_owns_byte BYTE

--who_owns_block BLOCK

173

-W BLOCK

$ sudo fileXray --who_owns_block 0Reserved

$ sudo fileXray --who_owns_block 1Allocation File




$ sudo fileXray --who_owns_block 12345Attributes B-Tree

$ sudo fileXray --who_owns_block 123456Unused

$ sudo fileXray --who_owns_block 1234567Unused

$ sudo fileXray --who_owns_block 12345678MacHD:/Users/johndoe/Documents/Virtual Machines.localized/\ Mac OS X 10.4.vmwarevm/Mac OS X 10.4-s005.vmdk

$ sudo fileXray --who_owns_block 123456789Specified location is beyond the volume's size.

fileXray

Documents

Transcript of fileXray