Zone Identifier Internals

The “Zone.Identifier” file is a common artefact observed when undertaking forensic examinations of Windows systems. More correctly, this isn’t a file. Rather, it is an Alternate Data Stream (ADS), attached to content downloaded from the internet by Internet Explorer. The stream’s purpose: to record the source of the file so that judgements about its level of trust can later on be made by the Windows OS, particularly when running downloaded executable files.

Raymond Chen describes using windows API’s to access this, and points to further background on this artefact.


CFP: Digital Forensics of Embedded Systems

The Journal of Digital Investigation is currently calling for papers for a Special Issue on Digital Forensics of Embedded Systems. The Guest Editors of this issue are Pavel Gladyshev, Ronald van der Knijff & Bradley Schatz.

We would welcome any novel research into aspects of digital forensics of embedded and mobile computing devices. Submissions are due 1 October 2013.

UPDATE: The due date for submissions for this CFP has been extended to 20 February 2014.

 

The Journal of Digital Investigation covers cutting edge developments in digital forensics and incident response from around the globe.

We welcome submissions for a special issue on Embedded Systems.

The issue will focus on the challenges presented to digital forensics and incident response in the shift from commodity monolithic operating systems and hardware platforms to bespoke and embedded computing devices.

We seek submissions including case studies, practitioner reports of what works in practice, survey articles covering state-of-the-art and future needs, objective tool reviews, and relevant legal analysis. We are also looking for work that proposes possible normalization and standardization in this area.

Deadline for submissions is 20 February 2014.

Visit the journal page for more information and to contribute to this special issue.
http://www.journals.elsevier.com/digital-investigation/call-for-papers/
An embedded system is a computer system that controls operation of a special purpose machine or device, such as

  • automobile engine, brakes, navigator,
  • mobile phone,
  • SCADA/ICS device,
  • smart meter,
  • CCTV camera  recorder,
  • toy,
  • washing machine,
  • mini PABX,

Individual embedded systems are becoming increasingly networked forming the foundation of what is now called smart homes and smart cities. Being part of everyday human activities, embedded systems can be an important source of evidence in digital investigations.

The embedded systems range in complexity from primitive controllers with well defined, fixed functionality, to 32-bit systems capable of running full versions of Linux or Windows.  Most embedded systems are proprietary, bespoke systems, whose interfaces, data structures and internal operation are protected by NDAs.  As a result, the extraction and interpretation of the data from such systems is a major challenge of embedded system forensics.

Embedded systems are increasingly employed for acquisition & analysis tasks in forensic practice. There may be novel forensic applications of embedded systems with the potential to greatly enhance the efficiency of investigations.

In an effort to increase understanding and advance the state of the art, this special issue is dedicated to presenting the varying views regarding digital forensics of, and with, embedded systems.
Submit a case study, survey paper, tool review, or other contribution now.
Visit www.elsevier.com/locate/diin for more information.


Mobile phone forensic analysis–analysis of JTAG and Chip Off images of Android YAFFS Flash

On 18 October 2012 I presented, at the Breakpoint 2012conference, some preliminary results of research I have been undertaking in the area of forensic acquisition and analysis of mobile phones. Specifically I have been focusing on Android phones using NAND flash memory and the YAFFS2 file system. The seminar principally addressed methods of acquisition (JTAG and Chip Off) and the fundamental challenges of reconstructing YAFFS2 file systems from said acquisitions. The slides from the presentation can be found here.

 

Object Headers Slide Screenshot

If you are currently undertaking work in this area and having trouble interpreting any flash images, I would be happy to hear from you.


Android forensic analysis lecture at Breakpoint2012 (AU)

I will be presenting a lecture on Android forensics, focusing on flash acquisition and YAFFS2 filesystem analysis at the Breakpoint 2012 conference in Melbourne, Australia, this October 18.

The speaker lineup is looking fascinating, with leaders in the area of mobile security (both IOS and Android), hardware reverse engineering and Windows internals being on my list of lectures to attend.


Digital forensic evidence chapter published in Expert Evidence text

My chapter on digital evidence has recently been published in the Australian authority on Expert Evidence. The chapter joins technical treatment of over 75 other areas of expert evidence.

The chapter aims to inform the legal professional and fact finder as to the foundations, context, principles, practices, limitations and challenges of the field of digital forensics, in order that they may understand the field enough to effectively engage with the digital forensic expert. It is anticipated that this chapter will additionally be of interest to practitioners and researchers in the field.

The chapter is currently available only to subscribers of the loose leaf service and online via Westlaw AU and Thomson Legal Online. The chapter will be individually purchasable via the above website in due course; if you wish to purchase a copy in the short term, please contact Thomson Reuters via email.


Digital Evidence and Computer Crime 3rd edition – book chapter in press

I just received in the mail an author’s advance copy of Eoghan Casey’s "Digital Evidence and Computer Crime". Originally published in 2000, this update sees the book now in its third edition. Amongst a wide range of significant updates  is a chapter Eoghan and I co-authored. The focus of the chapter is on methods of conducing digital investigations.

Identifying methods of reliably transitioning from investigative goals or claims to substantiated facts has been a significant preoccupation within the field over the last decade. Perspectives have ranged across extremes: from those that deny such methods exist (“it’s an art”) to those that attempt to characterise method as a system or recipe  (“it’s a process”). Only in recent years have clear inroads been made into the relationship between digital forensics and the scientific method in general.

The chapter begins with a comparison of a wide range of perspectives on digital investigation methodologies, and follows with practical guidance on applying the scientific method as a methodology for each step of a digital investigation. The chapter concludes with an investigative scenario demonstrating how the scientific method may be applied in the context of an actual case.


Finding Object Roots in Vista (direct from dump file)

The last post discussed finding object roots in Vista using the self referential semantics of the Kernel Processor Control Region (KPCR). Object roots are the starting points that structural interpretation approaches use to begin to interpret kernel structures, in much the same way that one might use the MBR of a hard disk to find partitions on a drive, or the NTFS boot sector to find the MFT area in a filesystem.

The KPCR scanning approach is general purpose in nature. Assuming an appropriate value for the Directory Table Base of the kernel address space, it will yield potential KPCR structures. It is, however, time consuming. Windbg is able to almost instantly find the KPCR in images stored as dump files, so it obviously isn’t employing scanning as a method of identification.

Looking further into the Microsoft dump file format, the dump file has a header which contains a field called KdDebuggerDataBlock, which is a pointer to the kernel virtual address of a KDDEBUGGER_DATA64 structure. This is the same structure which is the end goal of the KPCR trick.

Accordingly, when run against a dump file, the experimental version of volatility currently uses the KdDebuggerDataBlock as the object root, rather than KPCR.


Finding Object Roots in Vista (KPCR)

This is the third of a series of posts describing how  the volatility memory forensics application was ported to a new Windows operating system version.

Apart from the inevitable changes in kernel data structures which typically come with a new kernel version, Vista brought with it a change which broke one of volatility’s key techniques for identifying kernel objects. The change was Address Space Layout Randomisation (ALSR). Thanks to Gil Peterson for sharing this detail.

Structural interpretation approaches to volatile memory analysis rely on finding an initial kernel object from which one may traverse to other objects, in order to find  objects of interest to the investigator. With XP, volatility employed “the KPCR trick”. XP reliably stored the “Kernel Processor Control Region” at a fixed kernel virtual address (0xffdff000). From the KPCR structure, one then traverses intermediate structures (KdVersionBlock, then DebuggerDataList)  to access interesting structures such as the active process list.

In Vista, KPCR is not stored at a fixed address, so the first problem in porting volatility to Vista and above, to reliably get an initial reference to a valid KPCR structure.

Damien Aumaitre points out in his 2009 paper "A little journey inside Windows memory" that KPCR is self referencing. Based on this observation I undertook the following investigations:

1. I loaded up windbg to look at the KPCR of a XP dump file.

2. Set the symbol path

.sympath SRV*C:devsymbolcachesymbols*http://msdl.microsoft.com/download/symbols

.reload

3. Examined the KPCR

kd> !pcr
KPCR for Processor 0 at ffdff000:
    Major 1 Minor 1
    NtTib.ExceptionList: ed89dcf0
        NtTib.StackBase: ed89ddf0
       NtTib.StackLimit: ed89a000
     NtTib.SubSystemTib: 00000000
          NtTib.Version: 00000000
      NtTib.UserPointer: 00000000
          NtTib.SelfTib: 7ffdf000

                SelfPcr: ffdff000
                   Prcb: ffdff120
                   Irql: 00000000
                    IRR: 00000000
                    IDR: ffffffff
          InterruptMode: 00000000
                    IDT: 8003f400
                    GDT: 8003f000
                    TSS: 80042000

          CurrentThread: 86724020
             NextThread: 00000000
             IdleThread: 80552740

              DpcQueue:

4. Note that the first line indicates that KPCR is at VA 0xffdff000, which is to be expected. Note also that the SelfPcr attribute (at offset 0x1c from the start of the structure) has the same value at the address of the KPCR. This is what Aumaitre means by self referencing. There exists an additional self referencing property here related to the Prcb field which I will not go into.

5. Repeating the above on a Vista image gives the following. Note that the KPCR structure is not at 0xffdff000, and again, that the SelfPcr field points to the KPCR address.

kd> !pcr
KPCR for Processor 0 at 818f4700:
    Major 1 Minor 1
    NtTib.ExceptionList: 9a7bfcf0
        NtTib.StackBase: 00000000
       NtTib.StackLimit: 00000000
     NtTib.SubSystemTib: 80148000
          NtTib.Version: 000c3fac
      NtTib.UserPointer: 00000001
          NtTib.SelfTib: 7ffdf000

                SelfPcr: 818f4700
                   Prcb: 818f4820
                   Irql: 00000000
                    IRR: 00000000
                    IDR: ffffffff
          InterruptMode: 00000000
                    IDT: 822eb400
                    GDT: 822eb000
                    TSS: 80148000

          CurrentThread: 89d5eac0
             NextThread: 00000000
             IdleThread: 818f8300

              DpcQueue:

6. I checked the offsets for the SelfPcr as below:

kd> dt -r0 _KPCR 818f4700
ntdll!_KPCR
   +0×000 NtTib            : _NT_TIB
   +0×000 Used_ExceptionList : 0x9a7bfcf0 _EXCEPTION_REGISTRATION_RECORD
   +0×004 Used_StackBase   : (null)
   +0×008 Spare2           : (null)
   +0x00c TssCopy          : 0×80148000 Void
   +0×010 ContextSwitches  : 0xc3fac
   +0×014 SetMemberCopy    : 1
   +0×018 Used_Self        : 0x7ffdf000 Void
   +0x01c SelfPcr          : 0x818f4700 _KPCR
   +0×020 Prcb             : 0x818f4820 _KPRCB
   +0×024 Irql    
0;        : 0 ”
   +0×028 IRR              : 0
   +0x02c IrrActive        : 0
   +0×030 IDR              : 0xffffffff
   +0×034 KdVersionBlock   : 0x818f3c18 Void
   +0×038 IDT              : 0x822eb400 _KIDTENTRY
   +0x03c GDT              : 0x822eb000 _KGDTENTRY
   +0×040 TSS              : 0×80148000 _KTSS
   +0×044 MajorVersion     : 1
   +0×046 MinorVersion     : 1
   +0×048 SetMember        : 1
   +0x04c StallScaleFactor : 0×962
   +0×050 SpareUnused      : 0 ”
   +0×051 Number           : 0 ”
   +0×052 Spare0           : 0 ”
   +0×053 SecondLevelCacheAssociativity : 0 ”
   +0×054 VdmAlert         : 0
   +0×058 KernelReserved   : [14] 0
   +0×090 SecondLevelCacheSize : 0
   +0×094 HalReserved      : [16] 0
   +0x0d4 InterruptMode    : 0
   +0x0d8 Spare1           : 0 ”
   +0x0dc KernelReserved2  : [17] 0

Scanning KPCR in Vista memory dumps

With the former in mind, I set about writing a volatility scanner that scans the kernel address space for potential KPCR structures. I note here that in my development I have relied on dump files generated by Matthieu Suiche’s win32dd. Dump files store the physical address of the kernel page directory base, which is enough to reconstruct the kernel address space, however, other means will be necessary if operating on raw (DD) style memory images.

The scanner runs through each kernel address space memory region, looking for memory patterns that match the following constraints:

  1. The VA of the start of candidate bytes  >= 0×80000000 (i.e. it is in the kernel address space)
  2. The SelfPcr field at offset 0x1c from the start of the candidate bytes contains a pointer to the virtual address of the start of the KPCR structure
  3. The Prcb field at offset 0×20 within the structure contains a pointer to the start of the _KPRCB structure, which is embedded within the KPCR structure at offset 0×120.

I have observed the above scanning technique to identify one KPCR value per processor, which is consistent with the described function of the KPCR – it is a per processor structure.

Usage

The above scanner is usable as the volatility command kpcrscan, and the output of it used via the parameter –kpcr=

An example of usage is presented in the screen capture below:

image

Adding Vista structure definitions to Volatility

This post follows on from the last post. In the last post I described how I extended Volatility to work with the symbols for Window XP SP3. In this one, I describe how I applied the approach to Vista SP0.

1. Downloaded Windows Vista RTM x86 retail symbols from Microsoft. I installed them to C:devVistaSP0x86vista-x86

2. I then ran the tpi_vtypes.py program against the symbol file which corresponds to the general Vista kernel, ntkrnlmp.pdb, generating a python symbol definition file for volatility in the process.

C:mysrcpdbparse>c:Python25python.exe tpi_vtypes.py c:devVistaSP0x86symbolsEXEntkrnlmp.pdb > vista_sp0_x86_vtypes.py

3. Moved vista_sp0_x86_vtypes.py to the pluginsoverlayswindows folder within the volatility source tree.

4. Created a new profile implementation called vista_sp0_x86.py. I based this off the existing windows XP SP2 profile, modifying to suit. In essence, the new profile is composed of three things:

  1. a definition of native types (these appear to generally apply to 32bit windows operating systems (I simply took the exiting windows definitions);
  2. a definition of the specific structural types which I just generated (contained in vista_sp0_x86_vtypes.py);
  3. an overlay definition (I simply reused the existing XP SP2 one with the fixups from the last post.

vistasp0x86overlays = copy.deepcopy(vtypes.xpsp2overlays)

vistasp0x86overlays['_MMVAD_SHORT'][1]['Flags'][0] =  lambda x: x['u'][0]
vistasp0x86overlays['_CONTROL_AREA'][1]['Flags'][0] =  lambda x: x['u'][0]
vistasp0x86overlays['_MMVAD_LONG'][1]['Flags'][0] =  lambda x: x['u'][0]
vistasp0x86overlays['_MMVAD_LONG'][1]['Flags2'][0] =  lambda x: x['u'][0]

vista_sp0_x86_vtypes.ntkrnlmp_types.update(crashdump.crash_vtypes)
vista_sp0_x86_vtypes.ntkrnlmp_types.update(hibernate_vtypes.hibernate_vtypes)

class VistaSP0x86(xp_sp2.WinXPSP2):
    """ A Profile for Windows Vista SP0 x86 """
   native_types = vtypes.x86_native_types_32bit
   abstract_types = vista_sp0_x86_vtypes.ntkrnlmp_types
   overlay = vistasp0x86overlays

The next post will cover the modifications to volatility needed to find KdDebuggerDataBlock – the root of kernel objects required to find active modules and processes amongst other things.


Adding new structure definitions to Volatility

I am currently preparing for a day long tutorial on Windows Volatile Memory Forensics for Incident Response, which Michael Cohen and I are presenting at the AusCERT conference next week. A significant part of the analysis component of the tutorial will focus on the open source volatile memory analysis tool, Volatility.

A current limitation Volatility’s support for Windows is that it doesn’t support analysis of anything other than Windows XP. This comes down to two issues:

  1. The core kernel structures (symbols) which are related to information about processes, threads, open sockets and the like, tend to change from version to version of Windows; and
  2. The technique that Volatility uses to initially find those structures does not work on Vista and above.

In this post I will be relating the first steps of how I ported Volatility to accept new symbol definitions in support of Vista in x86 (ie 32 bit) systems.  Beta code (complete enough to run pslist amongst others) is now in subversion.

My approach involved integrating automatically generated type definitions for Windows XP SP3, before applying the approach to Vista. It may seem redundant to use automatically generated type definitions for XP SP3, given that volatility works fine with its own hand coded ones for this version, however, doing so provides a useful foundation step, highlighting any assumptions being made higher up in the code.

Generating new structure definitions from PDB files

Thankfully all of the heavy lifting has been done by others. Brendan Dolan-Gavit has produced pdbparse that generates volatility compatible structure definitions from the symbol files that Microsoft produces.

The steps I undertook are:

1. Downloaded Windows XP with Service Pack 3 x86 retail symbols from Microsoft. I installed them to C:devXPSP3_x86

2. Checked out via subversion the current beta of Volatility.

3. Checked out via subversion pdbparse.

4. I installed the python construct module, on which pdbparse relies.

5. I then ran the tpi_vtypes.py program from pdparse against the symbol file which corresponds to the Windows XP uni-processor kernel, ntoskrnl.pdb, generating a python symbol definition file for volatility in the process.

C:mysrcpdbparse>c:Python25python.exe tpi_vtypes.py c:devXPSP3_x86symbolsEXEntoskrnl.pdb > xp_sp3_x86_vtypes.py

Adding structure definitions to Volatility

Adding the new structure definitions to volatility is simplified by the new “profile” abstraction recently introduced. Profiles allow for defining sets of structure definitions specific to a particular envionment. In this case, I created a new profile called “WinXPSP3”.

The steps I took are as follow:

1. Moved xp_sp3_x86_vtypes.py to the pluginsoverlayswindows folder within the volatility source tree.

2. Created a new profile implementation called xp_sp3_x86.py. I based this off the existing windows XP SP2 profile, modifying to suit. In essence, the new profile is composed of three things:

  • a definition of native types (these appear to generally apply to 32bit windows operating systems (I simply took the exiting windows definitions);
  • a definition of the specific structural types which I just generated (contained in vista_sp0_x86_vtypes.py);
  • an overlay definition (I simply reused the existing XP SP2 one

class WinXPSP3(xp_sp2.WinXPSP2):
    """ A Profile for windows XP SP3 """
    native_types = vtypes.x86_native_types_32bit
    abstract_types = xp_sp3_x86_vtypes.ntoskrnl_types
    overlay = vtypes.xpsp2overlays

3. Running volatility against an appropriate image resulted in the following:

c:mysrcVolatility-1.4_beta1>c:Python26python.exe volatility.py –profile Win
XPSP3 -f "C:UsersbradleyDesktopimagesRAMXPSP3-x86.dmp" pslist
Volatile Systems Volatility Framework 1.4_beta1
invalid syntax (xp_sp3_x86_vtypes.py, line 63)

4. Inspecting the auto-generated structure definitions where the error was yielded the following:

‘Valid’ : [ 0x0, [UNIMPLEMENTED LF_BITFIELD]],

5. Reading the above error indicated that pdbparse does not handle bit-fields in structures.  I modified tpi_vtypes.py to simply ignore any structure features that pdbparse tagged as UNIMPLEMENTED.

def print_vtype(lf):
    print "  ‘%s’ : [ %#x, {" % (lf.name, lf.size)
    for s in lf.fieldlist.substructs:
    ms =  member_str(s.index)
    if (ms.find("UNIMPLEMENTED") >= 0):
        pass
    else:

            print "    '%s' : [ %#x, %s]," % (s.name, s.offset, ms)
    print "} ],"

6. I ran volatility again:

c:mysrcVolatility-1.4_beta1>c:Python26python.exe volatility.py –profile WinXPSP3 -f "C:UsersbradleyDesktopimagesRAMXPSP3-x86.dmp" pslist
Volatile Systems Volatility Framework 1.4_beta1
Error – Flags has no offset in object _MMVAD_SHORT. Check that vtypes has a concerete definition for it.
Error – Flags has no offset in object _CONTROL_AREA. Check that vtypes has a concerete definition for it.
Error – Flags2 has no offset in object _MMVAD_LONG. Check that vtypes has a concerete definition for it.
Error – Flags has no offset in object _MMVAD_LONG. Check that vtypes has a concerete definition for it.
‘NoneType’ object has no attribute ‘Signature’

7. Reading the above error indicated that the new structure definitions were missing some fields. These are most likely related to the removal of the bit-fields. Viewing the structures which correspond in the original vtypes.py file for XP SP2, we have:

‘_MMVAD_SHORT’ : [  0x18, {
  'StartingVpn' : [ 0x0, ['unsigned long']],
  ‘EndingVpn’ : [ 0x4, ['unsigned long']],
  ‘Parent’ : [ 0x8, ['pointer', ['_MMVAD']]],
  ‘LeftChild’ : [ 0xc, ['pointer', ['_MMVAD']]],
  ‘RightChild’ : [ 0x10, ['pointer', ['_MMVAD']]],
  ‘Flags’ : [ 0x14, ['unsigned long']],
} ],

8. The corresponding auto-generated structure definition is:

  ‘_MMVAD_SHORT’ : [ 0x18, {
    'StartingVpn' : [ 0x0, ['unsigned long']],
    ‘EndingVpn’ : [ 0x4, ['unsigned long']],
    ‘Parent’ : [ 0x8, ['pointer', ['_MMVAD']]],
    ‘LeftChild’ : [ 0xc, ['pointer', ['_MMVAD']]],
    ‘RightChild’ : [ 0x10, ['pointer', ['_MMVAD']]],
    ‘u’ : [ 0x14, ['__unnamed_1498']],
} ],

<
p>9. In the standard Volatility type definitions, offset 0×14 into the struct is an unsigned long type called Flags, whereas is the auto generated types, the same offset is a field called “u”, which has type ‘__unnamed_1498′. Further investigation reveals that the latter type appears to be a Union structure.

10. In order to resolve this discrepancy, my goal was to take the working field definition for “Flags” from the original “_MMVAD_SHORT” structure, and transplant it in the newly generated structures.

11. The problem with doing that is that editing automatically generated code tends to lead to problems down the line. Fortunately, recent work in the volatility framework has anticipated this kind of need, with the introduction of type “overlays”.

12. The existing XP type definitions already contained an overlay for the flags field, so Michael Cohen gave me the magic incantation necessary to patch the in memory version of the overlay structure. The patched version of the _MMVAD_SHORT overlay contains a field ‘Flags’, which, when the offset of it is read, returns the offset of the ‘u’ field of the auto-generated type definition. Note that I am glossing over significant complexity here.

xpsp3overlays['_MMVAD_SHORT'][1]['Flags'][0] =  lambda x: x['u'][0]

My next post will cover adding Vista structure definitions, and a final post will cover the key modifications needed to find the KPCR structure.