In the last two weeks, two of our favourite disk forensic tools integrated native read support for the AFF4 forensic format. Forensic Explorer released v4 of their product, with native AFF4 read support, and X-Ways Forensics released v19.5, which has a plugin API supporting our AFF4 read plugin.
This represents a big step forward towards general adoption of the next-generation image format.
Evimetry’s filesystem bridge provides a straightforward and efficient way of consuming AFF4 images from any commercial forensic tool, and results in faster analysis & processing than E01′s. Despite this, it is convenient to be able to open AFF4 images directly from tools without having this dependency.
For the last year and a half, Evimetry have been investing significant effort in growing the AFF4 ecosystem, by standardising the format, providing open-source implementations, integrations with leading open source forensic software, and working with commercial vendors to integrate read support.
In October we worked closely with X-Ways to define a plug-in API to support new forensic image formats, which X-Ways integrated in the 19.5 beta releases. We followed this up by producing an X-Ways plugin to read AFF4 images via our C++ based Evimetry libAFF4 Reader DLL. Around the same time, we provided the reader DLL’s to the folks behind Forensic Explorer (FEX). In no time, the v4 beta builds of FEX supported reading AFF4 images too.
Usage: X-Ways >= 19.5
Download the current Evimetry X-Ways AFF4 reader plugin, and copy the contents into the X-Ways installation folder. Make sure you have the Visual C++ 2015 Runtime installed.
CAVEAT: Only x64 is supported for now.
UPDATE: We now support x86 (32 bit) as well.
Usage: Forensic Explorer >= 4.0
The current FEX 4.0 build already integrates the Evimetry libAFF4 reader DLLs. This DLL contains a bug that has since been fixed in a later version of the DLL. We anticipate that this will make it into the next release of FEX. In the meantime replace the libaff4 DLL in Forensic Explorer with the one contained in the Evimetry libAFF4 reader DLL package.
Caveat: BETA code quality
Please note that the Evimetry libAFF4 reader DLLs are currently at BETA quality, while we undertake further testing and importantly, tuning. If you strike any issues, please submit a bug report to email@example.com .
You have been tasked with forensic acquisition of 6 servers in the AWS cloud, with a total of 2TB of storage. How do you do it?
This post will describe the method I applied in a recent case, where we collect the storage, acquired it into forensic images, and pulled down the images into our custody overnight. While I will be describing how I did it using Evimetry, the method is easily translatable to other tools.
Storage forensics in AWS
Unlike many cloud IAAS platforms, AWS provides us with the ability to take a Snapshot of the storage of a Virtual Computer (an Instance, in EC2 parlance). This gives you a point in time copy of the storage device. This isn’t a forensic image, as there isn’t a hash protecting the copy.
This enables us to quickly Collect a copy of the storage, without affecting the availability of the Target device. To truly collect the copy though, we need to take it under our control. To do this, we rely on the ability of AWS to share Snapshots between accounts.
Once we have access to the Snapshot in our own Security Domain (account), we can then shift to forensic acquisition of the copy. This is best achieved by generating a Volume from the Snapshot, attaching the Volume to a purpose-built acquisition server, and acquiring using regular forensic processes. Once a forensic image is acquired, we then Transfer it out of the cloud to store on a storage device that we Possess.
The following sections step through the process of undertaking the method.
Evidence Isolation & Location
First up, create your own Security Domain for Collecting the Snapshots into, and undertaking acquisition. In AWS, this is easily achieved by maintaining your own account, separate to the TARGET account. The below screenshot displays an account I have established under my own name, logged into the AWS Console.
I recommend running the two separate AWS security domains (the TARGET and EVIDENCE) using two separate web browser windows, one of them using private mode browsing so that you can use the TARGET’s credentials in one, and your security domain’s credentials in another.
Note the Account ID (993480464498) – this will be required to identify this security domain when we come to share a Snapshot from the TARGET to our security domain.
In the TARGET AWS console, use the left menu, “Instances” to show the instances, and find the instance that you want to collect.
In the screen capture below, one can see the TARGET instance, the instance ID (in this case i-065e4cd1fbf56c92e), the block device volume (vol-08c5f1566ec4ea6c5), and the Availability Zone “ap-southeast-2c”. These identifiers should be documented to establish the provenance of the evidence.
In the left menu, “Elastic Block Store”, select “Snapshots”, and then “Create Snapshot”.
Select the volume of our TARGET server (ol-08c5f1566ec4ea6c5), and describe the evidence.
We now have a snapshot of the block storage of the instance. We record the Snapshot ID (snap-0925cec0faee0659a) to maintain provenance of the evidence.
Now that we have an image (not a forensic image, as we don’t have a hash), we want to Collect it. This means taking possession of it, so that it can’t be modified. To do this, we share the image with the EVIDENCE security domain we created earlier.
Recalling that the Account ID of our acquisition security domain is (993480464498), we privately share the image with that account.
Note that the snapshot isn’t instantaneous and may take some minutes to complete.
Prepare Evidence Storage Server – Server provisioning
While the snapshot is going, switch browser windows (and security domains), and begin setting up your Evidence Server in the same datacentre as the target server. This will be running the Evimetry Cloud Agent, and will be co-located with the TARGET server for efficiency and speed. In the below AWS control panel, we set the location to “Sydney” which matches the TARGET server in this instance.
In the left menu we select “Instances” and then “Launch Instance”
To deploy the Evimetry Cloud Agent, we need an Ubuntu 14.04 instance. Select that.
The speed at which your acquisition will occur will depend on a number of factors, including the virtual disk size, the performance of the virtual storage, and the number of CPU’s you have in the server. In a future blog post, I will go into this in more detail, but for now, select a 4 CPU machine with moderate performance.
Next up, we want to make sure that the evidence storage server is as close as possible to the target. From before, we have identified that that the target is in “ap-southeast-2c”, so we make sure that the subnet matches. We also ensure that we enable the auto assignment of a public IP, so we can connect to the server. This is sufficient to then “Review & Launch”.
Finally, we launch the new Evidence Storage instance.
The final task in bringing up the evidence storage server is to establish a key pair for working with the server. We create a new key pair called “SF-Acquisition” below.
Download the key pair, and save it somewhere safely. Example shell commands follow.
The final thing to setup is a port forward so that we can connect through to the Evidence Server. Unlike some cloud services, EC2 Instances sit on a private IP address, behind a firewall.
In the “Network & Security” section of the console, go to “Security Groups”. Recalling from the Instance that is was started in the Security Group “Launch Wizard 3”, edit the inbound rules of that security group.
Create a rule forwarding the Evimetry Cloud Agent’s port (TCP 9982) to the Evidence Server.
Verify Access to Evimetry Cloud Agent
At this point, the Evimetry cloud agent is ready to be used. Using the public IP of the VM (188.8.131.52), connect in using the Evimetry Controller.
The agent will appear in the controller’s fabric nodes view (note that the IP of the Evidence Server is showing a 172.X.X.X private IP address). Visible underneath it is its storage, and an Evimetry Repository, which is located on its internal storage. We will acquire our images into this Repository.
Acquiring the image
We now go back to the EVIDENCE Security Domain, and access the “Elastic Block Store” | “Snapshots” section. Be sure to filter the view to “Private Snapshot” as it won’t be visible in the default setting.
The snapshot from the SUSPECT security domain will now be visible (check the Snapshot ID matches). We now transform the image into a Volume, which can then be added to a running instance in much the same way we plug removable storage into a computer. First, right click on the Snapshot and select “Create Volume”.
In the volume creation form, where we create the evidence storage instance, we choose the Availability Zone of “ap-southeast-2c”. Make sure that you choose this zone as the instance where the Volume is created, and then click on “Create Volume” to create the volume.
Note the volume ID, of the new Volume, which is vol-097e59361bd515f78. Follow the link.
Now we can attach the volume to our Evidence Server. Go to the “Elastic Block Store” | “Volumes” area and, noting the Volume ID, select Attach Volume.
Recalling that the instance ID of our Evidence Server is i-0af9148e32f37b8ac, attach the Volume as a virtual disk.
Refreshing the Cloud Agent instance listed in the Evimetry Controller now shows the disk attached to the agent as /dev/xvdf . Note that the newly attached disk is locked against mounting and writing. Right click on the disk and select Acquire.
The acquisition settings dialog will appear. Select a full linear acquisition of the attached drive, and add the Repository on the Storage Server as the container location. Give the Image a name using your standard image naming scheme, and document the original Volume ID and Instance ID associated with this image. Then click OK.
Acquisition is now underway. The screenshot below shows an acquisition using the “Provisioned IOPS SSD” as Volume storage, which proceeds at around 90MB/s, constrained by the storage of the infrastructure. Our testing shows that using “General Purpose SSD’s” as storage gives a trickling rate of around 10MB/s (that’s 4x slower than USB2!). A future post will focus on scaling this speed.
When the acquisition (including verification) completes, Evimetry will display a completion dialog.
We then transfer the image locally to the lab using Evimetry, by flipping to the “Images” tab of the Controller, and right clicking on the newly created image.
After choosing the destination, the image downloads locally.
This post has described a methodology for acquiring storage in the EC2 cloud. Using EC2 Snapshots in conjunction with Snapshot Sharing enables one to quickly Collect copies of Target storage. Acquisition can then be undertaken in the Cloud, so that the evidence is protected by a hash at the earliest opportunity, while minimising the amount of data that needs to be copied.
In future posts, I will follow up on how virtual disk selection affects the speed of acquisition; how to acquire volatile memory in AWS; and how to undertake analysis in the cloud.
Late last year I had the pleasure of attending the F3 conference in Gloucestershire, UK. It is quite unlike any other digital forensics conference I have ever been to; a community run, practitioner focused, 2 day conference situated in a stately manor in the English countryside. I can thoroughly recommend it.
The Advanced Forensic Format 4 Working Group (AFF4 WG) is calling for interested parties to join the second working group meeting, to be co-located at the DFRWS Conference 2017, in Austin, TX.
Originally proposed in 2009 by Michael Cohen, Simson Garfinkel, and Bradley Schatz, the AFF4 forensic container enables new approaches to forensics, unparalleled forensic acquisition speeds and more accurate representation of evidence. The AFF4 WG has recently released v1.0 of the AFF4 Standard, including canonical images, specification, and open source libraries for implementers. Current AFF4 implementations include Rekall, Evimetry, Sleuth Kit, Volatility and GRR.
For more information, please see the working group mailing list, or contact Bradley Schatz or Michael Cohen.
Co-Chair: Dr Bradley L Schatz, Schatz Forensic/Evimetry, [ bradley <at> schatzforensic <dot> com ]
Co-Chair: Dr Michael Cohen, Google, [ scudette <at> google <dot> com ]
AFF4 working group mailing list: https://groups.google.com/forum/#!forum/aff4-wg
We recently contributed patches to the Sleuth Kit to read AFF4 images. While we are waiting for those to be pulled into the main distribution, the following recipe should suffice for compiling a stand alone copy on MacOS.
The following dependencies are needed to compile libAFF4 on OSX. I use MacPorts, and the corresponding packages i needed to install are:
* tclap (missing *.pc file – place in /opt/local/lib/pkgconfig/)
Clone and compile LibAFF4 (C/C++)
Use the following to clone the current release of libaff4, configure it, and install.
git clone https://github.com/google/aff4.git
git submodule update –init third_party/gtest
git reset –hard
./configure CC=clang CXX=clang++ CXXFLAGS=”-std=c++11 -stdlib=libc++ -O2 -g0 -I/opt/local/include” LDFLAGS=”-stdlib=libc++ -L/opt/local/lib”
sudo make install
Clone and compile the Sleuth Kit
Use the following to compile the sleuthkit with libaff4 support.
git clone https://github.com/blschatz/sleuthkit.git
git checkout release-4.4
autoreconf –force –install –verbose
sudo make install
We recently released Evimetry 3, the newest release of our revolutionary approach to forensic acquisition and analysis.
The big news is that we now support remote volatile memory acquisition. This means that in addition to being able to acquire remote disks, you can now acquire the volatile memory of live Windows, MacOS, and Linux hosts. We primarily support Windows XP and above (x86 and x64) and OSX Mountain Lion and above (x64). The coverage for Linux memory acquisition is limited to 64 bit Intel machines where the kmem driver is enabled.
Get straight to analysis.
In addition to acquiring the physical memory, we also acquire and store the entry points needed to find the kernel page tables and base kernel data structures. The benefit of this is that time-consuming scanning for these entry points (which are fundamental to further analysis) can be bypassed getting you to analysing evidence sooner.
We have developed patches to the leading volatile memory analysis frameworks, Volatility and Rekall, to support reading these images, and the patches for Volatility have been contributed to the main Volatility project on GitHub.
We take full advantage of Evimetry’s advanced compression to transport memory over the network at maximal rates. The effects of latency, a killer of network performance over long distance links, can be negated by pushing our networked evidence storage agents into the same network as the suspect computer.
Originally proposed in 2009 by Michael Cohen, Simson Garfinkel, and Bradley Schatz, the AFF4 forensic container enables new approaches to forensics, unparalleled forensic acquisition speeds and more accurate representation of evidence. These are enabled through next-generation forensic image features such as storage virtualisation, arbitrary metadata, and partial, non-linear and discontiguous images. The standard is the culmination of research spanning 6 years and 4 scientifically peer reviewed papers.
The release of these is a significant step forwards to the wider adoption of the format, enabling a large portion of the open source forensic toolchain to access AFF4 forensic images, and commercial implementers the ability to support reading the format by integration of a single unencumbered library.
Evimetry Community Edition provides a subset of the Evimetry system for free. The purpose of this is to grow the AFF4 ecosystem, firstly by providing a pain free path for Evimetry licensees to provide AFF4 images to non-licensees. Secondly, we wanted to provide practitioners, researchers and educators a freely available implementation of the AFF4 standard v1.0 which can be used to gain familiarity with the format. Schatz Forensic, the creators of Evimetry, drove the standardisation effort behind the AFF4 Standard v1.0.
With the Community Licenced Evimetry Controller, you can create Linear AFF4 Images on your Windows based analysis system, verify the integrity of AFF4 images, and convert between AFF4, E01/EWF and Raw images. You can also mount AFF4 images as virtual disks and analyse with your preferred forensic tools.
Using the Community Licenced Evimetry Filesystem Bridge, you can access entire repositories of AFF4 images as virtual raw files, enabling straightforward consumption with your existing forensic toolkit.
The release of Evimetry Community Edition coincides with the release by Schatz Forensic of open source implementations of the AFF4 format, patches to the Sleuth Kit supporting AFF4 images, and the release of the AFF4 Standard v1.0.
To gain access to the initial release of Evimetry Community Edition, email us at firstname.lastname@example.org .
Today marks the release of the Advanced Forensic Format 4 (AFF4) Standard v1.0.
Originally proposed in 2009 by Michael Cohen, Simson Garfinkel, and Bradley Schatz, the AFF4 forensic container enables new approaches to forensics, unparalleled forensic acquisition speeds and more accurate representation of evidence. These are enabled through next-generation forensic image features such as storage virtualisation, arbitrary metadata, and partial, non-linear and discontiguous images. The standard is the culmination of research spanning 8 years and 4 scientifically peer reviewed papers.
This release of a standard specification for the file format is a milestone towards the wider adoption of the format, providing implementers an unambiguous and straightforward path to implementation. The release of the AFF4 Standard coincides with the limited release of Evimetry Community Edition, a freely licensed subset of the AFF4 based forensic tool, and in the coming days, a C++ implementation and patches to the Sleuth Kit, and support for Volatility and Rekall.
The standard specification and reference images are available at , the python implementation at , and aff4.org  becoming the central point of publication.
While users of Evimetry are able to exploit the benefits afforded by AFF4 seamlessly with their regular forensic tools, we believe that native support for the format across both opensource and commercial tools will accelerate forensic workflow even further.
The screenshot below demonstrates a non-linear partial physical image (containing only the allocated blocks from the target disk) being interpreted by the SleuthKit.
We will be releasing patches for libaff4 (C++) and Sleuth Kit shortly.