Digital Forensics Solutions

Registry Decoder Bug Fix Release

2012-02-02T12:14:00.001-06:00

Registry Decoder 1.2, SVN revision 96, has just been released! As usual, it can be downloaded from the Google Code repository:

http://code.google.com/p/registrydecoder/downloads/list

We recommend that all Registry Decoder users upgrade as the following bugs are fixed:

1) Application hang if start/end filtering dates are entered with certain invalid input
2) Mactime timelinining format fix
3) Changed confusing timezone output for browsing, searching, and reports
4) Tabs generated from differencing operations can be closed by the user

We also have two new features:
1) Comma separated value (CSV) report format
2) Timeline data can be written directly to Excel (tab separated)

If you have sent us a bug report since version 1.1 you should receive an email this morning with confirmation that the issue was fixed. Please write back if you do not receive one of these emails.

Android Mind Reading: Memory Acquisition and Analysis with DMD and Volatility

2012-01-29T16:51:00.001-06:00

I would like to thank everyone who attended our talk at Shmoocon yesterday or watched the live stream. The turnout was amazing and we've gotten some great feedback. The slides from the talk are available here. We will be releasing the memory acquisition tool later this week, and the volatility support will soon follow.

New Paper - Acquisition and Analysis of Volatile Memory from Android Devices

2012-01-09T10:22:00.001-06:00

We are happy to announce that our paper on Android memory forensics has just been published in the Journal of Digital Investigations! This paper covers a number of topics that we believe will be of interest to both practitioners and researchers in the memory forensics field.

The two main contributions of the paper are:

A kernel module that is able to acquire a complete memory capture from Android devices as well as other Linux computers. This module is also unique in that it operates solely within the kernel and does not require userland interaction. This preserves memory much more effectively than other kernel modules, and a complete comparison of the efficiency is given in the paper. The kernel module can also acquire memory over the network, which prevents the investigator from having to save to the phone’s internal storage or SD card.
Additions to the Volatility memory analysis framework that allow it to analyze Android kernel memory. This allows all of the Linux analysis plugins to be used against Android memory captures.

There is also discussion on the difficulty of performing generic memory analysis of Android devices as well as the differences of the ARM versus Intel architecture, where a majority of previous memory forensics research has been performed.

If you are interested in this research and are going to be at Shmoocon, Joe Sylve (@jtsylve) will be there presenting the memory acquisition module as well as the Volatility capabilities. You can also leave comments on the blog or find us on Twitter.

DFS January Speaking Events: SANS East, DoD Cybercrime, Shmoocon, and more!

2012-01-04T13:36:00.000-06:00

We wanted to update you on a number of presentations and speaking events that we have in the month of January.

First on January 19, 2012, Joe Sylve (@jtsylve) will be presenting at the Tech-Knowledge e-Buisiness Workshop on malware and it's dangers to the workplace.

At SANS Security East, Andrew Case (@attrc) will be giving a workshop on using Volatility to analyze Linux memory captures on January 20, 2012. Two days later Lodovico Marziale and Andrew will be giving a Registry Decoder hands-on workshop.

On January 24, 2012 DFS will be exhibiting at the MSET Supplier & Service Provider Expo.

Then on January 26th, Lodovico will be presenting ways to incorporate Registry Decoder into the forensics process and how to leverage it for forensics research at the DOD Cybercrime Conference.

Finally, Joe will present our work on Android memory acquisition and analysis at Shmoocon on January 28th. We will also release and open-source the tools we developed to acquire full physical memory captures of RAM from Linux-based devices (including Android) and Volatility functionality to analyze these captures.

We've got a very busy month ahead of us and we hope many of our blog readers are able to attend some of the events.

Registry Analysis with Reglookup

2011-11-10T11:37:00.001-06:00

Now that Registry Decoder 1.1 is released, we wanted to highlight two libraries that are used during Registry Decoder's pre-processing phase. The first of these is reglookup and will be the subject of this blog post. Our next post will cover pytsk.

Reglookup is developed by Timothy Morgan and is both a library as well as a set of tools. We will first discuss the tools.

Tools

reglookup [1]

The reglookup binary is used to list the contents of a registry into a comma separated format. By default it will list all the paths, last write times, and name/value pairs contained within the registry. The –s option enables printing of security descriptor information as well.

The –p option is the one we use most as it allows for filtering output to only keys and name/value pairs under the given path. For example –p /AccessData/Products/ would list entries under the Products key and not the entire registry. This can be very useful for limiting to certain sections of the registry such as USBSTOR, network shares, and more.

We have previously released a whitepaper detailing use of the reglookup tool to help with recovery and analysis of deleted registry hives here.

reglookup-timeline

This tool is used to create a CSV timeline based on last write times within a hive.

reglookup-recover

This tool recovers deleted entries within registry hives, and then reports them in a CSV format similar to reglookup. The theory used to recover deleted entries is covered in Tim’s paper that can be found here. This capability has fairly obvious applications in forensics investigations, and investigators should consider adding reglookup-recover usage to their forensics process.

The Library

In Registry Decoder, we use the Python bindings for regfi (the C library) in order to enumerate every key, its last write time, and name/value pair in a particular hive. You can see how this is accomplished here. There is also documentation for the C library here and many example provided by the author for use with Python here.

In general, the library allows for completely programmatic exploration and analysis of registry hives using a very straightforward API. This includes querying keys, getting their name/value pairs, retrieving security attributes, recovering deleted records from within hives, and more. When using the python bindings, very powerful analysis can be performed in only a few lines of code.

Getting Reglookup

Source code and Windows binaries for reglookup can be found on the project downloads page. There are also packages for a number of Linux distributions, but they currently only have very old versions. We would advise acquiring reglookup from the author's webpage until the Linux distributions catch up.

References

[1] http://www.linuxcertif.com/man/1/reglookup/

Registry Decoder 1.1 Released!

2011-11-02T10:45:00.000-05:00

Digital Forensics Solutions is announcing the release of Registry Decoder 1.1, which has many completely new features and updates as well as bugfixes! Please see our previous blog post here for the initial release of Registry Decoder.

New Features include:

- Support for processing Encase (E01) files and split images
- Full wildcard searching
- Adding evidence after a case is created
- Exporting of paths and their key/value pairs
- Timelining of keys from the GUI into the Sleuthkit format
- Running plugins from the command line
- Running custom plugins outside the main executable/package
- Support for dual boot machines

Updates include:

- Reports now have their extension appended if the user doesn't enter them
- Reports can now be filtered by either deleting results or shift/ctrl selecting results
- Users can right click within the Browse View to search directly for paths
- The name/value box in the Browse View is now sortable
- We also have six new plugins from Kevin Moore of CERT!

Major changes since 1.0:

BROKEN BACKWARDS COMPATIBILITY
All evidence created by version 1.0 of the online tool (regdecoderlive) and cases created by previous versions of the offline tool WILL NOT BE compatible with version 1.1

We regret that we had to break compatibility with version 1.0 (and it won't happen again!), but the changes were significant and handling old data structures and databases would have required very ugly special-casing within the handling code.

The date format to filter searches has been changed to “yyyy/mm/dd” from “mm/dd/yyyy” so that dates can be directly copied & pasted from plugin and search outputs as well as from the Browse View.

Current Plans for Registry Decoder:

Registry Decoder version 1.1 currently has a “feature freeze” as we let the forensics community react and provide feedback to the new features. The only development that will continue will be that of plugins since they do not require any core changes or enhancements.

Plugin Development:

With this release, we are also releasing our official API documentation. The API is meant to be useable by even non-programmers, and many of our plugins are less than 10 lines of Python code. The latest version of the API can be found in the downloads section of Registry Decoder.

We also want to concentrate on reaching out to other practitioners and research groups (both professional and academic) in an attempt to proliferate Registry Decoder throughout these communities. We would appreciate any plugins contributed by these communities.

Existing plugins can be found in templates/template_files within the source code tree or can be viewed online at:

http://code.google.com/p/registrydecoder/source/browse/#svn%2Ftrunk%2Ftemplates%2Ftemplate_files

To make development easier, we have created the ability to run plugins from outside the core plugins directory as well as from the command line. Full details of how to accomplish this are explained in the plugins API documentation.

Mailing List:

We have created a mailing list for future updates, announcements, and ongoing discussion of the project. Please join the list http://groups.google.com/group/registry-decoder/ if you are interested.

Training:

We are now offering Registry Decoder training to all interested parties. Please see our training page for details and contact information.

Downloads and Instructions:

As always, the two tools, as well as their instructions, can be found on their respective Google code projects http://code.google.com/p/registrydecoder/ and http://code.google.com/p/regdecoderlive/.

Before ending this post, we would like to thank a few people who helped make this release a success. In no particular order... Tim Morgan, the author of reglookup, who helped us troubleshoot a few issues we had with the library; Michael Cohen, the author of pytsk, who helped develop in-library support for Encase and split images; and Kevin Moore of CERT, as he contributed a number of complex plugins to the project.

We would also like to thank all the beta testers that sent bug reports and feedback.

If you have any questions or comments, please either leave a comment on the blog or email registrydecoder@digdeeply.com

CyberSpeak Interview about Registry Decoder

2011-09-26T11:58:00.000-05:00

We are making a quick blog post to say that Registry Decoder co-developer Andrew Case (@attrc) was interviewed about the project this week on Ovie Carroll's (@ovie) CyberSpeak podcast. The recording can be downloaded here:

http://traffic.libsyn.com/cyberspeak/Cyberspeak-Show-141-2011-09-25.mp3

The interview gives some background on the project, explains the functionality it gives investigators, and discusses on-going development.

We hope that you can take a few minutes and listen, and please provide feedback in the comments section.

Recovering and Analyzing Deleted Registry Files

2011-09-16T12:21:00.000-05:00

I have been working on a case recently where we were asked to investigate possible data exfiltration from inside a corporate network. While investigating this type of scenario is not uncommon and has a number of investigative methods that can be used (see our previous post here), this scenario had a unique characteristic - namely that both computers we were asked to investigate had been reformatted/reinstalled since the investigated employee left.

This obviously was going to make the case more difficult and required some creative thinking in order to recover the necessary data to analyze. When attempting to document the process taken to perform a component of the analysis, what was meant to be a couple paragraph blog post, turned into a multiple page writeup. For that reason, I chose to simply convert the writeup to PDF and release it as a mini-whitepaper that can be found here:

http://www.digitalforensicssolutions.com/papers/recovering-and-analyzing-deleted-registry-hives.pdf

I hope you take a few minutes (should be 10-15 at most) to read the paper and hopefully learn something from it. From the feedback I got from other investigators who have seen the paper, (@kdpryor, @wyattroersma, @littlemac042), they all have liked it and found it interesting, and I believe you will too.

Please either comment on the blog or write me directly if you have any questions.

Andrew Case - @attrc

DFS Upcoming Speaking Events: DOD Cybercrime, SANs, BSidesDFW, and more!

2011-09-14T10:15:00.000-05:00

We wanted to give an update on a number of accepted presentations and other speaking events that we have upcoming in the next few months.

First, Golden Richard (@nolaforensix) will be presenting at the The Colloquium on IT Security, Cyber Forensics and Combating Cybercrime on Recent Advances in Live Forensics.

At BSidesDFW, Lodovico Marziale will be presenting on Registry Decoder and Andrew Case (@attrc) will be presenting on ways to investigate data exfiltration cases.

Then at the SANs Security East event in January, Lodovico and Andrew will be giving a Registry Decoder hands-on workshop, and the next day Andrew will be giving a workshop on using Volatility to analyze Linux memory captures.

And finally, Andrew will be presenting ways to incorporate Registry Decoder into the forensics process and how to leverage it for forensics research at the DOD Cybercrime Conf in January.

We hope that many of our blog readers will be able to attend some of these events. Be sure to drop us an email or twitter message if you will be there!

Announcing Registry Decoder

2011-09-06T12:18:00.002-05:00

Digital Forensics Solutions is pleased to announce Registry Decoder, an open source tool that automates the acquisition, analysis, and reporting of Microsoft Windows registry contents. The tool was initially funded by the National Institute of Justice (NIJ) and is now ready for public release. Please see our History Page for information about the project.

Registry Decoder consists of two components, the first of which is a tool for online acquisition of registry files from a running machine. The page for this project can be found here. To safely acquire files from a running machine, we ‘freeze’ a copy of the current registry files using the System Restore Facility. This places the files into a read-only location and ensures that the operating system will not have the files opened (which would prevent them from being copied to external storage).

Beyond the current set of registry files, the acquisition component can also acquire historical files from the running system. These historical files are acquired from XP machines through the System Restore Point facility and through the Volume Shadow Service on Vista and Windows 7 machines.

The second component of Registry Decoder performs offline analysis (on an investigator’s lab machine) of acquired registry files. This project can be found here. The current version of this tool can process raw disk images, partition images, individual registry files, and the database of hives acquired by the online component. When given a disk image, the Sleuthkit libraries are used to parse the image and read each registry hive. This includes the ability to acquire historical files from System Restore Points as well as the RegBack folder of Vista and 7 images. Individual registry hives are processed using libraries from the RegLookup project.

After being provided with all registry-oriented evidence for a particular case, which can be any combination of registry files, disk images, and acquired databases, Registry Decoder performs a one-time pre-processing of the evidence. During this process, it creates a number of databases and metadata files that contain all information needed to analyze the files.

The analysis section of the offline component contains a number of powerful features. The first feature is Search, which allows for powerful searching across registry hives. The searching abilities include:

Filtering by hive keys, name, and data
Filtering by the last write time of keys
Searching individual terms or with a newline delimited search term file
Exact or wildcard based search
Viewing of search results
Automated reporting of search contents to HTML, PDF, or XLS

Another important feature of Registry Decoder is its plug-in system. This facility is similar to the plugins provided by RegRipper, in that individual plugins provide very specific analysis of a subset of data contained within the registry. Output of the plugins can be automatically exported into reports, in the same manner as for registry searches.

A third feature provided by Registry Decoder is differencing of registry hives. This feature utilizes the search and plugin subsystems to illustrate differences and similarities between two registry hives. This allows for viewing of changes across time from the same computer or comparing results of searches or plugins against files from multiple computers.

Finally, Registry Decoder supports browsing of registry hives through the file view. This is very similar to AccessData’s Registry Viewer and provides the ability to browse hives, view data, and acquire the last write time of relevant registry keys.

We hope that Registry Decoder interests you and that you will try it out in cases that you are working on. We believe that Registry Decoder significantly reduces the time, effort, and skills needed to perform complex registry analysis. By being open source and well documented, we also think that Registry Decoder provides a strong platform for future research and development within the registry forensics field. If you decide to use Registry Decoder, we would love to hear your feedback either through the comments section of the blog or you can email directly to: registrydecoder@digdeeply.com.

The August NolaSec Meeting

2011-08-17T09:19:00.000-05:00

To all our New Orleans readers,

The next @NolaSec meeting has been planned for Thursday, August 25th, at 5:30PM at the Bridge Lounge.

Full details of the meeting can be found here:

https://sites.google.com/site/nolasecurity/announcements/august-meeting

As usual, the food is on us and feel free to invite anyone who you think may be interested.

We already have a number of RSVPs, and are expecting a large crowd.

Materials from our GFIRST and OMFW presentations

2011-08-10T15:18:00.000-05:00

We have had a busy couple of weeks presenting at a number of conferences and workshops, and now finally have the time to distribute our materials.

First is Andrew Case's (@attrc) presentation at the Open Memory Forensics Workshop which can be found here. This talk covered Volatility's current Linux memory analysis capabilities as well as functionality that will be incorporated in the near future. These new features include Android support and kernel-level rootkit detection.

Second is the slides from our GFIRST presentation on investigating coordinated data exfiltration (here). This talk was co-presented by Golden Richard (@nolaforensix) and Andrew Case. The purpose of this presentation was to show the steps we took when investigating a complex, real-life data exfiltration case. We cover analyzing a number of evidence sources, how to correlate the raw data, and a process for coherently reporting the findings.

Andrew Case also did a Volatility Linux workshop at the recent Blackhat Vegas Briefings, and the newly developed functionality and plugins will soon be incorporated into the Linux branch of Volatility 2.0.

Our GFIRST presentation

2011-06-30T10:14:00.001-05:00

Now that GFIRST has posted their schedule and agenda for this year (here), we would like to announce that we will be speaking there on investigating coordinated data exfiltration. This talk will be based on an investigation we performed last year for a large organization and that involved multiple insiders working in tandem to siphon data outside the company's network.

Our speaking slot is from 2:30-5:00PM on Tuesday, and the presentation will be delivered by Dr. Golden Richard (@nolaforensix) and Andrew Case (@attrc). Please be sure to stop by if you are attending and say hello.

Phishing Web-Based Email Services with HTML5

2011-06-22T07:45:00.001-05:00

We are writing this for our often co-researcher, Joe Sylve (@jtsylve), who has just released an interesting paper on a new phishing technique that utilizes "the programmable session history stack introduced in HTML 5." We found the paper and its results to be quite telling, and see it as another example of the security issues caused by the explosion of features put into the HTML5 standard with little or no security consideration.

The paper can be found here:

http://dl.dropbox.com/u/17627038/papers/html5phishing.pdf

The Open Memory Forensics Workshop (OMFW) is now open for registration

2011-06-08T11:41:00.001-05:00

The 2011 Open Memory Forensics Workshop is now open for registration:

https://www.volatilesystems.com/default/omfw

The workshop will be held along with DFRWS 2011 and Digital Forensics Solution's Andrew Case (@attrc) will be presenting. We will have more details on the presentation as the event gets closer. Anyone interested in recent advances and current research in memory forensics is highly encouraged to attend.

Android memory analysis research to be presented at SOURCE Seattle

2011-06-03T09:57:00.001-05:00

It has been a few weeks since we last posted, and we wanted to give readers a notice about new research that will be presented in two weeks at SOURCE Seattle (link). Digital Forensics Solutions' researcher Andrew Case (@attrc) will be discussing code he developed that performs automated memory analysis of Android's Dalvik virtual machine (link). This talk will cover Dalvik internals, how the data structures are accessed offline, and use of the developed functionality against a number of popular Android applications.

On a final note, we also recently found out that Andrew's Linux memory analysis workshop has been accepted for Blackhat Vegas later this summer (link). More news about this will be released in the coming weeks.

Honeynet Challenge Winners & Volatility Linux Support

2011-05-09T11:10:00.000-05:00

This post is a follow up to our previous entry about the HoneyNet challenge #7 (here).

We are now writing to say that the challenge results have been released (here), and that all of the winners (see the last five documents under "Attachment" at the previous link) used the Volatility Linux support documented in our previous blog post. Not only do these entries show the power of the developed Volatility functionality, but they also show how to use the Linux support in real-world case examples. Answering many of the challenge's questions, such as which process had the malicious connection open, which sockets were active, file system activity, etc, was as simple as running a plug-in.

We hope that the results of this challenge will inspire more people to give the Linux functionality a test and possibly to even contribute their own plug-ins.

PaulDotCom Appearance Tonight (4/28)

2011-04-28T09:22:00.001-05:00

We wanted to give a quick update to say that blog author Andrew Case (@attrc) will be on the PaulDotCom show tonight. He will be discussing research from his recent Blackhat DC presentation as well as some updates since first publication.

To listen in live to the show please use this link:

http://pauldotcom.com/live/

The show starts at 7PM Central time, and Andrew's work will be the focus of the Tech Segment.

Announcing Scalpel 2.0

2011-04-21T14:38:00.001-05:00

We are happy to announce the release of the next version of the Scalpel file carver, version 2.0 - the first public release in almost five years. There are a slew of performance enhancements and new features, focusing on improved carving accuracy and performance, and even more goodness is on the way.

Just some of the new features include:

Support for TRE-based regular expressions for headers and footers
Support for minimum carve sizes for recovered files
Parallel architecture to take full advantage of multicore processors
Beta support for NVIDIA CUDA-based GPU acceleration of header / footer searches
An asynchronous IO architecture for significantly faster IO throughput
Support for 32 and 64-bit Linux, Windows XP, Vista and 7, and OSX

Check back with us in the coming weeks for some ways to put these new features to good use, as well as for the introduction of some even newer functionality.

The new version can be downloaded from:

http://www.digitalforensicssolutions.com/Scalpel/

The download file contains pre-compiled Windows binaries as well as the project source code. If you find any bugs while using Scalpel please send an email to scalpel@digitalforensicssolutions.com. If you want to send comments to the authors, you can contact Golden Richard (golden@cs.uno.edu / @nolaforensix ) or Lodovico Marziale ( vico@digdeeply.com / http://www.linkedin.com/in/lodovicomarziale ).

If you are interested in the GPU research that went into this project, we published a paper at DFRWS that discusses both the CUDA architecture as well as the integration of it into Scalpel. It can be found here.

Interesting Registry Backup Feature of Windows 7, Vista, and Server 2008

2011-03-28T11:05:00.001-05:00

We have been doing quite a bit of registry-related research lately, and when I was investigating a Windows 7 machine, I noticed a folder "RegBack" under "C:\Windows\System32\config" (the normal directory where registry files are kept). This piqued my interest and upon viewing the folder, I noticed what looked like a backup of all the core registry files (system, software, security, sam), and they all had a last written time of about 8 days earlier.

Wanting to know what was controlling this folder, I Googled "RegBack", which resulted in about 77,000 hits related to registry tech support or anti-virus scan results. The key moment came when I saw a forum poster mention that this folder was controlled by the "RegIdleBackup" scheduled task. I then browsed my scheduled tasks library and found this task:

As can be seen in the picture, the "RegIdleBackup" task is scheduled to run every ten days and has a description of "Registry Idle Backup Task". Obviously, I did not create this task so I will assume its default in Windows 7. This would concur with many of the posts I found related to the RegBack folder.

I then decided to see if this behavior was the same on Vista and Server 2008. To my surprise, the RegBack folder and a registry backup existed on both of these operating system versions, but neither of them had the "RegIdleBackup" scheduled task. I then looked at the services list to see if any had a name related to registry functions, but did not find any. At this point I have yet to determine what controls the updating of this folder on Vista/2008 or when the update occurs. If anyone has insight into this please comment on the blog or email me about it and I will update the post.

After realizing that the all of the latest Windows versions contain a pristine, historical copy of the registry, I wanted to see if the existence of RegBack was known in the forensics community. I then emailed a few people who I know perform many related hands-on investigations and training sessions, and all of them said that they had not heard about the folder or its associated task. I then Googled for for terms such as "regback" "forensics" and "regback" "sans" and all results came back empty. The one reference I did find to RegBack examination was a CEIC 2007 presentation (here, slides 23-24) by Lance Mueller. The existence of this folder has obvious forensics implications, and can enable at least one revision of historical files to be gathered offline without having to interact with the volume shadow service.

Hopefully this post was interesting enough for a Monday morning read. If after reading the post you notice that your Windows 7 installation does not have the RegIdleBackup scheduled task or that your Vista/2008 installation does, I would be very interested in hearing about it.

Analyzing the New Honeynet Memory Analysis Challenge with Volatility

2011-03-03T11:05:00.001-06:00

Yesterday, someone in the Volatility IRC channel (#volatility on freenode), posted a link to a new honeynet challenge that involved Linux memory analysis (here), so I thought this would be a good way to showcase the new Linux features.....

Since many of the challenge questions can be solved directly with available plugins or minor manual analysis of plugin output, I think people should find it interesting.

For instance, the third question of the challenge is:
"What processes were running on targeted server? (2pts)"

This can be trivally solved with the linux_task_list_psaux plugin (output here), you can also use the kmem_cache plugin to get a partial list of exited processes

To analyze the memory capture yourself with Volatility, first download the memory capture ( here ) and then checkout the linux branch from SVN with this command:

svn checkout http://volatility.googlecode.com/svn/branches/linux-support vollinux

then "cd" to "vollinux", and run Volatility like this:

python volatility.py -f [path to downloaded memory dump] --profile=debian2626 [plugin_name]

If you were to use the linux_netstat plugin in the above command, you would see the equivalent of "netstat" on the target memory image, and it should go a long way in helping solve the challenge (hint hint)

If you have any questions/feedback then please comment on this post or use the normal methods to get in touch with Volatility developers.

Thanks and have fun!

Bringing Linux Support to Volatility

2011-03-01T11:50:00.002-06:00

Hello readers… This is going to be a long post where I (@attrc) attept to introduce ongoing (still very beta) work that I have been doing that allows for kernel-version generic processing of Linux memory images using the Volatility memory analysis framework. To keep things somewhat sane, this post will be broke into a couple of sections.

Let us begin…

1) Overview of functionality

First we will discuss the currently implemented plug-ins and features of the developed work. We will start with the plug-ins dealing with per-process data:

- linux_task_list_ps

- Gathers active tasks from the task_struct->tasks list

- linux_task_list_psaux

- Like linux_task_list_ps except gathers process command line information from userland, will eventually include the start time of each process (needs more research)

- linux_tasklist_kmem_cache

- Gathers active process through the kmem_cache (see this paper for more information)

- linux_list_open_files

- Lists open files per process

- linux_netstat

- Lists socket information per process

- linux_proc_maps

- Lists process map information (like /proc//maps) per process

Each of the plug-ins by default will analyze every active process. If you want to limit by a process or some subset of processes then you can indicate them by a comma separated list of process IDs with the –p option, such as –p 1,104,15 or –p 12345.

Next the plug-ins related to networking information:

- linux_arp

- Prints the ARP table

- linux_ifconfig

- Prints network interface information

- linux_route

- Prints the routing tables

- linux_route_cache

- Prints entries from the routing cache table (see this paper for more information)

And finally some miscellaneous plug-ins:

- linux_dmesg

- Prints the buffer shown by the dmesg command

- linux_lsmod

- Prints currently loaded kernel modules

- linux_mount

- Prints mounted devices as seen in /proc/mounts

2) Example command line usage

- Gathering active tasks

- python volatility.py -f [path to memory image] --profile=[name of profile ] linux_task_list_ps

For more information please see the Volatility README file in the branch root.

3) (Current) Caveats of version support

While the methodology for supporting the huge number of kernel version variations present in each Linux distribution is sound and automated, we currently have only generated profiles for our set of test kernels. Profiles are what allow Volatility to interact with a large number of kernels as they contain information about the investigated kernel including the System.map data and the in-memory layout of all structures. This allows for kernel version generic support and makes it trivial to support future kernels.

Currently we have generated profiles for the Ubuntu 2.6.32.25 and 2.6.32.27, Debian 2.6.26, and CentOS 2.6.9-89-EL kernels. These kernels and associated memory captures are what make up the current test bed and their version numbers are far apart enough to contain significant changes between kernel versions. Later in the post I will describe how to generate profiles for your own kernel if it is different from one of the above. Please do not try to use the current profiles on other kernels than the ones listed as they won’t work, even if they are closely related. Dealing with this issue will be substantially easier when a stable release with Linux support is done and is discussed later. The list of current profile names can be gathered by running python volatility.py –info .

Another caveat is that currently there is only kmem_cache support for SLAB based systems, so if you are using a distribution that utilizes SLUB you will receive an error along the lines of “Could not find a suitable allocator” when attempting to use kmem_cache based plug-ins. Again, SLUB support will be present in the first stable release.

The last discussed caveat is that currently the Linux support only works against 32 bit memory images as this is all Volatility supports. 64 bit support is currently planned and once the appropriate core Volatility functionality is developed, the Linux plug-ins will be thoroughly tested against 64 bit systems.

4) Developing and Testing still needed

Since the code is still in early beta stage there is need for further development and extensive testing. If you have coding and memory analysis skills, feel free to add your own plug-ins and research to the project. If you are looking for interesting plug-ins that still need to be developed then check the TODO file of the linux-support SVN branch. To get in touch with developers use the contact information found in the README file included in the branch root (IRC is usually best/quickest).

Testing is especially needed for popular distributions that are not included in our test bed (Redhat, SuSe, etc) and for the newest round of plug-ins (linux_arp, linux_route, linux_route_cache). If you are going to test one of the mentioned distros or your own kernel, you can generate a profile using two easy steps. In the following example I will assume you are going to generate a kernel for a 2.6.16 SuSe kernel.

First you need to install dwarfdump, which is available in source form or through most distributions’ package repositories. Once dwarfdump is still you then need the System.map for the running kernel and a debug version of the kernel (vmlinux). Most distributions package debug versions of their kernel within the repositories, and if you compile your own kernel, the compilation process will produce a file named “vmlinux” within the directory you type ‘make’ (usually /usr/src/linux).

You then run the command as:

dwarfdump –di vmlinux-file > dd-out

Next, you use the Python script included with Volatility tools/dwarfparse.py like so:

python tools/dwarfparse.py –s [System map file path] dd-out > suse_2_6_16_vtypes.py

Once the vtype file is created it then needs to be copied into the volatility/plug-ins/overlays/linux/ directory. To finish, simply copy and modify one of the existing profile scripts, such as centos.py, and name the profile appropriately to use your new vtypes.

If you need help creating a profile, please visit the Volatility IRC channel (#volatility on freenode) or comment on this post. In the stable release of Linux support this process will be fully automated and not require a debugging version of the kernel if the source code is present, which is the more common scenario. The stable release will also contain a large number of profiles and will hopefully minimize the need for users to create their own.

Obtaining a memory capture can be performed a few ways. The easiest is to use the suspend feature of Vmware Workstation (and possibly the free server), which creates a *.vmem file in the virtual machine’s data folder. This *.vmem file is a bit-for-bit copy of RAM and Volatility can be run directly against it. If you do not have access to Vmware you can run either the crash or fmem driver and then use dd to capture memory. If you choose to use fmem, please read its README file before attempting to use dd with it.

If you discover across a bug in the current code when testing, please file a bug using the online tracker which can be found here with instructions here .

5) Access to Source Code

Access to the current code can be found at http://code.google.com/p/volatility/source/browse/branches/linux-support/ and checked out by:

svn checkout http://volatility.googlecode.com/svn/branches/linux-support volatility-linux

If you encounter a bug or want to stay with the latest development, please be sure to update your checkout often as development is very active.

6) Future Plans

Besides the previously mentioned topics, future plans for Linux support can be found in the TODO file of the linux-support branch. Once the Volatility 1.4 is released, there will be a concentrated effort to get stable Linux support released, and we expect a number of improvements during that time.

7) Further Reading

If you are interested in Linux memory analysis and the theory behind the plug-ins there are a number of references to check. First are three books which deal extensively with Linux Internals, 1 2 3 . Second there are a number of published papers in the field of Linux memory analysis which can be found through these links 1 2 . You should also visit http://lxr.linux.no as it has a web-based LXR installation for all kernel versions and allows searching, cross referencing, and so on. It is an amazing time saver during research.

Hopefully this post was not too overwhelming and inspired people to give the new Linux support a try. In future posts we will be presenting some interaction with the new support including malware detection capabilities currently being developed. Be sure to follow the blog’s twitter account @dfsforensics in order to get the latest information.

Forensic Examination of Pointsec Encrypted Drives

2011-02-07T13:48:00.003-06:00

Many organizations use Pointsec (Check Point) full disk encryption in order to keep their data secure, especially in the case of laptops. As forensics investigators, we are occasionally tasked with creating forensically sound, decrypted images of Pointsec encrypted drives for preservation or investigation. This is a notoriously difficult task, as no existing forensics tool has the push-button ability to create a decrypted image from an encrypted one. While there are a few online resources documenting this process, they are contradictory and occasionally incorrect. In order to help out other investigators faced with this challenge, we are going to present a set of steps and pointers for acquiring a forensically sound (with caveats) decrypted image from a Pointsec encrypted drive image. Note that this post is not about breaking the encryption, just about creating a decrypted image assuming we have required security credentials. Also, since the process has failed a few, rare times in our lab, alternative methods will be presented in a future post.

Things you need:

A Pointsec encrypted raw drive image
A workstation with LiveView (and an underlying VMWare installation)
A custom BartPE CD with the Pointsec Dynamic Mount Utility (DMU) for mounting the encrypted image, and with FTK Imager. Instructions for creating this disk with DMU can be found in the Dynamic Mount Utility Administration Guide (currently HERE: https://updates.checkpoint.com/fileserver/SOURCE/direct/ID/11801/FILE/CP_2.0_FDE_Dynamic_Mount_Utility_AdminGuide.pdf). You will need to add FTK Imager to the disk as well.
A network share to write the decrypted image to.

The process:

Use LiveView to generate the configuration files only for the encrypted drive image.
Open the newly-generated configuration file (.vmx) with VMWare Workstation and:

Set the CD-ROM to the correct drive letter
Add and configure a Network Adapter (use NAT if in doubt)

Open the vmx file itself in a text editor and add the following line to the end:

Bios.BootDelay = “10000”

(this adds a 10 second boot delay, or else step 6 becomes difficult)

Load the BartPE CD into the physical computer’s CD ROM tray
Launch the virtual machine in LiveView, select to continue when prompted.
Hit ESC to enter the VMWare boot menu, and select the CD-ROM drive to boot the BartPE CD.
Once BartPE is fully loaded, configure networking at the prompt.

Set the correct network settings.
Map a network share for writing the decrypted image.

Run the DMU utility and select the encrypted drive.
Enter the require credentials (local Administrator has worked for us in the past).
Once the drive appears as “unlocked,” use FTK Imager to create an image of the unlocked local drive onto the mapped network drive.

Troubleshooting and implementation notes:

We have tried imaging the unlocked drive with dd and dcfldd, but neither run correctly inside BartPE.
You may have to configure networking and / or map the network drive in BartPE more than once before it sticks. Be sure to use the provided “PE Network Configurator.” Configuring with “NET USE” or “ipconfig” from the command line does not appear to work correctly (or at all).
Sometimes when booting BartPE or when running DMU, you get a BSOD “STOP 0x0500????”. These error codes do not appear to be documented anywhere, and to get around it you need to use one of the alternate methods we will present in the next installment.
DMU does not have an option to mount the encrypted volume read-only. Forensically, this means that the decrypted image will not be exactly as the encrypted image was, but since we were not booted to the encrypted image, and we are careful not to touch any files on the volume (right?), this method should be perfectly acceptable with proper documentation.

Hopefully this guide will help other investigators to gather a forensically sound disk image of Pointsec encrypted laptops. In future posts, we will present alternatives methods to the one presented that accomplish the same task.

Exploring The Sleuthkit's new tsk_loaddb feature

2011-01-25T08:58:00.002-06:00

The Sleuthkit's latest release (3.2.0) contains a new feature tsk_loaddb that loads all metadata about a disk image into an Sqlite database. This is a very interesting feature as a long standing issue with TSK is that you had to rerun commands over and over, and that information was not cached between invocations. tsk_loaddb fixes this as all information necessary to examine an image is saved in the created database.

Unfortunately the database structure is not documented, so in this post we aim to reveal important parts of the database and to show how this new feature can be used to examine cases with TSK much more efficiently.

In this example, we have created an image file with /dev/zero, formatted it ext3, and mounted it on loopback. We then used gen_fs.py to create a simple directory layout within the mounted disk image. After that, we created the TSK database with:

./tsk_loaddb disk.img

and this created an Sqlite database named disk.img.db. Upon examining this database we see the structure shown here.

Within this database structure we can observe usual TSK information such as the disk partition, files contained, and block level data. Now examination of TSK data is as simple as SQL queries, which we will show for a few common scenarios.

1) Locating blocks of a file ('march.xls' in this example) contained in the disk image

sqlite> select blocks.blk_start from tsk_fs_blocks as blocks, tsk_fs_files as files where files.name="march.xls" and files.file_id=blocks.file_id;
99331
sqlite>

Block 99331 is reported from the query and if we use blkcat to read this block, we see that it matches the information contained in the gen_fs.py script. If we are recovering a file that spans multiple blocks, we can simply include the blocks.blk_len column in the select statment and it will report the number of blocks used by the file.

2) Locating the filename for an offset in the disk

A very common scenario in forensics investigations is searching for keywords and then determining the context of matches. This often requires determining which file the matching keywords were found in or if they are in unallocated space. Before the tsk_loaddb feature, this step required using a number of TSK commands in succession for each offset found by a file carver or indexer. Again, with the new feature it's as simple as an SQL query.

First, we determine where on the disk our string of interest (EEEEE, see gen_fs.py) is located:

# strings -t d disk.img | grep EEEEE
101714944 EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE

Second, we get the block size from the image and divide our offset by it:
sqlite> select block_size from tsk_fs_info;
1024
sqlite>
101714944/1024 = 99331

Finally, we search for which file corresponds to this block:

sqlite> select files.name from tsk_fs_blocks as blocks, tsk_fs_files as files where blocks.blk_start <= 99331 and blocks.blk_start+blocks.blk_len >= 99331 and files.file_id=blocks.file_id;

This query works by finding a file that
1) starts either at or before our block of interest
2) ends at or after our block of interest
This allows for immediate retrieval of filenames related to keyword searches and file carving. As discussed at the end of this post, this process can easily be scripted for a large number of files.

3) Time based activity searching

Many cases involve searching for file activity based on timelines. Since the sleuthkit database contains MAC time information, we can now perform much of this work through SQL queries. For example a simple search of files created before a 8:23PM on 01/24/11 could look like:

sqlite> select name from tsk_fs_files where datetime(ctime,'unixepoch','localtime') < '2011-01-24 20:34:00';

Now, there are a few things to explain about this query. First, we are querying based on the create time of the file (ctime) and in order to have this file match our local time zone, instead of UTC, we have to use the Sqlite datetime function to format the date correctly. Second, the date we want to search against needs to be written as is formatted after the less than (<) sign. Once we have this syntax correct, we can quickly search for files based on complex, multiple parameter queries such as 'created on X but modified after Y' or 'accessed between Z and Y'.

In this post, we hope to have showed some of the power contained in the new tsk_loaddb feature as well as inspired people to construct new queries and research new automated capabilties. Obviously we have performed our work through the direct SQL interface, but scripting interactions in Python or another language would be trivial using the provided Sqllite libraries. We also have envisioned a number of higher level capabilities that can be easily made using this feature, and hope to implement them and share them on the blog in the future.

Speaking Materials from our talk at Blackhat DC

2011-01-21T13:56:00.005-06:00

Digital Forensics Solution's researcher Andrew Case recently presented forensics memory analysis of Linux Live CDs and the Tor anonymity project at the Blackhat DC 2011 conference.

The talk has already received considerable attention and Tor quickly fixed some of the issues that were discussed. Two bug reports ( 1 and 2 ) were filed and the most recent release contains patches that sanitize sensitive memory after use.

We are now working to get the presented live CD research integrated into Volatility so that the research becomes available to the general public. Once this occurs, investigators of all skill levels will be able to properly handle cases involving live CDs.

The white paper that was published can be found here and the slides that accompanied the presentation here.

If you have any questions or comments about the research please email Andrew or comment below. Also, check back Monday as we will have a writeup of other Blackhat presentations that caught our interest.

A New Blog About Digital Forensics and Computer Security

2011-01-21T12:23:00.004-06:00

Welcome to Digital Forensics Solution's blog about computer security and forensics. We plan on using this blog to document our research, presentations, software projects, and interesting things we discover along the way.

Our past work can be found on our research page and information about contributing authors can be found on our about us page.

We already have a number of topics that we plan on blogging about over the next week or two so please check regularly for updates.