We’re happy to announce that Basis Technology will be sponsoring Autopsy® 3 module development contests each semester for current university students! We think this is a great way to expand the functionality of Autopsy and get students involved in the open source development process.
If you are a teacher or student comfortable coding in Java, have a look at the contest page. If you are a teacher and want to use this contest as part of your class projects, then contact us. We're happy to help. Or, even if you don’t need anything from us, let us know and we’ll list you on the site as being part of the contest.
If you are not a student, then stay tuned for a module writing contest as part of the 2014 Open Source Digital Forensics Conference (OSDFCon). See the 2013 challenge page for last year's results and more details.
January - June 2014 Contest
There is a $600 cash prize as well as free admission to the 2014 OSDFCon that will be awarded to the winner, $400 for second place, and $200 for third. In true open source fashion, the final winners will be chosen using crowdsourcing techniques.
The point of these contests is to encourage creative thinking and provide some financial incentive to contribute useful (and free) functionality to the Autopsy community, and as a result the rules of the contests are minimal. The major restriction for participation is being a current undergraduate or graduate student with a .edu email address used for the submission to: firstname.lastname@example.org. The deadline for all submissions is June 27, 2014. For additional details about submission criteria and selection criteria see the contest page.
Also check out the existing modules page on sleuthkit.org, the github.com issues log for ideas, and the developers guide to get started!
Create your free online surveys with SurveyMonkey
, the world's leading questionnaire tool.
As readers of this blog know, Autopsy was designed to be a digital forensics platform that other open source developers can build modules for. To help motivate other developers to write Autopsy modules, Basis Technology created a module development challenge and we’re pleased to announce the winners.
The ground rules were simple:
- Make something useful and creative that can plugin to the Autopsy platform and release it as open source software
- Submit the module before the Open Source Digital Forensics Conference (OSDFCon).
- Present the module to the attendees of OSDFCon in person or via video.
- Profit! (in the form of cash prizes!)
We received two really great submissions. We were impressed by the amount of effort that went into each one of them (and note that we did not award a 3rd prize because there were not enough submissions, so you could have won some cash with even a basic module!). These modules have been tested by the Basis team and work with 3.0.7 and above in box 32 and 64-bit versions of Autopsy.
First Prize: $1,500
Author: Willi Ballenthin
Minimum version of Autopsy required: 3.0.7
Description: Willi wrote two modules that support registry analysis. One is an ingest module that detects registry hives and extracts the keys and values into “derived files” of the registry hive. This means that they are shown in the directory tree and you can navigate the registry structure and search its contents.
The second module was a new content viewer (the area in the lower right of Autopsy) that will show the tree of a registry hive and allow you to navigate it after you have selected the hive. If you use only this module, you will not see the registry expanded in the directory tree.
Both of these modules are great additions to the capabilities to Autopsy and provide the user with functions much like Regedit.exe to view registry hives.
License of source code: Apache 2
Second Prize: $500
Author: Petter Bjelland
Minimum Autopsy version: 3.0.7
Description: Petter developed a fuzzy hashing module based on sdhash. sdhash allows you to match files that are similar, but not necessarily exactly the same, as other files. With this ingest module and a new viewer, the investigator can match files against other files or sdhash reference sets during ingest, or search for similar files from the directory viewer or search results after ingest. Petter could not attend OSDFCon and instead submitted a video of the module in use. It is linked to below.
In addition to the great contribution to the community with this open source module, Petter also donated his cash prize to the Red Cross to benefit victims of Typhoon Haiyan in the Philippines.
Source URL: https://github.com/pcbje/autopsy-ahbm
Release Download: https://github.com/pcbje/autopsy-ahbm/releases
License: Apache 2.0
Video presentation: http://youtu.be/GBmZRufH_3o
We think these two modules show the power of the platform and the ability for it to change and evolve using the developer’s guide and some creative thinking.
See a list of all third party modules here: http://wiki.sleuthkit.org/index.php?title=Autopsy_3rd_Party_Modules
Congratulations and thanks to both Willi and Petter from the entire Autopsy community!
We’ll be doing this again next year alongside OSDFCon with the same rules, so feel free to start developing your modules now.
Basis Technology is offering an Autopsy training class directly following the Open Source Digital Forensics Conference (http://OSDFCon.org) in Chantilly, VA. This is part of an initial push to establish the premier training program for this powerful open source digital forensics tool.
For those who have already downloaded and tried Autopsy (and there are over 13,000 people who have done so for the current version) or those who have seen us present at conferences, you may be wondering why you need the training. After all, we’ve designed Autopsy to be easy to use and intuitive out of the box. The reason you want to attend this training is that you’ll learn about what is going on under the covers. Anyone can press buttons, but to testify about a tool, you need to understand what happens when you press the button.
You know you’ll get all of the details because it will be taught by a combination of engineers and examiners. The current plan is to combine the use case experience from one of our examiners with the implementation details from one of our engineers (likely Brian Carrier).
The course spans two days and includes both lectures and hands-on examples. Computers are provided with the training. During the class, we’ll cover:
- Autopsy set-up and overview
- In-depth coverage of each of the ingest modules, including:
- Keyword Search
- Hash Lookup
- Recent User Activity (registry, web, etc.)
- Archive Extractor
- Views and how to use them.
- Tagging and reporting
- Hands-on case study with tutorials and problem sets
- Harnessing automation and workflow feature
Register for the training here.
The current incarnation of the Autopsy training class does not provide a certification. We understand the benefits of being certified to use a tool and are working towards defining the criteria of the certification so that it is viewed as a respectable level of achievement. If you interested in getting involved with this process after you attend the class, let us know.
Training for Developers
One of the key things that we talk about with Autopsy is its extensibility and how we designed it to be a platform for others to build modules for. We know we can’t solve everyone’s problems. This 2-day course is designed for examiners who are going to use the tool. We didn’t forget about the developers though.
We have a ½ workshop at OSDFCon about developing modules for Autopsy. Register soon for that event if you want to learn how to integrate your existing tool (or a new tool) into Autopsy. This will enable you to reach a bigger audience and not have to worry about disk images and file systems.
Training for Trainers
At this point, we are doing all of the training ourselves. If you are interested in us coming to your site to conduct training, let us know. We will schedule more events for 2014 and feedback on where we have significant interest is important.
If you teach at a University and want to involve Autopsy in your curriculum, let us know. We are starting to provide some outreach on this topic and can facilitate sharing of curriculum materials between educators. Contact us if you are using Autopsy and want to share your resources with other educators.
Register for the training and get some more details, by clicking here.
Keyword searching is a common and widely used investigation technique across all varieties of digital investigations. On the surface, it seems fairly straightforward - figure out what names, places, things, activities, applications, etc. you want to search for and perform the search. Simple right? What about misspellings? Or patterns of text versus an exact match? Many of the commercial digital forensics tools help solve some of these considerations and so does Autopsy - but it does it for free. Autopsy uses the open source Apache Solr and Tika libraries for fast and efficient text indexing and searching.
There are two basic times when you can keyword search in Autopsy.
- Keyword lists can be used during ingest when the files are being added to the case.
- Individual ad-hoc queries can be performed during the course of the analysis.
These two methods can actually happen at the same time during an investigation because you can start your analysis while the ingest process continues to process files. If you find keywords that you want to add to your search, they are added to the ingest list and will occur in the background. This means you don’t have to wait to start your investigation until your tool finishes processing. Why wait? Investigate!
Ingest modules and Near Real-time results
When adding a disk image to a case, the user chooses the ingest modules that they want to run on the image. One of the standard modules is Keyword Search and it will extract the text from files and add them to a Solr index. It will periodically, default is every 5 minutes, query the index for a list of keywords that the user configures.
The user can search for plain text search strings (like “Jesse James”) or patterns (regular expressions - REGEX). Autopsy comes with a list of pre-defined regular expressions to find phone numbers, email addresses, IP addresses, and URLs. A common strategy is to load up the keyword lists with common misspellings of important words or develop REGEX that will find close matches words or patterns of interest.
Autopsy’s ingest modules run in the background and make every effort to stay out of the way during the investigation, but things like keyword hits and other artifacts could be really helpful and interesting to an investigator to know about as they are discovered during the file analysis process. Autopsy publishes the keyword hits it finds in two places:
- The evidence tree on the left-hand side of the main UI
- The ingest inbox, which has an icon in the top right of the main UI
The motivation for the ingest inbox is that it gives you a chronological perspective on what has been found. If you are focused on the user’s web activity, then you won’t notice that they keyword hit on a specific term went from 4 to 5. The ingest inbox though will tell you what has been found since you last opened it. The goal is to notify (but not annoy) an investigator that new evidence items of interest have been found by the background analysis tasks.
Ad-hoc Searching and Using an Index
You can never know all of the keywords that you will care about when you start the case. Autopsy makes a text index (using Apache Solr) of the text on the drive so that later searches are very fast. You can think of a text index like the index in a book. It allows for a direct mapping of words or concepts to pages and the locations those words appear in text. For instance, if you wanted to find all the mentions of the words “digital forensics” in a text book - you could page through each page regardless of whether it had your phrase of interest on it one by one and highlight the phrase when you found it, or you could look in the index and go to the pages it shows up on directly. Autopsy works like the second option when doing keyword searching which means you get your results fast.
Making ad-hoc queries happens through the search bar in the top right of the Autopsy interface. This accepts both REGEX patterns and plain search strings. You can also run new lists that you’ve created and loaded via the configuration options from this area of the interface. For each independent search, a new tab is created in the results viewer panel, which mean you can run multiple searches in parallel and review them independently.
There are two general methods for performing keyword searches in digital forensics:
- By interpreting the file types, extracting the text, converting it to Unicode (if needed), and matching it against the list of keywords.
- By coming up with all possible byte sequences of the keywords in the possible encodings and looking for all of those byte sequences at the lowest levels of the drive data.
Autopsy extracts the text using Apache Tika and some other open source libraries. For files in formats that we don’t support or for unallocated space, we extract the strings. In the Tools -> Options area of Autopsy, you can configure which languages that you want to extract strings from. The more languages that you select, the more false positive data that you will see.
The benefits of extracting text are:
- Finds text in compressed formats.
- Finds text in file formats that make up their own encoding (i.e. PDF)
Extensibility from Solr
A big design goal of Autopsy 3 has always been extensibility. Solr provides its own methods of extensibility. Some examples include:
- Autopsy currently runs a Solr server on your desktop system, which could get overburdened under heavy loads. In the future, we could allow Autopsy to use a central Solr instance if your environment could benefit from this.
- Extend the text analytics capabilities to obtain better results. We are using the standard library features, but better commercial ones also exist. For example, the text analytics side of Basis has Solr integration options for their lingustics components to get better results with non-English documents.
Start using this free and powerful keyword searching feature today - download Autopsy from sleuthkit.org/autopsy.
Windows Forensics Environment (WinFE) is a bootable operating system environment that can be used for forensic examinations. It provides a live boot environment that allows you to examine a suspect computer in a forensically sound way. We thought it would be a good exercise to determine if Autopsy 3 could run in this environment.
As it turns out, Autopsy 3 works pretty well out of the box in WinFE Lite, making it a good choice for doing triage analysis. WinFE Lite is build of WinFE.
Due to some dependencies in Autopsy that aren’t available in the WinFE Lite environment, not all functionality exists. Specifically, you will be unable to view videos or open zip files.
The instructions for installing and running Autopsy 3 in WinFE Lite on the SleuthKitWiki. Both WinFE Lite and Autopsy 3 are free tools that can provide a powerful triage forensics environment with a little pre-work. Having a bootable device with these tools on it, can be an invaluable resource to any investigator.
Basis Technology has just released v1.0 of a new module for the Autopsy Digital Forensics Platform - Video Triage for Autopsy. Video content analysis can be time consuming for an investigator. Normally an investigator would need to watch an entire video either at normal or accelerated speed or scrub through and potentially miss key components. The Video Triage module for Autopsy creates a storyboard of a video by grabbing key frames at equally spaced intervals in the video and displaying these as thumbnail images.
There are typically two major use cases for what the Video Triage module provides:
1) Getting a gist of what a video file contains in terms of its primary content
2) Identifying whether or not there is content embedded in the middle of an otherwise mundane video (read: abusive acts at minute 35 of a scenic home video)
It’s important to realize that using the Video Triage module isn’t a replacement for deep analysis, but given the potential amount of video footage analysis that some investigations require, it can provide a good first step towards data reduction and identifying candidates for deeper analysis.
This is the first time we’ve released a public module as an add-on to take advantage of the platform aspect of Autopsy. We’ve even included an auto-update feature for the module so that as new features are added and bug fixes get applied, as long as you have Internet connection, you’ll get a notification about the update and be able to apply it right from Autopsy (no external web browsing needed).
Check it out at the Autopsy Modules area of our site.
Extensibility of Autopsy
Autopsy was built as a platform specifically to allow for these types of modules to be created. Any organization or individual can develop these types of extensions. A great place to start is by looking at the developer docs for the project. We’ve also put forth an Autopsy developer challenge for OSDF 2013.
Another example of a similar content viewer module we’ve created integrates Basis Technology’s Rosette Language Platform to translate and highlight names, organizations, and locations from documents viewed in Autopsy across a variety of languages. Currently this module is in the proof-of-concept stage, but could be further expanded for a full featured module.
Both the Video Triage and RLP viewer modules are examples of content viewer modules. For a description on the other types of modules that Autopsy supports, check out the API docs.
Video Triage Future Updates
In v1.0 we’ve released the core of what we wanted to get out to the community, but there are still areas for improvement. Most notably, we’d like to include some user options to increase the size of the thumbnails that are generated as well as the number of thumbnails (currently set at 12 for each video). These decisions were initially made to balance some of the performance trade-offs that would accompany larger thumbnails and more key frames, but we’ve got some ideas on how to counter those such that we can leave it up to the user based on their preferences.
Other ideas include allowing sub-segments of the video to be played independently from where the thumbnail was captured to give more context to the investigator about the key frame and pre-processing video on ingest (versus on demand to improve performance).
If you download the module and have ideas for future improvement, we’d love to hear about them. Feel free to email us at email@example.com to share your ideas for this or any other modules you’d like to see developed.
Timelines are useful in digital forensics for identifying when activity occurred on a computer and are mainly used for data reduction or identifying specific state changes that have occurred on a computer. Amazingly, this broad usage helps answer a lot of questions different investigators can have. That’s why we recently added a timeline feature to Autopsy in the 3.0.5 release. This feature will evolve over time and it is still considered beta, but currently it collects file system times, displays the activity in bar charts, and allows you to view the file contents as text, image, or hex.
The main reason that it is still beta is that we need to make some improvements with memory handling. It loads the entire timeline into memory and that doesn't scale for large numbers of events. We’ll fix that soon though. We’ll also be expanding the feature to pull in data from other data sources, such as web history, log files, and the registry.
We hope the open source community will find our initial implementation helpful in their investigations and would love to have some conversations with the community about it, so please let us know if/how you end up using it by engaging at sleuthkit.org.
Using the Timeline Feature in Autopsy 3.0.5+
Once you’ve got Autopsy 3.0.5 or greater installed (sleuthkit.org/autopsy) and have added a disk image to a case, you can access the timeline through the Tools menu item and selecting “Make Timeline (beta)”.
Behind the scenes, it will make a Sleuth Kit body file and run mactime to sort the data into a text timeline. Autopsy then parses the “mactime” output and displays it in graphs.
Initially, the graphs show the number of events per year. Selecting a year shows bars for each month in the year. Selecting a month will then show the number of events in each day. The bottom part of the screen allows you to see all of the files that have activity and the contents of each file. From this view, you can still use the thumbnail view to see thumbnails of all images on a given day and you can play videos as well.
You can also right click on a file in the lower left and choose “View File In Directory” to bring you back to the parent folder in the main Autopsy interface. This allows you to identify a suspicious file from the timeline view and then see what else is in the folder.
Use Case: Timeline Analysis in Intrusion Investigations
In time critical investigations such as those that take place in a post-breach cyber forensics investigation, it is important to be able to quickly focus on the relevant data. One way of doing this is through the use of file system timeline analysis. There are two big ways of using timelines: macro and micro.
At the macro-level, timelines are useful for seeing the big picture of how a computer was used. Often, the investigator’s first encounter with a computer could be after it has been compromised and he or she may not know what is normal. By looking at the high-level activity for the past month, they may be able to identify what directories had activity. This helps to determine what user accounts are used and what applications are used. This can be correlated with data in the registry.
At the micro-level, timelines are useful to seeing all of the places that had activity in a given time range when intrusion activity occurred (assuming that the file system time stamps were not modified by the attacker). This can be useful when you have a time frame from an external data source (such as network packets or IDS) or when you want to see all of the places that the intruder placed files.
Timelines are useful for both live analysis (when the suspected computer is still running) and dead analysis (when the suspected computer has been powered off).
For more information or to download Autopsy and the graphical timeline feature for free, visit sleuthkit.org/autopsy.