Profile Based Digital Forensic Preview
The newest build of REAPER Preview (officially Alpha 2) includes quite a few changes, but one that I am especially excited about is Profile Based Preivew. First I will describe the new REAPER Preview process:
REAPER Preview is designed to be highly-automatic preview. When REAPER Preview starts an ‘autorun’ profile is detected. Any file type filters, hash databases, and keyword lists that an investigator always wants to search for, are automatically scanned. These lists, obviously, need to be pre-set by by an experienced investigator for items that ALWAYS need to be scanned for. This profile should be extremely generic.
While the automatic profile is running in the background, the investigator can choose a specific pre-created profile. A specific profile could be, for example, exploitation, hacking, financial, etc. Each of these profiles would have specific hashes, keywords and preferred file types to search for. By creating a profile you not only control what is automatically searched for, but what and how found items are displayed. For example, in an exploitation case hashes and images might be the most important, where music files may not be relevant. You can choose to remove music as a display option, forcing the first responder to focus on images/hashes (since that is all they have access to). The scanning is automatic so they do not need to do anything except click the link of the profile they would like to view.
I often hear that generic keyword lists make no sense. To my knowledge there has not been a study linking keyword lists to profiling machines. I know investigators that have certain keyword lists in their heads for certain types of cases… why not use these to attempt to profile a system? I do, however, see the need for manual keyword searching since static keyword lists would not include names, etc. Because of this each profile also supports manual keyword searching against file names and full disk.
Essentially REAPER Preview is a tiered system, from highly-generic (autorun profile), to case-type specific (pre-set profiles), to incident specific (manual searching). I believe these three layers of abstraction can help an investigator quickly dig deeply into a system while not missing important information that might be more general.
I welcome you to try REAPER Preview yourself - it can be downloaded from the REAPER Forensics project page: http://sourceforge.net/projects/reaperforensics/
Any and all feedback is most welcome! Want to see a specific feature? Found a problem? Let me know!
REAPER Preview Alpha 2 changelog
Gearing up for the official Alpha 2 release of REAPER Preview here is the change log and feature list:
Back-end:
<ul><li>REAPER Preview no longer loop-mounts suspect drives. All data is parsed directly from the raw disk. This not only faster, but we also do not need to worry about the issues talked about here.</li><li>Suspect disks are still set to read-only at the block level</li><li>Back-end is structured in a much more modular way. Programmers could easily insert a certain tool into the work-flow, if necessary</li><li>Preset automatic keyword searching</li><li>Preset automatic hashdb searching</li></ul>Front-End:
<ul><li>Whole code re-write. Front-end is now completely modular. Add or remove items with a simple include</li><li>Triage profiles supported! Will explain the concept of triage profiles in the next single</li><li>Greatly-improved Image/video gallery from Dynamic Drive implemented. The automatic image gallery I wrote was not powerful enough, and theirs is very nice. (Video previews are currently not working since the back-end switch).</li><li>Manual file-name and full disk keyword searching now available</li><li>Improved session logging (non-persistent)</li></ul>
Video Preview from Command Line with ffmpeg
Earlier I singleed about creating an animated preview gif from a given video. When using that method with a file list, ffmpeg would treat the file name as a data stream when read directly into the loop by piping the input file into the loop after done (see the first loop below). I like that method because it is easy, and uses ‘read’ which does not complain about spaces in file names.
You can read more about what I was trying to do here
The fix for the file-name-as-a-data-stream error is to dump each line of the input file into an array first. That is what I am doing below.
*Note: You have to set the IFS to something else if the file names have spaces oldIFS=$IFS
IFS=:
while read line; do
filenames=("${filenames[@]}" "$line")
done > $errorfile
fi
let COUNTER=COUNTER+1
if [ "$COUNTER" -eq "10000100" ]
then
echo "First 100 Thumbnails Generated: `date`" > $OUTPUT/videosdone
fi
done
fi
IFS=$oldIFS
Video Screenshot Preview gif Built from Command-Line Linux
Edit: This version will produce errors when using a file list. See this single for a more reliable way.
I have been searching for a while for a way to create a video preview from the command line in Linux. Not just a simple screen shot, but an animated gif of screen shots throughout the video. My thinking is that a screen shot of a video at a random time may not look suspicious, but the next frame may be something illegal. Essentially for a video I would like to take 4 - 6 screen shots regardless of the duration, compile these into an animated gif, and display the preview.
First I have been looking at my options:
I am on Debian ‘Lenny’, and while vlc might look like a good option, the lenny release is stuck at 0.8.6. The newest release is 1.1.4 (I think), but in 0.8.6 the –start-time switch is ignored. I tried upgrading using sid, but ran into a bunch of problems and decided not to mess with it.
I looked into mplayer which created screen shots, but I could not easily find how to divide the duration into 6, and quickly take the snap shot at those times. Basically I just got a bunch of sequential snapshots, and when I put them together would make the video again. I could delete some in the middle to get the desired effect, but thought there had to be an easier way. Also mplayer gui always wanted to start, and I did not want that.
Finally ffmpeg - with ffmpeg and imagemagick I was able to get something similar to what I wanted.
First the ffmpeg line
ffmpeg -i $file -ss 120 -t 120 -r 0.05 -s 90x90 f%d.jpg
What this does is takes the input video file ‘$file’, starts at 2 minutes (-ss 120), runs for 2 minutes (-t 120), sets a very low frame rate (-r 0.05), re-sizes the preview to 90x90px (-s 90x90), and names all the output images f#.jpg (f%d.jpg). Rather than calculating the duration, making the frame rate low gives a similar effect. I will write duration calculation later.
So once we run that we have a directory full of *.jpg files. We need to roll them into one animated gif. For this I use imagemagick. I have seen a lot of people who are using gimp for this. I love gimp, but imagemagick is easier converting a bunch of jpgs to an animated gif.
convert -delay 100 -loop 0 f*.jpg $file.gif
adapted from here
This will group all the jpg files in a loop with approx a 1 second pause per image. Works a treat!
Here is the first preview I tested (have only tested with .ogm and .mp4 so far)
Here is my full bash script to do the processing. It takes a file name as an argument - the loop is to deal with file names with spaces.
#!/bin/bash
echo "$1" | while read file
do
if [ -f "$file" ]; then
echo "Creating preview of $file"
ffmpeg -i "$file" -ss 120 -t 120 -r 0.05 -s 90x90 f%d.jpg
fn=$(echo ${file// /}) # Remove spaces in filename
convert -delay 100 -loop 0 f*.jpg $fn.gif
rm *.jpg
fi
done
exit 0
CarvFS on Mac OSx
A while ago I briefly used CarvFS on a linux system for testing. It was nice. Zero-storage carving can come in handy, especially when you are dealing with live CD systems. But installing on Mac would make experimentation and testing a bit more handy than running a VM. If you are reading this you might have had the experience of trying to compile CarvFS on Mac, have failed, and are stuck. Fear not!
Error when compiling on Mac
<blockquote>CMake Error at CMakeLists.txt:21 (MESSAGE):
No compatible (>= 1.0.0) version of libcarvpath found</blockquote>
First, a blog I really enjoy int for(ensic) blog has notes and downloads to install via Darwin ports. These can be found here: http://computer.forensikblog.de/en/2010/08/carvfs_on_a_mac.html
*note - if you use the Darwin ports method he uses patches for libcarvpath, carvfs, and the ewf module that I do not use!
But me being stubborn, I don’t like to use Darwin ports since I can compile what I want 95% of the time. Welcome to the 5%. So looking at forensikblog’s port file you can see what you need to change. By the errors it looks like it is only a library file, but it is also a bit more. So here is my non-Darwin ports CarvFS tutorial:
<ul>
<li>Install cmake: http://www.cmake.org/cmake/resources/software.html</li>
<li>Install FUSE: http://code.google.com/p/macfuse/</li>
<li>Install libcarvpath: http://sourceforge.net/projects/carvpath/files/LibCarvPath/libcarvpath1.0.0.tgz/download</li></ul><ul><li>Download and extract carvfs: http://sourceforge.net/projects/carvpath/files/</li></ul>
In the carvfs directory there is a ‘src’ sub-folder. Inside that replace the CMakeLists.txt file with this one [broken link, sorry]
Edit ‘carvfs.c’ where it says
<blockquote>sprintf(imgtypelib,”libmod%s.so”,imgtype);</blockquote>
to be
<blockquote>sprintf(imgtypelib,”libmod%s.dylib“,imgtype);</blockquote>
Then in the main carvfs directory run the command:
<blockquote>
<div style="text-align: left;">cmake -DCMAKE_INCLUDE_PATH:PATH=/usr/local/include -DCMAKE_LIBRARY_PATH:PATH=/usr/local/lib -DCMAKE_INSTALL_PREFIX:PATH=/usr/local -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON src</div>
</blockquote>
If everything is ok, you will get a make file. Then you just do the standard ‘make && sudo make install’
Thanks again to int for(ensic) blog.
SIMILE Timeplot graphing hours minutes seconds
All of the examples for SIMILE Timeplot are in YYYY/MM/DD format. I was wanting to plot data down to the minute/second. Looking around I found that the date format of the data (.txt) file should be ISO8601. I tried, but still had problems parsing the time part. Thanks to this blog I saw two problems I was having. First, the time format should be: YYYY-MM-DD HH:MM:SS. So a [time,data] file would look like [2009-02-12 15:10:00,23.407]
Next is with the index.html that loads the plot data. Look for the linetimeplot1.loadText(dataURL, " ", eventSource);
What I did not immediately recognize was that the “ “ bit is actually a field separator. So for comma separated values that line should be:timeplot1.loadText(dataURL, ",", eventSource);
Once done everything worked as expected. SIMILE Timeplot and Timeline are great tools. Hope this saves you some research time.
http://www.simile-widgets.org/timeplot/