Saturday, January 15, 2011

EMET Glob.. Got 30 seconds? I might save you 30 seconds ;-).

So EMET is an awesome tool, it really is. If you are floating about the Internet on a Windows machine I highly recommend it. Sure your AV software may have some similar features but the protections will probably not be as robust and/or you may not have as granular control over protections. I will include a couple of links to good articles I've found regarding configuring EMET below.

  1. Article on protection against an Adobe 0-day. Note: the problem is in a dll so the steps outlined here need to be applied to any application that loads the vulnerable dll
  2. Rationally Paranoid write-up on EMET.
  3. James Mcquaid write-up on EMET.

One annoying thing about the command line configuration tool EMET_conf.exe is that it doesn't support file globs. So I whipped up the following batch file in a couple of minutes to deal with this. Hopefully it will save you a couple of minutes when adding executables to EMET for compatibility testing your applications. It would be nice to be able to toggle EMET options via cli as well.. Perhaps I will work on this next.


@echo off
SETLOCAL EnableDelayedExpansion EnableExtensions

rem - change this to the path of your emet_conf executable.
SET EMETCMD="%PROGRAMFILES%\emet\emet_conf.exe"

if "%1" == "" goto error_missing_action
if %2 == "" goto error_missing_glob

if "%1" == "add" goto emet_add
if "%1" == "delete" goto emet_delete
echo.
rem - add the user_supplied glob to emet
:emet_add
echo going to add files matching glob %2 to emet
@echo on
for /f "tokens=*" %%i in ('dir /s/b/p %2') do %EMETCMD% --add "%%i"
@echo off
goto end

rem - delete the user_supplied glob to emet
:emet_delete
echo going to delete files matching glob %2 to emet
@echo on
for /f "tokens=*" %%i in ('dir /s/b/p %2') do %EMETCMD% --delete "%%i"
@echo off
goto end

rem - user must specify add or delete as action
:error_missing_action
echo missing action argument you must specify add or delete!
echo usage emet_glob.bat adddelete "%PROGRAMFILES%\adobe\*.exe"
goto end

rem - user must specify a second argument of a base path to recursively search for files to add
:error_missing_glob
echo missing action argument you must specify add or delete!
echo "usage emet_glob.bat adddelete "%PROGRAMFILES%\adobe\*.exe"
goto end

:end
echo.
echo Done.


Monday, April 19, 2010

New Suricata Release 0.8.2!

The release notes below say it all ;-)...

The OISF development team is proud to introduce the 3rd beta release of
Suricata, the Open Source Intrusion Detection and Prevention engine. The
first release candidate is currently scheduled for early May, but check
https://redmine.openinfosecfoundation.org/projects/roadmap/suricata for
the up to date schedule!

Get the new release here:
http://www.openinfosecfoundation.org/download/suricata-0.8.2.tar.gz

New features

- Support for the following keywords: detection_filter, http_client_body
- The HTTP parser can now set server personalities
- threshold.config support
- The experimental CUDA code now also works on x86_64
- IP address only rules for IPv6 are now supported as well
- Suricata can now write a pid file (pass --pidfile )
- A fuzzer script was added to the code base
- Policy lookup for defrag module

Improvements

- Much better average and worstcase performance in the detection engine
- Memory footprint was reduced
- More validation at signature loading stage
- Libnet 1.1 is now optional
- Negated uricontent and http_cookie matching is now supported
- Lots of fixes of issues found by Valgrind's DRD, CLANG and Parfait.
- Threads are named now in "top" (Linux only atm).
- Unified1 file handling is improved

Bugs fixed

Many :)
Several segmentation faults, upgrading is highly recommended.

See
https://redmine.openinfosecfoundation.org/projects/suricata/issues?fixed_version_id=6&set_filter=1&status_id=c

Known issues & missing features

We have made significant progress towards reaching our first full
(non-beta) release of Suricata. Your feedback is always important to us
and we appreciate your time and effort. As always, we are doing our
best to make you aware of continuing development and items within the
engine that are not yet complete. With this in mind, please notice the
list we have included of known items we are working on.

- Using the http_cookie keyword seems to cause a match on all packets.
- Currently we dont' support the dce option for byte_test and byte_jump.
- Stream reassembly is currently only performed for app-layer code.
- Inconsistent time stamps in http log file due to handling & updating
of the http state.
- DCE/RPC over udp is not currently supported.
- dce_stub_data does not respect relative modifiers.
- Engine does not work properly on big endian platforms.
- Time based stats are not calculated correctly.

See https://redmine.openinfosecfoundation.org/projects/suricata/issues
for an up to date list and to report new issues.

Friday, April 9, 2010

Help Us Make Our Meerkat Fuzzier!

If you checkout the latest version of the suricata from it's git repo, now included is a new qa/ directory. In this directory there now lives a perl script I created called wirefuzz.pl that is a suricata specific re-implementation of the wireshark fuzzing technique described here. The script can also be used as a shortcut to running the engine through valgrind tools etc.

In it's simplest "set it and forget it form" you can leave it running and it will loop through the pcaps provided continuing to mutate them with whatever error ratio you provide, i.e. if you pass -e=0.02 there is a 2% chance that each byte will be modified in one of four different ways. It will continue to loop until a invalid exit value is detected, at which point it will try to find the a core dump, parse it and save the output to a file for further inspection.

We have found this technique to be a very effective way of discovering bugs. We have multiple instances of this script running 24/7 using an extensive collection of pcaps as ammunition. With that said we will never be able to account for the "uniqueness" that exists in real-world environments both large and small.

if you want to help the project please don't hesitate to checkout the latest version of the repo and point the fuzzer at perhaps your rotating packet capture, or a sample of your network traffic.

If you have any questions please don't hesitate to ask on the oisf-users mailing list, which you can subscribe to here, or you can leave a comment and I will try to respond. Ladies and gentlemen start your fuzzers!

To check the latest development version of suricata:

git clone git://phalanx.openinfosecfoundation.org/oisf.git

The script relies on a couple of perl modules Capture::Tiny and Devel::GDB. Here are a couple of tips on getting them installed:


Ubuntu 9.10
sudo apt-get install libdevel-gdb-perl libcapture-tiny-perl

RedHatES/CentOS 5
yum -y install cpanspec perl-Module-Build
cpanspec --packager OISF -v -s --follow Capture::Tiny
cpanspec --packager OISF -v -s --follow Devel::GDB
rpmbuild --rebuild *.src.rpm
rpm -ivh /usr/src/redhat/RPMS/noarch/perl-Devel-GDB*.rpm
rpm -ivh /usr/src/redhat/RPMS/noarch/perl-Capture-Tiny*.rpm

Fedora Core 12
yum -y install perl-Capture-Tiny perl-Devel-GDB

Other debain based versions, try the Ubunutu instructions if this doesn't work try the following.
sudo apt-get install dh-make-perl
mkdir fuzzmodules && cd fuzzmodules
dh-make-perl --cpan Devel-GDB --build
dh-make-perl --cpan Capture-Tiny --build
sudo dpkg -i *.deb

Output from wirefuzz.pl -h:

-h or help <(this output)>
-r=<(filemask for pcaps to read)>
-n=<(optional) number of iterations or if not specified will run until error>
-s=<(optional) path to ids rules file will be passed as -s to suricata>
-e=<(optional) editcap error ratio to introduce if not specified will not fuzz. Valid range for this is 0.00 - 1.0>
-p=<(path to the suricata bin)>
-l=<(optional) log dir for output if not specified will use current directory.>
-v=<(optional) (memcheck|drd|helgrind|callgrind) will run the command through one of the specified valgrind tools.>
-y <(shuffle the array, this is useful if running multiple instances of this script.)>

Example usage:
First thing to do is download and build suricata from git with -O0 so vars don't get optimized out. See the example below:
git clone git://phalanx.openinfosecfoundation.org/oisf.git suricatafuzz1 && cd suricatafuzz1 && ./autogen.sh && CFLAGS="-g -O0" ./configure && make

Second thing to do is to edit suricata.yaml to fit your environment.

Third go ahead and run the script.

In the example below the script will loop forever until an error is encountered will behave in the following way.
1.-r Process all pcaps in subdirectories of /home/somepath/pcaps/
2.-s Tell suricata to use the rules file /home/somepath/current-all.rules
3.-y Shuffle the array of pcaps this is useful if running multiple instances of this script.
4.-c Tell suricata to use the suricata.yaml in the current dir.
6.-e Tell editcap to introduce a 2% error ratio, i.e. there is a 2% chance that a byte will be fuzzed see http://wiki.wireshark.org/FuzzTesting for more info.
7.-p Use src/suricata as our suricata bin file. The script will determin if the argument passed is a bin file or a txt wrapper and will adjust accordingly.

/usr/bin/wirefuzz.pl -r=/home/somepath/pcaps/*/* -s=/home/somepath/current-all.rules -y -c=suricata.yaml -e=0.02 -p src/suricata

If an error is encountered a file named ERR.txt will be created in the log dir (current dir in this example) that will contain output from stderr,stdout, and gdb.

Take a look at the opts make it work for you environtment and from the OISF QA team thanks for helping us make our meerkat fuzzier! ;-)

Monday, March 1, 2010

New release of Suricata and a brief Istanbul summary.

I know that I'm a little late with this but we have released a new version of Suricata version 0.8.1 which you can get here. It is still beta quality code but we have made some significant changes/improvements which you can read about here. In addition to the known issues it should be noted that the PF_RING code in this version works with versions of the api prior to 4.1.2. Victor currently has a patch in his inbox that will allow it to work with later versions and for the user to specify a PF_RING load balancing type for the cluster i.e. load balanced per-packet or per-flow.

Secondly, I met with most of the OISF team in Istanbul last week. We had some very productive meetings and I feel we really got a good grasp on what needs to be done in engine in the coming year(s). On a more personal note it was great to finally put faces with names for those individuals I had not met prior to last week. Victor and I also got the chance to speak at local OWASP meeting in Instanbul along with Brian Rectanus from OISF/Breach. These were a great group of guys and despite the fact that Victor and I were totally disorganized (Jonkman was originally supposed to speak ;-)) they didn't seem to mind to much or at least they hid it well.

Thursday, December 31, 2009

Suricata Release

We have done it! We have released the first version of our brand-spanking new IDP engine named Suricata. For more info please see the official release announcement on the OISF website. Everybody on the team has been working hard day and night for these last six months to get this far.

As Victor mentioned in his post if you find a bug or have any feedback please don't hesitate to let us know via the project's redmine page.

Sunday, November 29, 2009

Very quick look at zero-copy bpf in FreeBSD 8.0

So it appears as if they have finally integrated Zero-Copy bpf support into FreeBSD 8.0. I have not had any time to do any real performance tests but I thought I would just throw a few little notes up about it. Zero-Copy BPF is not enabled by default, to enable it you must do the following.

sysctl net.bpf.zerocopy_enable=1

Once you set this option you can fire up any libpcap based application and it should use the zero-copy functionality. It appears as if there was a patch to netstat that wasn't integrated for the 8.0 release but can be found here. A few little fixes namely converting the %lu printfs to %llu instead will give you a netstat that will produce stats about zero-copy operations.

without zero-copy sysctl option set to zero...
./netstat -s -B
tcpdump: pid 3402 on ed0:
376 packets received
376 packets matched receive filter
0 packets dropped
0 current hold buffer size
1146 current store buffer size
0 packets written
0 packets matched write filter
0 packet writes failed
0 zero copy operations

with zero-copy sysctl option set to 1

FreeBSD-32-bit# ./netstat -s -B
tcpdump: pid 3424 on ed0:
745 packets received
745 packets matched receive filter
0 packets dropped
0 current hold buffer size
830 current store buffer size
0 packets written
0 packets matched write filter
0 packet writes failed
1490 zero copy operations

Wednesday, November 11, 2009

clang static-analyzer == awesomeness

If you are looking for a good open source static analyzer for c/c++ you know that it is slim pickins out there. There are great tools for detecting memory issues like valgrind and all of it's included goodies. Most open source static code analyzers are out of date, are just to darn difficult to get working (cough* cough* splint), or don't scale well to large projects like flawfinder and it's hey you have a static buffer at line x you better make sure you do proper bounds checking.

So in my search for a static code analyzer I stumbled across the clang static-analyzer and I must say that it is pretty darn amazing. While it doesn't detect buffer overflows at the time of writing, it informs the user of a ton of other issues that when resolved can lead to cleaner more efficient code. I'm just going to summarize the steps that I went through to get it up and running. Most of these steps are on the clang static-analyzer site or came from this blog post.

1. Checkout llvm using subversion

svn co http://llvm.org/svn/llvm-project/llvm/trunk llvm

2. Checkout clang using subversion
cd llvm/tools
svn co http://llvm.org/svn/llvm-project/cfe/trunk clang

3. Build llvm and clang
cd ..
./configure --prefix=/opt/clang
make
sudo make install

4.Clang static-analyzer isn't installed with make install so lets move it to the location where we installed everything else.

sudo cp -Rf tools/clang/tools/scan-build /opt/clang/
sudo cp -Rf tools/clang/tools/scan-view /opt/clang/

5.Add the different clang dirs to your $PATH. Usually this can be done by adding a line similar to the following in /etc/profile.

PATH=$PATH:/opt/clang/bin:/opt/clang/libexec:/opt/clang/scan-build:/opt/clang/scan-view

6.Go into the directory where your source code resides. If you have code that follows the normal ./configure, make, make install type of build you will want to do the following.

scan-build ./configure
scan-build -o /var/www/html/testresults make

7. Once it is completed you should see a message like "x diagnostics generated." Fire up your browser and got to the /testresults/ dir on the web server where you dumped your results. The interface is amazing as once you click on a bug it will actually walk you through the code, and do things for you like expand macros etc.

Enjoy ;-)