Ruby Apache Log-Viewer

Ruby Apache Log-Viewer

README

Check also Apache Security Tips

I am a Security Analyst, and all the time in my work I found many request attempts searching vulnerabilities in web applications hosted by the company where I work; so I decided to write a simple Ruby script parser which will allow me to view the Apache2 Logs easily, and then, try to identify the fingerprint of the attackers. Also I advise of some Apache2 Security Tricks, that prevent common hacking attempts.

Well, maybe you couldn’t change the configuration that I will explain in this post, but if you understand the next settings you’ll be in capacity to request an specific administrator ticket, a hosting change for your company or maybe (why not) hack their servers.

With my previous Apache security settings I could make a HEAD request and get this output:

$ curl --head 'http://cixtor.com/'
  HTTP/1.1 200 OK
  Date: Sat, 28 Jul 2012 20:57:34 GMT
  Server: Apache/2.2.14 (Ubuntu) PHP/5.3.2-1ubuntu4.17 with Suhosin-Patch
  Vary: Accept-Encoding
  Content-Type: text/html;charset=UTF8

And after a few changes, the output has been converted to this, removing software versions, modules used and disallowing the directory listing in all the folders and sub-folders:

$ curl --head 'http://cixtor.com/'
  HTTP/1.1 403 Forbidden
  Date: Sat, 28 Jul 2012 20:57:34 GMT
  Server: Apache
  Vary: Accept-Encoding
  Content-Type: text/html;charset=UTF8

But anyway, a hacker could continue checking with a tool like nikto, websploit, htexploit, oscanner, w3af, webshag or in the worse cases with MetaSploit possible vulnerabilities in the web-applications hosted in my/your server. Then, what to do now? In a real case, an administrator could configure a security module like ModSecurity in Apache to prevent possible hack attempts, but in my case I can’t activate all the rules of this module because in the company where I work we administer many web-applications that let publish and upload multiple type of contents which could taken by the ModSecurity rules as a hacking attempt though not be.

Explanation

Then I decided to write a Ruby script to let me know if a request is malicious or not, I will use Nikto to check this script; Nikto is a pluggable web server and CGI scanner written in Perl, using rfp’s LibWhisker to perform fast security or informational checks, the official website of this tool http://cirt.net/nikto2. Features:

  • Easily updatable CSV-format checks database
  • Output reports in plain text or HTML
  • Available HTTP versions automatic switching
  • Generic as well as specific server software checks
  • SSL support (through libnet-ssleay-perl)
  • Proxy support (with authentication)
  • Cookies support

Lets execute this command with the parameter -C all to check all possible CGI vulnerabilities in my domain: nikto --host www.cixtor.com -C all I will monitor the requests sent by this program to my website with this command: tail -f /home/cixtor/access-logs/cixtor.com.log; a possible output could be this:

72.227.103.10 - - [28/Jul/2012:16:00:06 -0500] "HEAD / HTTP/1.1" 403 261 "-" "Mozilla/4.75 (Nikto/2.03 )"
72.227.103.10 - - [28/Jul/2012:16:00:07 -0500] "GET /Ei7N487W.html~ HTTP/1.0" 404 771 "-" "Mozilla/4.75 (Nikto/2.03 )"
72.227.103.10 - - [28/Jul/2012:16:00:22 -0500] "GET /cgi-sys/main.cgi?board=FREE_BOARD&command=down_load&filename;=../../../../../../../../../../etc/passwd HTTP/1.0" 404 771 "-" "Mozilla/4.75 (Nikto/2.03 )"

In this test Nikto could send 16544 requests in 66 seconds. This is the script in Ruby that I wrote to check the Apache logs:

#!/usr/bin/env ruby
filepath = !ARGV[0].nil? ? ARGV[0].to_s : nil
if !filepath.nil?
	filesource = File.new(filepath, 'r')
	filesource.each do |linetext|
		# puts linetext
		if match = linetext.match(/(.*) - - \[(.*)\] "(.*) (.*) (.*)" (\d+) (\d+) "(.*)" "(.*)"/)
			puts "Request information:"
			puts "  Remote address: #{match[1]}"
			puts "  Request datetime: #{match[2]}"
			puts "  Request method: #{match[3]}"
			puts "  Request url: #{match[4]}"
			puts "  HTTP version response: #{match[5]}"
			puts "  Response code: #{match[6]}"
			puts "  Response size: #{match[7]}"
			puts "  Unknown: #{match[8]}"
			puts "  User agent: #{match[9]}"
		end
	end
else
	puts "Usage: "+__FILE__+" /var/log/apache2/access.log"
end
Do you have a project idea? Let's make it together!