Find all unique url's from Apache log files

Posted on Tue 05 February 2013 in misc • Tagged with awk, sed

I needed to build a list of all unique hits that had been made on a website in Apache.

Here's what I came up with using awk and sed.  This should match any HTTP 2xx or 3xx requests and strip of any GET request parameters.

awk '\$9 \~/\^(2|3)/ {print …


Continue reading

In-place file editing with sed

Posted on Wed 17 March 2010 in misc • Tagged with sed

This took me a while to work out why this would not work. When using the -i option with sed you need to specify a backup extension to be used (at least on my 10.5 mac you do!)

e.g.

sed -i '' -e 's/something/something else/g' -e …


Continue reading