Sharing UNIX knowledge > Securities > Squid on OpenBSD

Published : 2012-09-10

Squid on OpenBSD

Proxy servers are now standard, to not say compulsory in enterprises (mainly for log collecting). The most used service is Squid, whether in v2.7 or v3.1, at this time (1 january 2013)

In this tutorial, we will study OpenBSD 5.2 and Squid 3.3.4, together SquidGuard 1.4p3. Squid configuration will not be detailed, in terms of security. Only system interaction elements, between Squid and SquidGuard, will be mentioned. We don’t use authentication.

We will optimise Squid by putting all data using disk accesses on a ramdisk, and resolve the associated problematics.The server host has 16Go RAM to store all “disk” cache.

Prerequisite: PF Configuration

You need to configure OpenBSD Firewall to allow 3128 TCP for your clients and allow HTTP and HTTPS requests from your proxy. We also redirect HTTP port to proxy for transparent requests. Here is a sample configuration file for Packet Filter

#
# Macros & Tables
#
proxy_ip="10.1.1.1"
outgoing_proxy_ip="195.25.36.159"
table <clients> { 10.50.0.0/16 192.168.0.0/16 172.16.0.0/18 }

#
# Default rules
#

block in log all
pass out all

#
# NAT
#

pass out quick to any port http rdr-to $proxy_ip port 3129

#
# Filtering
#

pass in proto tcp from <clients> to $proxy_ip port 3128
pass in proto tcp from any port { http https } to $outgoing_proxy_ip

Prerequisiste: configure OpenBSD limits

For supporting IO parallel requests, we must increase system limits for file descriptors. Open /etc/login.conf file and edit daemon class to increase file descriptors to 4096:

daemon:\
        :ignorenologin:\
        :datasize=infinity:\
        :maxproc=infinity:\
        :openfiles-cur=<span style="color: #ff0000;">4096</span>:\
        :stacksize-cur=8M:\
        :localcipher=blowfish,8:\
        :tc=default:

Installation

Please start by installing a fresh OpenBSD 5.2.

Now we install some prerequisited, especially Squid 2.7, SquidGuard, which will be used as authentication support.

export PKG_PATH="http://ftp.fr.openbsd.org/pub/OpenBSD/5.2/packages/amd64/"
pkg_add -i squid
pkg_add -i squidGuard
pkg_add -i wget

Now we download Squid 3.3.4 from squid-cache.org and compile it.

cd /root/
wget http://www.squid-cache.org/Versions/v3/3.3/squid-3.3.4.tar.gz
tar xvzf squid-3.3.4.tar.gz
cd squid-3.3.4
./configure  '--enable-pf-transparent' '--enable-follow-x-forwarded-for' '--with-large-files' '--enable-ssl' '--disable-ipv6' '--enable-esi' '--enable-kill-parent-hack' '--disable-snmp' '--with-pthreads'
make -j8
make install

For compile options:

If you don’t use transparent proxy (redirect 80 -> 3128), remove ‘–enable-pf-transparent’
if you use IPv6, remove ‘–disable-ipv6’

-j8 compile option define the compiler to use 8 processor core, speeding up the compilation.

Now we change Squid service path. Edit /etc/rc.d/squid and change daemon line to

daemon="/usr/local/squid/sbin/squid"

Change now service flags, in order to load our squid configuration file, and enable service. For this, open (or create) /etc/rc.conf.local

squid=YES
squid_flags="-f /etc/squid/squid.conf"

Now, copy squid.conf in a best directory for OpenBSD

mkdir /etc/squid/
cp /usr/local/squid/etc/squid.conf.documented /etc/squid/squid.conf

Create squidguard required directories

mkdir /etc/squidguard/mount/

Configuration

Squid Configuration

Now we configure Squid to use SquidGuard. Warn, since 3.2 version the child configuration is different.

url_rewrite_program /usr/local/bin/squidGuard -c /etc/squidguard/squidguard.conf
url_rewrite_children 192 startup=150 idle=10 concurrency=0

By default, we launch 150 SquidGuard processes, with maximum 192 processes, and 10 for spare.

If you want use transparent proxy, don’t forget to add intercept http_port.

http_port 3128
http_port 3129 intercept

Now we configure memory space for squid, and associated parameters.

# Real squid memory cache
cache_mem 2800 MB
maximum_object_size_in_memory 8 MB

# Squid disk cache
cache_dir ufs /var/squid/cache 2800 16 64
minimum_object_size 3 KB
maximum_object_size 6 MB

# IP & DNS names memory cache
ipcache_size 10240
fqdncache_size 10240

# File descriptor number
max_filedescriptors 4096

You can tune these parameters for your configuration. For example, divide by 2 each cache size if you have only 8Go RAM. Change maximum size of cached files following the type of pages,documents downloaded by your customers.

Next, configure other options like ACL, etc… as you want

SquidGuard configuration

Open /etc/squidguard/squidguard.conf file (if it doesn’t exist, create it and populate it). We only modify blacklist directory line.

dbhome /etc/squidguard/mount/blacklists

System configuration

Now, We configure system, boot and blacklist updates.

First, we create two ramdisk (mfs), to store squid “disk” cache, and SquidGuard blacklists. Please open /etc/fstab and add these two lines.

swap            /etc/squidguard/mount   mfs rw,nodev,nosuid,noatime,-s=2096900 0 0
swap            /var/squid/cache        mfs rw,nodev,nosuid,noatime,-s=9236000 0 0

Interesting options are noatime and -s.

noatime: we don’t write access times in the FS. It’s a temporary fs, it’s useless.
-s : it’s the number of FS blocks. Generally it’s 512k blocs. The two values are respectively 991MB and 4.3GB.

Now, mount the FS

mount /var/squid/cache
mount /etc/squidguard/blacklists

We create Squid cache tree, and modify rights for transparent pf.

/usr/local/squid/sbin/squid -f /etc/squid/squid.conf -z
chown root:_squid /dev/pf
chmod 0640 /dev/pf

We generate SquidGuard blacklists. Our base is Toulouse University

cd /etc/squidguard/
/usr/local/bin/wget ftp://ftp.univ-tlse1.fr/pub/reseau/cache/squidguard_contrib/blacklists.tar.gz
cd /etc/squidguard/mount/
tar xzf /etc/squidguard/blacklists.tar.gz
chown -R _squid:_squid /etc/squidguard/mount/
/usr/local/bin/squidGuard -b -c /etc/squidguard/squidguard.conf -C all
chown -R _squid:_squid /etc/squidguard/mount/

Après this, if you type df -h you must see the mfs fills.

mfs:185        4.3G    1.0M   4.3G    1%    /var/squid/cache
mfs:754       991M   78.8M    862M     8%    /etc/squidguard/mount

The solution is ready ! It remains a subsidiary step, what are we doing when there is a reboot (mfs => all is lost), and now update blacklists properly ?

I suggest you this script, used since a moment, in my CNRS lab, under our double Squid gateway:

#! /bin/sh

#
# Functions
#

gen_blacklists() {
        # extraction
        cd /etc/squidguard/mount/
        tar xzf /etc/squidguard/blacklists.tar.gz

        # apply rights
        chown -R _squid:_squid /etc/squidguard/mount/

        # Generation
        /usr/local/bin/squidGuard -b -c /etc/squidguard/squidguard.conf -C all

        # re-apply rights
        chown -R _squid:_squid /etc/squidguard/mount/

        # backup for reboot
        cp -Rp /etc/squidguard/mount/blacklists /etc/squidguard

        /etc/rc.d/squid start
}

# stop squid
/etc/rc.d/squid stop

# remove RAM datas
rm -R /etc/squidguard/mount/*
/sbin/umount /etc/squidguard/mount
# We wait volume umount
while [ $(df -h|grep /etc/squidguard/mount|wc -l|awk '{print $1}') -gt 1 ];
do
        sleep 1;
        /sbin/umount /etc/squidguard/mount
done

# We mount volume
mount /etc/squidguard/mount

# At boot we take the backup
if [ "$1" == "boot" ];
then
        if [ -d /etc/squidguard/blacklists ];
        then
                cp -Rp /etc/squidguard/blacklists /etc/squidguard/mount/
                /etc/rc.d/squid start
        else
                # If there isn't blacklist backup, we get the .tar.gz backup and regen
                if [ -f /etc/squidguard/blacklists.tar.gz ];
                then
                        gen_blacklists
                else
                        echo "/!\ Squid cannot be started because no blacklists found"
                fi
        fi
else
        # removing old blacklists
        if [ -d /etc/squidguard/blacklists ];
        then
                rm -R /etc/squidguard/blacklists
        fi

        # removing old archive
        rm /etc/squidguard/blacklists.tar.gz*

        # donwload BL
        cd /etc/squidguard/
        /usr/local/bin/wget ftp://ftp.univ-tlse1.fr/pub/reseau/cache/squidguard_contrib/blacklists.tar.gz

        gen_blacklists
fi

This script execute 2 tasks. It manage complete blacklist creation and the update, but it also make a blacklist backup.

It can happen in a continuity/recovery activity plan, your proxy and your hardware router crash. Generally OpenBSD boots faster than the routers, and it can’t download blacklists over the net, and then generate it, and squid won’t start.

This script make a blacklist backup and restore these at boot. If they don’t exist, we get the last .tar.gz, and do a generation process. If there is nothing, Squid won’t start, manually operation is required. This process is called by passing boot argument to script.

Now we apply the boot process, edit /etc/rc.local file and add these lines:

echo 'Started Squid Proxy Cache Service'
if [ -x /usr/local/squid/sbin/squid ]; then
        umount /var/squid/cache
        mount /var/squid/cache
        /usr/local/squid/sbin/squid -f /etc/squid/squid.conf -z
        chown root:_squid /dev/pf
        chmod 0640 /dev/pf
        /etc/squidguard/squidguard-update.sh boot
fi
echo ''

Also edit /etc/rc.shutdown file to proper squid shutdown

echo 'Stopping Squid Proxy Cache Service'
if [ -x /usr/local/sbin/squid ]; then
        /etc/rc.d/squid stop
fi
echo ''

To finish, we add blacklist update process, for example on sunday morning at 5AM

crontab -e
00      5       *     *     6       /etc/squidguard/squidguard-update.sh > /dev/null

Your caching proxy is now optimized !

Note: these actions are feasible under FreeBSD and Linux, you must change ramdisk technology, and file paths, but OpenBSD is the most secure and secured for this.