Package Proxy Cache

From ArchWiki
The printable version is no longer supported and may have rendering errors. Please update your browser bookmarks and please use the default browser print function instead.

Merge-arrows-2.pngThis article or section is a candidate for merging with Pacman/Tips_and_tricks#Network_shared_pacman_cache.Merge-arrows-2.png

Notes: Same topic (Discuss in Talk:Package Proxy Cache)

If you want to install the same Arch packages over and over - like for testing AIF profiles - it could help if you would not have to get the packages every time from the internet. This article shows you how to setup a Squid proxy that only caches arch packages and can be used with aif/pacman/wget/etc with minimal configuration on the client system.

Install Squid

Install squid.

Configure Squid

This is the minimum configuration to get squid cache arch packages.

Cache Rules

Before defining these rules, remove/comment (if you do not need them) all the default refresh_patterns

/etc/squid/squid.conf 
refresh_pattern \.pkg\.tar\.   0       20%     4320      reload-into-ims
refresh_pattern .              0       0%      0

That should define that *.pkg.tar.* gets cached, and anything else should not.

Maximum Filesize

Objects larger than this size will NOT be saved on disk:

/etc/squid/squid.conf 
maximum_object_size 256 MB

Cache Directory

Set the cache dir and its maximum size and subdirs:

/etc/squid/squid.conf 
cache_dir aufs /var/cache/squid 10000 16 256

Shutdown Lifetime

Time to wait until all active client sockets are closed:

/etc/squid/squid.conf 
shutdown_lifetime 1 seconds 
Note:

Every time you change the cache_dir path (and after fresh install), you need to (re)create this directory:

# squid -z

and it could be helpful to check the config file before running squid:

# squid -k parse

Start Squid

# systemctl start squid.service

or if squid is already running:

# systemctl restart squid.service
Note:

It could be helpful to check the config file before running:

# squid -k check

Follow Squid access log

To see the access to squid:

# tail -f /var/log/squid/access.log

You should see this for packages that are directed to original host:

...TCP_MISS/200...DIRECT...

and for packages that are delivered from the cache:

...TCP_HIT/200...NONE...

Manual Arch Install

Before running /arch/setup, add variables for your proxy. To do so, run on the console:

# export http_proxy='http://your_squid_machine_ip:3128/'
# export ftp_proxy='ftp://your_squid_machine_ip:3128/'

Now just use /arch/setup to normally install the system, and it should use your proxy. Watch the squid logs to verify this.

Note: If you want to use the proxy settings in the installed system, you need to add the http_proxy and/or ftp_proxy variables in an appropriate place on the installed system. (like /etc/profile.d/proxy.sh)

Intercepting local requests

If you want all HTTP requests on local machine automagically go through squid, we first need to add an intercepting port for squid:

/etc/squid/squid.conf 
http_port 3127 intercept

and iptables rules to redirect all (except the ones from squid) port 80 requests to squid:

# iptables -t nat -A OUTPUT -p tcp --dport 80 -m owner --uid-owner proxy -j ACCEPT
# iptables -t nat -A OUTPUT -p tcp --dport 80 -j REDIRECT --to-ports 3127
Note: if you get random slow download speeds in vagrant/packer/virtualbox, try using virtio network device type.