Search squid archive

Re: Problems with squid in a campus setup

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]




Hello,

first of all, thank you for your recommendations.

On Monday 26 March 2012 16:34:21 Marcus Kool wrote:
> Youtube may be hogging your pipe but it is better to know than to guess.

Of course, before we decided for the proxy setup we investigated bandwidth 
usage. HTTP Traffic was about 60-70% of our traffic, and a good chunk of that was 
youtube. That's why we decided, that we would try a squid setup with youtube 
caching. However it took a while before we found a solution for caching 
youtube, as 3.1 hasn't implemented the necessary features yet.

> The access.log shows content sizes so with a simple awk script it should
> be easy to find out.
> 
> I have also seen many sites where advertisements and trackers consume 15%
> bandwidth. This may vary. So blocking ads and trackers is a thing to
> consider.

Thanks for this insight! This would of course be a welcome saving of bandwidth 
in my personal opinion. I'm just not sure if we're allowed to do this, as the 
patron of the proxy is a public-law institution and as such bound to anti-
censorship laws. Need to check with a Legalese translator.

> 
> Do not expect too much from web caching. More and more websites
>  intentionally make their sites not cacheable. Look at the percentage of
>  TCP_MISS in access.log or use a second awk script to find out more about
>  cacheability.

Every bit counts. Before we apply for an increase of (expensive) uplink 
bandwidth we want to play every trick we have up our sleeve. At the moment our 
cache is still cold, because for getting the proxy running again I had to 
completely wipe the cache. At the moment we have a hit:miss ratio of about
1:5.  For youtube caching we have a saved bandwidth around 100 GB for the 27th 
of march (one video in particular had a size of 768 MB and was watched 19 
times). Online lectures are currently en vogue.

> 
> I recommend going for a newer Squid: 3.1.19 is stable and fixes issues that
> 3.1.10 has.

Will do so.

> On Linux, aufs has a better performance than diskd

Thanks again for this tip!

> 
> > Additional memory for storing objects is 2048 MB:
> >
> > cache_mem 2048 MB
> 
> Seems right. But you also need virtual memory for Squid being able to
> fork processes without issues. Do have have 8 GB swap ?

Yes. 10 GB actually.

> 
> But read the FAQ about memory usage and a large disk cache:
> http://wiki.squid-cache.org/SquidFaq/SquidMemory
> Squid uses an additional 512*14 MB = 7.1 GB for the index of the disk
>  cache. I suggest to downsize to 1 GB in-memory index which implies to
> use only 73 GB disk cache.

Ah okay, here's one of my initial mistakes. I used only 10 MB for my 
calculation but of course we use a 64bit squid. Out of curiosity and because I 
want to learn: the reasoning for shrinking the disk cache from 512 GB to 73 GB 
is because a big cache as we have it at the moment only leads to lots of stale 
objects in cache which additionally burden the CPU and RAM because of in-
memory metadata?


Best regards,
 - Christian Loth
 


[Linux Audio Users]     [Photo]     [Yosemite News]     [Samba]     [Video Projectors]     [Video Devices]     [Big List of Linux Books]     [LCD TVs]     [Webcams]     [Linux USB]

  Powered by Linux