Re: delay_access url_regex acl

[Marcus Kool <marcus.kool@xxxxxxxxxxxxxxx>]

Amos Jeffries wrote:
> On 28/05/11 00:46, Marc Nil wrote:
> > > > Hello,
> > > >
> > > > I am currently facing some troubles will using Squids
> > > > feature to manage bandwidth (delay_pools, delay_access, ...)
> > > >
> > > > I would like to apply a 50kbytes/s limitation to each
> > > > users and a global 3Mbytes/s limitation.
> > > > There is a authentication group (I am working with NTLM
> > > > authentication) for which I apply no limit.
> > > > Here is the conf :
> > > > acl InternetAdmin external InetGroup Internet_Admin
> > > >
> > > > delay_pools 1
> > > > delay_class 1 2
> > > > delay_parameters 1 3145728/3145728 51200/51200
> > > > delay_access 1 allow !InternetAdmin
> > > > delay_access 1 deny all
> > > >
> > > > This works fine.
> > >
> > > You sure? delay_access does not do external ACL lookups for the auth. 
> > > There must be a matching http_access test to perform the lookups and 
> > > cache them first.
> >
> > Yes the http_access rule is the following :
> > http_access allow InternetAdmin
> >
> > > > Now I want to allow all user to download at normal speed
> > > > on some sites.
> > > > acl whitelist.no_limit url_regex
> > > > "/etc/squid3/etc/whitelist.no_limit" # 
> > > > "/etc/squid3/etc/whitelist.no_limit contains list of sites with no 
> > > > BW limit
> > > >
> > > > delay_pools 1
> > > > delay_class 1 2
> > > > delay_parameters 1 3145728/3145728 51200/51200
> > > > delay_access 1 allow !whitelist.no_limit !InternetAdmin
> > > > delay_access 1 deny all
> > >
> > > This looks correct for delay_access. Is a bit non-intuitive though.
> > >
> > > Try this for easier reading later:
> > >    delay_access 1 deny whitelist.no_limit
> > >    delay_access 1 deny InternetAdmin
> > >    delay_access 1 allow all
> >
> > I also tested this option (with the sale result) I even removed the 
> > InternetAdmin line (still the same result)
>
> Definitely the regex bits then.

If you post the whitelist.no_limit we are able to see if there is
room for improvement.
Usually there is.

As a side note for Amos: I made the URL filter ufdbGuard 70% faster
with a regex optimiser that combines N REs into one large RE composed
of the N original REs with OR logic.  i.e.
   RE1
   RE2
   ...
   REn
becomes
   RE1|RE2|...|REn
The speed increase was measured with overall throughput so the actual
improvement for RE processing must be much higher than 70%.

> > > > However with such configuration all sites (even those in
> > > > whitelist.no_limit) are limited in bandwidth.
> > >
> > > I suspect some error in the regex patterns. Check for patterns that 
> > > are short and could match anywhere. Post the whitelist here if you 
> > > want any help with the check.
> >
> > I don't think there is any possible missmatch with those regex
> > (I used 
> > "http://download.tuxfamily.org/notepadplus/5.9/npp.5.9.Installer.exe"; 
> > to test a non-whitelisted url)
> >
> > cat /etc/squid3/etc/whitelist.no_limit
> > www.microsoft.com
> > cdimage.debian.org
>
> These are not suitable for regex. If working they will give high speed 
> web access anywhere just by appending "?wwwImicrosoftIcom" to the URL etc.
>
> Being domain names they are better suited for use in a dstdomain ACL.
>
> > > Alternatively if you added the whitelist to http_access as well it 
> > > could have screwed with the http_access login lookup. Making login 
> > > unavailable and always fail at the delay_access check.
> >
> > Most of the url in the whitelist.no_limit are also whitelisted 
> > somewhere in a http_access.
>
> Fine *if* the login http_access is still always being run.
>
>
> Amos