Re: Limit bandwidth per-user (uid/gid)

Linux Advanced Routing and Traffic Control

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



I saw a couple things go by on this thread that were interesting,
so let me try to answer them all in one go.

>I want to set up a server where each research project has an account
>where they send data via sftp or rsync; this data is then transferred
>overnight to a server in Europe. My idea is to use a separate cronjob
>or daemon for each user that runs with the user's privileges.

If you have control of both ends, you can run lossless by enabling ecn
on your shaping code (RED, PIE, Codel and FQ_codel support ECN), and
enabling tcp to use ecn on both sides via sysctl.conf

Rather than mucking with a network cgroup, you could just depriorize
rsync (port 883) into a background htb class, so it will use a minimal
amount of bandwidth by default and scale up to full bandwidth if its
available. Then add a fq system on top of that to get fairness between
users.

>1. Limit the total bandwidth that a group (GID) can generate. There
>  should be separate limits for inbound and outbound traffic.

>> You can't limit inbound traffic to Antactica unless you do it on the
>> other end of the satellite link (Europe). Well, you can limit the
>> inbound bandwidth by throwing away packets locally, but that's stupid
>> for packets which have already come over an expensive satellite link.

Dropping packets is the only way that a non-ecn-enabled TCP can slow
down, so although it is a "waste", it is a necessary one. Certainly this
is a good use case for ecn in support of the science.

>>Only the remote side decides how much bandwidth to send over the
>>satellite link. What has been sent by the remote side will go over the
>>satellite link and you can't undo this on the local side.

If you wanted total control over the satellite link you could set it
up as a tunnel from point A to point B and control matters better on
both sides that way. You'd then shape the tunnel on the remote side to
drop packets appropriately.

(the ecn suggestion is only going to work on endpoints you control)

>2. Limit the bandwidth per-user (UID), so if the GID is allowed to
>   generate 384kbps of traffic, and 3 users are using the network, each
>   user can at most benefit of 128kbps. If there's only one user he gets
>   all the 384kbps.
>   Again there should be different limits for inbound and outbound
>   traffic.
>   This should work regardless the number of connections the user makes.

As noted elsewhere in this thread, a "user" is not a network object
per se'. It does sound like you want per-user fairness not per flow
fairness in your design (fq_codel and sfq are per flow by default). I
think that is indeed doable with a tc filter for per user network
cgroups on egress nowadays but haven't tried it.

On ingress, stuck on ways to do it.

Anyway, cerowrt's swm-scripts work on openwrt and cerowrt, can also be
made to  work on mainline linux,

https://github.com/dtaht/ceropackages-3.3/tree/fc16330283cc3d3fc4d97819158fb2d9bd0ee45e/net/sqm-scripts

and this is a decent start at a shaper for mainline linuxes:

https://wiki.gentoo.org/wiki/Traffic_shaping

In your case I'd probably set the fq_codel target at 60ms or so, and interval
pretty high as well (700ms RTTs? really?), and enable ecn in both directions

I recently wrote a rant on everything wrong with wondershaper.

http://www.bufferbloat.net/projects/cerowrt/wiki/Wondershaper_Must_Die


-- 
Dave Täht
--
To unsubscribe from this list: send the line "unsubscribe lartc" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html




[Index of Archives]     [LARTC Home Page]     [Netfilter]     [Netfilter Development]     [Network Development]     [Bugtraq]     [GCC Help]     [Yosemite News]     [Linux Kernel]     [Fedora Users]
  Powered by Linux