[forum] Re: bloatware

[jeroen <jeroen@fox-toolkit.org>]

> Date: Sun, 30 Mar 2003 13:06:57 -0800 (PST)
> From: <Jim.Gettys@hp.com>
> To: forum@XFree86.Org
> Cc: Martin Konold <martin.konold@erfrakon.de>
> Subject: Re: [forum] bloatware
> Reply-To: forum@XFree86.Org
>
> > Sender: forum-admin@XFree86.Org
> > From: Martin Konold <martin.konold@erfrakon.de>
>
> material elided.
>
> > > Various stuff has bloated over the years, but it is generally on the
> > > applications side.  Sometimes you think the X server is bloating, when
> > > it is really that applications have a pixmap leak, for example.
> >
> > In order to show that X is bloated I generally kill all apps except a
> > single xterm and then check the memory usage.
>
> To begin with, what you see is VM; if the memory is never referenced
> again, it is fine for it to page out.
>
> X has to allocate space for all the memory clients ask.  There is no good
> way to reclaim it right now.  Your VM usage of the X server will be:
>   o the size of X itself
>   o however much I/O space is required to map the hardware registers and
> VRAM o the heap used for allocation of X data structures and Pixmaps.
>
> So what you are seeing is ***the high water mark of what all the bloatware
> has asked the X server on their behalf*** added up to.
>
> Ok, what can be done about this?
>
> A number of things...
>
> 1) We can try, as a stop-gap, to see if cranking down the MMAP threshold
>    in glibc (the point at which glibc calls mmap to allocate pages of
>    memory to satisfy allocations) helps.  Then if many pixmaps are larger
>    than that threshold, at least the VM gets returned when the pixmaps
>    get freed.  But we can't just set this arbitrarily low, as memory
>    allocation is very performance sensitive in respect to any windowing
>    operation (the region code in the X server pounds on malloc/free),
>    and a few apps do some amazing things with regions.
>
> 2) If we separate pixmap allocation from the general heap, it would be
>    easy to have the X server do a copying garbage collector and reclaim
>    large amounts of VM, as all pixmap access is through internal data
>    structures.  This would also avoid large amounts of heap fragmentation
>    where window data structures currently get scattered through the VM of
>    the X server (around the much larger pixmap allocations), and is
>    worth doing anyway as you'd like pixmap allocation to
>    be something a device driver can do anyway (so they'd be out in VRAM
>    when appropriate).   When clients exit, it would probably then
>    be worth copying pixmaps around to compact the virtual memory address
>    space, and one could then free the left over pages easily, returning
>    VM to the operating system.
>
> I was in the middle of 1) when things recently blew up, so I don't have
> anything to report yet; maybe something in a few weeks.  2) will take
> more work, and if someone is interested, please let us (Keith and I) know;
> it shouldn't be too hard to implement.
>
> All this was much less of a concern when displays were typically 1 or
> 8 bits/pixel, but these days most displays are 16 or 32 bits/pixel,
> compounding the heap usage problem.
>
>                                     - Jim


Is it possible to collect memory allocated by the X Server on behalf of
some client? Like a malloc-zone.  When a client disconnects, one could
then simply deallocate the entire zone.

Presumably, such a zone could be a mmap region which is dubdivided into
mallocs related to a certain client.

I don't know how feasible this is, one would have to keep track of
which memory was allocated for whom.  However, I feel it could improve
VM locality of reference and simplify garbage collection quite a bit.

I believe NeXTSTEP had something like this, too.


	regards,


			- Jeroen



-- 
+----------------------------------------------------------------------------+
| Copyright (C) 18:30 03/30/2003 Jeroen van der Zijp.   All Rights Reserved. |
+----------------------------------------------------------------------------+