[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Postgres batch write very slow - what to do



I tried maxpages = 0 and  full_page_writes=off and it seemed to be taking
forever.
All other tricks I have already tried.

At this point I wondering if its a jdbc client side issue - I am using the
latest 8.1.
(as I said in an earlier post - I am using addBatch with batch size of 100).
But just in case - I am missing something.

If 17k record/sec is right around expected then I must say I am little
disappointed from the "most advanced open source database".

thanks for all your help.

-Sanjay


Merlin Moncure-2 wrote:
> 
> On 3/15/07, femski <hypertree@xxxxxxxxx> wrote:
> well, what sql is the jdbc driver creating exactly?  It is probably
> running inserts in a transaction.   your load is about 17k inserts/sec
> which about right for postgres on your hardware.  you have the
> following options to play increase insert performance:
> 
> * tweak postgresql.conf
>   fsync: off it is not already
>   wal_segments: bump to at least 24 or so
>   maintenance_work_mem: if you create key after insert, bump this high
> (it speeds create index)
>   bgwriter settings: you can play with these, try disabling bgwriter
> first (maxpages=0)
>   full_page_writes=off might help, not 100% sure about this
> 
> * distribute load
>   make load app multi thrreaded.
> 
> * use copy for bulk load
>   [is there a way to make jdbc driver do this?]
> 
> * use multi-line inserts (at least 10 rows/insert)...nearly as fast as
> copy
> 
> * if jdbc driver is not already doing so, prepare your statements and
> execute.
> 
> merlin
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 1: if posting/reading through Usenet, please send an appropriate
>        subscribe-nomail command to majordomo@xxxxxxxxxxxxxx so that your
>        message can get through to the mailing list cleanly
> 
> 

-- 
View this message in context: http://www.nabble.com/Postgres-batch-write-very-slow---what-to-do-tf3395195.html#a9492938
Sent from the PostgreSQL - performance mailing list archive at Nabble.com.



[Postgresql General]     [Postgresql PHP]     [PHP Users]     [PHP Home]     [PHP on Windows]     [Kernel Newbies]     [PHP Classes]     [PHP Books]     [PHP Databases]     [Home]     [Yosemite]

Powered by Linux