NetBSD Problem Report #54181

From www@netbsd.org  Thu May  9 08:06:49 2019
Return-Path: <www@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 9D49E7A1B2
	for <gnats-bugs@gnats.NetBSD.org>; Thu,  9 May 2019 08:06:49 +0000 (UTC)
Message-Id: <20190509080648.71A047A1D6@mollari.NetBSD.org>
Date: Thu,  9 May 2019 08:06:48 +0000 (UTC)
From: dogan@tesmer.org.tr
Reply-To: dogan@tesmer.org.tr
To: gnats-bugs@NetBSD.org
Subject: postgresql/pg_dump causes memory exhaustion
X-Send-Pr-Version: www-1.0

>Number:         54181
>Category:       kern
>Synopsis:       postgresql/pg_dump causes memory exhaustion
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu May 09 08:10:00 +0000 2019
>Last-Modified:  Thu May 09 21:00:01 +0000 2019
>Originator:     Mustafa Dogan
>Release:        NetBSD 8_STABLE
>Organization:
TESMER
>Environment:
NetBSD localhost 8.0_STABLE NetBSD 8.0_STABLE (GENERIC) #0: Fri Mar  8 03:35:49 +03 2019  root@localhost:/root/OBJ/sys/arch/amd64/compile/GENERIC amd64
>Description:
This PR seems to pkgsrc problem, but I think this problem is related to vm.
Some of packages causes memory exhaustion and DoS. For example PostgreSQL and samba3, which I tried, uses all of available memory, swap, and causes server unusable.
sysctl vm.{anon,file,exec}{min,max} doesn't help.
My scenario is follows:
I installed and used postgresql for approximately 15 years on NetBSD. When I upgrade NetBSD 8 (in fact, I started using NetBSD 8.0_BETA, and it has the same problem too), problems arises. There weren't any problem with NetBSD 7. My database size is bigger than RAM size. Server RAM is 128 GB. While I trying to dump database, (pg_dumpall) RAM usage is increasing very fast. After dump size exceeds RAM size, server starts using swap. After dump size exceeds RAM+swap size, server crashes. Without reboot, can't reduce RAM size.
A similar scenario is applicable for samba3. Transfering file(s) causes DoS.
top shows memory usage as follows:
load averages:  0.12,  0.20,  0.20;               up 7+07:20:28                                                                              10:59:48
227 processes: 225 sleeping, 2 on CPU
CPU states:  0.0% user,  0.0% nice,  0.4% system,  0.0% interrupt, 99.4% idle
Memory: 100G Act, 212K Inact, 1104K Wired, 50M Exec, 120G File, 7120M Free
S
>How-To-Repeat:
On NetBSD 8, Install postgresql and have big database(s). Run pg_dump, pg_dumpall, or pg_basebackup. Then, monitor memory usage.
>Fix:

>Audit-Trail:
From: Joerg Sonnenberger <joerg@bec.de>
To: gnats-bugs@netbsd.org
Cc: kern-bug-people@netbsd.org, gnats-admin@netbsd.org,
	netbsd-bugs@netbsd.org
Subject: Re: kern/54181: postgresql/pg_dump causes memory exhaustion
Date: Thu, 9 May 2019 22:55:50 +0200

 On Thu, May 09, 2019 at 08:10:00AM +0000, dogan@tesmer.org.tr wrote:
 > Some of packages causes memory exhaustion and DoS. For example PostgreSQL and samba3, which I tried, uses all of available memory, swap, and causes server unusable.
 > sysctl vm.{anon,file,exec}{min,max} doesn't help.

 Can you try restricting your system to 64GB and see if that helps? There
 have been previous discussions about some parts of UVM not keeping up on
 systems with lots of memory, and very high "active" count would support
 that.

 Joerg

Home	PR Database Search