NetBSD Problem Report #57767

From www@netbsd.org  Mon Dec 11 05:39:04 2023
Return-Path: <www@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 0F76A1A9238
	for <gnats-bugs@gnats.NetBSD.org>; Mon, 11 Dec 2023 05:39:04 +0000 (UTC)
Message-Id: <20231211053903.121F01A923C@mollari.NetBSD.org>
Date: Mon, 11 Dec 2023 05:39:03 +0000 (UTC)
From: toku@tokugawa.org
Reply-To: toku@tokugawa.org
To: gnats-bugs@NetBSD.org
Subject: blacklistd data inconsistency
X-Send-Pr-Version: www-1.0

>Number:         57767
>Category:       bin
>Synopsis:       blacklistd data inconsistency
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    bin-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Mon Dec 11 05:40:00 +0000 2023
>Last-Modified:  Sat Dec 30 18:25:01 +0000 2023
>Originator:     Yoshitaka Tokugawa
>Release:        NetBSD 9.3
>Organization:
>Environment:
NetBSD hpms.tokugawa.org 9.3 NetBSD 9.3 (NETBSD) #4: Fri Dec  8 15:45:00 JST 2023  toku@hpms.tokugawa.org:/usr/src/sys/arch/amd64/compile/NETBSD amd64
>Description:
When blacklistd is restarted with the -r option, the blacklistd ruleset for npf  is updated based on information in the database file. At this time, the new id assigned by npfctl is retained in blacklistd's memory, but the database file is not updated, resulting in inconsistencies with the information displayed by blacklistctl dump command. To resolve this problem, I propose the following modifications.

>How-To-Repeat:
1. restart blacklistd with -r
2. compare id number by following commands

npfctl rule blacklistd list
blacklistctl dump -b

This problem occurs when blacklistd is restarted after some old rules have been deleted by blacklistd.
>Fix:
*** blacklistd.c.orig   Mon Dec 11 11:57:04 2023
--- blacklistd.c        Mon Dec 11 11:53:25 2023
***************
*** 396,402 ****
--- 396,404 ----
                        continue;

                (void)run_change("add", &c, dbi.id, sizeof(dbi.id));
+               state_put(state, &c, &dbi); /* toku */
        }
+       state_sync(state); /* toku */
  }

  int

>Audit-Trail:
From: Yoshitaka Tokugawa <toku@tokugawa.org>
To: gnats-bugs@netbsd.org
Cc: Yoshitaka Tokugawa <toku@tokugawa.org>
Subject: Re: bin/57767: blacklistd data inconsistency
Date: Tue, 12 Dec 2023 08:37:35 +0900

 Sorry, My fix was incomplete.
 According to the dbopen man page, "the position of the cursor is not 
 affected by calls to the del, get, put, or sync routines.”.

 However, if I put a record to the database in the middle of reading 
 records sequentially by (*seq)(), it seems to (*seq)() sometimes return 
 the same record. Why?

 I am not sure if my fix is the proper way, but if I open the database 
 with O_RDONLY and process the records, the "npfctl rule blacklistd show" 
 and "blacklistctl dump -b" information will remain consistent.

 *** blacklistd.c.orig   Mon Dec 11 11:57:04 2023
 --- blacklistd.c        Tue Dec 12 08:35:34 2023
 ***************
 *** 387,402 ****
    static void
    rules_restore(void)
    {
          struct conf c;
          struct dbinfo dbi;
          unsigned int f;

 !       for (f = 1; state_iterate(state, &c, &dbi, f) == 1; f = 0) {
                  if (dbi.id[0] == '\0')
                          continue;

                  (void)run_change("add", &c, dbi.id, sizeof(dbi.id));
          }
    }

    int
 --- 387,407 ----
    static void
    rules_restore(void)
    {
 +       DB *db; /* toku */
          struct conf c;
          struct dbinfo dbi;
          unsigned int f;

 !       db = state_open(dbfile, O_RDONLY, 0); /* toku */
 !       for (f = 1; state_iterate(db, &c, &dbi, f) == 1; f = 0) { /* toku */
                  if (dbi.id[0] == '\0')
                          continue;

                  (void)run_change("add", &c, dbi.id, sizeof(dbi.id));
 +               state_put(state, &c, &dbi); /* toku */
          }
 +       state_sync(state); /* toku */
 +       state_close(db); /* toku */
    }

    int

 On 2023/12/11 14:40, gnats-admin@netbsd.org wrote:
 > Thank you very much for your problem report.
 > It has the internal identification `bin/57767'.
 > The individual assigned to look at your
 > report is: bin-bug-people.
 >
 >> Category:       bin
 >> Responsible:    bin-bug-people
 >> Synopsis:       blacklistd data inconsistency
 >> Arrival-Date:   Mon Dec 11 05:40:00 +0000 2023

From: "Christos Zoulas" <christos@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/57767 CVS commit: src/external/bsd/blocklist/bin
Date: Sat, 23 Dec 2023 16:53:54 -0500

 Module Name:	src
 Committed By:	christos
 Date:		Sat Dec 23 21:53:54 UTC 2023

 Modified Files:
 	src/external/bsd/blocklist/bin: blocklistd.c

 Log Message:
 PR/57767: Yoshitaka Tokugawa: When restoring, do so from a readonly copy
 of the database and update the read-write copy with the new firewall ids.
 Before we did not update the state file so it contained the old firewall ids.


 To generate a diff of this commit:
 cvs rdiff -u -r1.3 -r1.4 src/external/bsd/blocklist/bin/blocklistd.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

From: "Martin Husemann" <martin@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/57767 CVS commit: [netbsd-10] src/external/bsd/blocklist/bin
Date: Mon, 25 Dec 2023 18:59:46 +0000

 Module Name:	src
 Committed By:	martin
 Date:		Mon Dec 25 18:59:46 UTC 2023

 Modified Files:
 	src/external/bsd/blocklist/bin [netbsd-10]: blocklistd.c

 Log Message:
 Pull up following revision(s) (requested by kim in ticket #519):

 	external/bsd/blocklist/bin/blocklistd.c: revision 1.4

 PR/57767: Yoshitaka Tokugawa: When restoring, do so from a readonly copy
 of the database and update the read-write copy with the new firewall ids.

 Before we did not update the state file so it contained the old firewall ids.


 To generate a diff of this commit:
 cvs rdiff -u -r1.3 -r1.3.2.1 src/external/bsd/blocklist/bin/blocklistd.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

From: "Martin Husemann" <martin@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/57767 CVS commit: [netbsd-9] src/external/bsd/blacklist/bin
Date: Sat, 30 Dec 2023 18:24:50 +0000

 Module Name:	src
 Committed By:	martin
 Date:		Sat Dec 30 18:24:50 UTC 2023

 Modified Files:
 	src/external/bsd/blacklist/bin [netbsd-9]: blacklistd.c

 Log Message:
 Pull up following revision(s) (requested by kim in ticket #1781):

 	external/bsd/blocklist/bin/blocklistd.c: revision 1.4
 	(applied to external/bsd/blacklist/bin/blacklistd.c)

 PR/57767: Yoshitaka Tokugawa: When restoring, do so from a readonly copy
 of the database and update the read-write copy with the new firewall ids.
 Before we did not update the state file so it contained the old firewall ids.


 To generate a diff of this commit:
 cvs rdiff -u -r1.38 -r1.38.2.1 src/external/bsd/blacklist/bin/blacklistd.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

Home	PR Database Search