NetBSD Problem Report #46419

From www@NetBSD.org  Sun May  6 18:59:41 2012
Return-Path: <www@NetBSD.org>
Received: from mail.netbsd.org (mail.netbsd.org [149.20.53.66])
	by www.NetBSD.org (Postfix) with ESMTP id 0E81F63B915
	for <gnats-bugs@gnats.NetBSD.org>; Sun,  6 May 2012 18:59:41 +0000 (UTC)
Message-Id: <20120506185939.9B9A663B86B@www.NetBSD.org>
Date: Sun,  6 May 2012 18:59:39 +0000 (UTC)
From: er.abhinav.upadhyay@gmail.com
Reply-To: er.abhinav.upadhyay@gmail.com
To: gnats-bugs@NetBSD.org
Subject: Inconsistent information in man.db for man page aliases after updation
X-Send-Pr-Version: www-1.0

>Number:         46419
>Category:       bin
>Synopsis:       Inconsistent information in man.db for man page aliases after updation
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    bin-bug-people
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sun May 06 19:00:00 +0000 2012
>Closed-Date:    Wed May 09 05:33:46 +0000 2012
>Last-Modified:  Wed May 09 05:33:46 +0000 2012
>Originator:     Abhinav Upadhyay
>Release:        NetBSD 6.0_BETA
>Organization:
>Environment:
NetBSD  6.0_BETA NetBSD 6.0_BETA (GENERIC) i386
>Description:
makemandb(8) maintains an index of all the different aliases of the man pages. This information is maintained in the mandb_links table in the database. But there is a small bug in there. When old man pages are removed, the corresponding entry for the aliases of those man pages are not removed from the mandb_links table. 

Similarly, in case a man page is updated to add new aliases or if one or more of its aliases are removed, the mandb_links table won't get updated to reflect this change properly, leading to inconsistent information. 

Note that, the mandb_links table is not being used right now for any purpose but it might be of use in future. Attached patch should fix this problem.
>How-To-Repeat:

Edit a man page, for example /usr/share/man/man1/ls.1 to add a new alias:

.Nm ls, ls2

Run makemandb to update the index

Run: sqlite3 /var/db/man.db "SELECT * FROM mandb_links WHERE target='ls';"

See the entry for the new alias in the output.

Now, edit again the man page and remove that alias.
Re-run makemandb to update index.
Re-run the SQL query and see the alias entry still present.




>Fix:
Index: DBSCHEMA
===================================================================
RCS file: /cvsroot/src/usr.sbin/makemandb/DBSCHEMA,v
retrieving revision 1.1
diff -u -p -r1.1 DBSCHEMA
--- DBSCHEMA    7 Feb 2012 19:13:32 -0000       1.1
+++ DBSCHEMA    6 May 2012 18:13:10 -0000
@@ -50,3 +50,4 @@ There are three tables in the database a
   3. section        The section number  
   4. machine        The machine architecture (if any) for which 
                     the page is relevant
+  5. md5_hash       MD5 Hash of the target man page.
Index: apropos-utils.c
===================================================================
RCS file: /cvsroot/src/usr.sbin/makemandb/apropos-utils.c,v
retrieving revision 1.4
diff -u -p -r1.4 apropos-utils.c
--- apropos-utils.c     15 Apr 2012 15:56:52 -0000      1.4
+++ apropos-utils.c     6 May 2012 18:13:10 -0000
@@ -172,7 +172,7 @@ create_db(sqlite3 *db)
                            "file UNIQUE, md5_hash UNIQUE, id  INTEGER PRIMARY KEY); "
                                //mandb_meta
                        "CREATE TABLE IF NOT EXISTS mandb_links(link, target, section, "
-                           "machine); ";       //mandb_links
+                           "machine, md5_hash); ";     //mandb_links

        sqlite3_exec(db, sqlstr, NULL, NULL, &errmsg);
        if (errmsg != NULL)
@@ -181,7 +181,9 @@ create_db(sqlite3 *db)
        sqlstr = "CREATE INDEX IF NOT EXISTS index_mandb_links ON mandb_links "
                        "(link); "
                        "CREATE INDEX IF NOT EXISTS index_mandb_meta_dev ON mandb_meta "
-                       "(device, inode)";
+                       "(device, inode); "
+                       "CREATE INDEX IF NOT EXISTS index_mandb_links_md5 ON mandb_links "
+                       "(md5_hash);";
        sqlite3_exec(db, sqlstr, NULL, NULL, &errmsg);
        if (errmsg != NULL)
                goto out;
Index: makemandb.c
===================================================================
RCS file: /cvsroot/src/usr.sbin/makemandb/makemandb.c,v
retrieving revision 1.8
diff -u -p -r1.8 makemandb.c
--- makemandb.c 4 May 2012 23:50:26 -0000       1.8
+++ makemandb.c 6 May 2012 18:13:11 -0000
@@ -780,6 +780,8 @@ update_db(sqlite3 *db, struct mparse *mp

        sqlstr = "DELETE FROM mandb_meta WHERE file NOT IN"
                 " (SELECT file FROM metadb.file_cache);"
+                "DELETE FROM mandb_links WHERE md5_hash NOT IN"
+                " (SELECT md5_hash from mandb_meta);"
                 "DROP TABLE metadb.file_cache;"
                 "DELETE FROM mandb WHERE rowid NOT IN"
                 " (SELECT id FROM mandb_meta);";
@@ -1726,9 +1728,9 @@ insert_into_db(sqlite3 *db, mandb_rec *r
                                ln[strlen(ln) - 1] = 0;

                        str = sqlite3_mprintf("INSERT INTO mandb_links"
-                                             " VALUES (%Q, %Q, %Q, %Q)",
+                                             " VALUES (%Q, %Q, %Q, %Q, %Q)",
                                              ln, rec->name, rec->section,
-                                             rec->machine);
+                                             rec->machine, rec->md5_hash);
                        sqlite3_exec(db, str, NULL, NULL, &errmsg);
                        sqlite3_free(str);
                        if (errmsg != NULL) {

>Release-Note:

>Audit-Trail:
From: Matthew Mondor <mm_lists@pulsar-zone.net>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: bin/46419: Inconsistent information in man.db for man page
 aliases after updation
Date: Sun, 6 May 2012 20:44:57 -0400

 On Sun,  6 May 2012 19:00:01 +0000 (UTC)
 er.abhinav.upadhyay@gmail.com wrote:

 > makemandb(8) maintains an index of all the different aliases of the man pages. This information is maintained in the mandb_links table in the database. But there is a small bug in there. When old man pages are removed, the corresponding entry for the aliases of those man pages are not removed from the mandb_links table. 

 Sorry that I didn't yet read the whole implementation, thus this
 question:

 I see that a new column is added to the new table, which gets created
 if it doesn't exist.  Is this table deleted and recreated at every
 makemandb run?  Otherwise, some migration code might be needed to alter
 the existing table and add the new column (or to delete the existing
 tables if they're not of the intended version/format)...

 Or perhaps this doesn't matter as there was no official release yet
 with the new code, and UPDATING could hold a note with instructions for
 the admins to delete the old db (if that is a problem, that is)?

 Thanks,
 -- 
 Matt

From: Abhinav Upadhyay <er.abhinav.upadhyay@gmail.com>
To: gnats-bugs@netbsd.org
Cc: gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: bin/46419: Inconsistent information in man.db for man page
 aliases after updation
Date: Mon, 7 May 2012 12:23:11 +0530

 On Mon, May 7, 2012 at 6:15 AM, Matthew Mondor <mm_lists@pulsar-zone.net> w=
 rote:
 > The following reply was made to PR bin/46419; it has been noted by GNATS.
 >
 > From: Matthew Mondor <mm_lists@pulsar-zone.net>
 > To: gnats-bugs@NetBSD.org
 > Cc:
 > Subject: Re: bin/46419: Inconsistent information in man.db for man page
 > =A0aliases after updation
 > Date: Sun, 6 May 2012 20:44:57 -0400
 >
 > =A0On Sun, =A06 May 2012 19:00:01 +0000 (UTC)
 > =A0er.abhinav.upadhyay@gmail.com wrote:
 >
 > =A0> makemandb(8) maintains an index of all the different aliases of the =
 man pages. This information is maintained in the mandb_links table in the d=
 atabase. But there is a small bug in there. When old man pages are removed,=
  the corresponding entry for the aliases of those man pages are not removed=
  from the mandb_links table.
 >
 > =A0Sorry that I didn't yet read the whole implementation, thus this
 > =A0question:
 >
 > =A0I see that a new column is added to the new table, which gets created
 > =A0if it doesn't exist. =A0Is this table deleted and recreated at every
 > =A0makemandb run? =A0Otherwise, some migration code might be needed to al=
 ter
 > =A0the existing table and add the new column (or to delete the existing
 > =A0tables if they're not of the intended version/format)...
 >
 > =A0Or perhaps this doesn't matter as there was no official release yet
 > =A0with the new code, and UPDATING could hold a note with instructions fo=
 r
 > =A0the admins to delete the old db (if that is a problem, that is)?
 >
 > =A0Thanks,
 > =A0--

 Yes, good catch. I should also change the schema version in
 apropos-utils.h and probably a note should be added to UPDATING to run
 makemandb with the -f flag to force recreation of the db.

 --
 Abhinav

From: "Thomas Klausner" <wiz@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/46419 CVS commit: src/usr.sbin/makemandb
Date: Mon, 7 May 2012 11:18:16 +0000

 Module Name:	src
 Committed By:	wiz
 Date:		Mon May  7 11:18:16 UTC 2012

 Modified Files:
 	src/usr.sbin/makemandb: DBSCHEMA apropos-utils.c apropos-utils.h
 	    makemandb.c

 Log Message:
 PR 46419 by Abhinav Upadhyay using his updated patch:
 Clean up after removing man page aliases.


 To generate a diff of this commit:
 cvs rdiff -u -r1.1 -r1.2 src/usr.sbin/makemandb/DBSCHEMA
 cvs rdiff -u -r1.4 -r1.5 src/usr.sbin/makemandb/apropos-utils.c
 cvs rdiff -u -r1.2 -r1.3 src/usr.sbin/makemandb/apropos-utils.h
 cvs rdiff -u -r1.8 -r1.9 src/usr.sbin/makemandb/makemandb.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: open->pending-pullups
State-Changed-By: wiz@NetBSD.org
State-Changed-When: Mon, 07 May 2012 11:21:19 +0000
State-Changed-Why:
Committed, pullup #229 requested.


From: Matthew Mondor <mm_lists@pulsar-zone.net>
To: gnats-bugs@NetBSD.org
Cc: wiz@NetBSD.org
Subject: Re: bin/46419 (Inconsistent information in man.db for man page
 aliases after updation)
Date: Mon, 7 May 2012 18:46:11 -0400

 On Mon,  7 May 2012 11:21:20 +0000 (UTC)
 wiz@NetBSD.org wrote:

 > Synopsis: Inconsistent information in man.db for man page aliases after updation
 > 
 > State-Changed-From-To: open->pending-pullups
 > State-Changed-By: wiz@NetBSD.org
 > State-Changed-When: Mon, 07 May 2012 11:21:19 +0000
 > State-Changed-Why:
 > Committed, pullup #229 requested.

 Please note that as Abhinav confirmed, a note should be added to
 UPDATING saying that  makemandb -f  should be run to rebuild the whole
 database for the new SQL changes to take effect.

 Thanks,
 -- 
 Matt

From: Thomas Klausner <wiz@NetBSD.org>
To: NetBSD bugtracking <gnats-bugs@NetBSD.org>
Cc: 
Subject: Re: bin/46419 (Inconsistent information in man.db for man page
 aliases after updation)
Date: Tue, 8 May 2012 00:47:54 +0200

 On Mon, May 07, 2012 at 06:46:11PM -0400, Matthew Mondor wrote:
 > Please note that as Abhinav confirmed, a note should be added to
 > UPDATING saying that  makemandb -f  should be run to rebuild the whole
 > database for the new SQL changes to take effect.

 Already done...

 Cheers,
  Thomas

From: Matthew Mondor <mm_lists@pulsar-zone.net>
To: gnats-bugs@NetBSD.org
Cc: wiz@NetBSD.org
Subject: Re: bin/46419 (Inconsistent information in man.db for man page
 aliases after updation)
Date: Mon, 7 May 2012 18:54:24 -0400

 On Mon,  7 May 2012 22:50:06 +0000 (UTC)
 Thomas Klausner <wiz@NetBSD.org> wrote:

 >  On Mon, May 07, 2012 at 06:46:11PM -0400, Matthew Mondor wrote:
 >  > Please note that as Abhinav confirmed, a note should be added to
 >  > UPDATING saying that  makemandb -f  should be run to rebuild the whole
 >  > database for the new SQL changes to take effect.
 >  
 >  Already done...

 Oh, sorry about that, and thanks! :)
 -- 
 Matt

From: "Jeff Rizzo" <riz@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/46419 CVS commit: [netbsd-6] src/usr.sbin/makemandb
Date: Wed, 9 May 2012 03:41:00 +0000

 Module Name:	src
 Committed By:	riz
 Date:		Wed May  9 03:41:00 UTC 2012

 Modified Files:
 	src/usr.sbin/makemandb [netbsd-6]: DBSCHEMA apropos-utils.c
 	    apropos-utils.h makemandb.c

 Log Message:
 Pull up following revision(s) (requested by wiz in ticket #229):
 	usr.sbin/makemandb/makemandb.c: revision 1.9
 	usr.sbin/makemandb/DBSCHEMA: revision 1.2
 	usr.sbin/makemandb/apropos-utils.c: revision 1.5
 	usr.sbin/makemandb/apropos-utils.h: revision 1.3
 PR 46419 by Abhinav Upadhyay using his updated patch:
 Clean up after removing man page aliases.


 To generate a diff of this commit:
 cvs rdiff -u -r1.1 -r1.1.2.1 src/usr.sbin/makemandb/DBSCHEMA
 cvs rdiff -u -r1.2.2.1 -r1.2.2.2 src/usr.sbin/makemandb/apropos-utils.c
 cvs rdiff -u -r1.2 -r1.2.2.1 src/usr.sbin/makemandb/apropos-utils.h
 cvs rdiff -u -r1.2.2.4 -r1.2.2.5 src/usr.sbin/makemandb/makemandb.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: pending-pullups->closed
State-Changed-By: wiz@NetBSD.org
State-Changed-When: Wed, 09 May 2012 05:33:46 +0000
State-Changed-Why:
Pulled up. Thanks for the contribution!


>Unformatted:

Home	PR Database Search