NetBSD Problem Report #53344

From www@NetBSD.org  Tue Jun  5 15:03:13 2018
Return-Path: <www@NetBSD.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 868107A151
	for <gnats-bugs@gnats.NetBSD.org>; Tue,  5 Jun 2018 15:03:13 +0000 (UTC)
Message-Id: <20180605150312.343CE7A1EE@mollari.NetBSD.org>
Date: Tue,  5 Jun 2018 15:03:12 +0000 (UTC)
From: tr@vispaul.me
Reply-To: tr@vispaul.me
To: gnats-bugs@NetBSD.org
Subject: sysutils/salt 2018.3.0 pkgin module fixes
X-Send-Pr-Version: www-1.0

>Number:         53344
>Category:       pkg
>Synopsis:       sysutils/salt 2018.3.0 pkgin module fixes
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    tpaul
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Tue Jun 05 15:05:00 +0000 2018
>Closed-Date:    Thu Jun 21 11:21:08 +0000 2018
>Last-Modified:  Thu Jun 21 11:21:08 +0000 2018
>Originator:     Travis Paul
>Release:        current
>Organization:
>Environment:
NetBSD n7.local 7.1.2 NetBSD 7.1.2 (GENERIC.201803151611Z) amd64

>Description:
Salt's pkgin module has some issues:
- pkg.file_dict crashes
- pkg.latest_version doesn't actually work

Patches accepted upstream (also added some unit tests upstream): https://github.com/saltstack/salt/pull/47814

I was informed that these fixes will not make it into 2018.3.1, and will be available 2 months after 2018.3.1 is released at the earliest.

salt-2017.7.4 in pkgsrc 2018Q1 is also broken but the fixes for pkg.latest_version don't work there and I haven't been able to fix the issue in that version yet. 
>How-To-Repeat:
pkgin reports that nginx version 1.14.0 is available but not yet installed:

# pkgin se nginx
nginx-1.14.0         Lightweight HTTP server and mail proxy server

=: package is installed and up-to-date
<: package is installed but newer version is available
: installed package has a greater version than available package

However, pkg.latest_version returns nothing:

# salt \* pkg.latest_version nginx
salt.n7test.local:


pkg_info shows the installed files for an installed package:

# pkg_info -qL nginx
/opt/pkg/current/man/man8/nginx.8
/opt/pkg/current/sbin/nginx
/opt/pkg/current/share/examples/nginx/conf/fastcgi.conf
/opt/pkg/current/share/examples/nginx/conf/fastcgi_params
/opt/pkg/current/share/examples/nginx/conf/koi-utf
/opt/pkg/current/share/examples/nginx/conf/koi-win
/opt/pkg/current/share/examples/nginx/conf/mime.types
/opt/pkg/current/share/examples/nginx/conf/nginx.conf
/opt/pkg/current/share/examples/nginx/conf/win-utf
/opt/pkg/current/share/examples/nginx/html/50x.html
/opt/pkg/current/share/examples/nginx/html/inde



However, pkg.file_dict throws an exception:

# salt \* pkg.file_dict nginx

salt.n7-test.local:
   The minion function caused an exception: Traceback (most recent call last):
     File "/opt/pkg/current/lib/python2.7/site-packages/salt/minion.py", line 1600, in _thread_return
       return_data = minion_instance.executors[fname](opts, data, func, args, kwargs)
     File "/opt/pkg/current/lib/python2.7/site-packages/salt/executors/direct_call.py", line 12, in execute
       return func(*args, **kwargs)
     File "/opt/pkg/current/lib/python2.7/site-packages/salt/modules/pkgin.py", line 684, in file_dict
       for field in ret:
   RuntimeError: dictionary changed size during iteration
>Fix:
Index: distinfo
===================================================================
RCS file: /cvsroot/pkgsrc/sysutils/salt/distinfo,v
retrieving revision 1.35
diff -u -r1.35 distinfo
--- distinfo    12 May 2018 11:29:35 -0000      1.35
+++ distinfo    5 Jun 2018 09:05:25 -0000
@@ -5,4 +5,5 @@
 SHA512 (salt-2018.3.0.tar.gz) = 3d64a4d12d4ff7effe644b682b138d8f49fd00523e9cfe3e80bebff9c944a089fb41cbac92e63139633425f49c1eebeb99145290a0bdd1b4f306da86abe9b2e5
 Size (salt-2018.3.0.tar.gz) = 13448567 bytes
 SHA1 (patch-salt_grains_core.py) = 735a71cd9616be34cce6b015da7a70cace08accd
+SHA1 (patch-salt_modules_pkgin.py) = c2c2d10d7657f3a64d7e1f3292de0d8cb798480a
 SHA1 (patch-salt_version.py) = 1827dac3609a938fae38ee5dfd2a873c9723dfbd
Index: patches/patch-salt_modules_pkgin.py
===================================================================
RCS file: patches/patch-salt_modules_pkgin.py
diff -N patches/patch-salt_modules_pkgin.py
--- /dev/null   1 Jan 1970 00:00:00 -0000
+++ patches/patch-salt_modules_pkgin.py 5 Jun 2018 09:05:25 -0000
@@ -0,0 +1,38 @@
+$NetBSD$
+
+Fixes 2 bugs in the pkgin module:
+- pkg.latest_version doesn't return a version for an uninstalled package.
+- pkg.file_dict crashes.
+https://github.com/saltstack/salt/pull/47814
+
+--- salt/modules/pkgin.py.orig 2018-04-02 16:35:12.000000000 +0000
++++ salt/modules/pkgin.py
+@@ -181,7 +181,9 @@ def latest_version(*names, **kwargs):
+
+         out = __salt__['cmd.run'](cmd, output_loglevel='trace')
+         for line in out.splitlines():
+-            p = line.split(',' if _supports_parsing() else None)
++            if line.startswith('No results found for'):
++                return pkglist
++            p = line.split(';' if _supports_parsing() else None)
+
+             if p and p[0] in ('=:', '<:', '>:', ''):
+                 # These are explanation comments
+@@ -190,7 +192,7 @@ def latest_version(*names, **kwargs):
+                 s = _splitpkg(p[0])
+                 if s:
+                     if not s[0] in pkglist:
+-                        if len(p) > 1 and p[1] == '<':
++                        if len(p) > 1 and p[1] in ('<', '', '='):
+                             pkglist[s[0]] = s[1]
+                         else:
+                             pkglist[s[0]] = ''
+@@ -681,7 +683,7 @@ def file_dict(*packages):
+                 continue  # unexpected string
+
+     ret = {'errors': errors, 'files': files}
+-    for field in ret:
++    for field in list(ret):
+         if not ret[field] or ret[field] == '':
+             del ret[field]
+     return ret

>Release-Note:

>Audit-Trail:
From: "Travis Paul" <tpaul@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/53344 CVS commit: pkgsrc/sysutils/salt
Date: Wed, 20 Jun 2018 11:50:56 +0000

 Module Name:	pkgsrc
 Committed By:	tpaul
 Date:		Wed Jun 20 11:50:56 UTC 2018

 Modified Files:
 	pkgsrc/sysutils/salt: Makefile distinfo
 Added Files:
 	pkgsrc/sysutils/salt/patches: patch-salt_grains_core.py
 	    patch-salt_modules_pkgin.py

 Log Message:
 salt: don't crash if no swap is configured, apply pkgin module fixes

 patches/patch-salt_grains_core.py:
 - Reapply patch in a NetBSD/OpenBSD specific code block after package update
   to 2018.3.1, from PR pkg/53278
 - upstream pull #47600

 patches/patch-salt_modules_pkgin.py:
 - Apply patch in pkgin specific code block from PR pkg/53344
 - upstream pull #47814

 bump PKGREVISION
 ok <leot>


 To generate a diff of this commit:
 cvs rdiff -u -r1.62 -r1.63 pkgsrc/sysutils/salt/Makefile
 cvs rdiff -u -r1.36 -r1.37 pkgsrc/sysutils/salt/distinfo
 cvs rdiff -u -r0 -r1.5 pkgsrc/sysutils/salt/patches/patch-salt_grains_core.py
 cvs rdiff -u -r0 -r1.1 \
     pkgsrc/sysutils/salt/patches/patch-salt_modules_pkgin.py

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

Responsible-Changed-From-To: pkg-manager->tpaul
Responsible-Changed-By: tpaul@NetBSD.org
Responsible-Changed-When: Thu, 21 Jun 2018 11:21:08 +0000
Responsible-Changed-Why:
I applied the patch


State-Changed-From-To: open->closed
State-Changed-By: tpaul@NetBSD.org
State-Changed-When: Thu, 21 Jun 2018 11:21:08 +0000
State-Changed-Why:
Patch was applied and tested

>Unformatted:

NetBSD Home
NetBSD PR Database Search

(Contact us) $NetBSD: query-full-pr,v 1.43 2018/01/16 07:36:43 maya Exp $
$NetBSD: gnats_config.sh,v 1.9 2014/08/02 14:16:04 spz Exp $
Copyright © 1994-2017 The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.