[gnome-continuous-yocto/gnomeostree-3.28-rocko: 4424/8267] verify-bashisms: fix problems with tinfoil2
- From: Emmanuele Bassi <ebassi src gnome org>
- To: commits-list gnome org
- Cc:
- Subject: [gnome-continuous-yocto/gnomeostree-3.28-rocko: 4424/8267] verify-bashisms: fix problems with tinfoil2
- Date: Sun, 17 Dec 2017 02:01:03 +0000 (UTC)
commit 010f9fa86a993554e2b9134e6baf96899649a2ef
Author: Patrick Ohly <patrick ohly intel com>
Date: Tue Jan 31 13:50:29 2017 +0100
verify-bashisms: fix problems with tinfoil2
tinfoil2 is based on a client/server architecture, which broke the
verify-bashisms script:
- The tinfoil instance and its data proxies can't be pickled, so
all interaction with the bitbake server has to run in the main
script process and only processing of the plain scripts can
be done with multiprocessing:
_pickle.PicklingError: Can't pickle <class
'bb.tinfoil.TinfoilCookerAdapter.TinfoilRecipeCacheAdapter'>: attribute lookup TinfoilRecipeCacheAdapter on
bb.tinfoil failed
- The multiprocessing pool has to be created before initializing
tinfoil, otherwise the pool workers end up trying to communicate
with the bitbake server during shutdown:
ERROR: UI received SIGTERM
Process ForkPoolWorker-2:
Traceback (most recent call last):
File "/usr/lib/python3.4/multiprocessing/process.py", line 257, in _bootstrap
util._exit_function()
File "/usr/lib/python3.4/multiprocessing/util.py", line 286, in _exit_function
_run_finalizers(0)
...
File "/usr/lib/python3.4/multiprocessing/process.py", line 131, in is_alive
assert self._parent_pid == os.getpid(), 'can only test a child process'
AssertionError: can only test a child process
- func() needs to defined before creating the pool to avoid:
AttributeError: Can't get attribute 'func' on <module '__main__' from
'/work/openembedded-core/scripts/verify-bashisms'>
(From OE-Core rev: aa439f11c7f414774843720d68ebe0a6d3375ea6)
Signed-off-by: Patrick Ohly <patrick ohly intel com>
Signed-off-by: Richard Purdie <richard purdie linuxfoundation org>
scripts/verify-bashisms | 44 +++++++++++++++++++++++++-------------------
1 files changed, 25 insertions(+), 19 deletions(-)
---
diff --git a/scripts/verify-bashisms b/scripts/verify-bashisms
index ed0a563..613f174 100755
--- a/scripts/verify-bashisms
+++ b/scripts/verify-bashisms
@@ -71,6 +71,20 @@ if __name__=='__main__':
print("Cannot find checkbashisms.pl on $PATH, get it from
https://anonscm.debian.org/cgit/collab-maint/devscripts.git/plain/scripts/checkbashisms.pl")
sys.exit(1)
+ # The order of defining the worker function,
+ # initializing the pool and connecting to the
+ # bitbake server is crucial, don't change it.
+ def func(item):
+ fn, scripts = item
+ result = []
+ for key, script in scripts:
+ r = process(fn, key, script)
+ if r: result.extend(r)
+ return fn, result
+
+ import multiprocessing
+ pool = multiprocessing.Pool()
+
tinfoil = get_tinfoil()
# This is only the default configuration and should iterate over
@@ -83,32 +97,24 @@ if __name__=='__main__':
else:
initial_pns = sorted(pkg_pn)
- pns = []
- print("Generating file list...")
+ pns = {}
+ print("Generating scripts...")
for pn in initial_pns:
for fn in pkg_pn[pn]:
# There's no point checking multiple BBCLASSEXTENDed variants of the same recipe
realfn, _, _ = bb.cache.virtualfn2realfn(fn)
if realfn not in pns:
- pns.append(realfn)
-
-
- def func(fn):
- result = []
- data = tinfoil.parse_recipe_file(fn)
- for key in data.keys():
- if data.getVarFlag(key, "func") and not data.getVarFlag(key, "python"):
- script = data.getVar(key, False)
- if not script: continue
- #print ("%s:%s" % (fn, key))
- r = process(fn, key, script)
- if r: result.extend(r)
- return fn, result
+ data = tinfoil.parse_recipe_file(realfn)
+ scripts = []
+ for key in data.keys():
+ if data.getVarFlag(key, "func") and not data.getVarFlag(key, "python"):
+ script = data.getVar(key, False)
+ if script:
+ scripts.append((key, script))
+ pns[realfn] = scripts
print("Scanning scripts...\n")
- import multiprocessing
- pool = multiprocessing.Pool()
- for pn,results in pool.imap(func, pns):
+ for pn, results in pool.imap(func, pns.items()):
if results:
print(pn)
for message,source in results:
[
Date Prev][
Date Next] [
Thread Prev][
Thread Next]
[
Thread Index]
[
Date Index]
[
Author Index]