Use reference to err to avoid deletion in Py3.4

[ashgillman]

Update on #1446, I was still getting an error on Python 3.4.4

See also PR #1447

[coveralls]

Coverage increased (+0.0006%) to 72.326% when pulling 8b3c8df on ashgillman:FixMapNodePy3.4 into 3ed411b on nipy:master.

[chrisgorgo]

@oesteban how does this relate to your py3 efforts?

[oesteban]

In Circle we were not testing python 3.4. In travis it didn't complain. I guess there is no test to exercise that exception catching.

[ashgillman]

Here is an MWE to demonstrate my issue.

from nipype.pipeline.engine import MapNode, Workflow from nipype.interfaces import Function def subworkflow_wrapper(arg): import os from nipype.pipeline.engine import Node, Workflow from nipype.interfaces import Function from nipype.utils.filemanip import loadpkl crash_program = """\ def crash_program(arg): raise Exception('Important message, {}'.format(arg)) """ sub_wf = Workflow(name='sub_wf') sub_wf.base_dir = '.' crash = Node(Function(function_str=crash_program, input_names=['arg'], output_names=['res']), name='crash') sub_wf.add_nodes([crash]) sub_wf.inputs.crash.arg = arg sub_wf.run() results = loadpkl( os.path.join(sub_wf.name, crash.name, 'result_{}.pklz'.format(crash.name))) return results.inputs['_outputs']['res'] wf = Workflow(name='wf') sub_wf_each = MapNode( Function(function=subworkflow_wrapper, input_names=['arg'], output_names=['res']), iterfield=['arg'], name='sub_wf_each') wf.add_nodes((sub_wf_each, )) wf.inputs.sub_wf_each.arg = [1, 2, 3] wf.run() 

Th important part of the log being:

160909-09:23:31,375 workflow INFO: Traceback (most recent call last): File "/nix/store/gdq8zym3pd538si7mrfp4av14zggis1i-python3.4-nipype-0.12.0/lib/python3.4/site-packages/nipype/pipeline/plugins/linear.py", line 39, in run node.run(updatehash=updatehash) File "/nix/store/gdq8zym3pd538si7mrfp4av14zggis1i-python3.4-nipype-0.12.0/lib/python3.4/site-packages/nipype/pipeline/engine/nodes.py", line 394, in run self._run_interface() File "/nix/store/gdq8zym3pd538si7mrfp4av14zggis1i-python3.4-nipype-0.12.0/lib/python3.4/site-packages/nipype/pipeline/engine/nodes.py", line 1306, in _run_interface updatehash=updatehash)) File "/nix/store/gdq8zym3pd538si7mrfp4av14zggis1i-python3.4-nipype-0.12.0/lib/python3.4/site-packages/nipype/pipeline/engine/nodes.py", line 1169, in _collate_results for i, node, err in nodes: File "/nix/store/gdq8zym3pd538si7mrfp4av14zggis1i-python3.4-nipype-0.12.0/lib/python3.4/site-packages/nipype/pipeline/engine/nodes.py", line 1162, in _node_runner yield i, node, err UnboundLocalError: local variable 'err' referenced before assignment 

This patch (replicating the PR) fixes it:

import nipype from nipype.utils.misc import str2bool def _node_runner(self, nodes, updatehash=False): for i, node in nodes: err = None try: node.run(updatehash=updatehash) except Exception as this_err: err = this_err # save reference if str2bool(self.config['execution']['stop_on_first_crash']): self._result = node.result raise finally: yield i, node, err nipype.pipeline.engine.nodes.MapNode._node_runner = _node_runner 

[ashgillman]

I realised I was using 12.0 in that last example, but the same error holds for 12.1

160909-09:27:35,148 workflow INFO: Traceback (most recent call last): File "/nix/store/qa2c825midsqi30k0kkz68klw67yp8d7-python3.4-nipype-0.12.1/lib/python3.4/site-packages/nipype/pipeline/plugins/linear.py", line 39, in run node.run(updatehash=updatehash) File "/nix/store/qa2c825midsqi30k0kkz68klw67yp8d7-python3.4-nipype-0.12.1/lib/python3.4/site-packages/nipype/pipeline/engine/nodes.py", line 394, in run self._run_interface() File "/nix/store/qa2c825midsqi30k0kkz68klw67yp8d7-python3.4-nipype-0.12.1/lib/python3.4/site-packages/nipype/pipeline/engine/nodes.py", line 1306, in _run_interface updatehash=updatehash)) File "/nix/store/qa2c825midsqi30k0kkz68klw67yp8d7-python3.4-nipype-0.12.1/lib/python3.4/site-packages/nipype/pipeline/engine/nodes.py", line 1169, in _collate_results for i, node, err in nodes: File "/nix/store/qa2c825midsqi30k0kkz68klw67yp8d7-python3.4-nipype-0.12.1/lib/python3.4/site-packages/nipype/pipeline/engine/nodes.py", line 1162, in _node_runner yield i, node, err UnboundLocalError: local variable 'err' referenced before assignment 

Similarly for Py3.3

160909-09:29:43,881 workflow INFO: Traceback (most recent call last): File "/nix/store/pfdgknf2q99j3rb3qgr4f69a9ynn2zl7-python3.3-nipype-0.12.1/lib/python3.3/site-packages/nipype/pipeline/plugins/linear.py", line 39, in run node.run(updatehash=updatehash) File "/nix/store/pfdgknf2q99j3rb3qgr4f69a9ynn2zl7-python3.3-nipype-0.12.1/lib/python3.3/site-packages/nipype/pipeline/engine/nodes.py", line 394, in run self._run_interface() File "/nix/store/pfdgknf2q99j3rb3qgr4f69a9ynn2zl7-python3.3-nipype-0.12.1/lib/python3.3/site-packages/nipype/pipeline/engine/nodes.py", line 1306, in _run_interface updatehash=updatehash)) File "/nix/store/pfdgknf2q99j3rb3qgr4f69a9ynn2zl7-python3.3-nipype-0.12.1/lib/python3.3/site-packages/nipype/pipeline/engine/nodes.py", line 1169, in _collate_results for i, node, err in nodes: File "/nix/store/pfdgknf2q99j3rb3qgr4f69a9ynn2zl7-python3.3-nipype-0.12.1/lib/python3.3/site-packages/nipype/pipeline/engine/nodes.py", line 1162, in _node_runner yield i, node, err UnboundLocalError: local variable 'err' referenced before assignment 

[chrisgorgo]

Could you try current master? A lot of py3 improvements have been fixed
recently.

On Thu, Sep 8, 2016 at 4:31 PM, Ashley Gillman notifications@github.com
wrote:

I realised I was using 12.0 in that last example, but the same error holds
for 12.1

160909-09:27:35,148 workflow INFO:
Traceback (most recent call last):
File "/nix/store/qa2c825midsqi30k0kkz68klw67yp8d7-python3.4-nipype-0.12.1/lib/python3.4/site-packages/nipype/pipeline/plugins/linear.py", line 39, in run
node.run(updatehash=updatehash)
File "/nix/store/qa2c825midsqi30k0kkz68klw67yp8d7-python3.4-nipype-0.12.1/lib/python3.4/site-packages/nipype/pipeline/engine/nodes.py", line 394, in run
self._run_interface()
File "/nix/store/qa2c825midsqi30k0kkz68klw67yp8d7-python3.4-nipype-0.12.1/lib/python3.4/site-packages/nipype/pipeline/engine/nodes.py", line 1306, in _run_interface
updatehash=updatehash))
File "/nix/store/qa2c825midsqi30k0kkz68klw67yp8d7-python3.4-nipype-0.12.1/lib/python3.4/site-packages/nipype/pipeline/engine/nodes.py", line 1169, in _collate_results
for i, node, err in nodes:
File "/nix/store/qa2c825midsqi30k0kkz68klw67yp8d7-python3.4-nipype-0.12.1/lib/python3.4/site-packages/nipype/pipeline/engine/nodes.py", line 1162, in _node_runner
yield i, node, err
UnboundLocalError: local variable 'err' referenced before assignment

Similarly for Py3.3

160909-09:29:43,881 workflow INFO:
Traceback (most recent call last):
File "/nix/store/pfdgknf2q99j3rb3qgr4f69a9ynn2zl7-python3.3-nipype-0.12.1/lib/python3.3/site-packages/nipype/pipeline/plugins/linear.py", line 39, in run
node.run(updatehash=updatehash)
File "/nix/store/pfdgknf2q99j3rb3qgr4f69a9ynn2zl7-python3.3-nipype-0.12.1/lib/python3.3/site-packages/nipype/pipeline/engine/nodes.py", line 394, in run
self._run_interface()
File "/nix/store/pfdgknf2q99j3rb3qgr4f69a9ynn2zl7-python3.3-nipype-0.12.1/lib/python3.3/site-packages/nipype/pipeline/engine/nodes.py", line 1306, in _run_interface
updatehash=updatehash))
File "/nix/store/pfdgknf2q99j3rb3qgr4f69a9ynn2zl7-python3.3-nipype-0.12.1/lib/python3.3/site-packages/nipype/pipeline/engine/nodes.py", line 1169, in _collate_results
for i, node, err in nodes:
File "/nix/store/pfdgknf2q99j3rb3qgr4f69a9ynn2zl7-python3.3-nipype-0.12.1/lib/python3.3/site-packages/nipype/pipeline/engine/nodes.py", line 1162, in _node_runner
yield i, node, err
UnboundLocalError: local variable 'err' referenced before assignment

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#1551 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAOkp4PE4Fv-pR_N_vOV7L06FdGYtaBSks5qoJrAgaJpZM4JVyIY
.

[ashgillman]

On master (0f9a620)

160909-10:20:32,360 workflow INFO: Traceback (most recent call last): File "/nix/store/mgzabyx9n2fhdg9zfb276f0dgzjmd7s4-python3.3-nipype-master/lib/python3.3/site-packages/nipype/pipeline/plugins/linear.py", line 43, in run node.run(updatehash=updatehash) File "/nix/store/mgzabyx9n2fhdg9zfb276f0dgzjmd7s4-python3.3-nipype-master/lib/python3.3/site-packages/nipype/pipeline/engine/nodes.py", line 377, in run self._run_interface() File "/nix/store/mgzabyx9n2fhdg9zfb276f0dgzjmd7s4-python3.3-nipype-master/lib/python3.3/site-packages/nipype/pipeline/engine/nodes.py", line 1294, in _run_interface updatehash=updatehash)) File "/nix/store/mgzabyx9n2fhdg9zfb276f0dgzjmd7s4-python3.3-nipype-master/lib/python3.3/site-packages/nipype/pipeline/engine/nodes.py", line 1157, in _collate_results for i, node, err in nodes: File "/nix/store/mgzabyx9n2fhdg9zfb276f0dgzjmd7s4-python3.3-nipype-master/lib/python3.3/site-packages/nipype/pipeline/engine/nodes.py", line 1150, in _node_runner yield i, node, err UnboundLocalError: local variable 'err' referenced before assignment 

160909-10:34:17,162 workflow INFO: Traceback (most recent call last): File "/nix/store/95dlwka1jjbzca2fz232m6rwi1453svr-python3.4-nipype-master/lib/python3.4/site-packages/nipype/pipeline/plugins/linear.py", line 43, in run node.run(updatehash=updatehash) File "/nix/store/95dlwka1jjbzca2fz232m6rwi1453svr-python3.4-nipype-master/lib/python3.4/site-packages/nipype/pipeline/engine/nodes.py", line 377, in run self._run_interface() File "/nix/store/95dlwka1jjbzca2fz232m6rwi1453svr-python3.4-nipype-master/lib/python3.4/site-packages/nipype/pipeline/engine/nodes.py", line 1294, in _run_interface updatehash=updatehash)) File "/nix/store/95dlwka1jjbzca2fz232m6rwi1453svr-python3.4-nipype-master/lib/python3.4/site-packages/nipype/pipeline/engine/nodes.py", line 1157, in _collate_results for i, node, err in nodes: File "/nix/store/95dlwka1jjbzca2fz232m6rwi1453svr-python3.4-nipype-master/lib/python3.4/site-packages/nipype/pipeline/engine/nodes.py", line 1150, in _node_runner yield i, node, err UnboundLocalError: local variable 'err' referenced before assignment 

160909-11:22:58,974 workflow INFO: Traceback (most recent call last): File "/nix/store/a58j94iacgx2ip52y1i6ff6n7dq3fpdb-python3.5-nipype-master/lib/python3.5/site-packages/nipype/pipeline/plugins/linear.py", line 43, in run node.run(updatehash=updatehash) File "/nix/store/a58j94iacgx2ip52y1i6ff6n7dq3fpdb-python3.5-nipype-master/lib/python3.5/site-packages/nipype/pipeline/engine/nodes.py", line 377, in run self._run_interface() File "/nix/store/a58j94iacgx2ip52y1i6ff6n7dq3fpdb-python3.5-nipype-master/lib/python3.5/site-packages/nipype/pipeline/engine/nodes.py", line 1294, in _run_interface updatehash=updatehash)) File "/nix/store/a58j94iacgx2ip52y1i6ff6n7dq3fpdb-python3.5-nipype-master/lib/python3.5/site-packages/nipype/pipeline/engine/nodes.py", line 1157, in _collate_results for i, node, err in nodes: File "/nix/store/a58j94iacgx2ip52y1i6ff6n7dq3fpdb-python3.5-nipype-master/lib/python3.5/site-packages/nipype/pipeline/engine/nodes.py", line 1150, in _node_runner yield i, node, err UnboundLocalError: local variable 'err' referenced before assignment 

I hadn't realised it was all the Py3 versions. It seems strange no one else has noticed this, can any one else replicate?

[alexsavio]

Yes, the same happens to me. I didn't see it, since there is not test for this. Could you make a test from your MWE and push it along your fix?

[ashgillman]

Test added, fails w/o change, passes with new change.