fix[venom]: fix invalid `phi`s after SCCP #4181

HodanPlodky · 2024-07-25T15:09:26Z

What I did

Fix for issues #4162 and #4070

How I did it

Removed the phi nodes which were not in use after SCCP pass. This could happen when the SCCP would replace jnz by jmp instruction and by this removing need for phi instruction (this is the case in #4162). More over the when merging the basic block in the SimplifyCFGPass there could be problem when renaming the labels in phi nodes since it went only into the first cfg_out basic block (this is the case in #4070)

How to verify it

So far both of the POC from compile correctly

Commit message

This commit reduces `phi` nodes which, after SCCP, refer to a
basic block which is unreachable.  This could happen when the
SCCP would replace a `jnz` by a `jmp` instruction, resulting in a
`phi` instruction which refers to a non-existent block, or a `phi`
instruction in a basic block which only has one predecessor.  This
commit reduces such `phi` instructions into `store` instructions.

charles-cooper · 2024-07-26T01:42:41Z

shouldn't we fix up the phi nodes during sccp directly?

vyper/venom/passes/remove_invalid_phi_nodes.py

vyper/venom/passes/sccp/sccp.py

harkal · 2024-09-06T10:37:54Z

vyper/venom/passes/sccp/sccp.py

@@ -305,6 +311,9 @@ def _replace_constants(self, inst: IRInstruction):
                inst.opcode = "jmp"
                inst.operands = [target]
                self.cfg_dirty = True
+                for bb in inst.parent.cfg_out:
+                    if bb.label == target:
+                        self.cfg_phi_fix_needed.add(bb)


Why not self._fix_phi_bb_r(bb, OrderedSet()) here and remove all the need for self.cfg_phi_fix_needed and iterating later?

The reason was not to iterate over some basic block multiple times. Since solutions only detects start from which the occurrence of faulty phi node could be possible but not the end there could be a lot of overlap. In this case since I iterate over possible positions of the faulty phi nodes at the end I visit every possible basic block only once.

codecov · 2024-09-17T11:12:30Z

Codecov Report

Attention: Patch coverage is 70.45455% with 13 lines in your changes missing coverage. Please review.

Project coverage is 88.22%. Comparing base (03095ce) to head (a867595).

Files with missing lines	Patch %	Lines
vyper/venom/passes/sccp/sccp.py	70.58%	9 Missing and 1 partial ⚠️
vyper/venom/passes/simplify_cfg.py	70.00%	1 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #4181      +/-   ##
==========================================
- Coverage   91.35%   88.22%   -3.14%     
==========================================
  Files         109      109              
  Lines       15641    15672      +31     
  Branches     3443     3454      +11     
==========================================
- Hits        14289    13826     -463     
- Misses        920     1314     +394     
- Partials      432      532     +100

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

charles-cooper · 2024-09-18T13:13:03Z

vyper/venom/passes/sccp/sccp.py

+        from_src_bb = filter(lambda x: x[0] == src_bb.label, phi_inst.phi_operands)
+        operands = list(map(lambda x: x[1], from_src_bb))


please use a list comprehension here

Suggested change

from_src_bb = filter(lambda x: x[0] == src_bb.label, phi_inst.phi_operands)

operands = list(map(lambda x: x[1], from_src_bb))

operands = [op for label, op in phi_inst.phi_operands if label == src_bb.label]

charles-cooper · 2024-09-18T13:13:15Z

vyper/venom/passes/sccp/sccp.py

+
+        assert len(operands) == 1
+        assert isinstance(operands[0], IRVariable)
+        assert phi_inst.output is not None


Suggested change

assert phi_inst.output is not None

assert phi_inst.output is not None

charles-cooper · 2024-09-18T13:15:29Z

vyper/venom/passes/sccp/sccp.py

@@ -305,6 +311,9 @@ def _replace_constants(self, inst: IRInstruction):
                inst.opcode = "jmp"
                inst.operands = [target]
                self.cfg_dirty = True
+                for bb in inst.parent.cfg_out:
+                    if bb.label == target:
+                        self.cfg_phi_fix_needed.add(bb)


Suggested change

self.cfg_phi_fix_needed.add(bb)

self.cfg_phi_fix_needed.add(bb)

break

charles-cooper · 2024-09-18T13:19:31Z

vyper/venom/passes/sccp/sccp.py

+        for next_bb in bb.cfg_out:
+            self._fix_phi_bb_r(next_bb, visited)
+
+    def _fix_phi_node_inst(self, phi_inst: IRInstruction):


Suggested change

def _fix_phi_node_inst(self, phi_inst: IRInstruction):

def _fix_phi_inst(self, phi_inst: IRInstruction):

charles-cooper · 2024-09-18T13:56:01Z

vyper/venom/passes/sccp/sccp.py

@@ -329,6 +338,40 @@ def _replace_constants(self, inst: IRInstruction):
                if isinstance(lat, IRLiteral):
                    inst.operands[i] = lat

+    def _fix_phi_nodes(self):
+        visited: OrderedSet[IRBasicBlock] = OrderedSet()


why keep track of visited, instead of just iterating over fn.get_basic_blocks()?

charles-cooper · 2024-09-18T14:00:07Z

vyper/venom/passes/sccp/sccp.py

+        assert len(operands) == 1
+        assert isinstance(operands[0], IRVariable)
+        assert phi_inst.output is not None
+        phi_inst.output.value = operands[0]


why modify the instruction if it gets removed on the next line?

charles-cooper · 2024-09-18T14:12:52Z

vyper/venom/passes/sccp/sccp.py

+            self._fix_phi_bb_r(next_bb, visited)
+
+    def _fix_phi_node_inst(self, phi_inst: IRInstruction):
+        if len(phi_inst.parent.cfg_in) != 1:


when can this happen? shouldn't we handle this case?

charles-cooper · 2024-09-18T15:19:35Z

i pushed some cleanup here: HodanPlodky#2. it feels like a kludge though, since it seems like we really should be fixing up these phi nodes inside of _handle_SSA_work_item.

vyper/venom/passes/sccp/sccp.py

simplify the cleanup - transforming phis to store instead of removing them

charles-cooper

pushed some stuff, lgtm now. @harkal mind taking a look?

harkal

lgtm

This commit reduces `phi` nodes which, after SCCP, refer to a basic block which is unreachable. This could happen when the SCCP would replace a `jnz` by a `jmp` instruction, resulting in a `phi` instruction which refers to a non-existent block, or a `phi` instruction in a basic block which only has one predecessor. This commit reduces such `phi` instructions into `store` instructions. --------- Co-authored-by: Harry Kalogirou <harkal@nlogn.eu> Co-authored-by: Charles Cooper <cooper.charles.m@gmail.com>

HodanPlodky added 4 commits July 18, 2024 17:03

fix[venom]: test fix for issue vyperlang#4070

9ceb3e3

fix[venom]: ternary issue

c18ca96

fix[venom]: ternary issue

dff03be

fix[venom]: removing problematic phi nodes

98652d0

github-advanced-security bot found potential problems Jul 26, 2024

View reviewed changes

vyper/venom/passes/remove_invalid_phi_nodes.py Fixed Show fixed Hide fixed

fix[venom]: moved phi fix to sccp pass

db53eec

HodanPlodky changed the title ~~Fix/simplify cfg phi rename~~ fix/simplify cfg phi rename Aug 15, 2024

HodanPlodky changed the title ~~fix/simplify cfg phi rename~~ fix[venom]: simplify cfg phi rename Aug 15, 2024

HodanPlodky marked this pull request as ready for review August 16, 2024 15:18

harkal reviewed Aug 30, 2024

View reviewed changes

vyper/venom/passes/sccp/sccp.py Show resolved Hide resolved

github-advanced-security bot found potential problems Sep 2, 2024

View reviewed changes

vyper/venom/passes/sccp/sccp.py Fixed Show fixed Hide fixed

HodanPlodky added 2 commits September 2, 2024 15:09

better phi fixing

37dc99d

better phi fixing (lint)

4bd7fb7

harkal requested changes Sep 6, 2024

View reviewed changes

harkal approved these changes Sep 17, 2024

View reviewed changes

Merge branch 'master' into fix/simplify-cfg-phi-rename

a867595

charles-cooper reviewed Sep 18, 2024

View reviewed changes

simplify the cleanup

4f57fa1

charles-cooper reviewed Sep 18, 2024

View reviewed changes

vyper/venom/passes/sccp/sccp.py Outdated Show resolved Hide resolved

charles-cooper and others added 2 commits September 18, 2024 11:48

transform phis into stores

a39bbe1

Merge pull request #2 from charles-cooper/simplify-cfg-rename

59f64f0

simplify the cleanup - transforming phis to store instead of removing them

simplify some code

1128c60

charles-cooper requested a review from harkal September 22, 2024 23:53

charles-cooper approved these changes Sep 22, 2024

View reviewed changes

Merge branch 'master' into fix/simplify-cfg-phi-rename

826a0d3

harkal approved these changes Sep 26, 2024

View reviewed changes

fix a comment

c552757

charles-cooper changed the title ~~fix[venom]: simplify cfg phi rename~~ fix[venom]: fix invalid phis after SCCP Sep 26, 2024

charles-cooper added 2 commits September 26, 2024 20:59

add test for cfg simplification

ab38dea

add test for phi reduction

4599ea1

charles-cooper merged commit e2f6001 into vyperlang:master Sep 27, 2024
155 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix[venom]: fix invalid `phi`s after SCCP #4181

fix[venom]: fix invalid `phi`s after SCCP #4181

HodanPlodky commented Jul 25, 2024 •

edited by charles-cooper

Loading

charles-cooper commented Jul 26, 2024

harkal Sep 6, 2024

HodanPlodky Sep 9, 2024

codecov bot commented Sep 17, 2024

charles-cooper Sep 18, 2024

charles-cooper Sep 18, 2024

charles-cooper Sep 18, 2024

charles-cooper Sep 18, 2024

charles-cooper Sep 18, 2024

charles-cooper Sep 18, 2024

charles-cooper Sep 18, 2024

charles-cooper commented Sep 18, 2024

charles-cooper left a comment

harkal left a comment

		from_src_bb = filter(lambda x: x[0] == src_bb.label, phi_inst.phi_operands)
		operands = list(map(lambda x: x[1], from_src_bb))

	from_src_bb = filter(lambda x: x[0] == src_bb.label, phi_inst.phi_operands)
	operands = list(map(lambda x: x[1], from_src_bb))
	operands = [op for label, op in phi_inst.phi_operands if label == src_bb.label]

	assert phi_inst.output is not None
	assert phi_inst.output is not None

	self.cfg_phi_fix_needed.add(bb)
	self.cfg_phi_fix_needed.add(bb)
	break

	def _fix_phi_node_inst(self, phi_inst: IRInstruction):
	def _fix_phi_inst(self, phi_inst: IRInstruction):

fix[venom]: fix invalid phis after SCCP #4181

fix[venom]: fix invalid phis after SCCP #4181

Conversation

HodanPlodky commented Jul 25, 2024 • edited by charles-cooper Loading

What I did

How I did it

How to verify it

Commit message

charles-cooper commented Jul 26, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Sep 17, 2024

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

charles-cooper commented Sep 18, 2024

charles-cooper left a comment

Choose a reason for hiding this comment

harkal left a comment

Choose a reason for hiding this comment

fix[venom]: fix invalid `phi`s after SCCP #4181

fix[venom]: fix invalid `phi`s after SCCP #4181

HodanPlodky commented Jul 25, 2024 •

edited by charles-cooper

Loading