-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Chopping and Dicing #925
Draft
EagleoutIce
wants to merge
11
commits into
main
Choose a base branch
from
902-dicing
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+450
−9
Draft
Chopping and Dicing #925
Changes from 10 commits
Commits
Show all changes
11 commits
Select commit
Hold shift + click to select a range
6a32f48
feat(dicing): basic test and start of naive approach
Ehcsan a67b08d
lint-fix(dicing): linter fixes
Ehcsan 580c276
Merge branch 'main' into 902-dicing
EagleoutIce e600e24
feat-fix(dicing): corrected dicing result
Ehcsan a6f0706
Merge branch '902-dicing' of https://github.com/flowr-analysis/flowr …
Ehcsan a575d06
test-fix(dicing): removed .only from test
Ehcsan 7dccd66
feat(dicing): implemented pipeline for dicing
Ehcsan 6defc92
feat(dicing): finished naive dicing
Ehcsan b00497f
test(dicing): added more tests for dicing
Ehcsan d8f8436
Update test/functionality/slicing/slicing.spec.ts
EagleoutIce c15b349
feat(dicing): added intersection, difference
Ehcsan File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
| import type { DeepReadonly } from 'ts-essentials' | ||
| import type { DataflowInformation } from '../../../../dataflow/info' | ||
| import type { NormalizedAst } from '../../../../r-bridge/lang-4.x/ast/model/processing/decorate' | ||
| import type { SlicingCriteria } from '../../../../slicing/criterion/parse' | ||
| import { PipelineStepStage } from '../../pipeline-step' | ||
| import type { IPipelineStep } from '../../pipeline-step' | ||
| import { internalPrinter, StepOutputFormat } from '../../../print/print' | ||
| import { staticDicing } from '../../../../slicing/static/dicer' | ||
|
|
||
| export interface DiceRequiredInput { | ||
| /** The dicing criterion is only of interest if you actually want to Dice the R code */ | ||
| readonly startingCriterion: SlicingCriteria, | ||
|
|
||
| readonly endCriterion: SlicingCriteria, | ||
|
|
||
| /** How many re-visits of the same node are ok? */ | ||
| readonly threshold?: number | ||
| } | ||
|
|
||
| function processor(results: { dataflow?: DataflowInformation, normalize?: NormalizedAst }, input: Partial<DiceRequiredInput>) { | ||
| return staticDicing((results.dataflow as DataflowInformation).graph, results.normalize as NormalizedAst, input.endCriterion as SlicingCriteria, input.startingCriterion as SlicingCriteria, input.threshold) | ||
| } | ||
|
|
||
| export const STATIC_DICE = { | ||
| name: 'dice', | ||
| humanReadableName: 'static dice', | ||
| description: 'Calculate the actual static dice from the dataflow graph and the given slicing criteria', | ||
| processor, | ||
| executed: PipelineStepStage.OncePerRequest, | ||
| printer: { | ||
| [StepOutputFormat.Internal]: internalPrinter | ||
| }, | ||
| dependencies: [ 'dataflow' ], | ||
| requiredInput: undefined as unknown as DiceRequiredInput | ||
| } as const satisfies DeepReadonly<IPipelineStep<'dice', typeof processor>> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,116 @@ | ||
| import { initializeCleanEnvironments } from '../../dataflow/environments/environment' | ||
| import { edgeIncludesType, EdgeType, shouldTraverseEdge, TraverseEdge } from '../../dataflow/graph/edge' | ||
| import type { DataflowGraph } from '../../dataflow/graph/graph' | ||
| import { VertexType } from '../../dataflow/graph/vertex' | ||
| import type { NormalizedAst } from '../../r-bridge/lang-4.x/ast/model/processing/decorate' | ||
| import type { NodeId } from '../../r-bridge/lang-4.x/ast/model/processing/node-id' | ||
| import { guard } from '../../util/assert' | ||
| import { expensiveTrace } from '../../util/log' | ||
| import { convertAllSlicingCriteriaToIds, type SlicingCriteria } from '../criterion/parse' | ||
| import { envFingerprint } from './fingerprint' | ||
| import { handleReturns, sliceForCall } from './slice-call' | ||
| import type { SliceResult } from './slicer-types' | ||
| import { slicerLogger, staticSlicing } from './static-slicer' | ||
| import { VisitingQueue } from './visiting-queue' | ||
|
|
||
|
|
||
|
|
||
| export function staticDicing(graph: DataflowGraph, ast: NormalizedAst, endCriteria: SlicingCriteria, startCriteria: SlicingCriteria, threshold = 75): Readonly<SliceResult> { | ||
| const backwardsSlice = staticSlicing(graph, ast, endCriteria, threshold) | ||
| const forwardSlice = forwardSlicing(graph, ast, startCriteria, threshold) | ||
|
|
||
| const diceResult = new Set([...backwardsSlice.result].filter(i => forwardSlice.result.has(i))) | ||
| //console.log(diceResult) | ||
| const dicingResult = { timesHitThreshold: backwardsSlice.timesHitThreshold + forwardSlice.timesHitThreshold, result: diceResult, decodedCriteria: backwardsSlice.decodedCriteria.concat(forwardSlice.decodedCriteria) } | ||
| return dicingResult | ||
| } | ||
|
|
||
| function forwardSlicing(graph: DataflowGraph, ast: NormalizedAst, criteria: SlicingCriteria, threshold = 75): Readonly<SliceResult> { | ||
| const idMap = ast.idMap | ||
|
|
||
| guard(criteria.length > 0, 'must have at least one seed id to calculate slice') | ||
| const decodedCriteria = convertAllSlicingCriteriaToIds(criteria, idMap) | ||
| expensiveTrace(slicerLogger, | ||
| () => `calculating slice for ${decodedCriteria.length} seed criteria: ${decodedCriteria.map(s => JSON.stringify(s)).join(', ')}` | ||
| ) | ||
|
|
||
| const queue = new VisitingQueue(threshold) | ||
|
|
||
| let minDepth = Number.MAX_SAFE_INTEGER | ||
| const sliceSeedIds = new Set<NodeId>() | ||
| // every node ships the call environment which registers the calling environment | ||
| { | ||
| const emptyEnv = initializeCleanEnvironments() | ||
| const basePrint = envFingerprint(emptyEnv) | ||
| for(const { id: startId } of decodedCriteria) { | ||
| queue.add(startId, emptyEnv, basePrint, false) | ||
| // retrieve the minimum depth of all nodes to only add control dependencies if they are "part" of the current execution | ||
| minDepth = Math.min(minDepth, idMap.get(startId)?.info.depth ?? minDepth) | ||
| sliceSeedIds.add(startId) | ||
| } | ||
| } | ||
|
|
||
| const visitedIds = [] | ||
| while(queue.nonEmpty()) { | ||
| const current = queue.next() | ||
| const { baseEnvironment, id, onlyForSideEffects } = current | ||
| const baseEnvFingerprint = envFingerprint(baseEnvironment) | ||
|
|
||
| //This is for debug only | ||
| visitedIds.push(id) | ||
|
|
||
| const currentInfo = graph.get(id, true) | ||
| if(currentInfo === undefined) { | ||
| slicerLogger.warn(`id: ${id} must be in graph but can not be found, keep in slice to be sure`) | ||
| continue | ||
| } | ||
|
|
||
| const [currentVertex, currentEdges] = currentInfo | ||
| const ingoingEdges = graph.ingoingEdges(id) | ||
| if(ingoingEdges === undefined) { | ||
| continue | ||
| } | ||
|
|
||
| // we only add control dependencies iff 1) we are in different function call or 2) they have, at least, the same depth as the slicing seed | ||
| if(currentVertex.controlDependencies && currentVertex.controlDependencies.length > 0) { | ||
| const topLevel = graph.isRoot(id) || sliceSeedIds.has(id) | ||
| for(const cd of currentVertex.controlDependencies.filter(({ id }) => !queue.hasId(id))) { | ||
| if(!topLevel || (idMap.get(cd.id)?.info.depth ?? 0) <= minDepth) { | ||
| queue.add(cd.id, baseEnvironment, baseEnvFingerprint, false) | ||
| } | ||
| } | ||
| } | ||
|
|
||
| if(!onlyForSideEffects) { | ||
| if(currentVertex.tag === VertexType.FunctionCall && !currentVertex.onlyBuiltin) { | ||
| sliceForCall(current, currentVertex, graph, queue) | ||
| } | ||
|
|
||
| const ret = handleReturns(queue, currentEdges, baseEnvFingerprint, baseEnvironment) | ||
| if(ret) { | ||
| continue | ||
| } | ||
| } | ||
|
|
||
| for(const [target, { types }] of ingoingEdges) { | ||
| if(edgeIncludesType(types, EdgeType.NonStandardEvaluation)) { | ||
| continue | ||
| } | ||
| const t = shouldTraverseEdge(types) | ||
| if(t === TraverseEdge.Always) { | ||
| queue.add(target, baseEnvironment, baseEnvFingerprint, false) | ||
| } else if(t === TraverseEdge.DefinedByOnCall) { | ||
| const n = queue.potentialArguments.get(target) | ||
| if(n) { | ||
| queue.add(target, n.baseEnvironment, envFingerprint(n.baseEnvironment), n.onlyForSideEffects) | ||
| queue.potentialArguments.delete(target) | ||
| } | ||
| } else if(t === TraverseEdge.SideEffect) { | ||
| queue.add(target, baseEnvironment, baseEnvFingerprint, true) | ||
| } | ||
| } | ||
| } | ||
| //console.log('\n\nvisitedIds: %s\n\n', visitedIds) | ||
|
|
||
| return { ...queue.status(), decodedCriteria } | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,57 @@ | ||
| import { assertDiced, withShell } from '../../_helper/shell' | ||
| import type { SlicingCriteria } from '../../../../src/slicing/criterion/parse' | ||
| import type { TestLabel } from '../../_helper/label' | ||
| import { label } from '../../_helper/label' | ||
|
|
||
| describe.only('Simple', withShell(shell => { | ||
| describe('Base Dicing Cases', () => { | ||
| const testcases: { label: TestLabel, input: string, endCriterion: SlicingCriteria, startCriterion: SlicingCriteria, expected: string }[] | ||
| = [ | ||
| { label: label('Simple Example for a', ['assignment-functions', 'binary-operator']), input: 'a <- 3\nb <- 4\nc <- a + b', endCriterion: ['3@c'] as SlicingCriteria, startCriterion: ['1@a'] as SlicingCriteria, expected: 'a <- 3\nc <- a + b' }, | ||
| { label: label('Simple Example for b', ['assignment-functions', 'binary-operator']), input: 'a <- 3\nb <- 4\nc <- a + b', endCriterion: ['3@c'] as SlicingCriteria, startCriterion: ['2@b'] as SlicingCriteria, expected: 'b <- 4\nc <- a + b' }, | ||
| { label: label('Extended Example', ['assignment-functions', 'binary-operator']), input: 'a <- 3\nb <- 4\nc <- a + b\nd <- 5\ne <- d + c', endCriterion: ['5@e'] as SlicingCriteria, startCriterion: ['4@d'] as SlicingCriteria, expected: 'd <- 5\ne <- d + c' }, | ||
| { label: label('Multiple Start Points', ['assignment-functions', 'binary-operator']), input: 'a <- 3\nb <- 4\nc <- a + b\nd <- 5\ne <- d + c', endCriterion: ['5@e'] as SlicingCriteria, startCriterion: ['4@d', '3@c'] as SlicingCriteria, expected: 'c <- a + b\nd <- 5\ne <- d + c' }, | ||
| { label: label('Multiple End Points', ['assignment-functions', 'binary-operator']), input: 'a <- 3\nb <- 4\nc <- a + b\nd <- b + 5\ne <- d + c', endCriterion: ['4@d', '3@c'] as SlicingCriteria, startCriterion: ['2@b'] as SlicingCriteria, expected: 'b <- 4\nc <- a + b\nd <- b + 5' }, | ||
| ] | ||
|
|
||
| for(const testcase of testcases) { | ||
| assertDiced(testcase.label, shell, testcase.input, testcase.startCriterion, testcase.endCriterion, testcase.expected) | ||
| } | ||
| }) | ||
|
|
||
| describe('Dicing for Loops', () => { | ||
| const fibWhile = `x <- 1 | ||
| y <- 1 | ||
| i <- 0 | ||
| while (i < 10) { | ||
| h <- x | ||
| x <- x + y | ||
| y <- h | ||
| i <- i + 1 | ||
| } | ||
| cat(x)` | ||
|
|
||
| assertDiced(label('Simple while', ['assignment-functions', 'binary-operator', 'while-loop']), shell, fibWhile, ['2@y'], ['10@x'], 'y <- 1\nwhile(i < 10) x <- x + y\nx') | ||
| assertDiced(label('Complex while', ['assignment-functions', 'binary-operator', 'while-loop']), shell, fibWhile, ['2@y', '1@x'], ['10@x'], 'x <- 1\ny <- 1\nwhile(i < 10) x <- x + y\nx') | ||
| assertDiced(label('End in while', ['assignment-functions', 'binary-operator', 'while-loop']), shell, fibWhile, ['1@x'], ['7@y'], 'x <- 1\nwhile(i < 10) {\n h <- x\n x <- x + y\n y <- h\n}') | ||
| assertDiced(label('Start in while', ['assignment-functions', 'binary-operator', 'while-loop']), shell, fibWhile, ['6@x'], ['10@x'], 'while(i < 10) x <- x + y\nx') | ||
| assertDiced(label('Dice in while', ['assignment-functions', 'binary-operator', 'while-loop']), shell, fibWhile, ['5@x'], ['7@y'], 'while(i < 10) {\n h <- x\n y <- h\n}') | ||
|
|
||
| }) | ||
|
|
||
| describe('Dicing with functions', () => { | ||
| const code = `x <- function(a, b) { | ||
| y <- 10 | ||
| y <- y + a | ||
| y * b | ||
| } | ||
| c <- 2 | ||
| d <- 10 | ||
| z <- x(d, c)` | ||
|
|
||
| assertDiced(label('Simple function', ['assignment-functions', 'binary-operator', 'function-calls', 'function-definitions']), shell, code, ['6@c'], ['8@z'], 'x <- { y * b }\nc <- 2\nz <- x(d,c)') | ||
| assertDiced(label('Start in function paramenter', ['assignment-functions', 'binary-operator', 'function-calls', 'function-definitions']), shell, code, ['1@a'], ['8@z'], 'x <- function(a, b) {\n y <- a\n y * b\n }\nx(d, c)') | ||
| assertDiced(label('Start in function', ['assignment-functions', 'binary-operator', 'function-calls', 'function-definitions']), shell, code, ['3@a'], ['8@z'], 'y <- a\ny * b') | ||
| assertDiced(label('Cuts out function parameter', ['assignment-functions', 'binary-operator', 'function-calls', 'function-definitions']), shell, code, ['1@x'], ['8@z'], 'x <- { y * b }\nx(d, c)') | ||
| }) | ||
| })) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please add a new dicing category/context for the tests.