Narrow deep paths mutation #4

nandikaK · 2017-02-03T14:14:46Z

No description provided.

joernhees

please rebase on master and cleanup...

joernhees · 2017-02-08T15:43:01Z

graph_pattern.py

@@ -709,6 +711,90 @@ def to_count_var_over_values_query(self, var, vars_, values, limit):
        res += 'LIMIT %d\n' % limit
        return self._sparql_prefix(res)

+    def to_find_edge_var_for_narrow_path_query(self, edge_var, node_var,
+                                               vars_, filter_node_count,
+                                               filter_edge_count, limit_res):


joernhees · 2017-02-08T15:43:31Z

graph_pattern.py

+               '(dbr:Adult dbr:Child)' \
+               '(dbr:Angel  dbr:Heaven)' \
+               '(dbr:Arithmetic dbr:Mathematics)' \
+               '}\n' % (' '.join([v.n3() for v in vars_]))


that should be replaced by the values part... also dbr:<SPACE>Anything won't work...

joernhees · 2017-02-08T15:48:31Z

graph_pattern.py

+EDGE_VAR_COUNT = Variable('edge_count_var')
+NODE_VAR_COUNT = Variable('node_count_var')
+MAX_NODE_COUNT = Variable('maximum node count')
+PRIO_VAR = Variable('priority')


PEP8 (2 newlines above functions)

joernhees · 2017-02-08T15:51:20Z

requirements.txt

 requests==2.9.1
 rdflib==4.2.1
 scikit-learn==0.17.1
-scipy==0.17.0
+scipy


joernhees · 2017-02-08T15:52:34Z

tests/test_gp_learner_offline.py

+    ])
+    filter_node_count = 10
+    filter_edge_count = 1
+    limit_res = 32


move to config...

joernhees · 2017-02-08T15:52:36Z

tests/test_gp_learner_offline.py

+    filter_node_count = 10
+    filter_edge_count = 1
+    limit_res = 32
+    vars_ = {SOURCE_VAR,TARGET_VAR}


add values here...

joernhees · 2017-02-08T15:52:58Z

tests/test_gp_learner_offline.py

+    # test_mutate_deep_narrow_path()
+    test_to_find_edge_var_for_narrow_path_query()


always have a newline at end of file...

joernhees · 2017-02-08T15:53:08Z

graph_pattern.py

+        res += '    SELECT DISTINCT %s %s (COUNT(%s) AS %s) WHERE {\n' % (
+                   ' '.join([v.n3() for v in vars_]),
+                   edge_var.n3(), node_var.n3(), NODE_VAR_COUNT.n3())
+        # res += self._sparql_values_part(values)


why not use that? set indent accordingly...

joernhees

minor style and bugs...

joernhees · 2017-02-13T12:21:09Z

config/defaults.py

 # for import in helpers and __init__
+


keep whitespace intact

joernhees · 2017-02-13T12:21:26Z

gp_learner.py

@@ -41,7 +41,8 @@
 from cluster import expected_precision_loss_by_query_reduction
 from cluster import select_best_variant
 import config
-from gp_query import ask_multi_query
+from gp_query import ask_multi_query, \
+    variable_substitution_deep_narrow_mut_query


use same style as surrounding code please

joernhees · 2017-02-13T12:22:03Z

gp_learner.py

+            '\n'.join([' %d: %s' % (c, v.n3())
+                       for v, c in substitution_counts.most_common()]),
+        )
+        fixed = 'Y'


fixed = True?

joernhees · 2017-02-13T12:22:12Z

gp_learner.py

+        mutate_fix_var_filter(substitution_counts)
+        if not substitution_counts:
+            # could have happened that we removed the only possible substitution
+            fixed = 'N'


fixed = False?

joernhees · 2017-02-13T12:24:23Z

gp_learner.py

@@ -420,7 +421,7 @@ def _mutate_expand_node_helper(node, pb_en_out_link=config.MUTPB_EN_OUT_LINK):
        new_triple = (node, var_edge, var_node)
    else:
        new_triple = (var_node, var_edge, node)
-    return new_triple, var_node
+    return new_triple, var_node, var_edge


changing this needs adaptation of all users of this function...

joernhees · 2017-02-13T12:25:07Z

gp_learner.py

+            logger.debug("tried to fix a var %s without result:\n%s"
+                         "seems as if the pattern can't be fulfilled!",
+                         edge_var, child.to_sparql_select_query())
+            fixed = 'N'


fixed = False?

joernhees · 2017-02-13T12:25:25Z

gp_learner.py

+        test_gp = gp + [new_triple]
+        test_gp, fixed = _mutate_deep_narrow_path_helper(
+            sparql, timeout, gtp_scores, test_gp, var_edge, var_node)
+        if fixed == 'Y':


if fixed:?

joernhees · 2017-02-13T12:27:16Z

gp_query.py

@@ -426,6 +426,21 @@ def variable_substitution_query(
    )


+def variable_substitution_deep_narrow_mut_query(
+            sparql, timeout, graph_pattern, edge_var, node_var,
+        source_target_pairs, limit_res, batch_size=config.BATCH_SIZE):


…nd args passing, also tons of minor things

…oc string example

… nodes

joernhees requested changes Feb 8, 2017

View reviewed changes

joernhees requested changes Feb 13, 2017

View reviewed changes

nandikaK force-pushed the narrow_deep_paths_mutation branch from 45dc6e5 to 94f9ee2 Compare February 16, 2017 16:02

nandikaK and others added 5 commits February 17, 2017 03:47

WIP: narrow deep paths mutation

1fd4b97

deep narrow paths mutation conf var docs, cleanup and renames

62f45c1

refactoring of deep-narrow-paths query, sparql (uses template now), a…

0ec39a7

…nd args passing, also tons of minor things

test_mutate_deep_narrow_path moved to online tests

a20eca6

test_deep_narrow_paths_query now checks formed query string against d…

72fc46a

…oc string example

joernhees force-pushed the narrow_deep_paths_mutation branch from 94f9ee2 to 72fc46a Compare February 17, 2017 02:47

nandikaK force-pushed the narrow_deep_paths_mutation branch from 72fc46a to 1b74508 Compare February 27, 2017 17:03

WIP: added backtracking to deep narrow path mutation

3e7b352

joernhees force-pushed the narrow_deep_paths_mutation branch from 1b74508 to 3e7b352 Compare March 3, 2017 16:36

joernhees and others added 3 commits March 3, 2017 19:33

mutate helper tells what it fixed a var to and actually orders by prios

8595524

made the deep_narrow_paths mutation recursive with retry attemps

b479257

deep_narrow_paths mutation now tries to connect expanded gp to target…

9544083

… nodes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Narrow deep paths mutation #4

Narrow deep paths mutation #4

nandikaK commented Feb 3, 2017

joernhees left a comment

joernhees Feb 8, 2017

joernhees Feb 8, 2017

joernhees Feb 8, 2017

joernhees Feb 8, 2017

joernhees Feb 8, 2017

joernhees Feb 8, 2017

joernhees Feb 8, 2017

joernhees Feb 8, 2017

joernhees left a comment

joernhees Feb 13, 2017

joernhees Feb 13, 2017

joernhees Feb 13, 2017

joernhees Feb 13, 2017

joernhees Feb 13, 2017

joernhees Feb 13, 2017

joernhees Feb 13, 2017

joernhees Feb 13, 2017

		# test_mutate_deep_narrow_path()
		test_to_find_edge_var_for_narrow_path_query()

Narrow deep paths mutation #4

Are you sure you want to change the base?

Narrow deep paths mutation #4

Conversation

nandikaK commented Feb 3, 2017

joernhees left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

joernhees left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment