Skip to content

check pending 5 behind level requests #25

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
Kumarvels merged 38 commits into from
Jul 31, 2025
Merged

check pending 5 behind level requests #25

Kumarvels merged 38 commits into from
Jul 31, 2025

Conversation

@Kumarvels
Copy link
Collaborator

@Kumarvels Kumarvels commented Jul 25, 2025

old pending requests behind the current level at least 5 of them

Kumarvels and others added 30 commits July 25, 2025 04:54
@Kumarvels
Merge pull request #21 from Kumarvels/feature/data-quality-checks
6ba3434 
Feature/data quality checks
@Kumarvels
Create dependabot.yml
d1ef6ee 
Signed-off-by: Kumar Vel <11884941+Kumarvels@users.noreply.github.com>
@Kumarvels
Merge pull request #22 from Kumarvels/feature/data-quality-checks
a743086 
test: Run Streamlit Web UI
@Kumarvels
Merge pull request #23 from Kumarvels/feature/data-quality-checks
09abe2d 
fix: Fix Web UI loading error
@Kumarvels
Merge pull request #24 from Kumarvels/feature/data-quality-checks
487256d 
Refactor and simplify the Unified Workflow Web UI
@Kumarvels
Update README.md
fd22d94 
Signed-off-by: Kumar Vel <11884941+Kumarvels@users.noreply.github.com>
@Kumarvels
Update README.md
7d3d7a8 
Signed-off-by: Kumar Vel <11884941+Kumarvels@users.noreply.github.com>
@Kumarvels
Update README.md
6c281ab 
Add readme file update about why Trust based systems are better

Signed-off-by: Kumar Vel <11884941+Kumarvels@users.noreply.github.com>
@Kumarvels
Create unified_plugin_manager.py
af37c2f 
Enhanced Plugin System (src/core/unified_plugin_manager.py)

Signed-off-by: Kumar Vel <11884941+Kumarvels@users.noreply.github.com>
@Kumarvels
Create trustllm_plugin.py
bde6b87 
TrustLLM Comprehensive Plugin (src/plugins/trustllm_plugin.py)

Signed-off-by: Kumar Vel <11884941+Kumarvels@users.noreply.github.com>
@Kumarvels
Create llm_reliability_plugin.py
5f53176 
LLM-Specific Reliability Plugin (src/plugins/llm_reliability_plugin.py)

Signed-off-by: Kumar Vel <11884941+Kumarvels@users.noreply.github.com>
@Kumarvels
Create trust_timeline.py - Trust Evolution Timeline System
42fa52c 
Trust isn't static - it evolves over time and contexts

Signed-off-by: Kumar Vel <11884941+Kumarvels@users.noreply.github.com>
@Kumarvels
Create cross_model_correlation.py - Cross-Model Trust Correlation Engine
e29afcb 
AI systems don't operate in isolation - they interact and influence each other

Signed-off-by: Kumar Vel <11884941+Kumarvels@users.noreply.github.com>
@Kumarvels
Create trust_decision_matrix.py - Trust Decision Matrix System
3629c66 
The Problem: Different stakeholders need different trust criteria for the same system

Signed-off-by: Kumar Vel <11884941+Kumarvels@users.noreply.github.com>
@Kumarvels
Create trust_simulation.py - Trust Simulation and Stress Testing System
585fba1 
The Problem: Need to test trust under extreme conditions before deployment

Signed-off-by: Kumar Vel <11884941+Kumarvels@users.noreply.github.com>
@Kumarvels
Create trust_orchestrator.py - Trust Orchestration and Deployment Pip…
91ee58a 
…eline

The Problem: Need seamless integration from development to production with trust monitoring

Signed-off-by: Kumar Vel <11884941+Kumarvels@users.noreply.github.com>
@Kumarvels
Create trust_api.py - Trust API and Microservices Architecture
3f3d2a5 
The Problem: Need to scale trust evaluation across distributed systems

Signed-off-by: Kumar Vel <11884941+Kumarvels@users.noreply.github.com>
@Kumarvels
Create trust_dashboard.py - Trust Visualization and Dashboard System
f408cdf 
The Problem: Need intuitive visualization of complex trust metrics

Signed-off-by: Kumar Vel <11884941+Kumarvels@users.noreply.github.com>
@Kumarvels
Update README.md
971d43f 
OpenTrustEval from a simple evaluation tool into a comprehensive trust management platform

Signed-off-by: Kumar Vel <11884941+Kumarvels@users.noreply.github.com>
@Kumarvels
Update README.md
4cf8d18 
Updates: OpenTrustEval from a simple evaluation tool into a comprehensive trust management platform

Signed-off-by: Kumar Vel <11884941+Kumarvels@users.noreply.github.com>
@google-labs-jules
Retest web UI and validate libraries
6abd328
@Kumarvels
Merge pull request #26 from Kumarvels/retest-webui
1fd41c8 
Retest web UI and validate libraries
@google-labs-jules
Retest web UI and validate libraries
df27c76
@google-labs-jules
Retest web UI and validate libraries
1c27654
@google-labs-jules
Retest web UI and validate libraries
6f7f472
@Kumarvels
Merge branch 'main' into retest-webui
399bd83
@Kumarvels
Merge pull request #27 from Kumarvels/retest-webui
5b86c4e 
Retest web UI and validate libraries
@Kumarvels
testing scripts move into tests folder
482f726
@google-labs-jules
I am moving the test files.
0239ecb
@Kumarvels
Merge pull request #28 from Kumarvels/retest-webui
291d449 
I am moving the test files.
google-labs-jules bot and others added 6 commits July 26, 2025 16:13
@google-labs-jules
I am re-testing the web UI and moving the test files.
b13b0f5
@Kumarvels
Merge branch 'main' into retest-webui
fc37ddf
@Kumarvels
Merge pull request #29 from Kumarvels/retest-webui
4163ccf 
I am re-testing the web UI and moving the test files.
@Kumarvels
Create SECURITY.md
943a0a3 
Security policy  addition

Signed-off-by: Kumar Vel <11884941+Kumarvels@users.noreply.github.com>
@Kumarvels
Update issue templates
43680cc 
Bug report and  features request templates
@Kumarvels
Create codeql.yml
2726c2e 
codeql configuration

Signed-off-by: Kumar Vel <11884941+Kumarvels@users.noreply.github.com>
@github-advanced-security
Copy link
Contributor

github-advanced-security bot commented Jul 31, 2025

This pull request sets up GitHub code scanning for this repository. Once the scans have completed and the checks have passed, the analysis results for this pull request branch will appear on this overview. Once you merge this pull request, the 'Security' tab will show more code scanning analysis results (for example, for the default branch). Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results. For more information about GitHub code scanning, check out the documentation.

@Kumarvels @github-advanced-security
Potential fix for code scanning alert no. 27: Code injection
603e371 
security code fix

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Signed-off-by: Kumar Vel <11884941+Kumarvels@users.noreply.github.com>
github-advanced-security[bot]
github-advanced-security bot found potential problems Jul 31, 2025
View reviewed changes
data_engineering/dataset_integration.py Outdated
condition = params['condition']
if not self.is_safe_query(condition, df.columns):
raise ValueError("Unsafe filter condition detected. Only simple column comparisons are allowed.")
df = df.query(condition)

Check failure

Code scanning / CodeQL

Code injection Critical

This code execution depends on a
user-provided value
Loading
.

Copilot Autofix

AI 6 months ago

To fix the code injection vulnerability, we should avoid passing user-provided strings directly to df.query(). Instead, for filter operations, we should require the user to specify the column, operator, and value as separate fields in the transformation JSON. Then, construct the filter using pandas boolean indexing, which does not evaluate code. This change should be made in the process_dataset method in data_engineering/dataset_integration.py, specifically in the block handling the 'filter' operation.

Steps:

  • Update the 'filter' operation to expect params with keys: column, operator, and value (instead of a single condition string).
  • Validate that the column exists, the operator is one of a safe set (==, !=, <, >, <=, >=), and the value is a simple literal.
  • Construct the filter using pandas boolean indexing, e.g., df[df[column] <op> value].
  • Update the placeholder and documentation in the web UI to reflect the new expected format for filter transformations.

Required changes:

  • In process_dataset, replace the df.query(condition) block with code that parses and applies the filter using boolean indexing.
  • Optionally, update the Gradio UI placeholder to show the new filter format (in data_engineering/scripts/easy_dataset_webui.py).
Suggested changeset 2
data_engineering/dataset_integration.py

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/data_engineering/dataset_integration.py b/data_engineering/dataset_integration.py
--- a/data_engineering/dataset_integration.py
+++ b/data_engineering/dataset_integration.py
@@ -347,6 +347,24 @@
             elif operation == 'filter':
-                condition = params['condition']
-                if not self.is_safe_query(condition, df.columns):
-                    raise ValueError("Unsafe filter condition detected. Only simple column comparisons are allowed.")
-                df = df.query(condition)
+                # Expect params: {'column': ..., 'operator': ..., 'value': ...}
+                column = params.get('column')
+                operator = params.get('operator')
+                value = params.get('value')
+                allowed_operators = ['==', '!=', '<', '>', '<=', '>=']
+                if column not in df.columns:
+                    raise ValueError(f"Column '{column}' not found in dataset.")
+                if operator not in allowed_operators:
+                    raise ValueError(f"Operator '{operator}' is not allowed.")
+                # Apply filter using boolean indexing
+                if operator == '==':
+                    df = df[df[column] == value]
+                elif operator == '!=':
+                    df = df[df[column] != value]
+                elif operator == '<':
+                    df = df[df[column] < value]
+                elif operator == '>':
+                    df = df[df[column] > value]
+                elif operator == '<=':
+                    df = df[df[column] <= value]
+                elif operator == '>=':
+                    df = df[df[column] >= value]
             elif operation == 'sort':
EOF
@@ -347,6 +347,24 @@
elif operation == 'filter':
condition = params['condition']
if not self.is_safe_query(condition, df.columns):
raise ValueError("Unsafe filter condition detected. Only simple column comparisons are allowed.")
df = df.query(condition)
# Expect params: {'column': ..., 'operator': ..., 'value': ...}
column = params.get('column')
operator = params.get('operator')
value = params.get('value')
allowed_operators = ['==', '!=', '<', '>', '<=', '>=']
if column not in df.columns:
raise ValueError(f"Column '{column}' not found in dataset.")
if operator not in allowed_operators:
raise ValueError(f"Operator '{operator}' is not allowed.")
# Apply filter using boolean indexing
if operator == '==':
df = df[df[column] == value]
elif operator == '!=':
df = df[df[column] != value]
elif operator == '<':
df = df[df[column] < value]
elif operator == '>':
df = df[df[column] > value]
elif operator == '<=':
df = df[df[column] <= value]
elif operator == '>=':
df = df[df[column] >= value]
elif operation == 'sort':
data_engineering/scripts/easy_dataset_webui.py
Outside changed files

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/data_engineering/scripts/easy_dataset_webui.py b/data_engineering/scripts/easy_dataset_webui.py
--- a/data_engineering/scripts/easy_dataset_webui.py
+++ b/data_engineering/scripts/easy_dataset_webui.py
@@ -250,3 +250,3 @@
             label="Transformations (JSON)",
-            placeholder='[{"operation": "filter", "params": {"condition": "age > 30"}}]',
+            placeholder='[{"operation": "filter", "params": {"column": "age", "operator": ">", "value": 30}}]',
             lines=5
EOF
@@ -250,3 +250,3 @@
label="Transformations (JSON)",
placeholder='[{"operation": "filter", "params": {"condition": "age > 30"}}]',
placeholder='[{"operation": "filter", "params": {"column": "age", "operator": ">", "value": 30}}]',
lines=5
Copilot is powered by AI and may make mistakes. Always verify output.
Unable to commit as this autofix suggestion is now outdated
@Kumarvels @github-advanced-security
Potential fix for code scanning alert no. 33: Code injection
1acfc14 
code quality check

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Signed-off-by: Kumar Vel <11884941+Kumarvels@users.noreply.github.com>
@Kumarvels Kumarvels merged commit 801d1a9 into feature/data-quality-checks Jul 31, 2025
3 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@Kumarvels