Skip to content

Add polite guard #1223

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

Add polite guard #1223

wants to merge 6 commits into from

Conversation

Ankur-singh
Copy link

Description

Integrates Intel's Polite Guard model as one of the guardrail component/microservice.

Issues

n/a

Type of change

List the type of change like below. Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds new functionality)
  • Breaking change (fix or feature that would break existing design and interface)
  • Others (enhancement, documentation, validation, etc.)

Dependencies

No new dependency

Tests

Describe the tests that you ran to verify your changes.

I successfully tested the microservice by running it both as Python script and as Docker microservice. Both methods worked as expected, and I was able to verify the functionality of the component with different inputs.

@Ankur-singh Ankur-singh force-pushed the add-polite-guard branch 3 times, most recently from 3510bf5 to c561c25 Compare January 28, 2025 22:22
@Ankur-singh Ankur-singh force-pushed the add-polite-guard branch 2 times, most recently from 3b6569b to be910a0 Compare February 3, 2025 15:55
@letonghan
Copy link
Collaborator

letonghan commented Feb 10, 2025 •
edited
Loading

Hi @Ankur-singh , please check the failure of Compose file and dockerfile path checking. This polite guard needs to be added in .github/workflows/docker/compose/guardrails-compose.yaml file for release image building.

Also, please add a test scirpt here and follow the naming schema. Thanks!

@Ankur-singh Ankur-singh force-pushed the add-polite-guard branch 2 times, most recently from c37de4b to db947de Compare February 11, 2025 06:55
@Ankur-singh
Copy link
Author

Hi @letonghan, thank you for your help. I have made the requested changes. Fingers crossed, all checks should pass.

@Ankur-singh
Copy link
Author

Hi @letonghan can you please help me with the failed checks? I see that docker containers is created but the curl request fails. Other two services (llamaguard and hallucination_detection) are also failing. But I'm unable to figure out why, is it because of overlapping port usage? Are these services running in parallel?

Sorry about polluting the PR. Is there a way to run these workflows locally? I'll clear the next commit.

@xiguiw
Copy link
Collaborator

xiguiw commented Feb 25, 2025

@Ankur-singh

Are you still working on this?

The environments variables are not set.
Please set them in test script.
You can refer to other test scripts.

  • docker compose up guardrails-polite-guard-server -d
    time="2025-02-20T20:05:52Z" level=warning msg="The "no_proxy" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "http_proxy" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "https_proxy" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "http_proxy" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "https_proxy" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "no_proxy" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "no_proxy" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "http_proxy" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "https_proxy" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "SAFETY_GUARD_ENDPOINT" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "SAFETY_GUARD_MODEL_ID" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "https_proxy" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "no_proxy" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "http_proxy" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "http_proxy" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "https_proxy" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "no_proxy" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "no_proxy" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "http_proxy" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "https_proxy" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "no_proxy" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "http_proxy" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "https_proxy" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "http_proxy" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "https_proxy" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "LLM_MODEL_ID" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "vLLM_ENDPOINT" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "no_proxy" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "LLM_ENDPOINT_PORT" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "no_proxy" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "http_proxy" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "https_proxy" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "host_ip" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "host_ip" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "LLM_ENDPOINT_PORT" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "LLM_MODEL_ID" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "LLM_MODEL_ID" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "host_ip" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "LLM_ENDPOINT_PORT" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "host_ip" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "LLM_ENDPOINT_PORT" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "no_proxy" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "http_proxy" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "https_proxy" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "http_proxy" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "https_proxy" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "no_proxy" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "LLM_MODEL_ID" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "host_ip" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "LLM_ENDPOINT_PORT" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "host_ip" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "LLM_ENDPOINT_PORT" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "LLM_MODEL_ID" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "host_ip" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "LLM_ENDPOINT_PORT" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "host_ip" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "no_proxy" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "https_proxy" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "LLM_MODEL_ID" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "http_proxy" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "LLM_ENDPOINT_PORT" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "LLM_MODEL_ID" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "http_proxy" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "https_proxy" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "LLM_MODEL_ID" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "LLM_ENDPOINT_PORT" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "host_ip" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "LLM_MODEL_ID" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "LLM_ENDPOINT_PORT" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "host_ip" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "LLM_ENDPOINT_PORT" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "https_proxy" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "LLM_MODEL_ID" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "LLM_ENDPOINT_PORT" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "host_ip" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "http_proxy" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "LLM_MODEL_ID" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "LLM_ENDPOINT_PORT" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "host_ip" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="The "LLM_ENDPOINT_PORT" variable is not set. Defaulting to a blank string."
    time="2025-02-20T20:05:52Z" level=warning msg="Found orphan containers ([gpt-sovits-service]) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up."
    Container guardrails-polite-guard-server Creating
    Container guardrails-polite-guard-server Created
    Container guardrails-polite-guard-server Starting
    Container guardrails-polite-guard-server Started

  • sleep 15

  • echo 'Microservice started'

  • validate_microservice

  • echo 'Validate microservice started'
    Microservice started
    Validate microservice started

  • echo 'test 1 - Impolite'
    ++ curl localhost:11301/v1/polite -X POST -d '{"text":"He is stupid"}' -H 'Content-Type: application/json'
    test 1 - Impolite
    % Total % Received % Xferd Average Speed Time Time Time Current
    Dload Upload Total Spent Left Speed

    0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
    100 23 0 0 100 23 0 47131 --:--:-- --:--:-- --:--:-- 23000
    curl: (56) Recv failure: Connection reset by peer

  • result=

  • [[ '' == \V\i\o\l\a\t\e\d ]]

  • docker logs guardrails-polite-guard-server

  • exit 1
    Error: Process completed with exit code 1.

@Ankur-singh
Copy link
Author

Yes, still working on it. Sorry for the delay though.

@Ankur-singh
Copy link
Author

Ankur-singh commented Feb 28, 2025 •
edited
Loading

@xiguiw looks like ENV VARs were not the problem. Checked the log for other services (successful) and saw the same warning messages.

Tried to run bash ./guardrails/test_guardrails_polite_guard.sh locally and got the same error. After some digging, found out that the service takes more than 15 secs to start. Updated the script to check for application start, refer this. This solves the problem.

PS: @letonghan I believe, we should add similar checks for other services as well.

Update: Getting the following error, https://github.com/opea-project/GenAIComps/actions/runs/13624534401/job/38079510858?pr=1223#step:5:938

Are these tests executed in parallel? Does each service need to have an unique port number?

@Ankur-singh Ankur-singh changed the title [WIP] Add polite guard Add polite guard Feb 28, 2025
@xiguiw
Copy link
Collaborator

xiguiw commented Mar 26, 2025

Remind
rebase and trigger the CI

@Ankur-singh Ankur-singh force-pushed the add-polite-guard branch 3 times, most recently from bd25a5a to 792993e Compare March 28, 2025 15:39
Ankur-singh and others added 6 commits April 15, 2025 12:01
@Ankur-singh
integrate polite guard component
f09746b 
Signed-off-by: Ankur-singh <as.ankursingh3.1@gmail.com>
Signed-off-by: Ankur Singh <ankur1.singh@intel.com>
@Ankur-singh
add polite guard microservice readme
ebb1161 
Signed-off-by: Ankur-singh <as.ankursingh3.1@gmail.com>
Signed-off-by: Ankur Singh <ankur1.singh@intel.com>
@pre-commit-ci @Ankur-singh
[pre-commit.ci] auto fixes from pre-commit.com hooks
d9be607 
for more information, see https://pre-commit.ci

Signed-off-by: Ankur-singh <as.ankursingh3.1@gmail.com>
Signed-off-by: Ankur Singh <ankur1.singh@intel.com>
@Ankur-singh
Update README.md with corrected curl command and improved Python example
a5a7696 
Signed-off-by: Ankur-singh <as.ankursingh3.1@gmail.com>
Signed-off-by: Ankur Singh <ankur1.singh@intel.com>
@Ankur-singh
add polite_guard to tests, workflows, and service
1501a7f 
Signed-off-by: Ankur-singh <as.ankursingh3.1@gmail.com>
Signed-off-by: Ankur Singh <ankur1.singh@intel.com>
@Ankur-singh
Add retry logic for service startup in test script
09b44f9 
Signed-off-by: Ankur Singh <ankur1.singh@intel.com>
@joshuayao joshuayao added this to OPEA Apr 29, 2025
@joshuayao joshuayao added this to the v1.4 milestone Apr 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ftian1 ftian1 ftian1 approved these changes

@lvliang-intel lvliang-intel Awaiting requested review from lvliang-intel lvliang-intel is a code owner

@letonghan letonghan Awaiting requested review from letonghan letonghan is a code owner

@chensuyue chensuyue Awaiting requested review from chensuyue chensuyue is a code owner

@ZePan110 ZePan110 Awaiting requested review from ZePan110 ZePan110 is a code owner

At least 2 approving reviews are required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
OPEA
Status: No status
Milestone
v1.4
Development

Successfully merging this pull request may close these issues.
5 participants
@Ankur-singh @letonghan @xiguiw @ftian1 @joshuayao