pull_subquery_card() hangs in block_for_data_from_db() and never triggers do_post

[mobinakochaknia]

Hi team,

First of all, thank you for the great work on PilotScope and for making it publicly available—this is a very impressive and valuable tool!

I'm encountering an issue when trying to run benchmark_subquery in the PRICE project that uses your work for generating workloads, and I hope you can help clarify or provide suggestions for debugging.

Problem

When running the workload—specifically calling pull_subquery_card()—the process hangs in the block_for_data_from_db() function inside HttpInteractorReceiver.py and eventually times out.

Interestingly, when I run pull_record() instead, I do get the expected result.

What I've tried

• Tested on two separate servers to rule out environment-specific problems.
• Verified that all dependencies are installed correctly (the way that you said)
• Confirmed from PostgreSQL logs that the query request is indeed received successfully.
• Verified that the relevant ports are open and functional.
• Did some debugging and noticed that:
  • The do_post() method in HttpInteractorReceiver.py never gets called.
  • The receiver seems to be waiting indefinitely for a response that never comes.

Request

Do you have any idea what might be causing this behavior?
Any suggestions for debugging or known issues in this area would be much appreciated.

Thanks again for the excellent work—and looking forward to any guidance you can provide!

Best regards

[PengJiazhen408]

Hi,

Thank you for your feedback and detailed troubleshooting steps. I'm glad to hear that pull_record() is working for you, and I appreciate your thorough checks regarding dependencies, environments, and PostgreSQL logs.

The core difference between pull_subquery_card() and pull_record() is the communication mechanism: pull_subquery_card relies on HTTP communication, while pull_record uses SQLAlchemy. Since pull_record() works as expected, it suggests the issue is likely related to either the HTTP communication module between PilotScope and PostgreSQL, or the subquery generation process. ()

Based on your observations and our experience, here are some suggestions for further debugging:

1. Check HTTP Service Initialization on PostgreSQL Side

• Please review the PostgreSQL server logs:
  • If the HTTP service initializes successfully, you should see a log entry like: "Reach the maximum number of initing times!"
  • If not, you’ll see: "Init the http error!! Reiniting…!"
• This can help determine if the HTTP server is set up correctly.

2. Inspect send_and_resend Function

• If initialization is successful, try debugging or adding print statements in the send_and_resend function on the PostgreSQL side.
  • Pay special attention to the content of string_of_pilottransdata:
    • If it contains data, the HTTP communication module may be the source of the problem.
    • If it’s empty, the issue could be caused by subquery generation.

---

Another Test: Run a Controlled Test Case for Verification

• I also recommend running a simple test with the sample dataset:
  • Step 1: Install and load the test database as described in our official documentation)
  • Step 2: Try the example with pull_subquery_card (how-to here).
• With this test data, we can confirm that subquery generation works correctly:
  • If pull_subquery_card works under this setup, the issue is probably with the subquery generated in your case.
  • If it still fails, that points to a problem with the HTTP communication module.

Please try these approaches and let us know what you find. We're happy to help further once you have more details. Thanks again for your interest and support!