Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Run tests through kafka #2005

Merged
merged 14 commits into from
Jan 30, 2025
Merged

Run tests through kafka #2005

merged 14 commits into from
Jan 30, 2025

Conversation

Ark2307
Copy link
Contributor

@Ark2307 Ark2307 commented Jan 21, 2025

No description provided.

@Ark2307 Ark2307 force-pushed the feature/run_tests_through_kafka branch from 3a99143 to bcf7528 Compare January 21, 2025 11:22
ayushaga14
ayushaga14 reviewed Jan 21, 2025
View reviewed changes
apps/testing/src/main/java/com/akto/testing/testing_with_kafka/TestingConsumer.java Outdated
ObjectId testingRunResultSummaryId = new ObjectId(jsonObject.getString("testingRunResultSummaryId"));
ApiInfo.ApiInfoKey apiInfoKey = ApiInfo.getApiInfoKeyFromString(jsonObject.getString("apiInfoKey"));
String subcategory = jsonObject.getString("subcategory");
List<TestingRunResult.TestLog> testLogs = JSON.parseArray(jsonObject.getString("testLogs"), TestingRunResult.TestLog.class);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

check how testlogs are used and stored

ayushaga14
ayushaga14 reviewed Jan 21, 2025
View reviewed changes
apps/testing/src/main/java/com/akto/testing/testing_with_kafka/CommonSingletonForTesting.java Outdated
import com.akto.dto.testing.TestingRunConfig;
import com.akto.store.TestingUtil;

public class CommonSingletonForTesting {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rename this

ayushaga14
ayushaga14 reviewed Jan 21, 2025
View reviewed changes
apps/testing/src/main/java/com/akto/testing/testing_with_kafka/TestingProducer.java Outdated
return null;
}

private static void deleteAllMessagesFromTopic(String bootstrapServers, String topicName)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

delete the topic itself, new message push wll create a new one

@Ark2307 Ark2307 force-pushed the feature/run_tests_through_kafka branch from bcf7528 to f6a8ff4 Compare January 21, 2025 11:45
ayushaga14
ayushaga14 reviewed Jan 21, 2025
View reviewed changes
apps/testing/src/main/java/com/akto/testing/testing_with_kafka/TestingProducer.java Outdated

public static final Kafka producer = new Kafka(Constants.LOCAL_KAFKA_BROKER_URL, 500, 1000);

public static Void pushMessagesToKafka(List<TestMessages> messages){
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

void

ayushaga14
ayushaga14 reviewed Jan 27, 2025
View reviewed changes
apps/testing/src/main/java/com/akto/testing/Main.java Outdated

String testingRunId = currentTestInfo.getString("testingRunId");
String testingRunSummaryId = currentTestInfo.getString("summaryId");
TestingRun testingRun = TestingRunDao.instance.findOne(Filters.eq(Constants.ID, new ObjectId(testingRunId)));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

review this and add try catch, if something panics, whole process will exit

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

ayushaga14
ayushaga14 reviewed Jan 27, 2025
View reviewed changes
apps/testing/src/main/java/com/akto/testing/TestExecutor.java Outdated
return null;
}

private Void doJobForTest(int accountId, String testSubCategory, ApiInfo.ApiInfoKey apiInfoKey,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

which job? rename

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

ayushaga14
ayushaga14 reviewed Jan 27, 2025
View reviewed changes
apps/testing/src/main/java/com/akto/testing/testing_with_kafka/TestingConsumer.java Outdated
executor.awaitTermination(maxRunTimeInSeconds, TimeUnit.SECONDS);
break;
}
Thread.sleep(100);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

100 seconds?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

100ms

ayushaga14
ayushaga14 reviewed Jan 27, 2025
View reviewed changes
apps/testing/src/main/java/com/akto/testing/testing_with_kafka/TestingConsumer.java Outdated
try {
future.get(4, TimeUnit.MINUTES);
} catch (InterruptedException e) {
logger.error("Task timed out: " + message);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there a way to know which message timed out?

ayushaga14
ayushaga14 requested changes Jan 27, 2025
View reviewed changes
Copy link
Contributor

@ayushaga14 ayushaga14 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

resolve comments

@Ark2307 Ark2307 merged commit e8b096c into master Jan 30, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ayushaga14 ayushaga14 ayushaga14 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@Ark2307 @ayushaga14