Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] Failing DateProcessorTests when building #3999

Open
GumpacG opened this issue Jan 22, 2024 · 3 comments
Open

[BUG] Failing DateProcessorTests when building #3999

GumpacG opened this issue Jan 22, 2024 · 3 comments
Labels
bug Something isn't working maintenance Issues to help maintain the project, such as improving builds, testing, etc.

Comments

@GumpacG
Copy link
Contributor

GumpacG commented Jan 22, 2024 •
edited
Loading

Describe the bug
When building the project locally on the main branch, multiple date-processor tests fail. It seems that GitHub actions have these tests pass. Results from Grade Build workflow:

<testsuite name="org.opensearch.dataprepper.plugins.processor.date.DateProcessorTests" tests="39" skipped="0" failures="0" errors="0" timestamp="2024-01-22T19:03:45" hostname="fv-az585-104" time="1.235">
<properties/>
<testcase name="match_with_missing_hours_minutes_seconds_adds_zeros_test()" classname="org.opensearch.dataprepper.plugins.processor.date.DateProcessorTests" time="0.676"/>
<testcase name="[1] en-US" classname="org.opensearch.dataprepper.plugins.processor.date.DateProcessorTests" time="0.022"/>
<testcase name="[2] zh-CN" classname="org.opensearch.dataprepper.plugins.processor.date.DateProcessorTests" time="0.102"/>
<testcase name="[3] it-IT" classname="org.opensearch.dataprepper.plugins.processor.date.DateProcessorTests" time="0.078"/>
<testcase name="match_with_wrong_patterns_return_same_record_test_without_timestamp()" classname="org.opensearch.dataprepper.plugins.processor.date.DateProcessorTests" time="0.009"/>
<testcase name="[1] yyyy MM dd HH mm ss" classname="org.opensearch.dataprepper.plugins.processor.date.DateProcessorTests" time="0.01"/>
<testcase name="date_when_does_not_run_date_processor_for_event_with_date_when_as_false()" classname="org.opensearch.dataprepper.plugins.processor.date.DateProcessorTests" time="0.007"/>
<testcase name="[1] MMM/dd" classname="org.opensearch.dataprepper.plugins.processor.date.DateProcessorTests" time="0.014"/>
<testcase name="[2] MM dd" classname="org.opensearch.dataprepper.plugins.processor.date.DateProcessorTests" time="0.009"/>
<testcase name="[1] epoch_second, 1705950226, epoch_milli, 1705950226000" classname="org.opensearch.dataprepper.plugins.processor.date.DateProcessorTests" time="0.012"/>
<testcase name="[2] epoch_second, 1705950226, epoch_nano, 1705950226000000000" classname="org.opensearch.dataprepper.plugins.processor.date.DateProcessorTests" time="0.008"/>
<testcase name="[3] epoch_second, 1705950226, yyyy-MMM-dd HH:mm:ss.SSS, 2024-Jan-22 19:03:46.000" classname="org.opensearch.dataprepper.plugins.processor.date.DateProcessorTests" time="0.008"/>
<testcase name="[4] epoch_second, 1705950226, yyyy-MM-dd'T'HH:mm:ss.SSSXXX, 2024-01-22T19:03:46.000Z" classname="org.opensearch.dataprepper.plugins.processor.date.DateProcessorTests" time="0.008"/>
<testcase name="[5] epoch_milli, 1705950226581, epoch_second, 1705950226" classname="org.opensearch.dataprepper.plugins.processor.date.DateProcessorTests" time="0.008"/>
<testcase name="[6] epoch_milli, 1705950226581, epoch_nano, 1705950226581000000" classname="org.opensearch.dataprepper.plugins.processor.date.DateProcessorTests" time="0.008"/>
<testcase name="[7] epoch_milli, 1705950226581, yyyy-MMM-dd HH:mm:ss.SSS, 2024-Jan-22 19:03:46.581" classname="org.opensearch.dataprepper.plugins.processor.date.DateProcessorTests" time="0.01"/>
<testcase name="[8] epoch_milli, 1705950226581, yyyy-MM-dd'T'HH:mm:ss.SSSXXX, 2024-01-22T19:03:46.581Z" classname="org.opensearch.dataprepper.plugins.processor.date.DateProcessorTests" time="0.007"/>
<testcase name="[9] epoch_nano, 1705950226818194066, epoch_second, 1705950226" classname="org.opensearch.dataprepper.plugins.processor.date.DateProcessorTests" time="0.008"/>
<testcase name="[10] epoch_nano, 1705950226818194066, epoch_milli, 1705950226818" classname="org.opensearch.dataprepper.plugins.processor.date.DateProcessorTests" time="0.008"/>
<testcase name="[11] epoch_nano, 1705950226818194066, yyyy-MMM-dd HH:mm:ss.nnnnnnnnnXXX, 2024-Jan-22 19:03:46.818194066Z" classname="org.opensearch.dataprepper.plugins.processor.date.DateProcessorTests" time="0.008"/>
<testcase name="[12] epoch_nano, 1705950226818194066, yyyy-MM-dd'T'HH:mm:ss.SSSXXX, 2024-01-22T19:03:46.818Z" classname="org.opensearch.dataprepper.plugins.processor.date.DateProcessorTests" time="0.008"/>
<testcase name="[13] yyyy-MMM-dd HH:mm:ss.nnnnnnnnnXXX, 2024-Jan-22 19:03:46.818194066Z, epoch_second, 1705950226" classname="org.opensearch.dataprepper.plugins.processor.date.DateProcessorTests" time="0.009"/>
<testcase name="[14] yyyy-MMM-dd HH:mm:ss.nnnnnnnnnXXX, 2024-Jan-22 19:03:46.818194066Z, epoch_milli, 1705950226818" classname="org.opensearch.dataprepper.plugins.processor.date.DateProcessorTests" time="0.008"/>
<testcase name="[15] yyyy-MMM-dd HH:mm:ss.nnnnnnnnnXXX, 2024-Jan-22 19:03:46.818194066Z, epoch_nano, 1705950226818194066" classname="org.opensearch.dataprepper.plugins.processor.date.DateProcessorTests" time="0.009"/>
<testcase name="[16] yyyy-MMM-dd HH:mm:ss.nnnnnnnnnXXX, 2024-Jan-22 19:03:46.818194066Z, yyyy-MM-dd'T'HH:mm:ss.SSSXXX, 2024-01-22T19:03:46.818Z" classname="org.opensearch.dataprepper.plugins.processor.date.DateProcessorTests" time="0.009"/>
<testcase name="match_with_epoch_second_pattern()" classname="org.opensearch.dataprepper.plugins.processor.date.DateProcessorTests" time="0.006"/>
<testcase name="[1] en_US" classname="org.opensearch.dataprepper.plugins.processor.date.DateProcessorTests" time="0.01"/>
<testcase name="[2] fr_FR" classname="org.opensearch.dataprepper.plugins.processor.date.DateProcessorTests" time="0.015"/>
<testcase name="[3] ja_JP" classname="org.opensearch.dataprepper.plugins.processor.date.DateProcessorTests" time="0.015"/>
<testcase name="match_with_default_destination_test()" classname="org.opensearch.dataprepper.plugins.processor.date.DateProcessorTests" time="0.005"/>
<testcase name="[1] MMM/dd/uuuu" classname="org.opensearch.dataprepper.plugins.processor.date.DateProcessorTests" time="0.008"/>
<testcase name="[2] yyyy MM dd" classname="org.opensearch.dataprepper.plugins.processor.date.DateProcessorTests" time="0.007"/>
<testcase name="[1] America/New_York" classname="org.opensearch.dataprepper.plugins.processor.date.DateProcessorTests" time="0.01"/>
<testcase name="[2] America/Los_Angeles" classname="org.opensearch.dataprepper.plugins.processor.date.DateProcessorTests" time="0.008"/>
<testcase name="[3] Australia/Adelaide" classname="org.opensearch.dataprepper.plugins.processor.date.DateProcessorTests" time="0.007"/>
<testcase name="[4] Japan" classname="org.opensearch.dataprepper.plugins.processor.date.DateProcessorTests" time="0.007"/>
<testcase name="from_time_received_with_custom_destination_test()" classname="org.opensearch.dataprepper.plugins.processor.date.DateProcessorTests" time="0.005"/>
<testcase name="from_time_received_with_default_destination_test()" classname="org.opensearch.dataprepper.plugins.processor.date.DateProcessorTests" time="0.003"/>
<testcase name="match_with_custom_destination_test()" classname="org.opensearch.dataprepper.plugins.processor.date.DateProcessorTests" time="0.005"/>
<system-out>
<![CDATA[ ]]>
</system-out>
<system-err>
<![CDATA[ SLF4J: No SLF4J providers were found. SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J: See https://www.slf4j.org/codes.html#noProviders for further details. ]]>
</system-err>
</testsuite>

To Reproduce
Steps to reproduce the behavior:

  1. Clone the repo
  2. Checkout main branch
  3. Run ./gradlew build
  4. See error:
> Task :data-prepper-plugins:date-processor:test

DateProcessorTests > match_without_year_test(String) > [1] MMM/dd FAILED
    java.lang.NullPointerException at DateProcessorTests.java:510

DateProcessorTests > match_with_different_input_output_formats(String, Object, String, Object) > [13] yyyy-MMM-dd HH:mm:ss.nnnnnnnnnXXX, 2024-Jan.-22 11:39:29.367126542-08:00, epoch_second, 1705952369 FAILED
    java.lang.AssertionError at DateProcessorTests.java:334

DateProcessorTests > match_with_different_input_output_formats(String, Object, String, Object) > [14] yyyy-MMM-dd HH:mm:ss.nnnnnnnnnXXX, 2024-Jan.-22 11:39:29.367126542-08:00, epoch_milli, 1705952369367 FAILED
    java.lang.AssertionError at DateProcessorTests.java:334

DateProcessorTests > match_with_different_input_output_formats(String, Object, String, Object) > [15] yyyy-MMM-dd HH:mm:ss.nnnnnnnnnXXX, 2024-Jan.-22 11:39:29.367126542-08:00, epoch_nano, 1705952369367126542 FAILED
    java.lang.AssertionError at DateProcessorTests.java:334

DateProcessorTests > match_with_different_input_output_formats(String, Object, String, Object) > [16] yyyy-MMM-dd HH:mm:ss.nnnnnnnnnXXX, 2024-Jan.-22 11:39:29.367126542-08:00, yyyy-MM-dd'T'HH:mm:ss.SSSXXX, 2024-01-22T11:39:29.367-08:00 FAILED
    java.lang.AssertionError at DateProcessorTests.java:334

DateProcessorTests > match_with_default_destination_test() FAILED
    java.lang.NullPointerException at DateProcessorTests.java:535

DateProcessorTests > match_with_different_year_formats_test(String) > [1] MMM/dd/uuuu FAILED
    java.lang.NullPointerException at DateProcessorTests.java:453

DateProcessorTests > match_with_custom_destination_test() FAILED
    java.lang.NullPointerException at DateProcessorTests.java:535

70 tests completed, 8 failed

> Task :data-prepper-plugins:date-processor:test FAILED

Expected behavior
All tests should pass locally.

Environment (please complete the following information):

  • OS: macOS Sonoma 14.2.1
  • main branch on commit 41eab73
  • Java version 17.0.4
@GumpacG GumpacG added bug Something isn't working untriaged labels Jan 22, 2024
@dlvenable
Copy link
Member

@GumpacG , I know we've had some issues with these tests in different time zones. What time zone are you running the tests in? Also, what locale does your machine?

@dlvenable dlvenable added maintenance Issues to help maintain the project, such as improving builds, testing, etc. and removed untriaged labels Jan 23, 2024
@dlvenable
Copy link
Member

@GumpacG , Would you be interested in helping with this issue? I suspect that our tests are not always normalizing the time zone and thus can fail depending on the machine's time zone (and maybe locale).

@GumpacG
Copy link
Contributor Author

GumpacG commented Jan 23, 2024

@dlvenable Sure, I wasn't sure what was causing the failures so I logged this issue. Thanks for your ideas above, I can take a look.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working maintenance Issues to help maintain the project, such as improving builds, testing, etc.
Projects
Data Prepper Tracking Board
Status: Unplanned
Milestone
No milestone
Development

No branches or pull requests
2 participants
@dlvenable @GumpacG