Spock AWS Example Tests

Issues Found Attempt to upload large number of files: queuing numbers of files to process seems to cause errors in lamda function -connection also times out when trying to upload lots of files

Wrong File location for errored files: invalid/missing fields arent being sent to error folder

Tech/languages/approach Spock - I have always found spock to give a good balance between abstraction and exposing QA’s to the code logic. Cucumber (for example) abstracts far too much code away by default resulting in QA’s being too detached from the underlying code (they only look at it when something fails). Spock’s readability is generally very good - much better than Junit. Groovy - I always write my automation with the mindset of making it understandable for the next person coming along. In my experience it can be challenging for QA’s who have mainly performed manual testing to start coding - java/groovy is (in my opinion) the easiest OO language to pickup. Structure - Helper classes and methods used where appropriate - there is a fine line between making things understandable for another QA and hiding logic away. Proper commenting helps.

Further Work - Encryption of any secrets (eg aws keys)

Automate testing of empty fields for each element

Remove hard coded sleeps (can we stream aws logs to check for file creation?)

Count number of occurrences of string in local csv to compare against json from s3 (see 'Upload Large Valid File' test)

Check code for exception states and ensure they are all covered by a test - when writing tests i make sure to go through the code to look for any exceptions that can be triggered - i will speak to the developer to consider non happy path and non functional testing.

Boundary Testing - Upper values to be tested and automated

Exploratory Testing - Testing as many variations of csv files as possible

Performance Testing - Load testing required for lambda function

Retrieving files from processed/error outbucket - the files are named with timestamp making grabbing them more difficult. As a (poor) workaround I delete files between test runs and used a method to grab files that are there (files added in the test) - a new method to check timestamp and grab relevant files (problematic as function takes a few seconds so timestamp is impossible to know) - Access to logs should expose file name making this much easier.

Possibly create a method to create csv file programmatically - might not be viable due to large amount of arguments required for various fields

Create a gradle task to run tests from terminal and in ci/cd pipeline

Most testers only care about testing error code/exceptions and not the log message itself - understandable as these change and then automation can fail. I prefer to keep the log messages in while the number of tests are manageable.

Test different file types - ensure only csv files are processed