Get delta table files

[keenanwells-tatari]

I checked the answer here: #623

but wondering whether the of inputFiles is a reliable enough method of getting all the files? https://spark.apache.org/docs/3.1.3/api/python/reference/api/pyspark.sql.DataFrame.inputFiles.html

Docs are a bit vague on how reliable this is, we hosting our tables on S3

[keenanwells-tatari]

Came across this answer in regards to reliability with delta: https://stackoverflow.com/a/77107953

[keenanwells-tatari]

I think this is much more useful for what I need: https://docs.databricks.com/en/ingestion/file-metadata-column.html#metadata-examples