-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Updated process_report.py to avoid warnings for SettingWithCopyWarnings and FutureWarnings #40
Conversation
4707b82
to
1b2716a
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure I understand the relationship between the title of this PR ("Patched pandas warnings for SettingWithCopyWarnings and FutureWarnings") and the changes it contains.
@larsks Without the changes, running the processing script will make Pandas print the 2 warnings mentioned in the title. The SettingWithCopyWarnings came from the fact that I was performing chained indexing on a few dataframes. Since chained indexing may return a copy of the dataframe or dataframe itself, this causes ambiguity on whether assignments to the indexed dataframes would change the original, or merely a copy. I resolved the warnings by explicitly using the As for the FutureWarnings, these were caused by assigning strings to empty columns, whose values default to NaN (float). Since float is incompatible with strings, this raises a warning. |
Ah, so "Update code to avoid SettingWithCopyWarnings and FutureWarnings". |
@QuanMPhm You should probably put that into the commit message because it is a good explanation of what's going on. |
1b2716a
to
2416397
Compare
@naved001 I've updated the commit message |
Without the patches, running the processing script will make Pandas print 2 warnings, SettingWithCopyWarnings and FutureWarnings. The SettingWithCopyWarnings came from the fact that chained indexing was performed on a few dataframes. Since chained indexing may return a copy of the dataframe or dataframe itself, this causes ambiguity on whether assignments to the indexed dataframes would change the original, or merely a copy. This is resolves the warnings by explicitly using the copy() function to make copies of the dataframes. As for the FutureWarnings, these were caused by assigning strings to empty columns, whose values default to NaN (float). Since float is incompatible with strings, this raises a warning. This is fixed by explicitly type casting certain columns
2416397
to
a76b353
Compare
Closes #34, I've added a few small changes which should remove all pandas SettingWithCopyWarning and FutureWarning warnings. These consisted of explicit type casting, and explicitly making copies of data-frames that would later be modified