17032023 all rules and first bug fixes

[tab1tha]

What's Changed

• XML to DataFrames by @tab1tha in XML to DataFrames #113
• [xml-ingress] ChildIdentifiersTable, ChildCharacteristicsTable, CINdetailsTable by @tab1tha in [xml-ingress] ChildIdentifiersTable, ChildCharacteristicsTable, CINdetailsTable #114
• black formatting by @tab1tha in black formatting #115
• Rule template: Demonstrate on Rule 8500 by @tab1tha in Rule template: Demonstrate on Rule 8500 #117
• setup devcontainer to install depenency packages by @willLPD2I in setup devcontainer to install depenency packages #119
• Rule 8510, created and working by @phoebefryer in Rule 8510, created and working #118
• add pre-commit to requirements by @tab1tha in add pre-commit to requirements #121
• Rule100 completed by @JohnCWaC in Rule100 completed #125
• Added Rule 1510 by @PhilDeakin in Added Rule 1510 #124
• Rule1530 by @PhilDeakin in Rule1530 #126
• Makes census period from collection_year by @tab1tha in Makes census period from collection_year #123
• Established rule1550 by @phoebefryer in Established rule1550 #127
• Added4220 by @PhilDeakin in Added4220 #128
• Added8790 by @PhilDeakin in Added8790 #129
• Rule1540 by @willLPD2I in Rule1540 #130
• Added8794 by @PhilDeakin in Added8794 #131
• Added8650 by @PhilDeakin in Added8650 #133
• Ops/GitHub actions by @tab1tha in Ops/GitHub actions #134
• Rule 4180 by @amyrichards0n in Rule 4180 #135
• Added rule_8617 Rule 8617 : Code 8A has been returned. This code is not a valid code.  #72 by @bellt101 in Added rule_8617 #72 #137
• Added8568 by @PhilDeakin in Added8568 #140
• define query RuleType by @tab1tha in define query RuleType #142
• rule-8870q by @phoebefryer in rule-8870q #141
• Generate tables from XML by @tab1tha in Generate tables from XML #145
• Run all rules on data files by @tab1tha in Run all rules on data files #146
• Type1 rule implementation by @tab1tha in Type1 rule implementation #150
• Added8545Q by @PhilDeakin in Added8545Q #143
• Added8905 by @PhilDeakin in Added8905 #148
• Rule_8840 by @willLPD2I in Rule_8840 #152
• use ReferenceDate in make_census_period by @tab1tha in use ReferenceDate in make_census_period #153
• Rule8715 by @GBurrel in Rule8715 #147
• Rule8930 Closes Rule 8930 : Child Protection Plan End Date must fall within the census year #106 by @amyrichards0n in Rule8930 Closes #106 #154
• Closes Rule Rule 8696 : Assessment end date must fall within the census year #65 by @GBurrel in Closes Rule #65 #151
• Added 8535Q by @phoebefryer in Added 8535Q #155
• No errors closes 8520 by @GBurrel in No errors closes 8520 #156
• Added8615 by @PhilDeakin in Added8615 #160
• Added rule 8608 by @WendyWigan in Added rule 8608 #159
• Revised4012Q by @PhilDeakin in Revised4012Q #157
• Rule structure: Multiple columns across multiple tables. by @tab1tha in Rule structure: Multiple columns across multiple tables. #166
• Working rule8614 by @Csmyth201 in Working rule8614 #164
• XMLtoCSV CLI command by @willLPD2I in XMLtoCSV CLI command #161
• Rule8805 by @Csmyth201 in Rule8805 #163
• Added rule 4011 by @phoebefryer in Added rule 4011 #158
• Added8630 by @PhilDeakin in Added8630 #170
• Edit rule 8630 by @tab1tha in Edit rule 8630 #174
• Rule_8640 complete closes#43 by @GBurrel in Rule_8640 complete closes#43 #177
• Added rule 1105 by @phoebefryer in Added rule 1105 #138
• Created rule 8915 by @phoebefryer in Created rule 8915 #181
• Implement type3 by @tab1tha in Implement type3 #182
• Rule 8896: example of type3 by @tab1tha in Rule 8896: example of type3 #172
• added 8935 - will's type 3 by @willLPD2I in added 8935 - will's type 3 #175
• Rule8525 by @willLPD2I in Rule8525 #176
• Rule8810 by @willLPD2I in Rule8810 #168
• Rule8841 by @willLPD2I in Rule8841 #171
• Rule8897 by @WendyWigan in Rule8897 #173
• Rule8730 by @willLPD2I in Rule8730 #178
• Proof of concept: type3 by @tab1tha in Proof of concept: type3 #185
• 8736 by @GBurrel in 8736 #179
• Fixes print statements, adds date input flexibility for reference date by @willLPD2I in Fixes print statements, adds date input flexibility for reference date #183
• fixed make_date fucntion to allow varied inputs by @willLPD2I in fixed make_date fucntion to allow varied inputs #188
• Rule_4004 by @phoebefryer in Rule_4004 #187
• Rule 2885: Example of type2. by @tab1tha in Rule 2885: Example of type2. #167
• Fix group keys by @willLPD2I in Fix group keys #191
• Feature/check per type by @tab1tha in Feature/check per type #194
• Feature/error df by @willLPD2I in Feature/error df #199
• Added rule 8750 by @SLornieCYC in Added rule 8750 #169
• rule 4008 working, formatted by @willLPD2I in rule 4008 working, formatted #190
• Rule8920 by @GBurrel in Rule8920 #192
• rule 8555 created by @phoebefryer in rule 8555 created #193
• Created rule 8585Q by @phoebefryer in Created rule 8585Q #195
• Rule1103 by @GBurrel in Rule1103 #197
• Created rule 8875 by @phoebefryer in Created rule 8875 #196
• Rule_8565 by @phoebefryer in Rule_8565 #200
• group keys set to false in rules raising warnings by @willLPD2I in group keys set to false in rules raising warnings #202
• Rule4013 by @Csmyth201 in Rule4013 #201
• cli command to select issues by ERROR_IDs by @willLPD2I in cli command to select issues by ERROR_IDs #210
• Added4015 by @PhilDeakin in Added4015 #184
• fixes group keys in 4008 by @willLPD2I in fixes group keys in 4008 #212
• Added rule 8740 by @SLornieCYC in Added rule 8740 #189
• Added rule 4001 by @SLornieCYC in Added rule 4001 #208
• Resolve erratic behaviour in run-all command by @tab1tha in Resolve erratic behaviour in run-all command #214
• Include run instructions in README.MD by @tab1tha in Include run instructions in README.MD #215
• Isolate functions from CLI by @tab1tha in Isolate functions from CLI #216
• Rule8620 by @GBurrel in Rule8620 #206
• rule 8530Q by @phoebefryer in rule 8530Q #204
• Added rule 8940 by @SLornieCYC in Added rule 8940 #209
• Rule8867 by @GBurrel in Rule8867 #205
• Rule8720 Issue Rule 8720 : Child Protection Plan Start Date missing or out of data collection period #101 by @amyrichards0n in Rule8720 Issue #101 #198
• Rule 8670Q by @phoebefryer in Rule 8670Q #203
• Added rule 4016 by @SLornieCYC in Added rule 4016 #207
• draft Rules df by @willLPD2I in draft Rules df #217
• Added rule 4017 (mirror of 4016) by @SLornieCYC in Added rule 4017 (mirror of 4016) #219
• Added rule 8839 by @SLornieCYC in Added rule 8839 #220
• Create RULE_PATTERNS.MD by @tab1tha in Create RULE_PATTERNS.MD #230
• Rule 8606 by @phoebefryer in Rule 8606 #227
• Added rule 8590 by @SLornieCYC in Added rule 8590 #221
• 8842Q by @willLPD2I in 8842Q #223
• rule 8832 working by @willLPD2I in rule 8832 working #224
• 4000 working by @willLPD2I in 4000 working #225
• Rule 1104 by @tab1tha in Rule 1104 #228
• Rule8675Q by @willLPD2I in Rule8675Q  #226
• 8869 working by @willLPD2I in 8869 working #232
• Added rule 8770Q by @SLornieCYC in Added rule 8770Q #241
• 8873 by @willLPD2I in 8873 #233
• Rule 4010v2 closes Rule 4010 : CIN Plan start date is missing or out of data collection period #79 by @GBurrel in Rule 4010v2 closes #79 #245
• 8899Q working by @willLPD2I in 8899Q working #234
• 4003 working by @willLPD2I in 4003 working #235
• AddsRule4014 by @tab1tha in AddsRule4014 #231
• 2990 by @willLPD2I in 2990 #238
• Feature/generate output by @tab1tha in Feature/generate output #236
• 8890 working by @willLPD2I in 8890 working #239
• 8820 working by @willLPD2I in 8820 working #240
• 8815 working by @willLPD2I in 8815 working #242
• 8831 by @willLPD2I in 8831 #243
• 8825Q working by @willLPD2I in 8825Q working #244
• Rule8600 by @GBurrel in Rule8600 #248
• 8775Q working by @willLPD2I in 8775Q working #246
• 8868 working by @willLPD2I in 8868 working #247
• 8540 working by @willLPD2I in 8540 working #250
• 8569 working by @tab1tha in 8569 working #252
• Implement return level validation by @tab1tha in Implement return level validation #256
• Rule 2886Q by @tab1tha in Rule 2886Q #257
• Rule 1520 by @tab1tha in Rule 1520 #258
• Rule2887Q by @tab1tha in Rule2887Q #259
• Correct file type: 2886Q by @tab1tha in Correct file type: 2886Q #260
• Rule2883 by @tab1tha in Rule2883 #262
• edits on 8866 by @tab1tha in edits on 8866 #263
• 8866 working by @willLPD2I in 8866 working #253
• 8610 working by @willLPD2I in 8610 working #255
• Added2889 by @PhilDeakin in Added2889 #136
• Rule 2888Q by @tab1tha in Rule 2888Q #265
• 2884 by @willLPD2I in 2884 #254
• 4009 working by @willLPD2I in 4009 working #266
• Code for review Closes Rule Rule 8863 : An Assessment is shown as starting when there is another Assessment ongoing  #69 by @GBurrel in Code for review Closes Rule #69 #211
• 8816 by @willLPD2I in 8816 #251
• 8898 working by @willLPD2I in 8898 working #267
• update comments: 8898 by @tab1tha in update comments: 8898 #268
• Rule8772 by @GBurrel in Rule8772 #261
• 2991Q working by @willLPD2I in 2991Q working #264
• Rule 1560Q by @tab1tha in Rule 1560Q #269
• Added8910 by @PhilDeakin in Added8910 #149
• Feature/rpc layer by @tab1tha in Feature/rpc layer #270
• Process date columns by @willLPD2I in Process date columns #277
• Process RemoteFile objects by @tab1tha in Process RemoteFile objects #278
• parse RemoteFile by @tab1tha in parse RemoteFile #279
• Refactor/integration fixes - Json Serialiser by @tab1tha in Refactor/integration fixes - Json Serialiser #281
• Refactor/rule sweep by @tab1tha in Refactor/rule sweep #229
• Process date columns - more efficient implimentation by @willLPD2I in Process date columns - more efficient implimentation #282
• Include child ID in issue report by @tab1tha in Include child ID in issue report #285
• typo correction: "sources" not "sourcing" by @tab1tha in typo correction: "sources" not "sourcing" #286
• Documentation by @willLPD2I in Documentation #237
• Docstrings for all central functions by @willLPD2I in Docstrings for all central functions #280
• feature/select rules by @tab1tha in feature/select rules #289
• Rule sweep top down 100-8614 by @willLPD2I in Rule sweep top down 100-8614 #287
• Contributor docs by @willLPD2I in Contributor docs #290
• Fix dateofinitialcpc bug by @willLPD2I in Fix dateofinitialcpc bug #293
• Refactor/rule sweep ascend by @tab1tha in Refactor/rule sweep ascend #288
• Fix 8816 - ValueError: attempt to get argmax of an empty sequence by @tab1tha in Fix 8816 - ValueError: attempt to get argmax of an empty sequence #295
• diagram.data flow in cin tool. by @tab1tha in diagram.data flow in cin tool. #296
• debug/issue_report: row_id as integers by @tab1tha in debug/issue_report: row_id as integers #298
• Refactor/remove data container by @tab1tha in Refactor/remove data container #299
• Feature/include bank holidays by @tab1tha in Feature/include bank holidays #300
• Fix/xml csv missing data by @tab1tha in Fix/xml csv missing data #301
• separate data processing from conversion by @tab1tha in separate data processing from conversion #302
• enable successful ingress with incomplete data by @tab1tha in enable successful ingress with incomplete data #303
• Fix/no error scenario by @tab1tha in Fix/no error scenario #304
• fix lint error. isort by @tab1tha in fix lint error. isort #308
• Feature/create report by @tab1tha in Feature/create report #306
• added simple timer to CLI for testing purposes by @willLPD2I in added simple timer to CLI for testing purposes #307
• Bugfix 8535Q by @willLPD2I in Bugfix 8535Q #327
• remove hardcoded cin_data by @tab1tha in remove hardcoded cin_data #332
• 1560Q now correctly checks not by @willLPD2I in 1560Q now correctly checks <FormerUPN> not <UPN> #325
• 4180 bugfix by @willLPD2I in 4180 bugfix #326
• 8832 bugfix by @willLPD2I in 8832 bugfix #328
• bugfix 8897 by @willLPD2I in bugfix 8897  #329
• bugfix 8869 by @willLPD2I in bugfix 8869 #330
• bugfix 8868 by @willLPD2I in bugfix 8868 #333
• 8620 bugfix by @willLPD2I in 8620 bugfix #334
• removes select-issue-by-id feature by @tab1tha in removes select-issue-by-id feature #335
• Bugfix 8863 by @willLPD2I in Bugfix 8863 #338
• Bugfix 8820 by @willLPD2I in Bugfix 8820 #340
• Bugfix 8839 by @willLPD2I in Bugfix 8839 #339
• bugfix 4001, added check for CINplanStartDate by @willLPD2I in bugfix 4001, added check for CINplanStartDate #337
• bugfix 8890 by @willLPD2I in bugfix 8890 #336
• Docs/typos by @tab1tha in Docs/typos #349
• Bugfix 2885 by @willLPD2I in Bugfix 2885 #341
• Bugfix/fix inequalities by @willLPD2I in Bugfix/fix inequalities #351
• 2324ruleset/errors to queries by @willLPD2I in 2324ruleset/errors to queries #347
• Bugfix/fix merges by @willLPD2I in Bugfix/fix merges #352
• bugfix 4017 by @willLPD2I in bugfix 4017  #354
• bugfix 1510 by @willLPD2I in bugfix 1510 #356
• fixed 8535Q by @willLPD2I in fixed 8535Q #357
• bugfix 1560Q by @willLPD2I in bugfix 1560Q #358
• remove TODOs by @tab1tha in remove TODOs #350
• 1560Q fix pytest by @willLPD2I in 1560Q fix pytest #359
• Fix/key error 4001 by @tab1tha in Fix/key error 4001 #360
• mvp init file by @tab1tha in mvp init file #361
• suppress false-positive SettingWithCopyError by @tab1tha in suppress false-positive SettingWithCopyError #362
• ensure dates are passed as strings to fe by @tab1tha in ensure dates are passed as strings to fe #363
• apply datetime_to_str recursively by @tab1tha in apply datetime_to_str recursively #364
• bugfix 4014 to flag duplicate cin plans by @willLPD2I in bugfix 4014 to flag duplicate cin plans #386
• Bugfix 8816 by @willLPD2I in Bugfix 8816 #387
• Bugfix/8839 by @willLPD2I in Bugfix/8839 #389
• bugfix 8675Q, by @willLPD2I in bugfix 8675Q, #392
• Bugfix 2888Q by @tab1tha in Bugfix 2888Q #394
• bugfix 8820 correctly flags duplicates by @willLPD2I in bugfix 8820 correctly flags duplicates #388
• bugfix 2991Q by @willLPD2I in bugfix 2991Q  #395
• bugfix 8670Q by @willLPD2I in bugfix 8670Q #391
• replace truthy check with explicit len check by @tab1tha in replace truthy check with explicit len check #401

New Contributors

• @willLPD2I made their first contribution in setup devcontainer to install depenency packages #119
• @phoebefryer made their first contribution in Rule 8510, created and working #118
• @JohnCWaC made their first contribution in Rule100 completed #125
• @PhilDeakin made their first contribution in Added Rule 1510 #124
• @amyrichards0n made their first contribution in Rule 4180 #135
• @bellt101 made their first contribution in Added rule_8617 #72 #137
• @GBurrel made their first contribution in Rule8715 #147
• @WendyWigan made their first contribution in Added rule 8608 #159
• @Csmyth201 made their first contribution in Working rule8614 #164
• @SLornieCYC made their first contribution in Added rule 8750 #169

Full Changelog: https://github.com/data-to-insight/CIN-validator/commits/0.1.1

---

This discussion was created from the release 17032023 all rules and first bug fixes.

[tab1tha]

@SLornieCYC a new version of the tool is live.
If you get a chance, could you check whether the bug fixes worked?

@willLPD2I

[WillLP-code]

note: this isn't everything you raised Stephen, if somethin is still in issues we haven't made a PR to deal with it yet

[SLornieCYC]

@tab1tha The implemented bug fixes all look to be working as expected from a quick scan through (comparing pre- and post-fix CIN validator error outputs and then matching to DFE portal output as well).

The only one I'm not 100% sure about is the working days part on 8670Q. #377 (comment)

[tab1tha]

@SLornieCYC @willLPD2I a new version of the tool is now live.
If you get a chance, do check whether it works well now.

Happy Easter !