Skip to content

UrbanInstitute/nccs-regulation

BranchesTags

Repository files navigation

Nonprofit Regulation Project (Legal Compendium)

A repository for the CNP/NCCS Nonprofit Regulatory Environment project.

Data Files

The code here focuses on using hand-collected data produced by research collaborators and entered into Excel sheets, which are then read into R to clean and restructure.

The raw files that were provided by the research team at in data-raw/tidy-states.

Data Preparation Files

The Digitized Legal Compendium file is the primary data wrangling file for the first stages. It also suggests some standards for how to structure the file and format legal citations moving forward to scale teh work in the next phase.

The semi-clean file called 02-data-inter/ALL-STATES-FORMATTED.CSV is here: DOWNLOAD

The big changes that were made:

  • Variables were renamed for consistency and Legal Compendium descriptive labels were added.
  • IDs were inconsistent in the raw files (state names were incorrect leading to duplicate IDs, other ID names were inconsistent throughout the individual state files)
  • Text encoding from copied HTML was causing problems, so text fields were put through text sanitizers to remove any exotic encodings and ensure text is clean and consistent when stored in CSV files.
  • Section numbering fields, which consist of numbers and punctuation, were being interpretted as dates or getting corrupted in the usual excel ways. To preserve the integrity a text tag needs to be added to the start of each text field. For example, the section fields now start with {SS}:
  • The Section Symbol is non-unicode and thus was getting muddled, so it should be used sparingly (for example, it crashed this page when it was added).

Outstanding issues:

  • The section fields need to be cleaned and reconciled.
  • The Regulation Indicator field (reg_indicator) was inconsistent and it's unclear if it is accurate.
  • Please review variable names to see if we can make them intuitive.

Currently we group the regulations as:

  • reg_type (the high level column group in the legal compendium)
  • reg_rule (each column of the LC)
  • reg_body (which body is responsible for administering the regulation: AG, Other, and I added "Not Specified" for cases that were blank)
  • reg_action (describes what type of regulation it is - requirement, exemption, remedy, etc.)

I debated instead using "reg_group and reg_type instead of reg_type and reg_rule). Open to suggestions.

Demo Reports

I played with descriptive stats but could not find a lot that was interesting or insightful beyond the basic coverage for specific types of laws and variance in the extensiveness of the regulatory environments across states.

Otherwise I think people might want to query the statutes:

  1. By a regulatory type to see how laws vary across geographies
  2. By a geograph to see what laws exist
  3. By a combination of type and geography to look up a specific law

These report templates were an attempt to render the text data in a way that makes it easy to digest. It also highlights the data integrity issues that need to be fixed.

I can still add the user interface but I need some feedback on what is actually useful.

We can create tables that look similar to the original LC Excel file with YES/NO in the cells and hyperlinks to the notes and text on each section.

REG1 REG2
STATE1 link link
STATE2 link link

Just really wasn't sure what people want for an interface.

State-Level Reports

Report templates that summarize nonprofit statutes covered within each state.

See the STATE-REPORT-TEMPLATE.QMD file for details on replicating.

###
###  RENDER ALL REPORTS 
 
wd       <- "file/path/here/REPORTS/STATE/"
fn       <- "ALL-STATES-FORMATTED.CSV"
TEMPLATE <- "STATE-REPORT-TEMPLATE.qmd"

setwd( wd )

render_state_report <- function( STATE ){
  outfn <- paste0( STATE, ".html" )
  quarto::quarto_render(
    input = paste0( wd, "/", TEMPLATE ),
    output_file = outfn,
    execute_params = list(
      state  = STATE ) )
}

STATE <-
c("AL", "AK", "AZ", "AR", "CA", "CO", "CT", "DE", "FL", "GA", 
"HI", "ID", "IL", "IN", "IA", "KS", "KY", "LA", "ME", "MD", "MA", 
"MI", "MN", "MS", "MO", "MT", "NE", "NV", "NH", "NJ", "NM", "NY", 
"NC", "ND", "OH", "OK", "OR", "PA", "RI", "SC", "SD", "TN", "TX", 
"UT", "VT", "VA", "WA", "DC", "WV", "WI", "WY")

# ONE AT A TIME
render_state_report( STATE="WY" )

# ALL TOGETHER
purrr::walk( STATE, render_state_report )

Regulation Type Reports

Reports that summarize existing legislation on each individual regulatory type across all states.

See RULE-REPORT-TEMPLATE.QMD for the replication template.

###
###  RENDER REPORTS 
### 

wd <- "your/file/path/REPORTS/RULES"
setwd( wd )

TEMPLATE <- "RULE-REPORT-TEMPLATE.QMD*"

render_rule_report <- function( RULE ){
  outfn <- paste0( RULE, ".html" )
  quarto::quarto_render(
  input = paste0( wd, "/", TEMPLATE ),
  output_file = outfn,
  execute_params = list(
    rule  = RULE,
    fpath = fpath ) )
}

rules <-
c("BIFURC", "REGIOF", "ASSETS", "MERGER", "AMMEND", "VOLUNT", 
"JUDICI", "ADMINI", "STATUT", "OVERSI", "FILING", "AUDITS", "THRESH", 
"REGIST", "RELIGI", "SMALLO", "EDUCAT", "GOVMNT", "HOSPIT", "VETERA", 
"FNDYES", "FNDNOS", "CTRUST", "PTOEDU", "CONGRE", "NONSOL", "MEMFRA", 
"POLITI", "OTHTYP", "DISSOL", "BRDRMV", "COMREG", "COUNSL", "VENTUR", 
"NOTICE", "DDONOR", "CNTRCT", "ANNUAL", "FINANC", "BONDNG")

# ONE AT A TIME
render_rule_report( RULE = "BIFURC" )

# ALL TOGETHER
purrr::walk( rules, make_links2 )

About

A repository for the CNP Nonprofit Regulatory Environment project.

urbaninstitute.github.io/nccs-regulation/

Resources

Readme
Activity
Custom properties

Stars

0 stars

Watchers

7 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

No packages published

Languages