Comparative Analysis

Online regulations tools comparative analysis

Key findings

Overview

eRegulations (eRegs) is an open-source web-based application built by the Consumer Financial Protection Bureau and 18F. The eRegs platform makes regulations easier for agencies, states, and individuals to find, read, and understand. CMCS has initiated a pilot project to adapt eRegs for the needs of CMCS and State Medicaid staff.

Process

In order to gain a better understanding of what regulation tools people currently use, we performed a comparative analysis with a focus on navigability, supporting content, and search. We reviewed the following online tools:

Regulation tools owned by individual agencies

• CFPB interactive regulations
• ATF eRegs
• FEC eRegs
• GSA Acquisition.gov - FAR (Federal Acquisition Regulation)

The Government Printing Office

• eCFR
• Beta eCFR
• CFR online book

A third-party nonprofit hosted by a university

• Legal Information Institute CFR

For a spreadsheet version of this analysis, please refer to the comparative analysis spreadsheet.

Key findings

Navigability

Section headings and structure

• Regulations have a particular structure and have numbers (sometimes as Roman numerals) and names that describe the subject area within the regulation.
• eCFR has a mix of numbers and names everywhere. The navigation is structured by subchapter, but exposes (and emphasizes) parts as the main entry-point for browsing. Full-text displays of parts are the default.
  • The table of contents indicates where to go for CMS programs, but users would need to know the number of the correct part and that “Medical Assistance Programs” means Medicaid.
• Beta eCFR does the opposite and leaves out the numbers too often. Once a user selects Title 42, it brings them to chapters and subchapters, and then to parts. The structure is very much like an outline, but a user would need to know to click on the full name of a part (vs the part number) to get its full-text.
• Agency-owned eRegs use a combination of section numbers and names, though the emphasis is on section numbers. Similar to beta eCFR, the navigation emphasis is by subchapter, not part, but instead of an outline format, the left sidebar is used for navigation.
• Acquisition.gov displays both numbers and names, but it emphasizes part and section numbers for navigation in both the left sidebar and top navigation bar.

Areas to consider:

• Aim for displaying the part & section numbers prominently for navigation, and ensure that numbers are prominent enough to support our target users’ information seeking needs.
• Show wherever possible the names as well for users who are less familiar with regulations, including plain language additions that further comprehension, such as showing the official names and supplemental information in different places.
  • For example, showing names next to numbers whenever possible, while putting plain language in the sidebars or reg part homepages.
• Determine whether it’s better to break up content by section (eRegs default) or whether it should be structured differently.
• Consider linking out to beta eCFR for a full-text view of parts when and where appropriate, since eRegs is designed to present content chunked out by section.

Internal and external links

There are a few link types used in regulations tools: internal links, external links, and definition links. Definitions will be reviewed separately in the Supporting Content section.

ATF eRegs

• Internal links are underlined in blue and open in the same tab.
• External links are in blue text with an external link icon and open in a new tab.
• Words with definitions
 are underlined in gray and open up in the right sidebar.

FEC eRegs

• Internal links are underlined in blue and open in the same tab.
• External links are underlined in gray with an external link icon and open in a new tab.
• Words with definitions
 are blue and open up in the right sidebar.

CFPB interactive regulations

• Internal links are in blue text; visited links change from blue to green.
• There are no external links within the regulations, but there is a dedicated page that links to eCFR for CFPB-specific regulations eCFR.
• Links exist at the Title, Part, Subpart, and Section levels for navigation, but within the main content there doesn't seem to be linkage. beta eCFR.
• There are no links out, so just one style of links, since the site contains all the regulations within itself. 

If you hover over a subparagraph notation (for lack of a better word), you can click to copy the citation url. Online reg book.
• The only linking function is a share button for Facebook, Twitter, email, or copying the url. Cornell Law
• Section symbol cross-references link to a section within the site for navigation. 

* Non-section symbol links go to a citation link maker.

Acquisition.gov

• Internal links are underlined in blue and open in the same tab.
• External links are in blue text with an external link icon and open in a new tab.

Areas to consider:

• Differentiating between internal and external links needs further research around when it might be appropriate to link internally vs link externally to beta eCFR.
  • For example, one option to consider is linking internally within eRegs whenever we cover that reg part or section within eRegs & link to beta eCFR for parts we don't cover (like Title 45).

Changes over time

Users interact with online regulation tools to answer questions around effective dates or to compare changes between two versions. ATF, CFPB, and the beta eCFR offered some variations on how users can do this. ATF eRegs uses the annual edition of the regs, and can go back several years. It supports two ways of viewing regulations changes across time:

• The timeline Date Picker allows users to find what revision was effective on a specific date.
• Compare any two revisions of the regulation by selecting a date and clicking Show Differences. A combination of colors and text treatments indicate a "diff”: green italicized text for additions and gray strikethrough text for deletions.

Beta eCFR uses the daily version of the regs, so it can only go back a few years. Similar to ATF, it supports two main ways of viewing regulations changes across time:

• The Change View - a timeline date picker that allows users to find what a section looked like on a specific date, as well as any specific changes in the section (which can be many).
• The Compare View where users can choose two versions that display the changes between them. Removals of content are displayed as strike-through text with a red background and additions feature a green background. This feature is customizable, as users can change the dates to compare as well as how the changes are visually displayed.

CFPB offers current and past versions of regulations, but does not have the ability to pick a date or compare revisions between dates.

Acquisition.gov offers its full navigation interface only for the current version of the regulations. It offers downloadable files for past versions under an "Archives" link in the header.

Areas to consider:

• Identify a set of specific realistic user needs for interacting with regulation changes over time.
• Test whether the eRegs timeline and comparison feature is sufficient for our target users.

Supporting content

Definition aids

It’s standard for regulations to have words defined within a definition section. Every tool we reviewed included definition sections for viewing, with the exception of the Online reg book, whose sections were only available for download (not viewing). Some tools offered additional definition aids for users:

ATF eRegs

• Within the regulations text, ATF eRegs has underlined definitions in gray for users to click and open up in a supporting content sidebar.

FEC eRegs

• There is a site-wide glossary as well as gray underlined words that open to a definition in the sidebar, but they don’t appear to match (“meeting” is not defined in the site-wide glossary, but it does appear in the sidebar when selected in the section).

CFPB interactive regulations

• There are no definition aids other than the definitions within the text itself.

eCFR

• There are no definition aids other than the definitions within the text itself.

eCFR beta

• There are no definition aids other than the definitions within the text itself.

Cornell Law’s regulations tool

• There are linked definitions that appear in a popup with a source and an attempt to crowdsource feedback. Some definitions are missing (i.e., the popups are empty).

Acquisition.gov

• There are no definition aids other than the definitions within the text itself.

Areas to consider:

• Research user needs around definitions.
• Test the stock definition feature of eRegs to see if it is sufficient or whether a different handling of a glossary is needed (such as a site-wide glossary).

Inline content

CFPB is the only online tool that supports official interpretations “inline” -- placed within the regulations text itself.

Areas to consider

• Research information needs around inline content, such as what kind of inline content might be the most beneficial.

Search

Two search patterns we investigated are search query term highlighting and search aids.

Search query terms

All but one of the regs tools we reviewed highlights a search query term on the search results page, which is useful for helping users find and evaluate results and choose the best one. However, the highlighted term does not carry forward to the result. Forcing users to remember their search term query and scan (or use the browser’s “find” function) to locate their term asks a lot of users.

Only one regs tool was the opposite: the original eCFR does not highlight the search query term in the search results, but the search query term appears in red on the result.

Areas to consider:

• Highlight search query terms on Search Results as well as on the result.

Search aids

Most of the online regulations tools we reviewed have search aids for users. These aids come in a few different forms. Aids can be found within close proximity to a search box, such as suggested search topics, a tooltip near a search box, or help text within the search box itself. More involved search aids come in the form of search tips or a guide found elsewhere on a regulation website.

ATF eRegs

• For every title, there is a home page of “quick links” for important topics as well as example topic searches below the search box.


• On the About page, there are search instructions (which could probably be linked to from the search sidebar tab).

FEC

• A keyword example is visible underneath the search box in the left sidebar.
• There is a “more info” tool tip for refining keywords above search box on results page.
• “More keyword options" link displays on the search results page and opens a modal with a kind of advanced search.

CFPB

• Search is available via a link on each subpart (or individual section), and takes the user to a separate page (vs a persistent search box in the sidebar or above the main content).

eCFR

• There is simple or advanced (proximity or Boolean) search.
  • Simple search to search the entire CFR, limited to one CFR title. 
 * Proximity search adds the ability to search for words near/not near other words or followed by/not followed by other words within 2, 5, 10, 20, 30, or 50 words.
  • Boolean goes further: you can ask for up to three terms to be retrieved only when they appear within specific regions of the CFR.

Beta eCFR

• Search is complex, and there is help content for it under the Reader's Aid guides.

Online reg book

• There is a persistent search box in the upper right corner, and users can refine results with filters such as publish date, collection, and more. There is also advanced search for refining by date, collection, and allows for changing the scope from full-text to branch, category, citation, government author, series, SuDoc class number, and title.

Cornell Law

• There is a persistent search box in the upper right corner, but there are no search aids for users.

Acquisition.gov

• There is a persistent search box in the upper right corner.
• The search results page offers filtering of search results by part number, along with filtering by "Document Type" (subtopic, part, index, appendix, subchapter) and other search filters.

Areas to consider:

• Research information needs to determine appropriate search aids, such as suggested search topics and help tips.