Tutorial | ISSUES | The Ohio State University

How to Analyze Caselaw Using CourtListener's API in R

Fri, 01 Aug 2025 00:00:00 +0000

I previously showed how to pull Indian Child Welfare Act (ICWA) cases using the Caselaw Access Project (Harvard Law). That API has since changed. This post updates the workflow using the Free Law Project’s CourtListener, which now partners with Harvard to make this data freely available.

Finding the Cases

Start at the CourtListener website and search for “Indian Child Welfare Act” (or "icwa"). You’ll see a page of opinion results.

Click Select Jurisdiction → State. Clear all, then select Ohio Supreme Court, Ohio Court of Appeals, and Ohio Court of Claims. This gives you a quick check on the expected volume to validate the code’s output below.

The site reported 36 cases from Ohio. Note that the exact number may change over time as new opinions are added or metadata is corrected.

Getting the Data into R

The Search API returns clean JSON results by querying for opinions that contain the exact phrase "indian child welfare act", restricting to published opinions, ordering by relevance, and (optionally) limiting results to three Ohio courts.

library(httr2)
library(purrr)
library(dplyr)
library(tibble)
library(tidyr)
library(stringr)

SEARCH_BASE <- "https://www.courtlistener.com/api/rest/v4/search/"
UA <- "R/httr2 (your-email-address) – ICWA metadata collection"

query_params <- list(
 q = '"indian child welfare act"',
 type = "o",
 order_by = "score desc",
 stat_Published = "on",
 court = "ohio ohioctapp ohioctcl" # optional: specify court
)

A few helper functions keep the parsing tidy:

pull_chr <- function(x, ...) {
 v <- purrr::pluck(x, ..., .default = NA)
 if (is.null(v) || length(v) == 0) return(NA_character_)
 as.character(v)
}

pull_num <- function(x, ...) {
 v <- purrr::pluck(x, ..., .default = NA_real_)
 suppressWarnings(as.numeric(v))
}

get_json <- function(url) {
 resp <- request(url) |> req_user_agent(UA) |> req_url_query() |> req_perform()
 resp_check_status(resp)
 resp_body_json(resp, simplifyVector = FALSE)
}

This function handles pagination automatically, following the next cursor until all pages are exhausted:

fetch_search_pages <- function() {
 req <- request(SEARCH_BASE) |>
 req_user_agent(UA) |>
 req_url_query(!!!query_params)

 dat <- req |> req_perform() |>
 (\(r){ resp_check_status(r); resp_body_json(r, simplifyVector = FALSE) })()
 pages <- list(dat)

 while (!is.null(dat$`next`)) {
 dat <- request(dat$`next`) |> req_user_agent(UA) |> req_perform() |>
 (\(r){ resp_check_status(r); resp_body_json(r, simplifyVector = FALSE) })()
 pages <- append(pages, list(dat))
 }
 pages
}

pages <- fetch_search_pages()
results <- pages |> map("results") |> flatten()

results is a list of per-case records from the API. Each element includes the case name, filing date, court identifiers, citation strings, a public URL where the caselaw text is available, and other metadata.

For Ohio, this produced ~35 unique case names versus 36 displayed on the site. Minor discrepancies happen because search hits can include multiple opinions or sibling opinions per matter, and because the index is updated over time.

Downloading the Full Text

Each opinion has an API endpoint with plain-text and HTML fields. A small loop can request the text for each case URL, with a short pause and retry between requests, and save one .txt file per opinion:

download_opinion_text <- function(url, out_dir = "opinions") {
 dir.create(out_dir, showWarnings = FALSE)
 slug <- basename(url)
 path <- file.path(out_dir, paste0(slug, ".txt"))

 resp <- request(url) |> req_user_agent(UA) |> req_perform()
 resp_check_status(resp)
 body <- resp_body_json(resp, simplifyVector = FALSE)

 text <- body$plain_text %||% body$html_with_citations %||% ""
 writeLines(text, path)
 Sys.sleep(0.5) # be considerate of rate limits
 invisible(path)
}

walk(map_chr(results, "absolute_url"), download_opinion_text)

Here are the Ohio ICWA cases downloaded to a local folder:

Visualizing the Results

You can use ggplot2 to visualize the results. Below are two examples: filings by year and volume by court.

What You Can Do Next

After downloading all data — including caselaw text and metadata — you can:

Natural Language Processing — examine sentiment or common words across opinions
Topic Modeling — explore latent themes in ICWA cases using LDA or STM
Map filings by jurisdiction — visualize geographic patterns of ICWA litigation
Integrate with other sources — link to child welfare administrative data or Census indicators

Remember to be considerate of rate limits and to include a User Agent string with your contact information in every request.

The complete workflow is available on my GitHub.

Happy Coding!

A PostgreSQL Tutorial for Analyzing Racial and Geographic Disparities in HMDA Applications

Wed, 16 Jul 2025 00:00:00 +0000

My lab recently published a paper examining the association between redlining, reinvestment, racial segregation, and firearm-related injury in Chicago, Illinois. You can read the paper here: Redlining, reinvestment, and racial segregation: a Bayesian spatial analysis of mortgage lending trajectories and firearm-related violence.

To develop reverse redlining measures, we relied on data from the Home Mortgage Disclosure Act (HMDA).

If you’re interested in patterns of lending discrimination, the HMDA dataset is an invaluable resource. It contains millions of individual loan application records submitted annually by financial institutions, making it one of the most extensive and detailed public data sources on mortgage lending in the United States. The 2024 nationwide Modified Loan/Application Register (LAR) alone contains over 12 million records and 85 fields that detail applicant and co-applicant demographics, loan terms, and property characteristics — including race, income, loan amount, interest rate, property value, and loan outcomes.

Each record includes the census tract of the loan origination, which allows researchers to situate applicants within their geographic context. This spatial identifier can be linked to other data sources (e.g., Census or ACS data), enabling robust analyses of racial disparities in lending, neighborhood-level disinvestment, and broader patterns of residential inequality.

I highly recommend The Color of Law by Richard Rothstein for anyone interested in the deep-rooted history of discriminatory housing practices in the United States. The book compellingly explains how federal, state, and local policies intentionally created and reinforced de jure segregation. Chicago, one of the most highly segregated urban areas in the country, is prominently featured.

Due to its sheer volume, the HMDA data is not well-suited for analysis using spreadsheets like Excel — particularly when analyzing multiple years or conducting joins. Instead, its relational structure lends itself to PostgreSQL, an open-source relational database management system (RDBMS) that enables efficient querying, filtering, and aggregation of large datasets. PostgreSQL also supports the PostGIS geospatial extension, enabling direct spatial querying and linking to geographic units such as census tracts or ZIP codes. Download PostgreSQL if you don’t already have it.

Step 1: Download the Dataset

The dataset is freely available to the public.

Navigate to https://ffiec.cfpb.gov/data-publication/modified-lar/2024
Choose Year: 2024 → Nationwide: Combined file
Check Include File Header, then click Download Combined Modified LAR with Header

Save or extract the file to: C:\Users\YOUR_USER_NAME\Downloads\2024_combined_mlar_header\

Step 2: Confirm the File Structure in PowerShell

The file is a pipe-delimited .txt file — it can be tricky to read in spreadsheet tools. Use Windows PowerShell to inspect the structure first:

cd "C:\Users\YOUR_USER_NAME\Downloads\2024_combined_mlar_header"

# View the header (column names)
Get-Content .\2024_combined_mlar_header.txt -TotalCount 1

# Count the number of columns (should be 85)
((Get-Content .\2024_combined_mlar_header.txt -TotalCount 1) -split '\|').Count

Step 3: Create the PostgreSQL Table

Launch your SQL interface (pgAdmin Query Tool or psql):

psql -U your_username -d your_database_name

Then define the 85-column table structure:

DROP TABLE IF EXISTS lar2024;
CREATE TABLE lar2024 (
 activity_year INTEGER,
 lei TEXT,
 loan_type INTEGER,
 loan_purpose INTEGER,
 preapproval INTEGER,
 construction_method INTEGER,
 occupancy_type INTEGER,
 loan_amount BIGINT,
 action_taken INTEGER,
 state_code TEXT,
 county_code TEXT,
 census_tract TEXT,
 applicant_ethnicity_1 INTEGER,
 applicant_ethnicity_2 TEXT,
 applicant_ethnicity_3 TEXT,
 applicant_ethnicity_4 TEXT,
 applicant_ethnicity_5 TEXT,
 co_applicant_ethnicity_1 INTEGER,
 co_applicant_ethnicity_2 TEXT,
 co_applicant_ethnicity_3 TEXT,
 co_applicant_ethnicity_4 TEXT,
 co_applicant_ethnicity_5 TEXT,
 applicant_ethnicity_observed INTEGER,
 co_applicant_ethnicity_observed INTEGER,
 applicant_race_1 INTEGER,
 applicant_race_2 TEXT,
 applicant_race_3 TEXT,
 applicant_race_4 TEXT,
 applicant_race_5 TEXT,
 co_applicant_race_1 INTEGER,
 co_applicant_race_2 TEXT,
 co_applicant_race_3 TEXT,
 co_applicant_race_4 TEXT,
 co_applicant_race_5 TEXT,
 applicant_race_observed INTEGER,
 co_applicant_race_observed INTEGER,
 applicant_sex INTEGER,
 co_applicant_sex INTEGER,
 applicant_sex_observed INTEGER,
 co_applicant_sex_observed INTEGER,
 applicant_age TEXT,
 applicant_age_above_62 TEXT,
 co_applicant_age TEXT,
 co_applicant_age_above_62 TEXT,
 income TEXT,
 purchaser_type INTEGER,
 rate_spread TEXT,
 hoepa_status INTEGER,
 lien_status INTEGER,
 applicant_credit_scoring_model INTEGER,
 co_applicant_credit_scoring_model INTEGER,
 denial_reason_1 TEXT,
 denial_reason_2 TEXT,
 denial_reason_3 TEXT,
 denial_reason_4 TEXT,
 total_loan_costs TEXT,
 total_points_and_fees TEXT,
 origination_charges TEXT,
 discount_points TEXT,
 lender_credits TEXT,
 interest_rate TEXT,
 prepayment_penalty_term TEXT,
 debt_to_income_ratio TEXT,
 combined_loan_to_value_ratio TEXT,
 loan_term TEXT,
 intro_rate_period TEXT,
 balloon_payment INTEGER,
 interest_only_payment INTEGER,
 negative_amortization INTEGER,
 other_non_amortizing_features INTEGER,
 property_value TEXT,
 manufactured_home_secured_property_type INTEGER,
 manufactured_home_land_property_interest INTEGER,
 total_units TEXT,
 multifamily_affordable_units TEXT,
 submission_of_application INTEGER,
 initially_payable_to_institution INTEGER,
 aus_1 INTEGER,
 aus_2 TEXT,
 aus_3 TEXT,
 aus_4 TEXT,
 aus_5 TEXT,
 reverse_mortgage INTEGER,
 open_end_line_of_credit INTEGER,
 business_or_commercial_purpose INTEGER
);

Step 4: Import the Data

In psql, use the following single-line import command:

\copy lar2024 FROM 'C:/Users/YOUR-USER-NAME/Downloads/2024_combined_mlar_header/2024_combined_mlar_header.txt' WITH (FORMAT csv, DELIMITER '|', HEADER true, NULL '');

This tells PostgreSQL to:

Read a pipe-delimited file
Use the header row to match columns
Treat empty strings as NULL

Note: This must be entered as a single line. You will get a confirmation message when the data import succeeds.

Step 5: Verify the Import

The official CFPB download page indicates that the 2024 nationwide file contains approximately 12 million rows. After importing, verify with:

SELECT COUNT(*) FROM lar2024;
SELECT * FROM lar2024 LIMIT 5;

If the row count is substantially less than 12 million, the import did not complete successfully.

Create the Chicago, Illinois Subset

With the full dataset loaded, you can create a focused subset with a CREATE TABLE AS SELECT statement. The query below selects conventional, owner-occupied, site-built single-family loans in Cook County (FIPS 17031), excluding missing or exempt values:

DROP TABLE IF EXISTS public.il2024f1;
CREATE TABLE public.il2024f1 AS
SELECT
 l.activity_year,
 l.state_code,
 l.county_code,
 l.census_tract,
 l.debt_to_income_ratio,
 l.combined_loan_to_value_ratio,
 l.income,
 l.property_value,
 l.loan_amount,
 l.interest_rate,
 l.applicant_ethnicity_1,
 l.applicant_race_1,
 l.denial_reason_1,
 l.rate_spread,
 l.hoepa_status,
 l.lien_status
FROM public.lar2024 l
WHERE
 l.loan_purpose = '1' -- Home purchase
 AND l.action_taken = '1' -- Loan originated
 AND l.loan_type = '1' -- Conventional
 AND l.occupancy_type = '1' -- Owner-occupied
 AND l.state_code = 'IL'
 AND l.county_code = '17031' -- Cook County
 AND l.property_value ~ '^[0-9]+$'
 AND l.income ~ '^[0-9]+$'
 AND l.loan_amount IS NOT NULL
 AND l.loan_amount <> 99999
 AND l.construction_method = '1' -- Site-built
 AND l.total_units IN ('1','2','3','4','');

Prepare for Additional Analysis

Once exported to CSV, I divided the number of Black applicants in each census tract by the total number of applicants in that tract to get the share of Black applicants. I also downloaded Median Household Income data using the tidycensus package in R (ACS 5-year estimates 2018–2023).

The maps below compare neighborhood income levels with the geographic concentration of Black mortgage applicants receiving loans in 2024. Do you observe any patterns?

The spatial overlap — or rather, the striking lack of it — between high-income areas and Black loan recipients is consistent with decades of research documenting structural barriers to homeownership for Black families in Chicago.

Conclusion

The HMDA dataset offers an unparalleled opportunity to examine lending patterns, racial disparities, and neighborhood-level investment across the United States. By importing the 2024 Modified LAR into PostgreSQL, researchers can efficiently manage, filter, and analyze millions of records while linking them to geographic and demographic data.

Next steps may include:

Join with Census shapefiles to visualize patterns by tract
Summarize by loan purpose, race, or geography
Export to PostGIS for spatial queries
Link to neighborhood typologies or redlining indices

Happy Coding!