Tutorials | ISSUES | The Ohio State University

A PostgreSQL Tutorial for Analyzing Racial and Geographic Disparities in Home Mortgage Disclosure Act (HMDA) Data

Wed, 16 Jul 2025 00:00:00 +0000

My lab recently published a paper examining the association between redlining, reinvestment, racial segregation, and firearm-related injury in Chicago, Illinois. You can read the paper here: Redlining, reinvestment, and racial segregation: a Bayesian spatial analysis of mortgage lending trajectories and firearm-related violence.

To develop reverse redlining measures, we relied on data from the Home Mortgage Disclosure Act (HMDA).

If you’re interested in patterns of lending discrimination, the HMDA dataset is an invaluable resource. It contains millions of individual loan application records submitted annually by financial institutions, making it one of the most extensive and most detailed public data sources on mortgage lending in the United States. The 2024 nationwide Modified Loan/Application Register (LAR) alone contains over 12 million records and 85 fields that detail applicant and co-applicant demographics, loan terms, and property characteristics, including race, income, loan amount, interest rate, property value, and loan outcomes.

Each record includes the census tract (effectively a neighborhood) of the loan origination, which allows researchers to situate applicants within their geographic context. This spatial identifier can be linked to other data sources (e.g., Census or ACS data), enabling robust analyses of racial disparities in lending, neighborhood-level disinvestment, and broader patterns of residential inequality.

I highly recommend the book titled “The Color of Law” by Richard Rothstein for anyone interested in understanding the deep-rooted history of discriminatory housing practices in the United States. The book compellingly explains how federal, state, and local policies — across multiple domains — intentionally created and reinforced de jure segregation. Chicago, which remains one of the most highly segregated urban areas in the country, is prominently featured.

Why PostgreSQL?

Due to its sheer volume in both size and number of fields, the HMDA data is not well-suited for analysis using spreadsheets like Excel, particularly when analyzing multiple years or conducting joins. Instead, its relational structure lends itself to the use of a tool like PostgreSQL, an open-source relational database management system (RDBMS) that enables efficient querying, filtering, and aggregation of large datasets.

PostgreSQL is especially well-suited for the HMDA dataset because it can:

Handle millions of rows
Perform complex joins and filters
Scale to support both interactive and batch analysis
Support geospatial extensions like PostGIS for direct spatial querying and linking to geographic units such as census tracts or ZIP codes

If you don’t already have it, download PostgreSQL now.

Motivation

My motivation for this analysis stems from a broader interest in how housing insecurity shapes health outcomes, consistent with a large body of research indicating that where you live significantly impacts your health. In my current work, I used HMDA data to create neighborhood typologies based on the race and income of applicants, and to generate measures of neighborhood change. While the process isn’t particularly difficult, it also isn’t obvious — hence this tutorial.

This walkthrough covers the complete process of downloading, inspecting, and importing the 2024 HMDA Modified LAR dataset into PostgreSQL using PowerShell and psql. It assumes you’re using a Windows machine.

Step 1: Download the Dataset

The dataset is freely available to the public. To download it follow these steps:

Navigate to https://ffiec.cfpb.gov/data-publication/modified-lar/2024
Choose:
- Year: 2024
- Nationwide: Combined file
Download the file: 2024_combined_mlar_header.txt
Save or extract the file to: C:\Users\YOUR_USER_NAME\Downloads\2024_combined_mlar_header\

Multiple years are available on the FFIEC website. You can download historical data to examine lending trends over time.

Step 2: Confirm the File Structure in PowerShell

The file structure of the HMDA dataset is not typical for most users or applications. It is a pipe-delimited text file (.txt), which can make it difficult to read or interpret using standard spreadsheet tools like Excel. The data appears in a long, unformatted stream, separated by vertical bars (|), and without intuitive formatting.

To extract the necessary information and prepare the data for import into PostgreSQL, I recommend using Windows PowerShell to inspect the file structure first. This allows you to confirm key details such as the number of fields (columns), the presence of a header row, and the overall formatting of the data.

Open PowerShell and run the following:

cd "C:\Users\YOUR_USER_NAME\Downloads\2024_combined_mlar_header"

# View the header (column names)
Get-Content .\2024_combined_mlar_header.txt -TotalCount 1

# Count the number of columns (should be 85)
((Get-Content .\2024_combined_mlar_header.txt -TotalCount 1) -split '\|').Count

Step 3: Create the PostgreSQL Table

To create a table called lar2024 in PostgreSQL using SQL, begin by launching your SQL interface. If you’re using a graphical tool like pgAdmin, open the Query Tool by right-clicking on your database and selecting “Query Tool.” If you’re using the psql command-line interface, connect to your target database like this:

psql -U your_username -d your_database_name

Replace the placeholders with your actual credentials.

Once you’re connected to your PostgreSQL database, you’ll use a CREATE TABLE SQL statement to define the structure of the lar2024 table:

DROP TABLE IF EXISTS lar2024;
CREATE TABLE lar2024 (
 activity_year INTEGER,
 lei TEXT,
 loan_type INTEGER,
 loan_purpose INTEGER,
 preapproval INTEGER,
 construction_method INTEGER,
 occupancy_type INTEGER,
 loan_amount BIGINT,
 action_taken INTEGER,
 state_code TEXT,
 county_code TEXT,
 census_tract TEXT,
 applicant_ethnicity_1 INTEGER,
 applicant_ethnicity_2 TEXT,
 applicant_ethnicity_3 TEXT,
 applicant_ethnicity_4 TEXT,
 applicant_ethnicity_5 TEXT,
 co_applicant_ethnicity_1 INTEGER,
 co_applicant_ethnicity_2 TEXT,
 co_applicant_ethnicity_3 TEXT,
 co_applicant_ethnicity_4 TEXT,
 co_applicant_ethnicity_5 TEXT,
 applicant_ethnicity_observed INTEGER,
 co_applicant_ethnicity_observed INTEGER,
 applicant_race_1 INTEGER,
 applicant_race_2 TEXT,
 applicant_race_3 TEXT,
 applicant_race_4 TEXT,
 applicant_race_5 TEXT,
 co_applicant_race_1 INTEGER,
 co_applicant_race_2 TEXT,
 co_applicant_race_3 TEXT,
 co_applicant_race_4 TEXT,
 co_applicant_race_5 TEXT,
 applicant_race_observed INTEGER,
 co_applicant_race_observed INTEGER,
 applicant_sex INTEGER,
 co_applicant_sex INTEGER,
 applicant_sex_observed INTEGER,
 co_applicant_sex_observed INTEGER,
 applicant_age TEXT,
 applicant_age_above_62 TEXT,
 co_applicant_age TEXT,
 co_applicant_age_above_62 TEXT,
 income TEXT,
 purchaser_type INTEGER,
 rate_spread TEXT,
 hoepa_status INTEGER,
 lien_status INTEGER,
 applicant_credit_scoring_model INTEGER,
 co_applicant_credit_scoring_model INTEGER,
 denial_reason_1 TEXT,
 denial_reason_2 TEXT,
 denial_reason_3 TEXT,
 denial_reason_4 TEXT,
 total_loan_costs TEXT,
 total_points_and_fees TEXT,
 origination_charges TEXT,
 discount_points TEXT,
 lender_credits TEXT,
 interest_rate TEXT,
 prepayment_penalty_term TEXT,
 debt_to_income_ratio TEXT,
 combined_loan_to_value_ratio TEXT,
 loan_term TEXT,
 intro_rate_period TEXT,
 balloon_payment INTEGER,
 interest_only_payment INTEGER,
 negative_amortization INTEGER,
 other_non_amortizing_features INTEGER,
 property_value TEXT,
 manufactured_home_secured_property_type INTEGER,
 manufactured_home_land_property_interest INTEGER,
 total_units TEXT,
 multifamily_affordable_units TEXT,
 submission_of_application INTEGER,
 initially_payable_to_institution INTEGER,
 aus_1 INTEGER,
 aus_2 TEXT,
 aus_3 TEXT,
 aus_4 TEXT,
 aus_5 TEXT,
 reverse_mortgage INTEGER,
 open_end_line_of_credit INTEGER,
 business_or_commercial_purpose INTEGER
);

Step 4: Import the Data

Now that the table structure is created, you can import the data. I prefer using the command-line interface, which is called psql in PostgreSQL, to import the data.

In psql, use the following single-line import command:

\copy lar2024 FROM 'C:/Users/YOUR-USER-NAME/Downloads/2024_combined_mlar_header/2024_combined_mlar_header.txt' WITH (FORMAT csv, DELIMITER '|', HEADER true, NULL '');

This tells PostgreSQL to:

Read a pipe-delimited file
Use the header row to match columns
Treat empty strings as NULL

This must be entered as a single line. You will get a notification that the data were successfully imported.

Step 5: Verify the Import

I always like to verify that what I am doing is correct. To verify the number of rows expected after importing the 2024 HMDA Modified Loan/Application Register (LAR) data into PostgreSQL, I consulted the official HMDA data publication website maintained by the Consumer Financial Protection Bureau (CFPB).

After importing the file using the \copy command in psql, run a simple SQL query to verify the total number of records successfully loaded:

SELECT COUNT(*) FROM lar2024;
SELECT * FROM lar2024 LIMIT 5;

Expected row count for the whole nation: ~12 million.

Create the Chicago, Illinois, Subset

I want my analysis to focus on a specific subset of the data (e.g., applicant race, loan originals that are single-family homes, etc., for Chicago). I can now create a sub-query containing a smaller subset of the data, export it to a .csv file, and then import it into R or QGIS.

The simple query I used to select the columns and filter the data to Cook County (which contains Chicago) is shown below:

DROP TABLE IF EXISTS public.il2024f1;
CREATE TABLE public.il2024f1 AS
SELECT
 l.activity_year,
 l.state_code,
 l.county_code,
 l.census_tract,
 l.debt_to_income_ratio,
 l.combined_loan_to_value_ratio,
 l.income,
 l.property_value,
 l.loan_amount,
 l.interest_rate,
 l.applicant_ethnicity_1,
 l.applicant_race_1,
 l.denial_reason_1,
 l.rate_spread,
 l.hoepa_status,
 l.lien_status
FROM public.lar2024 l
WHERE
 l.loan_purpose = '1'
 AND l.action_taken = '1'
 AND l.loan_type = '1'
 AND l.occupancy_type = '1'
 AND l.state_code = 'IL'
 AND l.county_code = '17031'
 -- Only accept property values that are purely numeric
 AND l.property_value ~ '^[0-9]+$'
 AND l.income ~ '^[0-9]+$'
 AND l.loan_amount IS NOT NULL
 AND l.loan_amount <> 99999
 -- Equivalent to derived_dwelling_category = 'Single Family'
 AND l.construction_method = '1'
 AND l.total_units IN ('1', '2', '3', '4', '')
;

This query selects conventional, owner-occupied, site-built single-family loans in Cook County, Illinois, excluding missing or exempt values.

Prepare File for Additional Analysis

However, we first need to prepare the file for further analysis. I will leave that to you since you can now do this easily in Excel, etc. Below, I divided the number of Black applicants in each census tract by the total number of applicants in that tract to get the share of Black applicants. I also downloaded Median Income data using the tidycensus package in R. This allows us to compare the average median household income 5-year estimates with the share of Black applicants receiving a loan in each neighborhood.

Do you observe any patterns? I do.

Conclusion

The HMDA dataset offers an unparalleled opportunity to examine lending patterns, racial disparities, and neighborhood-level investment across the United States. By importing the 2024 Modified LAR data into PostgreSQL, researchers can efficiently manage, filter, and analyze millions of loan records while linking them to geographic and demographic data.

Next Steps

Join with Census shapefiles to visualize by tract
Summarize by loan purpose, race, or geography
Export to PostGIS for spatial queries

Happy Coding!

How to Analyze Caselaw Using CourtListener's API in R

Fri, 01 Aug 2025 00:00:00 +0000

I previously showed how to pull ‘Indian’ Child Welfare Act (ICWA) cases using the Caselaw Access Project (Harvard Law). That API has since changed. This post updates the workflow using the Free Law Project’s CourtListener, which now partners with Harvard to make this data freely available.

Finding the Cases

Start at the CourtListener website and search for “Indian Child Welfare Act” (or “icwa”).

You’ll see a page of opinion results.

Filtering by Jurisdiction

Click Select Jurisdiction → State. Clear all, then select:

Ohio Supreme Court
Ohio Court of Appeals
Ohio Court of Claims

This gives you a quick check on the expected volume so you can validate the code’s output below.

The site reported 36 cases from Ohio when this tutorial was written. Note that the exact number may change over time as new opinions are added or metadata is corrected.

Getting the Data into R

The Search API returns clean JSON results by querying for opinions that contain the exact phrase “indian child welfare act”, restricting to published opinions, ordering by relevance, and (optionally) limiting the results to three Ohio courts.

Required Libraries

library(httr2)
library(purrr)
library(dplyr)
library(tibble)
library(tidyr)
library(stringr)

API Configuration

SEARCH_BASE <- "https://www.courtlistener.com/api/rest/v4/search/"
UA <- "R/httr2 (your-email-address) – ICWA metadata collection"

query_params <- list(
 q = '"indian child welfare act"',
 type = "o",
 order_by = "score desc",
 stat_Published = "on",
 court = "ohio ohioctapp ohioctcl" # optional: specify the court name
)

Helper Functions

pull_chr <- function(x, ...) {
 v <- purrr::pluck(x, ..., .default = NA)
 if (is.null(v) || length(v) == 0) return(NA_character_)
 as.character(v)
}

pull_num <- function(x, ...) {
 v <- purrr::pluck(x, ..., .default = NA_real_)
 suppressWarnings(as.numeric(v))
}

get_json <- function(url) {
 resp <- request(url) |>
 req_user_agent(UA) |>
 req_url_query() |>
 req_perform()
 resp_check_status(resp)
 resp_body_json(resp, simplifyVector = FALSE)
}

Fetching Search Results

fetch_search_pages <- function() {
 req <- request(SEARCH_BASE) |>
 req_user_agent(UA) |>
 req_url_query(!!!query_params)

 dat <- req |>
 req_perform() |>
 (\(r){
 resp_check_status(r)
 resp_body_json(r, simplifyVector=FALSE)
 })()

 pages <- list(dat)

 while (!is.null(dat$`next`)) {
 dat <- request(dat$`next`) |>
 req_user_agent(UA) |>
 req_perform() |>
 (\(r){
 resp_check_status(r)
 resp_body_json(r, simplifyVector=FALSE)
 })()
 pages <- append(pages, list(dat))
 }

 pages
}

pages <- fetch_search_pages()
results <- pages |> map("results") |> flatten()

results is a list of per-case records from the API. Each element includes:

Case name
Filing date
Court identifiers
Citation strings
Public URL where the caselaw text is available
Other relevant metadata

Creating a Dataframe

For Ohio, this produced ~35 unique case names versus 36 displayed on the site. In comparing the case_name variable to the search results, you can reconcile the discrepancy.

Minor discrepancies happen because search hits can include multiple opinions or siblings per matter, and because the index is updated over time.

Downloading Case Text

Each opinion has an API endpoint with plain text and HTML fields. A small function can loop over the URL field, request the text, and save one .txt file per opinion. A short pause between requests is included and retried if necessary.

# Example function to download case text
download_case_text <- function(case_url, output_dir) {
 # Add rate limiting
 Sys.sleep(1)

 # Request the case data
 case_data <- get_json(case_url)

 # Extract plain text
 case_text <- pull_chr(case_data, "plain_text")

 # Save to file
 file_name <- paste0(output_dir, "/",
 gsub("[^A-Za-z0-9]", "_", pull_chr(case_data, "case_name")),
 ".txt")
 writeLines(case_text, file_name)
}

Remember to be considerate of rate limits and to include a User Agent string with your contact information.

Visualizing the Results

You can use ggplot2 to visualize the results, for example:

Filings by Year

library(ggplot2)

# Assuming you have a dataframe with filing dates
cases_df %>%
 mutate(year = lubridate::year(date_filed)) %>%
 count(year) %>%
 ggplot(aes(x = year, y = n)) +
 geom_col(fill = "#bb0000") +
 labs(
 title = "ICWA Cases Over Time",
 x = "Year",
 y = "Number of Cases"
 ) +
 theme_minimal()

Volume by Court

# Top 25 jurisdictions
cases_df %>%
 count(court, sort = TRUE) %>%
 head(25) %>%
 ggplot(aes(x = reorder(court, n), y = n)) +
 geom_col(fill = "#bb0000") +
 coord_flip() +
 labs(
 title = "Top 25 Jurisdictions for ICWA Cases",
 x = "Court",
 y = "Number of Cases"
 ) +
 theme_minimal()

Next Steps

After downloading all data, including caselaw text and metadata, you can:

Use Natural Language Processing to examine sentiment or common words
Perform topic modeling to explore latent themes in ICWA cases
Map filings by jurisdiction
Integrate the data with other sources (e.g., demographic data, child welfare statistics)

Code Repository

The whole workflow is available on our GitHub repository.

Happy Coding!

SDOH Example

Wed, 10 Sep 2025 00:00:00 +0000

This is the SDOH example page.

Mapping the SDoH

Goal: Build a tract-level map of food insecurity and housing insecurity in Franklin County, Ohio using the CDC PLACES dataset, then export an sf/GeoJSON layer for reuse on the ISSUES Lab site.

Pipeline overview

Query PLACES (Socrata API) for two indicators filtered to Franklin County via FIPS.
Coerce indicators to numeric.
Download census tract polygons (TIGER/Line via tigris) and left_join on tract GEOID.
Mean-impute missing values for food insecurity (for quick visualization only).
Plot a fast choropleth and write a GeoJSON file.

ℹ️ The PLACES “GIS-friendly” table is attributes only. Socrata “location” endpoints typically expose centroids, not tract polygons—so we fetch attributes and join to TIGER/Line shapes.

Requirements (install once)

install.packages(c("RSocrata", "dplyr", "tigris", "sf"))

## Requirements (install once)

```r
install.packages(c("RSocrata", "dplyr", "tigris", "sf"))

## Load each session

- **RSocrata** — read data from data.cdc.gov (Socrata)
- **dplyr** — data manipulation
- **tigris** — tract geometries (TIGER/Line)
- **sf** — spatial classes, I/O (GeoJSON)
- **utils** — base R utilities

### Optional nicety

 options(tigris_use_cache = TRUE) # cache shapefiles locally between runs

#########################################
# Downloading PLACES data to #
# map the social determinants of health #
# for food and housing insecurity for #
# the ISSUES lab next paper :) #
# September 9, 2025 #
# Example #
#########################################

# install.packages("RSocrata") etc once
library(RSocrata) # read online data
library(dplyr) # manipulate dataframes
library(utils) # utilities

base <- "https://data.cdc.gov/resource/yjkw-uj5s.json"

# Note there is a geojson endpoint however the geometry is the centroid
# so it's better to download the non-spatial data and then get the census tract,
# and then merge the data to the census tract polygons.
fields <- c(
 "stateabbr","countyname","countyfips","tractfips",
 "foodinsecu_crudeprev","housinsecu_crudeprev"
)

select_q <- paste(fields, collapse = ",")

# Filter to Franklin County, Ohio (prefer FIPS for precision)
where_q <- "stateabbr = 'OH' AND countyfips = '39049'"

(url <- paste0(
 base, "?$select=", select_q,
 "&$where=", URLencode(where_q, reserved = TRUE)
))

df_franklin <- read.socrata(url) %>%
 mutate(
 foodinsecu_crudeprev = as.numeric(foodinsecu_crudeprev),
 housinsecu_crudeprev = as.numeric(housinsecu_crudeprev)
 )

glimpse(df_franklin)

library(tigris) # connect to census mapping files
options(tigris_use_cache = TRUE)

# We have census tract level data for Ohio (state FIPS = 39),
# Franklin County (county FIPS = 049). Using 2022 TIGER (2020 vintage).
oh_ct <- tracts(state = "Ohio", county = "Franklin", year = 2022, class = "sf")

df_franklin <- df_franklin %>% dplyr::rename(GEOID = tractfips)
oh_ct <- oh_ct %>% dplyr::left_join(df_franklin, by = "GEOID")

# Quick map (before imputation)
plot(oh_ct["foodinsecu_crudeprev"], main = "Food Insecurity (%) — Franklin County, OH")

# Mean-impute missing (visualization only)
m_food <- mean(oh_ct$foodinsecu_crudeprev, na.rm = TRUE)
oh_ct$foodinsecu_crudeprev[is.na(oh_ct$foodinsecu_crudeprev)] <- m_food

# Plot (after imputation)
plot(oh_ct["foodinsecu_crudeprev"], main = "Food Insecurity (%) — Franklin County, OH")

library(sf)
st_write(
 oh_ct,
 "C:/Users/barboza-salerno.1/Documents/lab/content/resources/datasets/ohio-franklin-sdoh.geojson",
 delete_dsn = TRUE
)

Tips & adaptations

Change geography: Edit where_q to your target stateabbr and countyfips (or drop the filter for entire states/US).
More indicators: Add PLACES columns to fields (e.g., lacktrpt_crudeprev, emotionspt_crudeprev).
Prettier maps: Use ggplot2 + geom_sf() with scale_fill_viridis_c() for publication graphics.
Missing data: Mean-imputation here is for a quick map; use a different imputation method.
Export path: Update st_write() to download the geojson file to your hard drive and map it with QGIS.

Downloads

R script is here: Download the R script
Mapping file is here: Download the geojson file for mapping

Spatial Access Analysis

Mon, 01 Jan 0001 00:00:00 +0000

Tutorials | ISSUES | The Ohio State University

A PostgreSQL Tutorial for Analyzing Racial and Geographic Disparities in Home Mortgage Disclosure Act (HMDA) Data

Why PostgreSQL?

Motivation

Step 1: Download the Dataset

Step 2: Confirm the File Structure in PowerShell

Step 3: Create the PostgreSQL Table

Step 4: Import the Data

Step 5: Verify the Import

Create the Chicago, Illinois, Subset

Prepare File for Additional Analysis

Conclusion

Next Steps

Related Resources

How to Analyze Caselaw Using CourtListener's API in R

Finding the Cases

Filtering by Jurisdiction

Getting the Data into R

Required Libraries

API Configuration

Helper Functions

Fetching Search Results

Creating a Dataframe

Downloading Case Text

Visualizing the Results

Filings by Year

Volume by Court

Next Steps

Code Repository

Related Resources

SDOH Example

Mapping the SDoH

Requirements (install once)

Tips & adaptations

Downloads

Spatial Access Analysis