Leveraging quality control sampling for your business

This is the first article in a three-part series. For more, read "Using data to manage repurchase risk and loan quality," (Part II of III) and "A primer on lifetime of loss reserving: Preparing for FASB" (Part III of III).

Freddie Mac and Fannie Mae (the GSEs) issued guidance beginning in September 2012 concerning changes in their respective representation and warranty framework.¹ The changes, effective for loans acquired by the GSEs on or after January 1, 2013, require lenders to report defects on various samples of loans delivered to the GSEs. These reports should be leveraged by lenders to monitor and mitigate their own risk of future repurchases.

Understanding sampling

Sampling is a process where users select a subset of observations (the “sample”) from all observations (the “population”) to draw conclusions about the population. For example, assume you make pizzas and want to know what percent of the U.S. population prefers pepperoni pizza over cheese pizza. It would be nearly impossible to ask every person in the U.S. which type of pizza they prefer; however, we could ask a smaller subset of the population to draw conclusions about the total. The more people we ask, the more confidence we have in the results. If we ask enough people, we could assume that the rate of people who prefer pepperoni pizza over cheese pizza in our sample represents the entire population.

Understanding the GSE samples

The GSEs, through self-assessments performed by mortgage sellers, use sampling to monitor the quality of loans purchased. Specifically, the GSEs require sellers to perform self-assessments on defects for loans purchased by the GSEs. The GSEs define three types of required samples to monitor defect rates:

• Random sampling
• Discretionary sampling
• Targeted sampling

Note, Freddie Mac has stated that if a mortgage is excluded from the sampling process it is not eligible for sale to Freddie Mac.

Random sampling

Random sampling is a type of sampling where the user sets the desired number of loans to be selected for review. One observation out of the entire population is randomly selected, usually using a random number generator, until the sample size is full. The purpose of random sampling is to develop a sample that is representative of the population; by randomly selecting a loan from the population, we expect, given a large enough sample size, any analysis on the sample is representative of the population.

Over each 12-month period, the random sample must include the full scope of:

• Product lines
• States of operation
• Branch offices
• Third party originators
• Loans with high risk characteristics

The GSEs require that at least 10% of the loans delivered to the GSEs are subject to random sampling.

For sellers with annual production in excess of 5,000 home mortgages per year, the seller may replace the 10% rule with a statistical sampling methodology that ensures a 95% confidence level with a 2% annual margin of error. Similarly, lenders who originate and/or underwrite more than 3,500 FHA loans also have the option of replacing the 10% rule with the 95%/2% statistical random sampling.

Discretionary sampling

Discretionary sampling is a non-random sampling process where a seller selects specific loans from their portfolio for review. Guidance from the GSEs state discretionary sampling should be used in the following instances:

• Review of loan production from a new employee, branch, or third party originator (TPO)
• Validation of the standards underlying new products
• Requests from the GSEs

Targeted sampling

Targeted sampling is a second type of non-random sampling process where an originator selects specific loans from their portfolio for review. Guidance from the GSEs and HUD state that loans that become 60 or more days delinquent in the first six months after the note date, defined as early payment defaults (EPDs), must be reviewed under a targeted sampling framework.

Leveraging sampling for your business

The risk of a repurchase is a significant financial risk associated with poor quality control on loans sold to the GSEs. The best way to efficiently protect oneself from that risk is to identify the sources of repurchase risk prior to GSE involvement. Mortgage originators can leverage their self-assessment reports in order to minimize this risk.

Random sampling

A quality control review of the random sample results in a loan-level dataset should flag each reviewed loan as having a defect or not.² Sellers can track and leverage this data to improve the quality of their loan production. First, sellers should track their overall defect rate on a periodic basis (e.g., monthly or quarterly) and monitor trends in their defect rate. Sellers should set an acceptable defect rate for their production and monitor actual defect rates to determine whether they’re above or below the acceptable range on an ongoing basis.

Sellers can analyze the data collected through these reviews to identify the drivers of defects. The data can be processed to:

1. Identify certain loan characteristics that are associated with higher defect rates

2. Identify the cause of the defect (e.g., appraisal, missing data, insufficient MI coverage, etc.)

3. Monitor the quality of loans delivered through TPOs or other channels

Such data analytics may help the seller identify process-driven risks that result in more defects (and ultimately repurchases) compared to other loans. For example, self-employed borrowers may have higher defect rates related to income documentation. A cost-benefit analysis must be made by the seller to correct the process or add pre-funding quality control reviews to loans with “higher risk” characteristics.³ In a second article to this series, we will expand upon ways to leverage the data collected from post-funding random sample quality control reviews.

Discretionary sampling

For discretionary sampling, annual quality control reviews on third party originators are particularly important. The seller does not have direct control over the third party’s underlying origination processes, but is at risk for repurchase if there is a demand (although this is typically passed through). An annual review of each TPO provides the seller with data to monitor the quality of production from external sources and benchmark TPOs against one another.

If a seller does identify a TPO with a consistently higher defect rate relative to peers, this type of information and benchmarking could help the lender begin a conversation with the TPO on potential remedies to correct the higher defect rate.

Targeted sampling

Targeted sampling can be used by sellers to monitor defect rate trends on “higher risk” loans. For example, assume a seller identified a particular TPO as delivering loans with higher defect rates compared to peers. The seller could over-sample loans from this TPO on a monthly basis in order to monitor the defect rate trend for loans delivered from this originator. If defect rates don’t improve over the near term, management could take corrective action such as refusing to fund or purchase loans from that TPO.

Additionally, sellers could rank loans according to the characteristics of the loans and perform targeted sampling on higher-risk loans on a pre-funding basis in order to minimize the risk of future repurchase requests for high-risk loans. A study by LoanLogics found that approximately 42% of all post-funding defects could have been identified and potentially corrected through a pre-funding audit.⁴

The GSEs perform targeted sampling based on two criteria:

• Individual loan risk
• Relative defect rates between lenders

The GSEs sample a larger percentage of loans from lenders with higher historical defect rates relative to peers.

By leveraging existing quality control sampling efforts, sellers can monitor and manage their defect risk through the use of data collected by processes already in place.

Footnotes

¹Fannie Mae’s Selling Guide Announcement 2012-08, Lender Letter 2012-05 and Freddie Mac’s Quality Control Best Practices (FreddieMac.com/singlefamily/quality_control.html)

²Such reviews also classify defects by the severity of defect such as: minor or material.

³Fannie Mae’s statistics show that 12 of the 20 top Fannie Mae lenders were able to reduce their defect rate to less than 1% by implementing a pre-funding review program.

⁴The study used a sampling of post-closing quality control reviews on loans closed in 2012 and 2013.