Insurance regulators are currently engaged in ongoing development of guidance and regulations regarding the utilization of big data, machine learning and artificial intelligence (AI) in the insurance industry. The Colorado Division of Insurance (CO DOI) took the lead in formulating AI regulation when, in July 2021, they enacted Senate Bill 21-169¹ (SB 21-169). The Bill mandates life insurance providers to assess their "external consumer data, information sources, algorithms, and predictive models" to prevent any biased treatment against consumers based on protected characteristics. SB 21-169 dictates that insurers must demonstrate to the CO DOI how they evaluate their data and predictive frameworks and confirm they do not cause unfair discrimination.

On September 28, 2023, CO DOI also released for public commentary a draft regulation titled “Quantitative Testing for Unfairly Discriminatory Outcomes for Algorithms and Predictive Models Used for Life Insurance Underwriting”² (the “Draft”). Should the Draft be adopted, insurers will have to infer the racial background of their prospective life insurance applicants. The process will be used to perform annual quantitative assessments of their data and models that utilize external consumer data and information sources (ECDIS) to detect potential unfair discrimination.

The proposed testing methodology outlined in the Draft identifies specific outcomes as unfairly discriminatory, mandating insurers to promptly take appropriate measures to address such instances. Stakeholders, including the American Academy of Actuaries (AAA), the American Property Casualty Insurance Association (APCIA), and the National Association of Mutual Insurance Companies (NAMIC), have submitted 17 comments³ during the stakeholder feedback period which concluded on October 26, 2023.

The Draft is applicable solely to life insurance companies authorized to conduct business in Colorado. These insurers must have integrated ECDIS, or utilized algorithms and predictive models employing ECDIS, which could also include data or algorithms from third-party entities “acting on behalf of the insurer.” A resulting Bill would make it mandatory for life insurers to furnish a progress report by June 1, 2024, as well as submit a compliance report confirming full adherence by December 1, 2024. Subsequently, such reports would be required on an annual basis.

The aim of the quantitative testing is to assess whether underwriting practices show unfair discrimination in terms of the race and ethnicity of insurance applicants. The Draft requires that race and ethnicity be estimated through the statistical approach known as Bayesian Improved First Name Surname Geocoding (BIFSG) - a methodology crafted by the RAND Corporation. (For additional insights into the BIFSG method, you can explore one of our recent blog posts⁴ written by Gary Wang, FCAS, AAA, CSPA, a senior consulting actuary at Pinnacle.) 

The Draft mandates two types of testing: 1) examining the approval decisions to determine if there's a statistically significant disapproval rate among Hispanic, Black, and API applicants compared to White applicants; and, 2) assessing the premium rates to ascertain statistically significant differences in rates for policies issued to insured Hispanic, Black, and API individuals versus insured White individuals. The evaluation of application approvals utilizes a logistic regression model, in which approval/disapproval represents the binary response variable. Meanwhile, the premium testing involves linear regression, with the premium rate per $1,000 serving as the continuous target variable of the model. To assess the evidence provided by the data in support of statistically significant differences in approval rates or premium rates, the Draft recommends the use of the 5% cut-off point for the probability of the observed data under the assumption of no difference.

If the 5% mark of significance testing has not been attained, additional testing is not required. If the five-percentage point threshold is reached, the next step is determining whether the differences in approval rates are five percentage points or greater than the overall average approval rate, and similarly whether the differences in premium rates are five percentage points or greater than the overall average premium rate. If these differences are within the 5% margin, additional testing is not required.

Provided the differences in approval or premium rates surpass the 5% benchmark, insurers are required to conduct further examination of their external or third-party data and algorithms to identify the specific variable(s) responsible for the observed distinctions. Insurers must employ two separate logistic regression models to assess approval or denial decisions or, alternatively, two linear regression models to evaluate premium rates respectively.

All variables, inclusive of EDCIS and traditional underwriting factors, must be encompassed within the initial model. The second model will incorporate an additional "dummy variable" representing the estimated race and ethnicity, in conjunction with the variables from the first model. Insurers need to compare coefficients from both models for each external data source variable. Should discrepancies emerge, both the variable itself and the algorithm or predictive model utilizing that variable will be deemed unfairly discriminatory. Insurers must promptly initiate corrective actions following the risk management framework rule. Furthermore, additional testing is mandated to validate the efficacy of the implemented remediation.

The CO DOI has undertaken a stakeholder engagement process to help enhance and finalize the Draft. The initial stakeholder meeting occurred on October 19, 2023, during which stakeholders orally conveyed their feedback on the Draft. Several comments from stakeholders prompted CO DOI to consider various aspects of the proposed testing regulation: a) assessing whether the 5% threshold for differences in application approvals and premium rates would accurately identify unfair discrimination; b) evaluating the feasibility of conducting the premium rates test for unfair discrimination, given the numerous variables involved in determining premium rates; and c) deliberating on whether the DOI should establish a regulatory safe haven to shield insurers from liability when they earnestly endeavor to comply with its provisions.

The diagram below provides a visualization of the testing process flow for premium rates.

The AAA was one of the responders, and it provided its comments⁵ on the Draft in a letter on October 26, 2023. The AAA’s primary suggestions include:

1. Use of BIFSG to estimated race and ethnicity.

The inference of race remains an actively evolving field of research. The Draft mandates companies to adopt the BIFSG technique for categorizing policyholders based on their race or ethnicity. Despite the widespread use of the BIFSG method for inferring race and ethnicity, various implementation approaches, such as the selection of Census data (such as 2010 or 2020) and different types of classification methods can yield contradicting conclusions concerning the existence of unfairly discriminatory practices.

2. Testing methods.

While the AAA is supportive of testing methods aimed at preventing unjustly discriminatory practices, they express apprehension regarding the methodologies employed and the potential associated risks and consequences. In Section 8 of the Draft, there's a specified procedure for modifying variables when the Section 6 testing reveals differences in acceptance rates exceeding 5%. From a statistical standpoint, introducing new variables could change the coefficients of variables already in the model, potentially leading to unsatisfactory outcomes in subsequent tests. The AAA urges Colorado to consider the challenges associated with mandating the use of p-values in this regulation.

While the current Draft for testing for unfair discrimination is applicable only to life insurers, it is expected that the CO DOI will propose a similar draft regulation for property and casualty insurers, starting with private passenger auto (PPA) as the first line of business to be subjected to such regulation. Colorado has already requested that auto insurers review the life governance document and provide input on the need for any modification to ensure it is appropriate for PPA. A stakeholder meeting regarding ‘Unfair Discrimination in Insurance Practices” focusing on underwriting practices in PPA insurance already took place on April 6, 2023. Given the varied challenges outlined in the responses and comments to the Draft, it will be interesting to see how the Draft, intended for life insurers, and any future regulation targeting the personal lines, continue to evolve and whether a consensus can be achieved among all stakeholders.

References:

1. https://leg.colorado.gov/sites/default/files/2021a_169_signed.pdf
2. https://drive.google.com/file/d/1BMFuRKbh39Q7YckPqrhrCRuWp29vJ44O/view
3. https://drive.google.com/drive/folders/1g7EqLp2sO_SFQaHtM9kvCJvNM6sZopuG
4. https://www.pinnacleactuaries.com/article/predictive-data-modeling-understanding-value-and-limitations-imputing-race-and-ethnicity

https://www.actuary.org/sites/default/files/2023-10/life-letter-colorado-draft-regulation.pdf=