Log in / My SOA

Evaluating Representation in Health Claims Data

Background and Purpose

Artificial intelligence (AI) models trained or informed on healthcare claims data are increasingly being used for various healthcare-related purposes. These models may be used by providers when delivering health care and by insurers when designing care management and disease management programs. Such AI models have the potential to enable care management or care delivery plans that are more comprehensive, more personalized, and more cost-effective than those that result from traditional approaches. However, if the healthcare claims data is not representative of the covered population (such as the provider’s patient base or the insurer’s members), output from the models may be poorly informed or misinformed. If the model output is relied on to make healthcare decisions with limited review, it could have a detrimental impact on the delivery of care and, ultimately, on morbidity and mortality.

Research Objective

The Society of Actuaries Research Institute (SOA) is seeking researchers to develop and document a:

  • Methodology and/or metrics for evaluating whether healthcare claims data that may be used to train or inform an AI model is representative of the covered population; and
  • A framework for evaluating whether healthcare data used to train and inform AI models is representative of the covered population; and
  • A quantitative or qualitative framework for evaluating the impact of any underrepresentation or overrepresentation that may be detected.

Please reflect in your proposal the difference in effort, timeline, and cost of including or excluding the third objective.

Proposal Requirements

To facilitate the evaluation of proposals, the following information should be submitted:

  • Resumes of the researcher(s), including any graduate student(s) expected to participate, indicating how their background, education and experience bear on their qualifications to undertake the research. If more than one researcher is involved, a single individual should be designated as the lead researcher and primary contact. The person submitting the proposal must be authorized to speak on behalf of all the researchers as well as for the firm or institution on whose behalf the proposal is submitted.
  • An outline of the approach to be used (e.g. literature search, model, etc.), emphasizing issues that require special consideration. Details should be given regarding the techniques to be used, collateral material to be consulted, and possible limitations of the analysis.
  • A description of the expected deliverables and any supporting data, tools or other resources.
  • Cost estimates for the research, including computer time, salaries, report preparation, material costs, etc. Such estimates can be in the form of hourly rates, but in such cases, time estimates should also be included. Any guarantees as to total cost should be given and will be considered in the evaluation of the proposal. While cost will be a factor in the evaluation of the proposal, it will not necessarily be the decisive factor.

Please note that as a policy, the SOA Research Institute generally does not provide funding to cover academic institution overhead expenses.

As a guide for developing the project budget, please review the Historical Project Cost Guide (see Appendix).

  • A schedule for completion of the research, identifying key dates or time frames for research completion and report submissions. The SOA is interested in completing this project in a timely manner. Suggestions in the proposal for ensuring timely delivery, such as fee adjustments, are encouraged.
  • Other related factors that give evidence of a proposer's capabilities to perform in a superior fashion should be detailed.

Selection Process

The SOA will appoint a Project Oversight Group (POG) to oversee the project. The POG is responsible for recommending the proposal to be funded. Input from other knowledgeable individuals also may be sought, but the POG will make the final recommendation, subject to SOA leadership approval. An SOA staff research actuary will provide staff actuarial support.

Questions

Any questions regarding this RFP should be directed to research-dei@soa.org with subject line: Evaluating Potential Underrepresentation in Health Claims Data.

Notification of Intent to Submit Proposal

If you intend to submit a proposal, please email written notification by Friday, May 9, 2025, to research-dei@soa.org with subject line: Evaluating Potential Underrepresentation in Health Claims Data.

Submission of Proposal

Please email your proposal to research-dei@soa.org with subject line: Evaluating Potential Underrepresentation in Health Claims Data; proposals must be received no later than Friday, May 23, 2025. It is anticipated that all proposers will be informed of the status of their proposal by the end of July 2025.

Conditions

The selection of a proposal is conditioned upon and not considered final until a Letter of Agreement is executed by both the Society of Actuaries Research Institute and the researcher.

The Society of Actuaries Research Institute reserves the right to not award a contract for this research. Reasons for not awarding a contract could include, but are not limited to, a lack of acceptable proposals or a finding that insufficient funds are available. The Society of Actuaries Research Institute also reserves the right to redirect the project as is deemed advisable.

The Society of Actuaries Research Institute plans to hold the copyright to the research and to publish the results with appropriate credit given to the researcher(s).

The Society of Actuaries Research Institute may choose to seek public exposure or media attention for the research. By submitting a proposal, you agree to cooperate with the [Society of Actuaries/sponsoring entity] in publicizing or promoting the research and responding to media requests.

The Society of Actuaries may also choose to market and promote the research to members, candidates and other interested parties. You agree to perform promotional communication requested by the Society of Actuaries Research Institute, which may include, but is not limited to, leading a webcast on the research, presenting the research at an SOA meeting, and/or writing an article on the research for an SOA newsletter.

Conflict of Interest

You agree to disclose any of your material business, financial and organizational interests and affiliations which are or may be construed to be reasonably related to the interest, activities and programs of the Society of Actuaries Research Institute.

Appendix

The cost ranges below are intended as a guide for budgeting project costs for proposals in response to SOA Research Institute Request for Proposals (RFP). Please note these figures span the 33rd to 66th percentiles for all projects as well as projects that involve a specific approach (lit review, survey, etc.). They are based on historical costs over several recent years. Expected costs for some RFPs may fall outside these ranges depending on the nature of the work and resources required for completion.

All Contracted Projects

This category includes all contracted projects that the Institute has undertaken within the last several years.

The 33rd-66th percentile project costs range is $25,000 - $50,000.

Literature Reviews

This category includes projects that involved only a literature review or the cost for the portion of a larger project that included a literature review.

The 33rd-66th percentile project costs range is $15,000 - $20,000.

Surveys

This category includes all projects that had a survey as their primary component.

The 33rd-66th percentile project costs range is $28,000 - $55,000.