Patients Rising Now is welcomes the opportunity to provide recommendations and comments about the pros and cons of ICER’s August 21st document outlining proposed changes to its Value Assessment Framework’s processes and methodology. As advocates for patients with life-threatening and life-altering conditions, we are making suggestions about opportunities for implementing and improving ICER’s processes and activities to better consider the needs of those patients.
Our comments are organized into the following sections:
- Patients’ Perspectives and Concerns
- Methodologies and Processes
- Health System Improvement and Policymakers’ Perspectives
- QALYs and Other Analytical Issues
- Additional Questions
Patients’ Perspectives and Concerns:
Summarizing the Patient Perspective
we have consistently stated, patients’ perspectives, concerns and viewpoints
are not adequately included in ICER’s methodology and overall activities. ICER
clearly needs to do more work on receiving and integrating input from
“patients, families, and patient organizations, as well as relevant sources of
patient-generated evidence.”[i] We believe that including a “Patient
Perspectives” chapter in ICER reports is a needed and helpful improvement; however,
we are concerned that ICER will only summarize relevant sources
rather than extensively present, discuss, and incorporate patients’
perspectives into its report, analytics, conclusions and metrics – leading to
ICER continuing to undervalue patients’ real concerns. As has been seen, such
incomplete evaluations can empower payers and others to reflexively deny
coverage to patients without organizations doing their own analyses or directly
obtaining patient input. To provide a more complete evaluation, we believe ICER
should be more explicit about how it will decide what are “relevant sources of
patient-generated evidence,” and how it will “summarize” that information in
Along those same
lines, we are also concerned about how ICER will go about obtaining those
perspectives. Specifically, ICER states in the document that it already seeks
“patient input through individual patient interviews, focus groups, partnering
with patient organizations to conduct surveys, and by requesting existing
resources from patient groups….. and will continue to do so,”[ii]
but in the past we have noted ICER’s inadequate attempt at a focus group that
included only three individuals.[iii]
A focus group must be robust enough to capture useful information, and a group
of three individuals simply isn’t sufficient. Therefore, we urge ICER to be
thoughtful in how it constructs and conducts focus groups, and would suggest it
contract with organizations experienced in conducting such research rather than
doing it themselves. We ask that you
please provide additional details on
how ICER will develop focus groups and related activities, how it will
incorporate information obtained from such activities into its analyses and
reports, the steps it will take to partner with patient and clinician
organizations for such activities, and its plans for retaining experts in
conducting such research to gather and analyze that information.
Real World Evidence
to use real world evidence (RWE) have historically been insufficient, so we are
encouraged that ICER will be seeking to use more RWE, and that ICER will be
using RWE as a companion to traditional research information since “randomized
controlled clinical trials have their own limitations and are often inadequate
to address all questions relevant to assessments of comparative clinical
Further, we are encouraged by ICER’s statement about its “commitment to explore forming collaborations with organizations to leverage these kinds of [real world] data for new analyses.”[v] We believe that ICER should work collaboratively with the most insightful experts about each of the disease areas in which it decides to work: clinicians who work with people who have those conditions, and the people who are actually living with those conditions.
proposing “to create a series of webinars that will describe the principles of
health technology assessment and economic modeling for a lay audience.”[vi]
Since this is part of ICER’s “commitment to facilitate effective stakeholder
engagement,” Patients Rising Now is uniquely qualified to
collaborate on such an undertaking, given our focus on providing
patients with such information in ways
that are useful to them. Patients Rising
Now would be very happy
to work with ICER to develop, produce and distribute such seminars, and we look forward to ICER’s feedback
on this partnership offer.
It appears that ICER is proposing to no longer examine treatments for ultra-rare conditions using $500,000/QALY as the upper limit for its cost-effectiveness threshold.[vii] We are very concerned that this could significantly reduce the implied incentives for developing new treatments for ultra-rare and orphan conditions. ICER should recognize the need for individualized approaches for analysis for rare and ultra-rare populations – including cost-effectiveness and cost thresholds. Development costs for rare and ultra-rare diseases are not less because they treat fewer people. We believe the framework update should reconsider these equity concerns. The small populations of rare and ultra-rare diseases is precisely why they need to be assessed with more consideration than people living with cardiovascular disease or diabetes.
Methodologies and Processes
are encouraged that ICER has recognized the need to update reports and note
ICER’s proposal to potentially update reports a year after they are finalized.
However, new discoveries and additional information about how to diagnose and
treat serious health conditions do not follow a yearly cycle. The FDA and other clinically focused
organizations (including the NIH and the CDC) release and publicize new
information to guide clinical care, self-care, and policy decisions as soon as
they can by updating their documents, websites and other vehicles for
communicating critical new health care knowledge to clinicians, patients and
policymakers. This contrasts with Medicare coverage update procedures and
timelines, and how most private sector companies update premiums and benefits
annually for each new plan year. ICER’s proposed arbitrary one-year anniversary
update serves neither purpose and is extremely problematic.
If ICER’s goal is to provide information for U.S. insurance companies to make coverage and payment decisions, then report updates should be made in the middle of each year so plans can integrate that information into their internal health technology assessment process for their next plan year’s offerings. However, if ICER’s goal is to provide information for policymakers, clinicians and patients, then report updates should be made whenever there is new information related to the benefits, risks, and use of specific treatment and diagnostic options. While NICE in the UK serves a very different function from that of ICER because it is directly connected to a health care payment and delivery system, we note that NICE has a very specific standard for updating its reports based upon new evidence: “Once published, all NICE guidance is regularly checked, and updated in the light of new evidence or intelligence if necessary.”[viii] We can see no reasonable rationale to explain how the 1-year anniversary updating policy would benefit specific stakeholders, and urge ongoing real-time updates.
Per-patient Budget Impact
proposal to “present a cumulative per-patient potential budget impact for each
year over the five-year time horizon”[ix]
is an interesting step toward more granular data analysis. However, we strongly suggest that ICER also
present information and analyses of patients in different types of insurance
coverage situations, e.g., Medicare versus private high-deductible insurance versus
ACA Marketplace Silver benchmark plans.
Health System Improvement and Policymakers’ Perspectives
“Real World” Health Care Delivery and Finance
We believe it is
extremely important for ICER as an organization to understand, communicate, and
incorporate into its work the multifaceted nature of health care delivery and
financing in the U.S. Health care
delivery and finance is complicated both by the patchwork nature of the
public-private system and the independent authorities of the 50 states, five
territories and the District of Columbia in regulating various aspects of
health care within their borders and as governed by their own constitutions,
laws and regulations. Anyone seeking to improve health care in the U.S. and
create a “more
effective, efficient, and just health care system”[x] must work within a reality-based
framework; otherwise, those efforts will not translate to the real world.
We are concerned that ICER’s thinking ignores the U.S. health financing structures and public and private expenditure decision processes. For example, with this statement: “Societal resources may be drawn into health spending from other domains of social spending that are much more important to people with lower incomes (such as public education). Some people with lower incomes are likely to be forced out of insurance markets all together,”[xi] ICER ignores the ACA’s low-income subsidies and Medicaid expansion. We are concerned, therefore, that ICER’s value framework will not be reality-based, and therefore will not be a reliable measure of the real world.
Inaccurate or Incomplete Understanding of Current Health Care Landscape
We found it
refreshing that ICER noted one of the unusual aspects of health care in the
U.S.: “estimates of demand based on
current funding may be distorted because health insurance is a tax-credited
employment benefit, meaning that health insurance coverage decisions do not
necessarily match population preferences.”[xii]
However, we are troubled that ICER appears to misunderstand the details of the
situation. The U.S. method for financially incentivizing health insurance for
individuals is primarily a tax deduction
rather than a tax credit; and the
tax deduction is only available within the realm of employer-provided health benefits
and the self-employed who show a profit. Further, employed individuals who
purchase their own health insurance (i.e., people who work for small employers)
and Medicare beneficiaries (who pay Medicare Part B and D premiums) only get a
tax deduction for total health
expenses (including premiums) that exceed 7.5% of their Adjusted Gross Income.[xiii]
On a related front, we are encouraged that ICER recognizes that the U.S. health care system is composed of multiple payers, but again, we are concerned ICER misunderstands the roles and responsibilities of policymakers as expressed in this statement: “While ICER acknowledges that different payer types may have different perspectives, we believe it is important for policymakers not to view health investment as less worthwhile if the return on investment is realized by a different (type of) payer. ICER’s economic evaluations will therefore continue to be conducted using a broad health care sector perspective, with a societal perspective as a scenario analysis. ICER may consider payer-specific analyses where considered particularly relevant and when data are available.”[xiv] This myopic and inaccurate perspective on how policymakers operate and the kind of information they need to enact good policy is alarming.
We are similarly perplexed by ICER’s misrepresentation of health care decision makers with this statement: “Population-level decisions and policies have always been made by life science companies, insurers, and clinical organizations looking at evidence in the same general way.”[xv] We are aware that companies of all sorts look at evidence; that is not in dispute. However, there are significant reasons that health care connected companies (life sciences, clinical, and insurers) do not look at overall population data and evidence “in the same general way,” but rather look at data and evidence that is relevant to their specific groups and geographic locations of interest. For example, the Veterans Administration does not look at children, while Medicaid programs clearly are concerned about populations of children. The Maine Medicaid program is not concerned about Dengue fever or Chagas, but the Texas Medicaid program might be concerned about such infectious diseases.
Similarly, a life science company developing a new treatment is concerned about the groups in the U.S. covered by different insurance programs, such as Medicaid, Medicare, private insurance, and so forth. That data is very important as they model how a new treatment might be used and how much revenue they might receive from the new treatment.
During the September 4th webinar, ICER’s President stated that “Our value framework is for the population. It’s really meant to reflect…. Yes, I mean it has the patient perspective built in, but it’s built in in a way that will in a sense support a population framework for policy making because we have one price across the population. You know, we can try to find subpopulations in which there might be a better value-based price, but there’s probably going to be one price. There’s going to be one coverage policy unless we can find good subpopulation data” (emphasis added).[xvi] We note that if this was the case, then the current controversy over surprise billing would not be happening, since those uniform prices would be known and all payers would already be paying the same for health care products and services.
Despite what ICER
seems to think, there is not one price
for medicines in the U.S., and each insurance program will be providing
different revenue amounts per patient, and the total amount from each type of
payer will depend on the number of people covered by each insurance program –
some treatments are used primarily by children with Medicaid, others by the
elderly with Medicaid, and so forth. We
are extremely alarmed at this fundamental mischaracterization of drug pricing
in the U.S., and urge ICER to adopt a more realistic approach to the issue
QALYs and Other Analytical Issues
We are encouraged that ICER recognizes that “Ethicists, health economists, and health technology assessment groups have long recognized that pure QALY maximization does not incorporate all the values that societies wish to consider when making prioritization decisions for health care spending.”[xvii] However, that statement belies the fact that ICER still uses QALYs as a central factor in its analyses, and is proposing to keep QALYs in a prominent position with its overall value framework process into 2020 and beyond.
Mischaracterization of QALY
describes the QALY as “the gold standard for measuring how well a medical
treatment improves and lengthens patients’ lives,”[xviii]
but that assertion is questionable at best.
While the QALY may have “served as a fundamental component of
cost-effectiveness analyses in the US and around the world for more than 30
its use as an analytical tool does not automatically make QALY appropriate or
beneficial as a decision guide for coverage or reimbursement. We urge ICER use rhetoric that more
appropriately describes the QALY, and also improve its description of what ICER’s
overall work and products actually are.
Additional Assessment Tools
has stated that it will “focus on the need for further research into methods
for quantitative and/or qualitative incorporation [of additional dimensions of
value] into technology assessments [because] methods for empirically
integrating them into a value-based price are not well established,”[xx]
but ICER has already started using evLYG, a new and untested assessment metric,
without such evaluation or review. Similarly, ICER changed how it calculates
the Budget Impact Threshold earlier this year by going from a 2-year average of
FDA approvals to a 5-year average.[xxi]
Previously, ICER has stated that it will make updates outside of its every 3-year
updates as it deems appropriate, but we are concerned that this means ICER will
continue to make adjustments to its processes and methodologies without
adequate – or any – outside consultation, particularly with patient or
clinician expert groups, and without reasonable announcements or explanation of
its rationale for the changes. By making
changes to its processes between announced updates, ICER is creating additional
confusion that erodes confidence in any products ICER may produce.
Budget Impact Calculations
In calculating budget impact, ICER will “assume that an equal proportion of patients (20%) would be treated with the new treatment each year over five years, arriving at a cumulative 100% uptake at five years.”[xxii] With this arbitrary assumption, ICER is ignoring existing knowledge and expert insights: a great deal of research has been done on adoption of innovations – both conceptual (e.g., Rogers[xxiii]), and practical (e.g., actual market adoption information for all manner of innovations). We recognize that ICER’s intent is “NOT meant to represent our assumptions of the budget impact of new interventions that are most likely in the real world,”[xxiv] but then this explicitly raises the question as to ICER’s thinking and rationale for how an analysis based on entirely unrealistic assumptions would be an appropriate starting place for competent and responsible policymakers. We are totally unaware of any policymakers who would ever consider basing their decisions on analyses that were NOT based on some factual information grounded in what was at least an attempt at real world assumptions and modeling – and particularly for their population or program of oversight or concern. We would welcome a specific explanation of the utility of ICER’s analytical vision as it relates to real world policymakers.
Also concerning is the fact that in setting its budget threshold amounts, ICER uses mean rather than median income as the basis for its QALY thresholds, which may not be appropriate. As Bang and Zhao have concluded, “mean and median-based QALYs
be considered together as complementary tools in CEA for informed
decision making, acknowledging the pros and cons of each.”[xxv]
And further, that the “median might be an important parameter that may be more
relevant to consumers than providers, more to patients and practicing
physicians than policymakers, and more on the individual level than the
societal level.”[xxvi]ICER should explain why it is using mean income ($54,420 in 2018) rather
than median ($63,179 in 2018),[xxvii]
particularly because most lower income people receive financial assistance to
cover health care costs, meaning that the major economic impact of health care
costs in the U.S. is on middle income individuals and families.
- In choosing to implement the Likert Scale for voting by appraisal committees, why was the 3-point scale chosen, and not a 5-point or 7-point scale, either of which would permit more nuanced information and prioritization of perspectives from advisors? What analysis did ICER undertake to determine that the 3-point scale was the appropriate range?
- It is our
understanding that the German rating system is based upon evaluation of actual
clinical trials, whereas ICER’s analysis is based more often on meta-analyses,
making the two incompatible. Why is ICER
proposing to also include ratings based on the German system?[xxviii]
- For the new Comparative Effectiveness Rating Matrix graphic, given our understanding of the new meanings for C++ and C-, we believe that the ordering in the graphic should be changed so that C++ is above C+, which would create a more orderly progression of potential benefit and certainty in the ratings scale ICER has chosen to deploy. If that is not correct, we ask that ICER please explain the new ratings matrix in more detail because it is unclear.
conclusion, we continue to be very concerned that ICER undervalues actual
patient perspectives, concerns, experiences, and views. If ICER’s goal is to
“help the United States evolve toward a health care system that provides fair
pricing, fair access, and a sustainable platform for future innovation,”[xxix]
then its process should support that goal.
The only way to do that is by
creating models and using assumptions that reflect the reality of the U.S.
health care ecosystem. Because the health
care “system” in the U.S. is multi-payer and pluralistic, continuing to conduct
unsophisticated assessments that suppose a uniform integrated system means ICER’s
work is not only unhelpful, it could be destructive if it leads to uninformed
decisions and bad policy.
[i] “2020 Value Assessment Framework – Proposed Changes,” August 21, 2019, P. 39
[ii] Ibid., P. 43
[iii] Patients Rising Now Letter to ICER “Draft Evidence Report “Esketamine for the Treatment of Treatment-Resistant Depression,” April 15, 2019
[iv] “2020 Value Assessment Framework – Proposed Changes,” August 21, 2019, P. 5
[v] Ibid., P. 6
[vi] Ibid., P. 46
[vii] Ibid., P. 15
[ix] “2020 Value Assessment Framework – Proposed Changes,” August 21, 2019, P. 37
[xi] “2020 Value Assessment Framework – Proposed Changes,” August 21, 2019, P. 17
[xii] Ibid., PP. 16-17
[xiv] “2020 Value Assessment Framework – Proposed Changes,” August 21, 2019, P. 26
[xv] Ibid., P. 3
[xvi] “ICER’s Value Framework and Patient Feedback: Q&A with Steve Pearson, September 4, 2019, https://icer-review.org/wp-content/uploads/2019/09/ICER%E2%80%99s-Value-Framework-Patient-Feedback-Webinar-0904196.mp4 [Time code 57:22]
[xvii] “2020 Value Assessment Framework – Proposed Changes,” August 21, 2019, P. 31
[xviii] Ibid., P. 11
[xix] Ibid., P. 11
[xx] Ibid., P. 13
[xxii] “2020 Value Assessment Framework – Proposed Changes,” August 21, 2019, P. 36
[xxiii] “Diffusion of Innovations,” by Everett M. Rogers
[xxiv] “2020 Value Assessment Framework – Proposed Changes,” August 21, 2019, P. 36
[xxv] “Median-Based Incremental Cost-Effectiveness Ratio (ICER),” J Stat Theory Pract. 2012 ; 6(3): 428–442
[xxvi] “Cost-Effectiveness Analysis: a proposal of new reporting standards in statistical analysis,” J Biopharm Stat. 2014; 24(2): 443–460
[xxviii] “2020 Value Assessment Framework – Proposed Changes,” August 21, 2019, P. 10
[xxix] Ibid., P. 2