Brendan O'Leary https://www.boleary.com/blog/ Posts from Brendan O'Leary quarto-1.8.27 Mon, 02 Feb 2026 15:00:00 GMT Graphing the FDA’s medical device review performance metrics (FY 2025) Brendan O'Leary https://www.boleary.com/blog/mdufa/202601/

Introduction

See the first post in this series for historical context. This post builds on that previous work, providing refreshed data and graphs for the 510(k) program, additional historical metrics for IVDs, and a new section on Pre-Submissions.

You can download the resulting datasets below. They include more than 40,500 performance metrics.

The plots in this post are based on the information in the December 10, 2018 MDUFA 3 report, the November 16, 2023 MDUFA 4 report, and the November 20, 2025 MDUFA 5 report. [1] [2] [3]

This post provides:

510(k) program metrics

  • Submission volume by review group over time
  • Which receipt cohorts have final data
  • How long 510(k)s spend under review at the FDA
  • On-time performance rates
  • How long 510(k)s spend on hold while applicants respond
  • Total review time from receipt to decision
  • How often the FDA requests additional information
  • Clearance and not-substantially-equivalent decision rates

Pre-Submission program metrics

  • Pre-Submission volume by review group over time
  • Which receipt cohorts have final data
  • FDA Days to written feedback
  • On-time performance for industry meeting minutes

Information about various factors that may affect this data and a glossary of MDUFA jargon are also included in this post.

Downloads

You can download all of the figures in this post as a single PDF that should print well on letter paper. Throughout this post, the captions for each figure include links to CSV files that contain the data used in the figures.

You can access data from the FDA’s MDUFA performance reports in spreadsheet form here:

Aside from some spot-checks, the datasets provided here have had limited verification and may include some errors.

In addition, data from some tables in the FDA’s MDUFA performance reports are missing from these datasets.

Use this information at your own risk and verify information using the reports provided directly by the FDA. To facilitate this, each data point provided includes information about its source, including a link to the FDA report from which it came, the relevant page number, and more. If you find a problem in these datasets, please report the problem here.

If you’re familiar with the R programming language, you can also access these datasets using the mdufa package, which is available on GitHub. You are welcome to use and modify this software, including for commercial purposes, under the MIT license. (In addition, this post is under the CC BY license, which permits sharing and adaptation, including commercially.)

Context for these datasets

NoteNew

Most of the information in this section is consistent with the information published in the first post in this series in 2024. Significant edits and new information are flagged in callouts like this.

Now, in early 2026, this section is inadequate. Significant damage has been inflicted on individual federal workers, on the broader federal workforce, and on the FDA as an institution. I can’t begin to catalog that here. Instead, I’ve focused new content on straightforward topics, like the impacts of shutdowns (which disrupt FDA operations) and predetermined change control plans (which reduce overall submission volume, although they may increase average review complexity).

The previous post already included a discussion about how receipt-cohort MDUFA metrics are “lagging indicators.” In addition, my personal view is that MDUFA metrics are among the last places we should expect to see significant evidence of trouble at the FDA (although we certainly did see that during the COVID-19 Public Health Emergency). First, dedicated career staff recognize the importance of their work at the FDA and are redoubling their efforts despite the circumstances. Second, as the saying goes, “What gets measured gets done.” The FDA’s medical device review programs are measured. Those measures are made public and are regularly analyzed by regulatory professionals and researchers. That makes it more likely that reviews will get done and that they will get done on time.

Will the reviews get done well? Yes, I hope and believe that the FDA’s medical device reviews will continue to be done well, but we won’t be able to determine that from the efficiency metrics shown here.

This post includes visualizations of performance metrics related to the “Medical Device User Fee Amendments” or “MDUFA.” Every five years since fiscal year (FY) 2003, Congress has authorized (or re-authorized) the FDA to collect user fees to support the FDA’s regulatory activities related to medical devices. Each of these five-year periods has been named sequentially: MDUFA or MDUFA I (FY 2003 through FY 2007), MDUFA II (FY 2008 through FY 2012), MDUFA III (FY 2013 through FY 2017), MDUFA IV (FY 2018 through FY 2022) and MDUFA V (FY 2023 through FY 2027).

This post will use “MDUFA 1,” “MDUFA 2,” “MDUFA 3,” “MDUFA 4,” and “MDUFA 5” to refer to these periods. The term “MDUFA” will be used to refer to any or all of these periods.

Before each re-authorization, the FDA has negotiated with representatives from the medical device industry, patient representatives, and other members of the public to arrive at a set of commitments that can be kept given the additional resources user fees provide. This results in a “MDUFA Commitment Letter,” a draft of which is provided to Congress in advance of re-authorization.

As a result of these commitments, since at least MDUFA 2, the FDA has published quarterly and annual “MDUFA Reports” that include performance metrics related to the FDA’s medical device regulatory activities with a particular focus on review times for premarket submissions for medical devices.

Definitions

The definitions below are based on the information in the MDUFA 5 Commitment Letter with some simplifications. [4]

Applicant
The person or organization providing a 510(k) or other submission to the FDA. [4, p. 32]
FDA Days
The number of calendar days that a submission was “under review” at the FDA, not including:
  • any time the FDA spent reviewing information that could not be accepted or filed because it was not administratively complete, or
  • any time the FDA spent waiting for an applicant to respond to a substantive request for additional information. [4, p. 32]
Fiscal Year (FY):
A period beginning on October 1 and ending on the following September 30. For example, the period between 2023-10-01 and 2024-09-30 is “FY 2024.”
Industry Days
The number of calendar days an administratively complete submission was “on hold” while the FDA waited for an applicant to respond to a substantive request for additional information.
MDUFA Decision
For 510(k)s, a decision that a device is substantially equivalent (“SE” or “cleared”) and can be marketed in the U.S. or a decision that a device is not substantially equivalent (“NSE” or “not cleared”) and cannot be marketed in the U.S. [4, pp. 32–33] Throughout this post, this is referred to simply as a “decision.”
MDUFA Performance Report Cutoff Date
The date as of which the performance metrics in a MDUFA performance report were calculated.
NoteNew

For the data included in this post, the cutoff dates are 2018-09-30 for MDUFA 3, 2023-09-30 for MDUFA 4, and 2025-09-30 for MDUFA 5. [1, p. 1] [2, p. 1] [3, p. 2]

Review Cycle
A period of FDA Days during which an accepted submission is “under review” at the FDA that is terminated by either a substantive request for additional information or a decision. For example, a submission that is cleared by the FDA without going “on hold” has one review cycle. A submission that the FDA places “on hold” through a substantive request for additional information, is then supplemented by the applicant with additional information, and then goes “under review” for a second time at the FDA before a final decision is made has two review cycles.1
Total Days
The number of calendar days from the FDA’s receipt of an administratively complete submission to a MDUFA Decision. When calculating summary statistics on this metric, the FDA excludes certain outlier submissions. The criteria that determine which outliers should be excluded change from year-to-year. [4, p. 35]

Metrics based on receipt cohorts are lagging indicators

Performance metrics included in the MDUFA reports are calculated across submissions grouped by the fiscal year in which they were received. These groups of submissions are called “receipt cohorts.” Grouping submissions by the fiscal year in which the FDA made a decision on the submissions results in groups of submissions called “decision cohorts.”

As of this writing, the most recent 510(k) receipt cohort includes many submissions for which the FDA has not made a final decision. This means that the receipt cohort is “open.” As review days continue to accumulate for these 510(k)s, the performance metrics that are based on those review times will change as well. Performance metrics calculated across open receipt cohorts continue to change until those cohorts “close,” which happens when each submission in the receipt cohort has received a decision.

Recent receipt cohorts are typically open. This means that performance metrics associated with these receipt cohorts, on their own, do not provide a particularly accurate indicator of final performance on those cohorts. It takes a while for them to stabilize. Before they stabilize, performance metrics associated with open receipt cohorts provide overly optimistic estimates of performance.2

NoteNew

In the graphs below, receipt cohorts with more submissions still awaiting decisions are plotted with greater transparency to indicate that their metrics may change.

While this post can help you understand how the FDA performed on closed receipt cohorts from past years, the MDUFA reports and this post will not provide you with much useful information about how things are going at the FDA right now, and data from open receipt cohorts should be interpreted with caution.

Reorganizations impacted these datasets

MDUFA 3 data is included for some review groups in the figures below and is not included for others. This is a result of reorganizations at the FDA. A major reorganization occurred on September 30, 2019, and a subsequent, smaller reorganization occurred later.

In some cases, the reorganizations took premarket review groups that already existed and were included in the MDUFA 3 reports and renamed them.3 This is essentially the case for OHT2, OHT5, OHT6, and OHT8, meaning that data for those review groups from the MDUFA 5 and MDUFA 4 reports can be aligned with data from the MDUFA 3 reports for corresponding review groups.

In other cases, the reorganizations took review groups that were included in the MDUFA 3 reports and divided up and separated their subgroups. This is essentially the case for OHT1, OHT3, and OHT4, meaning that MDUFA 5 and MDUFA 4 report data for those review groups cannot be aligned with MDUFA 3 report data.

In the case of OHT7, the reorganizations combined several review groups that already existed and were included in the MDUFA 3 reports into one larger group that is referenced in the MDUFA 4 and MDUFA 5 reports. This means that some data, such as the number of submissions received, can be summed across the corresponding groups in the MDUFA 3 reports to align the data with the larger organization that is included in the MDUFA 4 and MDUFA 5 reports. In other cases, aligning the data is more complex. In still other cases, it is not possible to align the data.

NoteNew

Additional historical metrics for in vitro diagnostic device reviews conducted under MDUFA 3 are included in this post. This required some additional data processing to account for reorganizations and differences in how the FDA’s MDUFA 3 reports and the FDA’s subsequent MDUFA 4 and MDUFA 5 reports break out data. For example, additional metrics based on sums and averages were calculated across the in vitro diagnostic review program from the division-level information included in MDUFA 3 reports. Other types of metrics, such as percentile metrics, can’t be combined in this way and are not included in this post.

While I was not able to identify a reference that maps the review groups included in the MDUFA 3 reports to those used in the MDUFA 4 and MDUFA 5 reports, Table 1 shows how I mapped the groups for the purposes of this post based on the names of the old review groups and the names of the new review groups.

Table 1: Mapping of review groups in the MDUFA 3 reports to review groups in the MDUFA 4 and MDUFA 5 reports.
MDUFA 3 report review group (Old) Corresponding MDUFA 4 and MDUFA 5 report review group(s) (New) Relationship between old and new review groups MDUFA 3 data available in figures
DAGRID: Division of Anesthesiology, General Hospital, Respiratory, Infection Control, and Dental Devices OHT1: Ophthalmic, Anesthesia, Respiratory, ENT, and Dental Devices
OHT3: GastroRenal, ObGyn, General Hospital, and Urology Devices
OHT4: Surgical and Infection Control Devices		 Old review group was split and is part of more than one new review group No
DCD: Division of Cardiovascular Devices OHT2: Cardiovascular Devices Old review group and new review group are equivalent Yes
DCTD: Division of Chemistry and Toxicology Devices OHT7: In Vitro Diagnostics Old review group is a subset of new review group Most sums and averages are available; percentiles are not available
DIHD: Division of Immunology and Hematology Devices OHT7: In Vitro Diagnostics Old review group is a subset of new review group Most sums and averages are available; percentiles are not available
DMD: Division of Microbiology Devices OHT7: In Vitro Diagnostics Old review group is a subset of new review group Most sums and averages are available; percentiles are not available
DMGP: Division of Molecular Genetics and Pathology Devices OHT7: In Vitro Diagnostics Old review group is a subset of new review group Most sums and averages are available; percentiles are not available
DNPMD: Division of Neurological and Physical Medicine Devices OHT5: Neurological and Physical Medicine Devices Old review group and new review group are equivalent Yes
DOD: Division of Orthopedic Devices OHT6: Orthopedic Devices Old review group and new review group are equivalent Yes
DOED: Division of Ophthalmic and Ear, Nose, and Throat Devices OHT1: Ophthalmic, Anesthesia, Respiratory, ENT, and Dental Devices Old review group was combined with parts of another old review group to form new review group No
DRGUD: Division of Reproductive, Gastro-Renal, and Urological Devices OHT3: GastroRenal, ObGyn, General Hospital, and Urology Devices Old review group was combined with part of another old review group to form new review group No
DRH: Division of Radiological Health OHT8: Radiological Health Old review group and new review group are equivalent Yes
DSD: Division of Surgical Devices OHT4: Surgical and Infection Control Devices Old review group was combined with part of another old review group to form new review group No

The COVID-19 Public Health Emergency impacted these datasets

A Public Health Emergency (PHE) was declared for COVID-19 beginning on 2020-01-27 (FY 2020 Q2) and was renewed until it expired on 2023-05-11 (FY 2023 Q3). Responding to the PHE had a major impact on the FDA’s work. Other related declarations and determinations also impacted and continue to impact the FDA’s work. To acknowledge the impact of the PHE on the FDA’s work, graphs in this post that plot metrics over time also show the PHE as a shaded region underlaying data points associated with FY 2020 through FY 2023.

NoteNew

New information about pre-submission impacts is included below.

In a series of “FDA Voices” blog posts, FDA leadership described the impact of the pandemic, including the review of thousands of Emergency Use Authorization (EUA) submissions for pandemic-related devices, on FDA’s work. [8] [9] [10]

In April 2021, FDA device review leaders noted:

  • “Unless IVD pre-submissions are related to COVID-19, companion diagnostics, a breakthrough designation request, or have a significant public health impact, we have been unable to review them.”
  • “For the remainder this year, we will be declining other IVD pre-submission requests that don’t fall into these categories for the present time.”
  • “Non-COVID-19 IVD submissions are experiencing longer-than-typical review timelines and delays in initiation of review.”
  • “[Divisions reviewing respirators, facemasks, gowns, and gloves] are experiencing longer review timelines for pre-submissions. We anticipate pre-submissions in these divisions to be completed within 120 days, rather than the usual 70 days.”

In December 2021, they wrote that for non-IVDs, “During 2022, CDRH expects non-IVD review divisions to be able to transition back to most of the MDUFA review timeframes, although some submissions may experience delays on a case-by-case basis.” For IVDs: “We’re pleased to announce that, as of January 1, 2022, CDRH plans to also accept pre-submissions requesting feedback for IVDs that require or would be likely to require a PMA or De Novo submission. Depending on the course of the pandemic, we hope to be able to accept all IVD pre-submission requests by the late spring or early summer.”

In May 2022, they announced reopening to all pre-submissions: “Starting June 1, CDRH will be accepting pre-submission requests of all types for all product areas. This means we will no longer have any limitations on pre-submission types that we will accept for IVDs.”

Other MDUFA timelines were also impacted.

Government shutdowns impacted these datasets

NoteNew

This section is new.

Government shutdowns may have impacted FDA review timelines. In particular, when a shutdown occurs at the beginning of a fiscal year, as was the case in FY2014 and FY2026, the FDA is unable to accept new user fees. This means that new submissions that require user fees cannot be logged in and reviews cannot begin. [11] When the government reopens, a large number of submissions get logged in all at once: Submissions that may have come in over time in a way that would not have strained the FDA’s capacity arrive all at once and may overwhelm the FDA’s capacity, impacting timeliness. Table 2 shows the shutdowns that are relevant to this post.

Table 2: Significant government shutdowns from MDUFA 3 onward. Asterisks indicate that a shutdown began on October 1, meaning that the FDA was not able to begin new reviews of submissions that require user fees.
Fiscal Year Start Date End Date Duration (Days)
2014* 2013-10-01 2013-10-17 16
2018 2018-01-20 2018-01-23 3
2019 2018-12-22 2019-01-25 35
2026* 2025-10-01 2025-11-12 43

Exemptions impacted these datasets

NoteNew

Clinical Electronic Thermometers were exempted in 2025. [12]

The FDA exempts lower-risk and well-characterized types of medical devices from 510(k) review from time to time. Examples include a large round of exemptions that occurred in 2017 and smaller rounds of exemptions in 2018 and 2020. [13] [14] [15] In addition, Clinical Electronic Thermometers were exempted in 2025. [12]

Exemptions can decrease the volume of 510(k)s that a review group receives. 510(k)s for medical devices that are not exempted and remain in the 510(k) program may include more complex or less well-characterized technologies that are more challenging to review. The review process for these 510(k)s may take more time on average, and this may affect performance metrics over time.

Predetermined Change Control Plans impacted these datasets

NoteNew

This section is new.

Predetermined Change Control Plans (PCCPs) allow manufacturers to pre-specify device modifications and have the FDA review them as part of a premarket submission. [16] [17] If the FDA authorizes a PCCP, the manufacturer can implement the pre-specified modifications without submitting a new premarket submission, provided the manufacturer follows the validation and verification protocols described in the PCCP. I maintain a list of devices with FDA-authorized PCCPs on this site.

While PCCPs decrease the FDA’s overall review workload by eliminating the need to review certain device modifications, submissions that include PCCPs may be more complex and may require more work to review. On average, this may serve to increase submission complexity over time, even as overall submission volume decreases or does not grow as much as it otherwise would.

These datasets focus on efficiency, not quality

The performance metrics in the MDUFA reports primarily focus on how long things take. Information about interim and final submission outcomes – such as the rate at which 510(k)s result in requests for additional information and the rate at which 510(k)s result in a medical device being cleared for market in the United States – are also included. More comprehensive information about submission and review quality, meaning how good a job applicants do in preparing submissions and how good a job the FDA does in reviewing them, is not included in the MDUFA reports or this post.

To provide the public with some additional insight into quality and other topics, the MDUFA commitments have included “independent assessments” by a government contractor. The MDUFA 4 assessment resulted in a report published in 2021. [18] Historically, the FDA and the GAO have also performed assessments. [19] [6]

Therefore, efficiency metrics associated with the amount of time submissions spend going through the FDA review process, especially when well-established submission and review quality metrics are not readily available, provide an incomplete picture of the FDA’s medical device review programs. In addition, differences in submission or review quality over time or across product areas may affect these metrics.

510(k) program metrics

The FDA provides visualizations of many 510(k) program performance metrics for the Center for Devices and Radiological Health (CDRH) as a whole, but the FDA does not provide these for review groups within CDRH. [3, pp. 111–118] This post provides visualizations for OHT1 through OHT8 and CDRH overall.

NoteNew

Plots for CDRH as a whole are now included in addition to plots for OHT1 through OHT8.

510(k) submission volume

The MDUFA reports include information about submissions received over time. This can be plotted, and future submission volume can be projected. The projections provided here are somewhat naive: They are linear models based on fiscal year and review organization.

Figure 1: 510(k) submission volume for CDRH by fiscal year received as of the corresponding MDUFA report cutoff dates. Raw data.
Figure 2: 510(k) submission volume by OHT and fiscal year received as of the corresponding MDUFA report cutoff dates. Raw data.

510(k) cohort closure percentage

See above for a discussion of what it means for a receipt cohort to be open or closed. As expected, Figure 3 and Figure 4 show that the most recent 510(k) receipt cohort had many open 510(k)s as of the most recent MDUFA performance report, meaning that performance metrics calculated on that cohort and shown in the figures throughout this post are likely to change substantially in future reports.

Figure 3: 510(k) cohort closure percentage for CDRH by fiscal year received as of the corresponding MDUFA report cutoff dates. Raw data.
Figure 4: 510(k) cohort closure percentage by OHT and fiscal year received as of the corresponding MDUFA report cutoff dates. Raw data.

510(k) FDA Days metrics

The MDUFA performance reports include summary statistics about how long reviews take across the population of 510(k)s received. In particular, the reports provide quintiles (the 20th, 40th, 60th, and 80th percentiles) of FDA Days, Industry Days, and Total Days. Quintiles are an unusual choice for summary statistics about a distribution, and I’m not aware of a standard approach to plotting them. They can’t be plotted with boxplots, for example, which use quartiles. The visualizations in this post are inspired by boxplots though: They use line segments to show the percentile metrics and points to show averages.

Figure 5: FDA Days performance metrics for CDRH by fiscal year received as of the corresponding MDUFA report cutoff dates. Raw data.
Figure 6: FDA Days performance metrics by OHT and fiscal year received as of the corresponding MDUFA report cutoff dates. Raw data.

The PHE had a major impact on the data in Figure 6 for OHT7. Figure 7 shows the same data within 0 through 90 FDA Days, making it easier to see the data associated with OHTs 1-6 and 8.

Figure 7: FDA Days performance metrics by OHT and fiscal year received as of the corresponding MDUFA report cutoff dates. Values above 90 FDA Days are not shown. Raw data.

510(k) on-time performance

The FDA has 90 FDA Days to review a 510(k).4 The percentage of 510(k)s reviewed within this time frame can be thought of as “on-time performance,” similar to how the metric is used in transportation. The MDUFA reports include on-time performance metrics for the 510(k) program, shown in Figure 8 and Figure 9.

Figure 8: FDA Days on-time performance for the 510(k) program (CDRH) by fiscal year received as of the corresponding MDUFA report cutoff dates. Raw data.
Figure 9: FDA Days on-time performance for the 510(k) program by review group and fiscal year received as of the corresponding MDUFA report cutoff dates. Raw data.

510(k) Industry Days metrics

By regulation, applicants have 30 days to respond to a request for additional information from the FDA before the FDA may consider a 510(k) to have been “withdrawn” by the applicant.5 By policy, the FDA allows up to 180 days for an applicant to respond. [20, p. 9] Figure 10 and Figure 11 provide information about how long these responses have taken by fiscal year and by the FDA review group with which an applicant is interacting.

Figure 10: Industry Days performance metrics for CDRH by fiscal year received as of the corresponding MDUFA report cutoff dates. Raw data.
Figure 11: Industry Days performance metrics by OHT and fiscal year received as of the corresponding MDUFA report cutoff dates. Raw data.

510(k) Total Days metrics

As described above, the FDA has 90 FDA Days to review a 510(k), and applicants have 180 Industry Days to respond to questions the FDA raises in the course of that review. This means that the total amount of time available for a 510(k) to go through the review process if these activities are completed on time is 270 Total Days. Figure 12, Figure 13, and Figure 14 show how long the process has taken by fiscal year.

Figure 12: Total Days performance metrics for CDRH by fiscal year received as of the corresponding MDUFA report cutoff dates. Raw data.
Figure 13: Total Days performance metrics by OHT and fiscal year received as of the corresponding MDUFA report cutoff dates. Raw data.

The PHE had a major impact on the data in Figure 13 for OHT7. Figure 14 shows the same data within 0 through 270 Total Days, making it easier to see the data associated with OHTs 1-6 and 8.

Figure 14: Total Days performance metrics by OHT and fiscal year received as of the corresponding MDUFA report cutoff dates. Values above 270 Total Days are not shown. Raw data.

510(k) review cycle metrics

The FDA reports the average number of review cycles for 510(k)s, which is related to the rate at which the FDA places 510(k)s “on hold” to request additional information from applicants.

Figure 15: Number of review cycles for 510(k)s (CDRH) over time as of the corresponding MDUFA report cutoff dates. Raw data.
Figure 16: Number of review cycles for 510(k)s over time as of the corresponding MDUFA report cutoff dates. Raw data.

While the FDA occasionally places a single 510(k) on hold more than once, this has been uncommon since at least 2014. [21, p. 8] It is not quite correct to assume that 510(k)s cannot have more than 2 review cycles, but it is a useful approximation for the purpose of estimating the rate at which the FDA places 510(k)s “on hold” to request additional information. With this assumption, that proportion is the Average Number of Review Cycles minus one.

Figure 17: Estimated rate at which additional information is requested for 510(k)s (CDRH) over time based on the average number of review cycles and an assumption that 510(k)s do not go on hold more than once. Raw data.
Figure 18: Estimated rate at which additional information is requested for 510(k)s over time based on the average number of review cycles and an assumption that 510(k)s do not go on hold more than once. Raw data.

510(k) decision metrics

In the MDUFA performance reports, the rates at which submissions receive an “SE” decision or receive an “NSE” decision are reported as percentages of submissions that receive either an SE or an NSE decision. The rates at which submissions are withdrawn by applicants or deleted by the FDA because an applicant did not respond to a request for additional information in a timely manner are reported as percentages of all accepted submissions. Because two different denominators are used to calculate these percentages in the MDUFA reports, adding the four percentages together results in a total that exceeds 100%.

This makes it difficult to understand what percentage of 510(k)s received by the FDA were cleared for market over time based on the rates in the MDUFA reports. However, the MDUFA reports also include the counts of submissions that received each of these decisions. Figure 19 and Figure 20 show the rates of cleared and not cleared (including withdrawn and deleted) 510(k)s.

Figure 19: The rate at which 510(k)s are cleared or not cleared for market (CDRH) over time as of the corresponding MDUFA report cutoff dates. Raw data.
Figure 20: The rate at which 510(k)s are cleared or not cleared for market over time as of the corresponding MDUFA report cutoff dates. Raw data.

Pre-Submission program metrics

The plots in this section start with MDUFA 4 because the Pre-Submission metrics changed significantly between MDUFA periods. In MDUFA 3, metrics measured how long it takes for the FDA and industry to meet. [1, pp. 247–262] Meetings might be scheduled or rescheduled for reasons unrelated to FDA preparation time, such as travel arrangements. In MDUFA 4, the metrics were updated to also include how long it takes for the FDA to send written feedback to industry before the meeting, which is a more direct and objective measure of the FDA’s review times for Pre-Submissions. [5, pp. 3–4] In MDUFA 5, metrics related to how long it takes for a meeting to occur were dropped. [4, pp. 2–5] As a result, there are fewer consistent metrics for this program over time than there are for the 510(k) program. The visualizations here focus on submission volume, a derived measure similar to cohort closure, and time to written feedback.

Pre-Submission volume

During the COVID-19 Public Health Emergency, CDRH did not accept pre-submissions for all products. In particular, IVD divisions declined most non-COVID pre-submissions from April 2021 through May 2022. See The COVID-19 Public Health Emergency impacted these datasets for details.

Figure 21: Pre-Submission submission volume for CDRH by fiscal year received as of the corresponding MDUFA report cutoff dates. Raw data.
Figure 22: Pre-Submission submission volume by OHT and fiscal year received as of the corresponding MDUFA report cutoff dates. Raw data.

Pre-Submission cohort closure percentage

Unlike 510(k) submissions, the FDA’s MDUFA performance reports do not include a pre-calculated cohort closure percentage for Pre-Submissions. The closure percentages shown here are calculated from the available metrics using different formulas for each MDUFA period:

  • MDUFA IV: Closure = Written Feedback Sent ÷ (Received − Closed Before RTA − Not Accepted)
  • MDUFA V: Closure = (Accepted − Non-MDUFA Action − Pending) ÷ (Accepted − Non-MDUFA Action)

These formulas approximate the percentage of Pre-Submissions in each fiscal year cohort that have received a final response as of the report date. The approach differs from 510(k) cohort closure, which is reported directly by the FDA and tracks whether submissions have received a decision.

When calculated values exceed 100% due to data inconsistencies, they are capped at 100%. For these visualizations, cohorts are marked as “Closed” and rounded to 100% when they reach 99.5% or higher.

Figure 23: Pre-Submission cohort closure percentage for CDRH by fiscal year received as of the corresponding MDUFA report cutoff dates. Raw data.
Figure 24: Pre-Submission cohort closure percentage by OHT and fiscal year received as of the corresponding MDUFA report cutoff dates. The MDUFA 4 performance report lists zero as the value for many Pre-Submission metrics for OHT3 in FY2022; while this may be erroneous, this figure displays the data as provided by the FDA. Raw data.

Pre-Submission FDA Days to Written Feedback

The MDUFA performance reports include summary statistics about how long it takes FDA to send written feedback across the population of Pre-Submissions received. In particular, the reports provide quintiles (the 20th, 40th, 60th, and 80th percentiles) of FDA Days to Written Feedback.

During the COVID-19 Public Health Emergency, pre-submission review timelines were extended. For divisions reviewing certain pandemic-related devices, the FDA anticipated completing pre-submissions in 120 days rather than 70 days. See The COVID-19 Public Health Emergency impacted these datasets for details.

Figure 25: FDA Days performance metrics for CDRH by fiscal year received as of the corresponding MDUFA report cutoff dates. Raw data.
Figure 26: FDA Days performance metrics by OHT and fiscal year received as of the corresponding MDUFA report cutoff dates. The MDUFA 4 performance report lists zero as the value for many Pre-Submission metrics for OHT3 in FY2022; while this may be erroneous, this figure displays the data as provided by the FDA. Raw data.

The PHE had a major impact on the data in Figure 26 for OHT7. Figure 27 shows the same data within 0 through 90 FDA Days, making it easier to see the data associated with OHTs 1-6 and 8.

Figure 27: FDA Days performance metrics by OHT and fiscal year received as of the corresponding MDUFA report cutoff dates. Values above 90 FDA Days are not shown. The MDUFA 4 performance report lists zero as the value for many Pre-Submission metrics for OHT3 in FY2022; while this may be erroneous, this figure displays the data as provided by the FDA. Raw data.

Pre-Submission Industry Meeting Minutes on-time performance

Under the MDUFA commitment, industry is expected to provide draft meeting minutes within 15 days of a Pre-Submission meeting. The MDUFA reports track the percentage of meetings for which industry provided minutes within this timeframe.

Figure 28: Industry on-time performance for providing Pre-Submission meeting minutes within 15 days (CDRH) by fiscal year as of the corresponding MDUFA report cutoff dates. Raw data.
Figure 29: Industry on-time performance for providing Pre-Submission meeting minutes within 15 days by review group and fiscal year as of the corresponding MDUFA report cutoff dates. Raw data.

Closing thoughts

The objective of this work was to make the information in the MDUFA reports easier to access, analyze, and use. This post provides visualizations of about 2,200 values from the FDA’s 510(k) and Pre-Submission program metrics over time and across review groups. These visualizations are intended to provide a more useful view of the tabular data provided throughout the FDA’s MDUFA reports while still presenting that data dispassionately and with appropriate context.

References

[1]
“MDUFA III (FY 2013-2017) performance report,” U.S. Food and Drug Administration, Report, Dec. 2018. Available: https://boleary-com-blog.s3.us-east-005.dream.io/mdufa-3_2018-12-10_quarterly-report.pdf
[2]
“MDUFA IV (FY 2018-2022) performance report,” U.S. Food and Drug Administration, Report, Nov. 2023. Available: https://boleary-com-blog.s3.us-east-005.dream.io/mdufa-4_2023-11-16_quarterly-report.pdf
[3]
“MDUFA V (FY 2023-2027) performance report,” U.S. Food and Drug Administration, Report, Nov. 2025. Available: https://boleary-com-blog.s3.us-east-005.dream.io/mdufa-5_2025-11-20_quarterly-report.pdf
[4]
“MDUFA performance goals and procedures, fiscal years 2023 through 2027,” U.S. Food and Drug Administration, User Fee Commitment Letter, Mar. 2022. Available: https://boleary-com-blog.s3.us-east-005.dream.io/mdufa-5_goals-and-procedures.pdf
[5]
“MDUFA performance goals and procedures, fiscal years 2018 through 2022,” U.S. Food and Drug Administration, User Fee Commitment Letter, Dec. 2016. Available: https://boleary-com-blog.s3.us-east-005.dream.io/mdufa-4_goals-and-procedures.pdf
[6]
M. Crosse et al., “Medical devices: FDA has met most performance goals but device reviews are taking longer,” U.S. Government Accountability Office, Report to Congressional Requesters GAO-12-418, Feb. 2012. Available: https://www.gao.gov/products/gao-12-418
[7]
“CDRH preliminary internal evaluations – volume I,” U.S. Food and Drug Administration, Report, 2010. Available: https://boleary-com-blog.s3.us-east-005.dream.io/fda-cdrh-510k-working-group-report-2010.pdf
[8]
J. Shuren and W. Maisel, “A year into the pandemic: How the FDA’s center for devices and radiological health is prioritizing its workload and looking ahead,” Apr. 2021. https://web.archive.org/web/20210415151209/https://www.fda.gov/news-events/fda-voices/year-pandemic-how-fdas-center-devices-and-radiological-health-prioritizing-its-workload-and-looking (accessed Apr. 15, 2021).
[9]
J. Shuren and W. Maisel, “Looking ahead to 2022 as FDA’s center for devices and radiological health manages a sustained increase in workload,” Dec. 2021. https://web.archive.org/web/20211222195626/https://www.fda.gov/news-events/fda-voices/looking-ahead-2022-fdas-center-devices-and-radiological-health-manages-sustained-increase-workload (accessed Dec. 22, 2021).
[10]
J. Shuren and W. Maisel, “FDA’s center for devices and radiological health’s continued efforts to return to normal: Reopening for all pre-submissions,” May 2022. https://web.archive.org/web/20220531154347/https://www.fda.gov/news-events/fda-voices/fdas-center-devices-and-radiological-healths-continued-efforts-return-normal-reopening-all-pre (accessed May 31, 2022).
[11]
U.S. Food and Drug Administration, “FDA FY 2026 lapse in funding information,” Oct. 01, 2025. https://web.archive.org/web/20251001131252/https://www.fda.gov/about-fda/fda-fy-2026-lapse-funding-information (accessed Oct. 01, 2025).
[12]
“90 FR 25889: Medical devices; exemptions from premarket notification: Class II devices; clinical electronic thermometers,” U.S. Food and Drug Administration, Federal Register Notice, Jun. 2025. Available: https://www.federalregister.gov/d/2025-11207
[13]
“82 FR 31976: Medical devices; exemptions from premarket notification: Class II devices,” U.S. Food and Drug Administration, Federal Register Notice, Jul. 2017. Available: https://www.federalregister.gov/d/2017-14453
[14]
“83 FR 25910: Medical devices; exemptions from premarket notification: Class II devices,” U.S. Food and Drug Administration, Federal Register Notice, Jun. 2018. Available: https://www.federalregister.gov/d/2018-11879
[15]
“85 FR 44186: Medical devices; exemptions from premarket notification: Class II devices,” U.S. Food and Drug Administration, Federal Register Notice, Jul. 2020. Available: https://www.federalregister.gov/d/2020-15256
[16]
“Marketing submission recommendations for a predetermined change control plan for artificial intelligence-enabled device software functions,” U.S. Food and Drug Administration, Final Guidance Document. Docket Number FDA-2022-D-2628, Dec. 2024. Available: https://www.regulations.gov/document/FDA-2022-D-2628-0036
[17]
“Predetermined Change Control Plans for medical devices,” U.S. Food and Drug Administration, Draft Guidance Document. Docket Number FDA-2024-D-2338, Aug. 2024. Available: https://www.regulations.gov/document/FDA-2024-D-2338-0002
[18]
“MDUFA IV phase 2 independent assessment final report,” U.S. Food and Drug Administration, Report, 2021. Available: https://boleary-com-blog.s3.us-east-005.dream.io/mdufa-4_independent-assessment-2021.pdf
[19]
“Analysis of premarket review times under the 510(k) program,” U.S. Food and Drug Administration, Report, Sep. 2011. Available: https://boleary-com-blog.s3.us-east-005.dream.io/mdufa-data-requests-2011.pdf
[20]
“FDA and industry actions on premarket notification (510(k)) submissions: Effect on FDA review clock and goals,” U.S. Food and Drug Administration, Final Guidance Document FDA-2003-D-0033, 2022. Available: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/fda-and-industry-actions-premarket-notification-510k-submissions-effect-fda-review-clock-and-goals
[21]
“Types of communication during the review of medical device submissions,” U.S. Food and Drug Administration, Final Guidance Document FDA-2013-D-0147, Apr. 2014. Available: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/types-communication-during-review-medical-device-submissions

Footnotes

  1. This term is not defined in the MDUFA V Commitment Letter, but this definition is consistent with its use in other documents. [5, p. 28] [6, pp. 4, 7] [7, p. 39] While the MDUFA 3 reports are silent on the issue, the MDUFA 4 and MDUFA 5 reports confirm that “review cycle” refers to a substantive review of an “accepted” 510(k). [2, p. 257] [3, p. 268] The FDA’s approach to analyzing requests for additional information suggests that this has been the practice since FY 2013. [3, p. 113]↩︎

  2. The FDA has used a number of analysis techniques to correct for this and get insights into recent performance based on data from open receipt cohorts. One approach is to look at historical data from when previous receipt cohorts that are now closed were still open. That information is compared to the final calculated performance metrics for the corresponding cohorts and to the receipt cohorts that are currently open to infer whether performance is better or worse now than it was then and to estimate where the final performance metric calculations for open cohorts may converge. The MDUFA reports include several figures based on this approach. [3, pp. 115–117] Best- and worst-case outcomes can also be estimated for some metrics.
    These complex techniques can be difficult to interpret. I favor looking at recently closed decision cohorts instead. Metrics similar to the Total Days metrics in the MDUFA reports can be calculated for cleared 510(k) receipt and decision cohorts using the FDA’s public 510(k) database. You can access this from R using the fdadata package, and I’ve also built a free web app you can use to explore clearance timeframes using this dataset.↩︎

  3. The reorganizations had other impacts as well, but this post is focused on their implications for MDUFA performance metric data.↩︎

  4. In addition to being a MDUFA commitment, this is in 21 USC § 360(n)(1).↩︎

  5. See 21 CFR 807.87(m).↩︎

Reuse

]]> FDA 510(k) devices datasets MDUFA https://www.boleary.com/blog/mdufa/202601/ Mon, 02 Feb 2026 15:00:00 GMT A Perspective on Artificial Intelligence for Molecular Pathologists Timothy J. O'Leary Brendan O'Leary Dianne P. O'Leary https://www.boleary.com/blog/posts/202503-ai-for-pathologists/ Published February 12 in the Journal of Molecular Diagnostics: A Perspective on Artificial Intelligence for Molecular Pathologists. [1] (Open access.)

Authors: Timothy J. O’Leary, Brendan O’Leary, and Dianne P. O’Leary.

References

[1]
T. J. O’Leary, B. J. O’Leary, and D. P. O’Leary, “A perspective on artificial intelligence for molecular pathologists,” The Journal of Molecular Diagnostics, Feb. 2025, doi: 10.1016/j.jmoldx.2025.01.005.
]]> ai https://www.boleary.com/blog/posts/202503-ai-for-pathologists/ Fri, 21 Mar 2025 04:00:00 GMT Data for researchers: Extracted text from FDA decision summaries for in vitro diagnostics Brendan O'Leary https://www.boleary.com/blog/posts/202501-ivd-decision-summaries/

Introduction

Each year, the FDA authorizes around 500 in vitro diagnostics (IVDs) for use in the United States through the 510(k) program, the De Novo program, and the Premarket Approval (PMA) program.1 When the FDA authorizes an IVD, the FDA often publishes a “decision summary” or a “summary of safety and effectiveness data” (SSED) on its website, providing valuable insight into its decision-making.

The FDA posts decision summaries and SSEDs for most medical devices that come to the market through the De Novo and PMA programs, and, in addition, the FDA posts decision summaries for most IVDs that come to the market through the 510(k) program. For 510(k) devices that are not IVDs, the FDA usually only posts information written by device developers (not by the FDA), as described in this post.

The FDA makes this information available in thousands of individual files on the FDA website. This makes it possible to find information about the individual decisions the FDA has made. Using the information for other purposes, such as for gaining insight into how the IVD review program has changed over time or for pre-training Large Language Models (LLMs), has been more difficult.

That work may be easier going forward: Here, you’ll find a dataset with the full text contents of more than 6,000 IVD decision summaries and SSEDs. This includes full embedded and OCR text from over 80,000 pages. It can be downloaded in CSV format.

FDA Regulatory Expertise:
I help technology developers put their best foot forward when they need to interact with the FDA.

As with other datasets offered on this site, check back for updates around once per month, and if you use this dataset, please cite this page.

Dataset description

This dataset was assembled by downloading decision summaries and SSEDs for 510(k)s, De Novos, and PMAs associated with the following IVD review panels: Clinical Chemistry, Clinical Toxicology, Hematology, Immunology, Microbiology, Molecular Genetics, and Pathology. The dataset provides one row per page for each decision summary or SSED and includes the following fields:

submission_number
The 510(k), De Novo, or PMA number for the submission associated with the document.
file_description
An indication of whether a document is a “Decision Summary” or a “Summary of Safety and Effectiveness” based on the language used to describe the document on the FDA’s website.
date_obtained
The date the document was obtained from the FDA website. The date is formatted according to ISO 8601.
page_number
The PDF page index from which the text was obtained.
text_embedded
The contents of any text embedded in the PDF.
text_ocr
The contents of any text found using optical character recognition (OCR).
Table 1: Decision dates associated with the most recent 510(k), De Novo, and PMA decision summary documents in the dataset. Data (CSV).

Download the data

NoteNo warranty

The information in this dataset is from sources in the public domain. It is provided here “as-is” without warranty of any kind. You may also want to refer to the FDA’s website.

The dataset is available as a gzip-compressed CSV file: ivd_decision_summary_text.csv.gz (Size: 58.4M, MD5: dd7584378ac472f15ca99ed6985efd12)

TipHow to get the PDFs

You can download individual decision summary and SSED PDFs from the FDA’s website. If you want all of the PDFs associated with decision summaries and SSEDs in this dataset, you can download the zip archives below:

In these archives, decision summaries and SSEDs are grouped by the fiscal years in which decisions were made. The FDA’s fiscal years begin in October and end in September. For example: FY2025 began on 2024-10-01.

These archives are large, and I may not host them here for long. Please don’t rely on them being here in the future.

Examples in the R programming language

Download and read the dataset

Here is a sample script in R that downloads and reads the dataset:

# Load libraries -----
library(utils)
library(readr)

# Download -----
utils::download.file(
  url =   
    paste0(
    "https://www.boleary.com/blog/posts/202501-ivd-decision-summaries/data/",
    "ivd_decision_summary_text.csv.gz"
  ),
  destfile = "ivd_decision_summary_text.csv.gz"
)

# Read -----
decision_summaries <- 
  readr::read_delim(
    file = "ivd_decision_summary_text.csv.gz",
    delim = ";",
    col_types = 
      readr::cols(
        submission_number = readr::col_character(),
        file_description = readr::col_character(),
        date_obtained = readr::col_date(), 
        page_number = readr::col_integer(),
        text_embedded = readr::col_character(),
        text_ocr = readr::col_character()
      )
  )

Exploring the availability of IVD decision summaries

Exploring which IVD review processes are associated with public decision summaries

After downloading the data using the code above, we can cross reference it with datasets provided by the FDA to see which IVD review processes are associated with decision summaries and SSEDs.

Code 
# First, download and read the data using the code in the previous example

if (!exists("decision_summaries")) {
  message(
    paste0(
      "Run the code from the previous example to download and read the ",
      "decision summary data"
    )
  )
}

# Load and install additional libraries -----
# Install the fdadata package from GitHub if it's missing
if (!require("fdadata", quietly = TRUE, warn.conflicts = FALSE)) {
  if (!require("devtools")) install.packages("devtools")
  devtools::install_github("bjoleary/fdadata")
}
library(dplyr, quietly = TRUE, warn.conflicts = FALSE)
library(tidyr, quietly = TRUE, warn.conflicts = FALSE)
library(janitor, quietly = TRUE, warn.conflicts = FALSE)

# Set up -----
# We will look at a specific date range. 
fys_of_interest <- seq.int(from = 2005, to = 2024, by = 1)

# We will look at submissions in IVD panels
panels_of_interest <- 
  c(
    "Clinical Chemistry",
    "Hematology",
    "Immunology",
    "Microbiology",
    "Clinical Toxicology",
    "Pathology",
    "Molecular Genetics"
  ) |> 
  sort()

# Build a list of IVD submissions of interest
submissions_of_interest <- 
  # Start with the information available in the fdadata package. 
  # Note: If you adapt this analysis to look at recent submissions, you 
  # will need to refresh this data rather than just using the dataset provided 
  # in the package. 
  fdadata::premarket |> 
  # Add a column for FY decision
  dplyr::mutate(fy_decision = fdadata::fy(.data$date_decision)) |> 
  # Combine the "track" and "type" columns into one column named 
  # "review_process"
  tidyr::unite(
    col = "review_process",
    "type",
    "track", 
    sep = ": ",
    na.rm = TRUE
  ) |> 
  dplyr::mutate(review_process = forcats::as_factor(.data$review_process)) |> 
  # Filter the dataset down to the submissions of interest based on the 
  # panels and FYs we are looking at. 
  dplyr::filter(
    .data$panel %in% panels_of_interest,
    .data$fy_decision %in% fys_of_interest
  ) |> 
  # Add a column named "summary_available" that indicates whether or not there 
  # is a decision summary or SSED for the submission in the decision_summaries 
  # dataset. 
  dplyr::mutate(
    summary_available = 
      .data$submission_number %in% decision_summaries$submission_number
  )

# Filter the decision_summaries dataset down to the submissions of interest 
# identified in the previous steps. 
summaries_of_interest <- 
  decision_summaries |> 
  dplyr::filter(
    .data$submission_number %in% submissions_of_interest$submission_number
  )

# Count the number of decision summaries available and not available by 
# review process
summary_availability <- 
  submissions_of_interest |> 
  # Group and count
  dplyr::group_by(
    .data$review_process,
    .data$summary_available,
    .drop = FALSE
  ) |> 
  dplyr::tally() 

# Calculate percentages of decision summaries available by review process
summary_availability_stats <- 
  summary_availability |> 
  tidyr::pivot_wider(
    names_from = "summary_available",
    values_from = "n",
    values_fill = 0L
  ) |>
  janitor::clean_names() |> 
  dplyr::mutate(
    total = .data$false + .data$true,
    summary_proportion = 
      dplyr::case_when(
        .data$false + .data$true > 0 ~
          .data$true / (.data$true + .data$false),
        # We also need to deal with processes that were not used for IVDs but 
        # are still available, where the proportion will be 0 / 0. We'll set 
        # this value to be negative to make it easier to sort them to the 
        # bottom. 
        TRUE ~ -Inf
      ),
    summary_proportion_string = 
      dplyr::case_when(
        .data$false + .data$true > 0 ~
          paste0(
            .data$true, " / ", .data$true + .data$false, " = ",
            plyr::round_any(.data$true / (.data$true + .data$false) * 100, 1),
            "%"
          ),
        # We also need to deal with processes that were not used for IVDs but 
        # are still available, where the proportion will be 0 / 0: 
        TRUE ~ paste0(.data$true, " / ", .data$true + .data$false)
      )
  )

# Sort by review process by frequency of use...
summary_availability_stats$review_process <-
  forcats::fct_reorder(
    .f = summary_availability_stats$review_process,
    .x = summary_availability_stats$total,
    .na_rm = FALSE
  )
# ...and then by decision summary availability
summary_availability_stats$review_process <-
  forcats::fct_reorder(
    .f = summary_availability_stats$review_process,
    .x = summary_availability_stats$summary_proportion,
    .na_rm = FALSE
  )

# Plot the results
sum_plot <- 
  ggplot2::ggplot(
    data = summary_availability_stats,
    mapping = 
      ggplot2::aes(
        x = .data$review_process,
        y = .data$summary_proportion,
        label = .data$summary_proportion_string,
      )
  ) + 
  ggplot2::geom_col(fill = "darkgray") + 
  # Add text 
  ggplot2::geom_text(
    mapping = 
      ggplot2::aes(
        y = 0.05
      ),
    hjust = "left",
    size = 2.5
  ) +
  # Keep the axis simple
  ggplot2::scale_y_continuous(
    breaks = c(0, 1), 
    labels = c("0", "100%"),
    # Add some padding near 100% so the axis label doesn't get clipped. 
    expand = ggplot2::expansion(add = c(0, 0.05))
  ) +
  # Add a dashed line to the plot to make it easy to see where 100% is. 
  ggplot2::geom_hline(yintercept = 1, linetype = "dashed") +
  # Add titles and such. 
  ggplot2::labs(
    title = "Decision summary availability by review process",
    subtitle = 
      paste0(
        "For FDA IVD decisions made in FY", 
        min(fys_of_interest),
        " through FY",
        max(fys_of_interest)
      ),
    x = "Review Process",
    y = "Percent of decisions with decision summaries",
    caption = 
      paste0(
        "Treat these results as estimates.\n",
        "Includes decision summaries for 510(k) and De Novo decisions and ",
        "SSEDs for PMA decisions."
      )
  ) +
  # Make the bars horizontal instead of vertical so the category labels and 
  # text are easy to read. 
  ggplot2::coord_flip() + 
  ggplot2::theme_classic()  

plot(sum_plot)
Figure 1: Availability of decision summaries by review process and fiscal year of decision

For IVDs, the FDA frequently issues decision summaries for original and panel track PMAs, for De Novos, and for 510(k)s.

Exploring which IVD review processes result in shorter and longer decision summaries

The FDA’s regulations and review processes are intended to result in the availability of medical devices that are safe and effective. Some decisions are more difficult to make than others, and different review processes tend to result in different amounts of information being included in decision summaries. One simple related measure is page count, which is readily available in this dataset:

Code 
# Calculate page counts for each summary
summary_page_count <- 
  dplyr::left_join(
    x = 
      summaries_of_interest |> 
      dplyr::group_by(
        .data$submission_number
      ) |> 
      # Identify the highest page number for each submission
      dplyr::summarise(
        page_count = max(.data$page_number)
      ),
    y = submissions_of_interest,
    by = c("submission_number" = "submission_number")
  ) |> 
  dplyr::group_by(
    .data$review_process,
    .data$page_count,
    # Remove unused review processes for this analysis. 
    .drop = TRUE
  ) |> 
  dplyr::tally() |> 
  # Set colors for higher an lower page counts
  dplyr::mutate(
    fill_color = 
      dplyr::case_when(
        .data$page_count <= 2L ~ "darkred",
        TRUE ~ "darkgray"
      )
  )

# For this plot, sort review processes alphabetically
summary_page_count$review_process <-
  forcats::fct_relevel(
    .f = summary_page_count$review_process,
    sort
  )

# Plot
ggplot2::ggplot(
  data = summary_page_count,
  mapping = 
    ggplot2::aes(
      x = .data$page_count,
      y = .data$n,
      fill = .data$fill_color
    )
) + 
  ggplot2::facet_wrap(
    # One facet per review process
    ~.data$review_process, 
    ncol = 3L,
    # Let the y-axis vary by review process, but hold the x-axis constant 
    # across facets. 
    scales = "free_y",
    drop = TRUE
  ) +
  ggplot2::geom_col() + 
  ggplot2::labs(
    title = "Number of pages in FDA IVD decision summaries by review process",
    subtitle = 
      paste0(
        "For summaries of decisions made in FY", 
        min(fys_of_interest),
        " through FY",
        max(fys_of_interest)
      ),    
    x = "Number of pages in decision summary",
    y = "Number of decision summaries",
    caption = 
      paste0(
        "Page counts that are less than or equal to 2 are highlighted in red.",
        "\n",
        "Treat these results as estimates.\n",
        "Includes decision summaries for 510(k) and De Novo decisions and ",
        "SSEDs for PMA decisions."
      )
  ) + 
  ggplot2::scale_y_continuous(
    # Keep the y-axis breaks and labels to a minimum
    breaks = scales::extended_breaks(3)
  ) +
  ggplot2::scale_fill_manual(values = c("darkgray", "red")) + 
  ggplot2::theme_classic() + 
  # No legend -- we'll use the figure caption instead. 
  ggplot2::guides(fill = "none")
Figure 2: Page counts of decision summaries by review process

The red bars in Figure 2 illustrate that decision summaries may only be a page or two long for many submissions that go through the Special and Abbreviated 510(k) review processes and for a significant subset of those that go through the Traditional 510(k) review process.

Special 510(k)s are “for certain well-defined device modifications where a manufacturer modifies its own legally marketed device, and design control procedures produce reliable results that can form, in addition to other 510(k) content requirements, the basis for substantial equivalence (SE).” [1, p. 4] Similarly, Abbreviated 510(k) reviews “[rely] on guidance documents, special controls, and/or voluntary consensus standards” [2, p. 3] and on “summary reports” based on these approaches. In both cases, it makes sense that the FDA’s summary of its decision-making may be brief.

For Traditional 510(k)s, single-page decision summaries tend to be associated with the FDA’s “Triage-Quick Review Program,” where the posted decision summary consists of template language that refers users to the 510(k) summary for information about the basis for the FDA’s decision. Triage-Quick Review Program decision summaries also note that “This program represents an internal workload management tool intended to reduce internal FDA review resources for 510(k) applications that are of good quality upon receipt by FDA.”2 The FDA’s use of this program for Traditional 510(k)s peaked for decisions made in FY2015.

NoteFiscal Years

The FDA’s fiscal years begin in October and end in September. For example: FY2025 began on 2024-10-01. For more on fiscal years and decision cohorts, see this post.

Code 
# Identify Traditional 510(k) decisions that may have gone through the 
# Triage-Quick Review Program, meaning that their decision summaries are less 
# than 3 pages long.
triage_quick_review_likely <- 
  dplyr::left_join(
    x = submissions_of_interest,
    y =   
      summaries_of_interest |> 
      dplyr::group_by(.data$submission_number) |> 
      dplyr::summarise(
        short_summary = max(.data$page_number) < 3
      ),
    by = c("submission_number" = "submission_number")
  ) |> 
  dplyr::filter(
    .data$review_process %in% c("510(k): Traditional"),
    .data$short_summary
  )

# Check for the presence of the word "Triage" in the summaries identified in 
# the previous step. 
triage_confirmed <- 
  decision_summaries |> 
  dplyr::filter(
    .data$submission_number %in% triage_quick_review_likely$submission_number
    ) |> 
  # Make one text field to search for each submission. 
  tidyr::unite(col = "text_to_search", "text_embedded", "text_ocr") |> 
  dplyr::select(
    "submission_number",
    "text_to_search"
  ) |> 
  dplyr::group_by(.data$submission_number) |> 
  dplyr::summarise(
    text_to_search = paste0(.data$text_to_search, collapse = " ")
  ) |> 
  # Check for the word "Triage"
  dplyr::mutate(
    triage_confirmed = 
      stringr::str_detect(
        string = .data$text_to_search, 
        pattern = stringr::fixed("Triage", ignore_case = TRUE)
      )
  ) |> 
  # Join in the submissions of interest dataset. 
  dplyr::right_join(
    y = submissions_of_interest, 
    by = c("submission_number" = "submission_number")
  ) |> 
    dplyr::filter(
    .data$review_process %in% c("510(k): Traditional")
  ) |> 
  # Group by fiscal year of decision and count
  dplyr::group_by(.data$fy_decision) |> 
  dplyr::summarise(
    n = n(),
    triage = sum(.data$triage_confirmed, na.rm = TRUE)
  ) |> 
  # Calculate percentages by year
  dplyr::mutate(
    percent = .data$triage / .data$n,
    string = 
      dplyr::case_when(
        .data$triage > 0 ~
          paste0(
            .data$triage, " / ", .data$n, "\n(", 
            plyr::round_any(.data$triage / .data$n * 100, 1),
            "%)"
          ),
        TRUE ~ " "
      )
  )

# Plot
ggplot2::ggplot(
  data = triage_confirmed,
  mapping = 
    ggplot2::aes(
      x = .data$fy_decision,
      y = .data$percent,
      label = .data$string
    )
) + 
  ggplot2::geom_col(fill = "darkgray") + 
  ggplot2::geom_text(size = 1.75, vjust = 1.25) + 
  ggplot2::scale_x_continuous(
    breaks = fys_of_interest[(fys_of_interest %% 2) == 0]
  ) + 
  ggplot2::scale_y_continuous(
    expand = c(0, NA),
    # Set the y-axis limit to a reasonable, human-friendly value
    limits = 
      c(
        0, 
        plyr::round_any(
          x = max(triage_confirmed$percent, na.rm = TRUE), 
          accuracy = 0.05,
          f = ceiling
        )
      ),
    labels = scales::label_percent()
  ) + 
  ggplot2::labs(
    title = "Estimated use of the Triage-Quick Review Program over time",
    subtitle = 
      paste0(
        "For IVD Traditional 510(k)s with decisions made in FY", 
        min(fys_of_interest),
        " through FY",
        max(fys_of_interest)
      ),    
    x = "FY Decision",
    y = "Traditional 510(k)s in Triage-Quick Review Program",
    caption = 
      paste0(
        "Treat these results as estimates.\n",
        "Triage-Quick Review submissions were identified based on the length ",
        "and content of decision summary documents."
      )
  ) + 
  ggplot2::theme_classic()
Figure 3: Use of the “Triage-Quick Review Program” for IVD Traditional 510(k)s over time.

Given that a summary that is only a page or two long may not be as informative or interesting, we can revise Figure 1 and visualize the rate at which longer decision summaries are available by review process:

Code 
# Similar to the first plot, but this time, limit it to those with decision 
# summaries longer than 2 pages. 
long_summary_availability <- 
  dplyr::left_join(
    x = submissions_of_interest,
    y =   
      summaries_of_interest |> 
      dplyr::group_by(.data$submission_number) |> 
      dplyr::summarise(
        long_summary = max(.data$page_number) > 2
      ),
    by = c("submission_number" = "submission_number")
  ) |> 
  # Calculate a composite metric that we can use to say whether or not a 
  # decision summary is available and is greater than 2 pages long. 
  dplyr::mutate(
    long_summary_composite = 
      dplyr::case_when(
        is.na(.data$long_summary) ~ FALSE,
        .data$long_summary == FALSE ~ FALSE,
        .data$summary_available == FALSE ~ FALSE,
        .data$long_summary == TRUE ~ TRUE
      )
  ) |> 
  # Group and count
  dplyr::group_by(
    .data$review_process,
    .data$long_summary_composite,
    .drop = FALSE
  ) |> 
  dplyr::tally() 

# Calculate percentages
long_summary_availability_stats <- 
  long_summary_availability |> 
  tidyr::pivot_wider(
    names_from = "long_summary_composite",
    values_from = "n",
    values_fill = 0L
  ) |>
  janitor::clean_names() |> 
  dplyr::mutate(
    total = .data$false + .data$true,
    summary_proportion = .data$true / (.data$true + .data$false),
    summary_proportion_string = 
      paste0(
        .data$true, " / ", .data$true + .data$false, " = ",
        plyr::round_any(.data$true / (.data$true + .data$false) * 100, 1),
        "%"
      )
  ) |> 
  dplyr::arrange(
    dplyr::desc(.data$total)
  ) 

# Sort by review process by frequency of use...
long_summary_availability_stats$review_process <-
  forcats::fct_reorder(
    .f = long_summary_availability_stats$review_process,
    .x = long_summary_availability_stats$total,
    .na_rm = FALSE
  )
# ...and then by decision summary availability
long_summary_availability_stats$review_process <- 
  forcats::fct_reorder(
    .f = long_summary_availability_stats$review_process, 
    .x = long_summary_availability_stats$summary_proportion,
    .na_rm = TRUE
  ) 

# Plot
long_sum_plot <- 
  ggplot2::ggplot(
    data = 
      long_summary_availability_stats |> 
      dplyr::filter(.data$summary_proportion > 0),
    mapping = 
      ggplot2::aes(
        x = .data$review_process,
        y = .data$summary_proportion,
        label = .data$summary_proportion_string,
      )
  ) + 
  ggplot2::geom_col(fill = "darkgray") + 
  ggplot2::geom_text(
    mapping = 
      ggplot2::aes(
        y = 0.02
      ),
    hjust = "left",
    size = 2.5
  ) +
  ggplot2::scale_y_continuous(
    breaks = c(0, 1), 
    labels = c("0", "100%"),
    expand = ggplot2::expansion(add = c(0, 0.05))
  ) +
  ggplot2::labs(
    title = 
      paste0(
        "Availability of decision summaries longer ",
        "than 2 pages \n",
        "by review process"
        ),
    subtitle = 
      paste0(
        "For FDA IVD decisions made in FY", 
        min(fys_of_interest),
        " through FY",
        max(fys_of_interest)
      ),
    x = "Review Process",
    y = "Percent of decisions with decision summaries >2 pages",
    caption = 
      paste0(
        "Treat these results as estimates.\n",
        "Includes decision summaries for 510(k) and De Novo decisions and ",
        "SSEDs for PMA decisions."
      )
  ) +
  ggplot2::geom_hline(yintercept = 1, linetype = "dashed") +
  ggplot2::coord_flip() + 
  ggplot2::theme_classic() 

plot(long_sum_plot)
Figure 4: Availability of decision summaries that are longer than 2 pages by review process and fiscal year of decision

Many decision summaries are (appropriately) brief. For example, while the FDA published decision summaries for 791 / 821 = 96% of IVD decisions made through the Special 510(k) process (see Figure 1), only 121 / 821 = 15% of IVD Special 510(k) decisions are associated with decision summaries that are longer than 2 pages.

Exploring the use of FDA-recognized CLSI publications

Many IVD reviews rely on the use of standards and guidelines from the Clinical and Laboratory Standards Institute (CLSI). The FDA “recognizes” many CLSI publications, meaning that the FDA has determined that the approaches described in these documents can help manufacturers meet regulatory requirements. [3, p. 2] The FDA maintains a helpful database of FDA-recognized standards, and this database allows one to export a list of FDA-recognized CLSI publications to a CSV file.

We can use the exported list of FDA-recognized CLSI publications to build search terms, and then we can see how often those terms appear in the FDA’s decision summaries.

We’ll look for each of these terms in decision summaries for decisions made in FY2023.

Code 
# Get the file from FDA.gov or download it here:
# fs::dir_create("data-raw/")
# utils::download.file(
#   url = 
#     paste0(
#       "https://www.boleary.com/blog/posts/202410-ivd-decision-summaries/",
#       "data/clsi_standards.csv"
#     ),
#   destfile = "data-raw/clsi_standards.csv"
# )

if (!exists("submissions_of_interest")) {
  message("Run the code from the previous examples first.")
}
if (!exists("decision_summaries")) {
  message("Run the code from the previous examples first.")
}

# Load additional libraries ----- 
library(stringr)

# Read in the standards data ----- 
clsi_standards <- 
  readr::read_delim(
    # Make sure your CLSI standards file is saved at this path: 
    file = "data-raw/clsi_standards.csv",
    col_types = 
      readr::cols(
        `Date of Entry` = readr::col_character(),
        `Specialty Task Group Area` = readr::col_character(),
        `Recognition Number` = readr::col_character(),
        `Extent of Recognition` = readr::col_character(),
        `Standard Developing Organization` = readr::col_character(),
        `Standard Designation Number/Date` = readr::col_character(),
        ` Title of Standard` = readr::col_character(),
        `Transition Expiration Date` = readr::col_character()
      )
  ) |> 
  janitor::clean_names() |> 
  dplyr::mutate(
    date_of_entry = lubridate::mdy(.data$date_of_entry),
    transition_expiration_date = 
      lubridate::mdy(.data$transition_expiration_date),
    standard_name = 
      stringr::str_extract(
        string = .data$standard_designation_number_date,
        # This pattern should grab the first "word" that is the main 
        # identifier for the standard. 
        pattern = "^[\\w/-]*"
      ) |> 
      stringr::str_remove(
        # This pattern should remove "-1st", "-2nd", "-3rd", "4th", etc.
        pattern = stringr::regex("-\\d*(st|nd|rd|th)")
      )
  ) |> 
  dplyr::mutate(
    standard_name_length = stringr::str_length(.data$standard_name)
  ) |> 
  dplyr::select(
    "standard_designation_number_date",
    "standard_name",
    "standard_name_length",
    dplyr::everything()
  ) 

# Confirm that the number of unique values in "standard_name" is the same as
# the number of rows in the table. 
duplicates <-   
  length(clsi_standards$standard_name) != length(unique(clsi_standards$standard_name))
if (duplicates) {
  message(
    paste0(
      "At least one search term is showing up more than once. ",
      "As of this writing, both the 5th and 6th edition of CLSI M23 are ",
      "recognized, so this is expected. Consider doing a manual check."
      ) |> 
      stringr::str_wrap(width = 75)
  )
}

# Search -----
search_terms <- 
  unique(sort(clsi_standards$standard_name))

# Put word boundaries around the search terms and make them case-insensitive
search_terms_regex <- 
  paste0(search_terms, "\\b[^-]") |> 
  stringr::regex(ignore_case = TRUE)

# Identify IVD decisions made in FY2023
ivd_2023 <- 
  submissions_of_interest |> 
  dplyr::filter(
    .data$date_decision >= lubridate::ymd("2022-10-01"),
    .data$date_decision < lubridate::ymd("2023-10-01")
  )

# Structure the data to be searched
search_domain <- 
  decision_summaries |> 
  dplyr::filter(.data$submission_number %in% ivd_2023$submission_number) |> 
  # Make one text field for each submission
  tidyr::unite(col = "text_to_search", "text_embedded", "text_ocr") |> 
  dplyr::select(
    "submission_number",
    "text_to_search"
  ) |> 
  dplyr::group_by(.data$submission_number) |> 
  dplyr::summarise(
    text_to_search = paste0(.data$text_to_search, collapse = " ")
  )

# For each search term, add a new column with the search results. 
for (i in seq_along(search_terms)) {
  column_name <- search_terms[[i]]
  search_domain <- 
    search_domain |> 
    dplyr::mutate(
      {{ column_name }} :=
        stringr::str_detect(
          string = .data$text_to_search,
          pattern = search_terms_regex[[i]]
        )
    )
}

# Tally up the search results for each term. 
search_results <- 
  search_domain |> 
  dplyr::select(-"text_to_search") |> 
  tidyr::pivot_longer(
    cols = tidyr::all_of(search_terms),
    names_to = "search_term",
    values_to = "result"
  ) |> 
  dplyr::group_by(.data$search_term) |> 
  dplyr::summarise(count = sum(.data$result)) |> 
  dplyr::filter(.data$count > 0) |> 
  # Sort by frequency
  dplyr::arrange(.data$count) |> 
  # Join in some metadata about the CLSI documents
  dplyr::left_join(
    y = 
      clsi_standards |> 
      dplyr::select("standard_name", "title_of_standard"),
    by = c("search_term" = "standard_name")
  ) |> 
  dplyr::mutate(search_term = forcats::as_factor(.data$search_term))

# Save the results
readr::write_excel_csv(
  x = search_results,
  file = "data/search_results.csv"
)

# Plot the top 10 results
top_number <- 10
ggplot2::ggplot(
  data = search_results |> tail(top_number),
  mapping = 
    ggplot2::aes(
      x = .data$search_term,
      y = .data$count
    )
) +
  ggplot2::geom_col(
    color = "lightgray",
    fill = "lightgray"
  ) + 
  ggplot2::geom_text(
    mapping = 
      ggplot2::aes(
        y = 1,
        label = 
          paste0(
            .data$title_of_standard |> 
              stringr::str_trunc(width = 100),
            "\n",
            "Referenced in ",
            .data$count,
            " decision summaries"
          ),
      ),
    hjust = 0,
    color = "black",
    size = 3
  )+
  ggplot2::labs(
    title = 
      paste0(
        "Top ", 
        top_number, 
        " CLSI documents referenced in FDA decision summaries"
      ),
    subtitle = "For IVD decisions made in FY2023",
    x = "CLSI document",
    y = "Count of FDA decision summaries found to reference the CLSI Document",
    caption = 
      paste0(
        "Treat these results as estimates.\n",
        "Includes decision summaries for 510(k) and De Novo decisions and ",
        "SSEDs for PMA decisions."
      )
  ) + 
  ggplot2::coord_flip() +
  ggplot2::theme_classic()
Figure 5: CLSI documents referenced in FDA decision summaries. Raw data (CSV).

CLSI’s “EP” documents (evaluation protocols) feature prominently in the results (CSV), with protocols for evaluating quantitative tests topping the list.

References

[1]
“The special 510(k) program,” U.S. Food and Drug Administration, Final Guidance Document. Docket Number FDA-2018-D-3304, Sep. 2019. Available: https://www.regulations.gov/document/FDA-2018-D-3304-0018
[2]
“The abbreviated 510(k) program,” U.S. Food and Drug Administration, Final Guidance Document. Docket Number FDA-2019-D-4015, Sep. 2019. Available: https://www.regulations.gov/document/FDA-2019-D-4015-0001
[3]
“Recognition and withdrawal of voluntary consensus standards,” U.S. Food and Drug Administration, Final Guidance Document. Docket Number FDA-2018-D-2936, Sep. 2020. Available: https://www.regulations.gov/document/FDA-2018-D-2936-0010

Footnotes

  1. I’ll use the term “IVD” throughout this post for convenience and readability, and, for the purposes of this post, “IVD” includes anything the FDA has reviewed and authorized under one of its IVD review panels: Clinical Chemistry, Clinical Toxicology, Hematology, Immunology, Microbiology, Molecular Genetics, and Pathology. As of publication, the FDA includes laboratory developed tests (LDTs) or laboratory developed procedures (LDPs) in its use of the term “IVD,” and the FDA has authorized some LDTs or LDPs through the premarket programs discussed here.↩︎

  2. See, for example: https://www.accessdata.fda.gov/cdrh_docs/reviews/K242540.pdf.↩︎

Reuse

]]> 510(k) datasets de novo FDA medical devices pma R regulatory science https://www.boleary.com/blog/posts/202501-ivd-decision-summaries/ Thu, 09 Jan 2025 05:00:00 GMT Finding FDA authorized devices with Predetermined Change Control Plans Brendan O'Leary https://www.boleary.com/blog/posts/202408-pccp/

Background

Note

The information in this post is based on sources in the public domain. It is provided here “as-is” without warranty of any kind. You may also want to refer to the FDA’s website.

Predetermined Change Control Plans (PCCPs) can enable significant modifications to be made to medical devices that have been reviewed by the FDA. When these modifications are made in accordance with a PCCP that the FDA has authorized, the modifications themselves do not require additional review by the FDA.

In December 2022, the Food and Drug Omnibus Reform Act of 2022 provided the FDA with new authority related to the authorization of PCCPs for medical devices.1 PCCPs are now described in section 515C of the Food, Drug, and Cosmetic Act (FD&C Act). Paraphrased and simplified, this says:

As a part of a premarket submission, the FDA may authorize PCCPs for “planned changes that may be made to the device” as long as “the device remains safe and effective without any such change” and, for 510(k) devices, “the device would remain substantially equivalent to the predicate.”2

Section 515C of the FD&C Act also describes what the FDA may require a PCCP to include.3 The FDA may require that a PCCP include information about:

  • Labeling required for safe and effective use as the device changes,

  • Notification requirements if the device does not function as intended, and

  • Performance requirements for changes made under the plan.

In addition, in April 2023, the FDA issued a draft guidance with proposed recommendations related to PCCPs for artificial intelligence and machine learning enabled medical devices.4 [1]

NoteUpdates

On 2024-12-03, the FDA revised and finalized its guidance on PCCPs for artificial intelligence and machine learning enabled medical devices. [2]

On 2024-08-21, the FDA published a new draft guidance on PCCPs for any device (not just AI or ML related devices) on its website: Predetermined Change Control Plans for Medical Devices. [3]

Even when a device developer does not plan to establish a PCCP for a medical device under development, the existence of a PCCP for a predicate device can impact the regulatory strategy for a device that will go through the 510(k) pathway. The statute says:

Only the version of the device cleared or approved, prior to changes made under the predetermined change control plan, may be used by a sponsor as a predicate device.5

See the example in Figure 1, which shows a potential predicate device that has been cleared by the FDA with a PCCP (Version 1), modified under the PCCP (Version 2), modified again without additional FDA review under the 510(k) modifications policy [4] (Version 3), cleared again by the FDA (Version 4), and then modified again under the PCCP (Version 5). If a device developer wants to use this device as a predicate device, the developer should compare the new device to either Version 1 or Version 4 because the other versions are not a “version of the device cleared or approved, prior to changes made under the predetermined change control plan”.

It is important for a device developer to know whether a device they are considering using as a predicate has a PCCP.

Finding devices with PCCPs using a search engine

One way to identify devices that may include a PCCP is through a site-specific search engine query. An example, showing results from Google as of 2024-04-19, is shown here6:

The figure shows Google search results for devices with PCCPs as of 2024-04-19. 21 results were found using a query for "PCCP OR "predetermined change" site:accessdata.fda.gov/cdrh_docs/". The first four results are shown, including DEN220063 from Caption Health, a 510(k) from Caption Health, a 510(k) from Apple, and a 510(k) from Medtronic.
Figure 2: Google search results for devices with PCCPs as of 2024-04-19.

This shows that there were 21 documents that included either the term “PCCP” or the term “predetermined change” on the site where the FDA posts submission and review summaries as of 2024-04-19. Some submissions returned multiple documents because, for example, both a submission summary document and a review summary document may be available for De Novos and for in vitro diagnostic 510(k)s. In addition, some of the results may be included because a 510(k) summary discusses a PCCP for a predicate device even if the device that was the subject of the 510(k) did not include a PCCP.

The search results show that the FDA authorized a few devices with PCCPs before the law was passed in December 2022. Several of these devices came to market through the De Novo program or in regulations recently established through the De Novo program, and the De Novo authorizations associated with these devices typically also established special controls related to PCCPs.

The results show that the FDA is including additional language in clearance letters when there is a PCCP. Specifically, many of these letters include the following paragraph:

FDA’s substantial equivalence determination also included the review and clearance of your Predetermined Change Control Plan (PCCP). Under section 515C(b)(1) of the Act, a new premarket notification is not required for a change to a device cleared under section 510(k) of the Act, if such change is consistent with an established PCCP granted pursuant to section 515C(b)(2) of the Act. Under 21 CFR 807.81(a)(3), a new premarket notification is required if there is a major change or modification in the intended use of a device, or if there is a change or modification in a device that could significantly affect the safety or effectiveness of the device, e.g., a significant change or modification in design, material, chemical composition, energy source, or manufacturing process. Accordingly, if deviations from the established PCCP result in a major change or modification in the intended use of the device, or result in a change or modification in the device that could significantly affect the safety or effectiveness of the device, then a new premarket notification would be required consistent with section 515C(b)(1) of the Act and 21 CFR 807.81(a)(3). Failure to submit such a premarket submission would constitute adulteration and misbranding under sections 501(f)(1)(B) and 502(o) of the Act, respectively. Additional information about changes that may require a new premarket notification are provided in the FDA guidance documents entitled “Deciding When to Submit a 510(k) for a Change to an Existing Device” (https://www.fda.gov/media/99812/download) and “Deciding When to Submit a 510(k) for a Software Change to an Existing Device” (https://www.fda.gov/media/99785/download).

This language is informative, and it can also help us find devices that have PCCPs. If the FDA did not include this language in its letters, we would need to rely on the 510(k) summary – which is written by the device developer and is not available for all devices – to determine whether a 510(k) clearance included a PCCP. Still, even though this language is appearing in letters from the FDA, we should not assume perfection: There may be cases where the FDA has included this language in a clearance letter when a PCCP was not part of the submission, and there may also be cases where the FDA did not include this language in a clearance letter when a PCCP was part of the clearance.7

The FDA also appears to be noting the presence of PCCPs in approval order statements for PMAs.8

PCCP authorizations over time and by device type

Results from the search shown in Figure 2, results from searching for the terms “PCCP” and “predetermined change” in the first several pages of publicly available clearance letters and De Novo reclassification orders using the dataset available here, and results from searching PMA approval order statements are available below.9 The results of this search suggest that, as of 2024-08-01, there had been about 35 authorizations of devices that included PCCPs.

It is important to treat these results as estimates for the reasons discussed in the previous section.

In addition, foundations for the PCCP framework were laid in earlier decisions that used similar concepts and approaches, but the search techniques shown here are not designed to find these precedents. Most of the search results were recently authorized devices:

For example, after authorizing P160045 for the Oncomine Dx Target Test from Thermo Fisher Scientific (approved on 2017-06-22), DEN170058 for the MSK-IMPACT tumor profiling test from the Memorial Sloan-Kettering Cancer Center (granted on 2017-11-15), and P170019 for the FoundationOne CDx from Foundation Medicine (approved on 2017-11-30), the FDA issued a Fact Sheet on how additional cancer mutations with potential clinical significance could be added to tumor profiling tests without further review by the FDA. [5]

Figure 3: Approximate number of submissions authorized by the FDA with a PCCP. Raw data (CSV).

Many of the artificial intelligence and machine learning enabled devices the FDA has authorized are cardiovascular and radiology devices10, and, based on this search, these device areas also appear to have many devices with PCCPs.

Figure 4: Approximate number of submissions authorized by the FDA with a PCCP by panel. Raw data (CSV).

The modifications permitted under authorized PCCPs can be consequential. They have included ML model retraining, breakpoint and cutoff changes, reporting of additional genetic variants, and more. You can explore the search results below. Notably, some of the types of changes proposed and included in authorized PCCPs may have already been permissible without a PCCP under established modifications policies. There are important tradeoffs device developers should consider before proposing to establish a PCCP for modifications that may already be permissible without review under one of the FDA’s existing policies.

FDA Regulatory Expertise:
I provide digital health and medical device regulatory strategy solutions to technology developers, healthcare organizations, trade and professional associations, and others. Book a meeting.

Devices found using this search

Note

This table shows search results as of about 2024-08-01. Updates will be posted here.

Table 1: Devices found using this search. Raw data (CSV).

References

[1]
“Marketing submission recommendations for a predetermined change control plan for artificial intelligence/machine learning (AI/ML)-enabled device software functions,” U.S. Food and Drug Administration, Draft Guidance Document. Docket Number FDA-2022-D-2628, Apr. 2023. Available: https://www.regulations.gov/document/FDA-2022-D-2628-0002
[2]
“Marketing submission recommendations for a predetermined change control plan for artificial intelligence-enabled device software functions,” U.S. Food and Drug Administration, Final Guidance Document. Docket Number FDA-2022-D-2628, Dec. 2024. Available: https://www.regulations.gov/document/FDA-2022-D-2628-0036
[3]
“Predetermined Change Control Plans for medical devices,” U.S. Food and Drug Administration, Draft Guidance Document. Docket Number FDA-2024-D-2338, Aug. 2024. Available: https://www.regulations.gov/document/FDA-2024-D-2338-0002
[4]
“Deciding when to submit a 510(k) for a change to an existing device,” U.S. Food and Drug Administration, Final Guidance Document. Docket Number FDA-2011-D-0453, Oct. 2017. Available: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/deciding-when-submit-510k-change-existing-device
[5]
“FDA fact sheet: CDRH’s approach to tumor profiling next generation sequencing tests,” U.S. Food and Drug Administration, Fact Sheet, Nov. 2017. Available: https://www.fda.gov/files/medical%20devices/published/CDRH’s-Approach-to-Tumor-Profiling-Next-Generation-Sequencing-Tests.pdf

Footnotes

  1. See section 3308 of the Food and Drug Omnibus Reform Act of 2022, Title III of Division FF of the Consolidated Appropriations Act, 2023, Pub. L. No. 117-328 (“FDORA”), enacted on December 29, 2022 and available at: https://www.congress.gov/117/plaws/publ328/PLAW-117publ328.pdf, pp. 5835-5836.↩︎

  2. See: FD&C Act Section 515C(a)(2) and (b)(2).↩︎

  3. See: FD&C Act Section 515C(a)(3) and (b)(3).↩︎

  4. See: https://www.fda.gov/medical-devices/medical-devices-news-and-events/cdrh-issues-draft-guidance-predetermined-change-control-plans-artificial-intelligencemachine.↩︎

  5. See: FD&C Act Section 515C(c).↩︎

  6. See: https://www.google.com/search?q=PCCP+OR+%22predetermined+change%22+site%3Aaccessdata.fda.gov%2Fcdrh_docs%2F.↩︎

  7. I shared a draft of this post with the FDA before publishing it, and the FDA subsequently issued corrections for some of the clearance letters that had been posted on the FDA’s website. These corrections are accounted for in the results provided here.↩︎

  8. See for example, P190016/S007 on 2024-02-01 and P160026/S040 on 2024-04-12.↩︎

  9. To search the approval order statements, I used my fdadata package for the R programming language: https://github.com/bjoleary/fdadata.↩︎

  10. See: https://www.fda.gov/medical-devices/software-medical-device-samd/artificial-intelligence-and-machine-learning-aiml-enabled-medical-devices.↩︎

Reuse

Citation

BibTeX citation:
@online{o'leary2024,
  author = {O’Leary, Brendan},
  title = {Finding {FDA} Authorized Devices with {Predetermined}
    {Change} {Control} {Plans}},
  date = {2024-08-20},
  url = {https://www.boleary.com/blog/posts/202408-pccp/},
  langid = {en},
  abstract = {In December 2022, the Food and Drug Omnibus Reform Act of
    2022 provided the FDA with new authority related to the
    authorization of Predetermined Change Control Plans (PCCPs). A PCCP
    can enable significant planned modifications to be made to a medical
    device after the device and the plans to modify the device have been
    reviewed by the FDA. This post provides a summary of the law,
    describes how to find devices that have been authorized with PCCPs,
    lists devices that I found using these approaches, shows the
    approximate rate at which the FDA has authorized these devices over
    time based on the search results, and shows the device areas where
    PCCPs are most commonly used based on the search results.}
}
For attribution, please cite this work as:
B. O’Leary, “Finding FDA authorized devices with Predetermined Change Control Plans,” Aug. 20, 2024. https://www.boleary.com/blog/posts/202408-pccp/
]]> AI/ML datasets FDA medical devices https://www.boleary.com/blog/posts/202408-pccp/ Tue, 20 Aug 2024 15:00:00 GMT List of FDA authorized devices with Predetermined Change Control Plans Brendan O'Leary https://www.boleary.com/blog/posts/202408-pccp/latest/
Note

The information on this page is based on sources in the public domain. It is provided here “as-is” without warranty of any kind. You may also want to refer to the FDA’s website.

In May 2025, the FDA added a PCCP search feature to its 510(k), De Novo, and PMA databases. This page continues to use the search method described here and may not match the FDA’s results.

For devices with 510(k)s, this list should include search results as of about 2026-02-27. For devices with De Novos, this list should include search results as of about 2026-02-11. For devices with PMAs, this list should include search results as of about 2026-03-20.

As explained in this post, the search results shown here may not include every device with a PCCP, and the search results shown here may include some devices that do not actually have a PCCP. It’s important to treat these search results, and any analyses based on these search results, as preliminary estimates. For example, the FDA sometimes sends authorization letters saying PCCPs are present when they aren’t. When the FDA finds these mistakes, they send new letters to correct them. This may lead to devices being removed from the list. You can download a CSV file with details on devices affected by these corrections: Corrections (CSV).

You can also download a CSV file with details about the devices on this list: Raw data (CSV).

Table 1: Search results – List of FDA authorized devices with PCCPs. Raw data (CSV).

Reuse

Citation

BibTeX citation:
@online{o'leary2024,
  author = {O’Leary, Brendan},
  title = {List of {FDA} Authorized Devices with {Predetermined}
    {Change} {Control} {Plans}},
  date = {2024-08-20},
  url = {https://www.boleary.com/blog/posts/202407-pccp/latest/},
  langid = {en},
  abstract = {This page lists medical devices that have FDA authorized
    Predetermined Change Control Plans (PCCPs) based on searches of
    device authorization letters and approval order statements.}
}
For attribution, please cite this work as:
B. O’Leary, “List of FDA authorized devices with Predetermined Change Control Plans,” Aug. 20, 2024. https://www.boleary.com/blog/posts/202407-pccp/latest/
]]> datasets FDA medical devices https://www.boleary.com/blog/posts/202408-pccp/latest/ Tue, 20 Aug 2024 15:00:00 GMT A transatlantic take on medical device regulation Brendan O'Leary https://www.boleary.com/blog/posts/202407-rsm-podcast/ I enjoyed having this conversation with Dr. Annabelle Painter and Johan Ordish about AI and medical device regulation. The RSM Digital Health Section Podcast is always worth a listen, and you should subscribe.

This post originally appeared on LinkedIn.

]]> digital health FDA https://www.boleary.com/blog/posts/202407-rsm-podcast/ Tue, 16 Jul 2024 04:00:00 GMT Listener Effort is a promising digital measure in ALS research Brendan O'Leary https://www.boleary.com/blog/posts/202406-listener-effort/ This preprint [1] covers several important advances in the development of new digital-enabled measures of ALS progression:

  • ALS symptoms can result in speech that can be difficult to understand (dysarthria)

  • “Listener effort” is a well-established measure related to dysarthria in speech-language pathology

  • Speech-language pathologists can assess listener effort reliably and rapidly

  • Listener effort correlates with well-established measures of ALS progression in individuals who have bulbar symptoms (e.g., symptoms that affect speech and swallowing)

  • Speech-language pathologist assessment of listener effort may help measure disease progression in these individuals with more granularity than can be provided by traditional measures alone

  • Machine learning models trained to measure listener effort also show promising performance

A key quote:

Not only is [listener effort] a reliable, quantitative endpoint of bulbar function, it is also an inherently clinically meaningful endpoint. The effect of dysarthria on communication and, in turn, quality of life, is well accepted in clinical settings. The EFNS-ALS guidelines suggest assessment of communication and treatment with communication support systems. People living with ALS worry about losing the ability to communicate. Decreased speech function on the ALSFRS-R question 1 (speech), is associated with poorer quality of life (QoL) on the ALS-Specific QoL Questionnaire (ALSSQoL). Furthermore, augmentative communication devices stabilize or improve both the quality of life and mood in people with dysarthria due to ALS Similar impacts of progressive dysarthria on quality of life have been demonstrated in Parkinson’s disease and other neurological disorders.1

The preprint is worth reading, and the data is available to other researchers through Everything ALS.

A version of this post originally appeared on LinkedIn.

References

[1]
I. N. Bingham et al., “Listener effort quantifies clinically meaningful progression of dysarthria in people living with amyotrophic lateral sclerosis,” medRxiv, 2024, doi: 10.1101/2024.05.31.24308140.

Footnotes

  1. See the preprint for additional context and references.↩︎

Reuse

]]> FDA digital health AI/ML drugs https://www.boleary.com/blog/posts/202406-listener-effort/ Mon, 03 Jun 2024 04:00:00 GMT “We don’t have a national active monitoring system for pandemic potential diseases, and it’s kind of crazy…” Brendan O'Leary https://www.boleary.com/blog/posts/202405-pandemic-potential/ Steven Roach of the Food Animal Concerns Trust, as quoted in STAT’s article on the USDA and FDA turf battles that are hampering responses to outbreaks like H5N1 bird flu:

I would say the biggest problem is we don’t have a national active monitoring system for pandemic potential diseases, and it’s kind of crazy that we don’t after Covid.

This post originally appeared on LinkedIn.

]]> FDA https://www.boleary.com/blog/posts/202405-pandemic-potential/ Tue, 14 May 2024 04:00:00 GMT EHRs bury critical information about hearing status Brendan O'Leary https://www.boleary.com/blog/posts/202403-can-you-hear-me/ We rely on Electronic Health Records (EHRs) to inform the care team about an individual’s hearing loss or deafness. EHR systems bury this critical information, and care teams need more conspicuous and persistent indicators both within and beyond the EHR. In this JAMA Internal Medicine Viewpoint, Dianne O’Leary and Timothy O’Leary highlight the alarming patient safety and health equity implications of the current approach and offer solutions. [1]

This post originally appeared on LinkedIn.

References

[1]
D. P. O’Leary and T. J. O’Leary, “Communicating With Patients With Hearing Loss or Deafness—Can You Hear Me?” JAMA Internal Medicine, Feb. 2024, doi: 10.1001/jamainternmed.2023.8563.
]]> digital health ehr health equity https://www.boleary.com/blog/posts/202403-can-you-hear-me/ Sun, 03 Mar 2024 05:00:00 GMT Downloadable spreadsheets and graphs of the FDA’s medical device review performance metrics Brendan O'Leary https://www.boleary.com/blog/mdufa/202401/

Introduction

Each quarter, the FDA publishes public performance reports on regulatory activities associated with the medical device review program. These reports are published as PDFs, which can make it difficult to analyze and use the data. And there’s a lot of data. For example, the most recent MDUFA IV performance report includes more than 14,000 individual performance metrics. When finished, the FDA is on track to publish more than 13,000 individual performance metrics from the MDUFA 5 program. For contrast, the MDUFA 2 program only resulted in about 240 published metrics. To make this information easier to access, analyze, and use, I’ve written software to scrape the PDF reports and extract most of the data.

You can download the resulting datasets below. They include more than 24,600 performance metrics.

This post also provides:

  • A plot of 510(k) volume by review group over time, including projections of future 510(k) volume
  • A plot of which 510(k) receipt cohorts are associated with final data
  • Plots that show how long 510(k)s spend “under review” with different review groups at the FDA
  • A plot that shows how long 510(k)s spend “on hold” while applicants respond to the FDA’s questions by FDA review group
  • Plots that show how long the 510(k) review process takes overall by FDA review group
  • Plots that show about how often different FDA review groups make substantive requests for additional information from 510(k) applicants
  • A plot that shows the rates at which different FDA review groups have cleared 510(k)s over time

These plots are based on the information in the December 10, 2018 MDUFA 3 report and the November 16, 2023 MDUFA 4 and MDUFA 5 reports. [1] [2] [3] Information about various factors that may affect this data and a glossary of MDUFA jargon are also included in this post.

Downloads

You can download all of the figures in this post as a single PDF that should print well on letter paper. Throughout this post, the captions for each figure include links to CSV files that contain the data used in the figures.

You can access data from the FDA’s MDUFA performance reports in spreadsheet form here:

FDA Regulatory Expertise:
I provide digital health and medical device regulatory strategy solutions to technology developers, healthcare organizations, trade and professional associations, and others. Book a consultation.

Aside from some spot-checks, the datasets provided here have had limited verification and may include some errors.

In addition, data from some tables in the FDA’s MDUFA performance reports are missing from these datasets.

Use this information at your own risk and verify information using the reports provided directly by the FDA. To facilitate this, each data point provided includes information about its source, including a link to the FDA report from which it came, the relevant page number, and more. If you find a problem in these datasets, please report the problem here.

If you’re familiar with the R programming language, you can also access these datasets using the mdufa package, which is available on GitHub. You are welcome to use and modify this software, including for commercial purposes, under the MIT license. (In addition, this post is under the CC BY license, which permits sharing and adaptation, including commercially.)

Context for these datasets

This post includes visualizations of performance metrics related to the “Medical Device User Fee Amendments” or “MDUFA.” Every five years since fiscal year (FY) 2003, Congress has authorized (or re-authorized) the FDA to collect user fees to support the FDA’s regulatory activities related to medical devices. Each of these five-year periods has been named sequentially: MDUFA or MDUFA I (FY 2003 through FY 2007), MDUFA II (FY 2008 through FY 2012), MDUFA III (FY 2013 through FY 2017), MDUFA IV (FY 2018 through FY 2022) and MDUFA V (FY 2023 through FY 2027).

This post will use “MDUFA 1,” “MDUFA 2,” “MDUFA 3,” “MDUFA 4,” and “MDUFA 5” to refer to these periods. The term “MDUFA” will be used to refer to any or all of these periods.

Before each re-authorization, the FDA has negotiated with representatives from the medical device industry, patient representatives, and other members of the public to arrive at a set of commitments that can be kept given the additional resources user fees provide. This results in a “MDUFA Commitment Letter,” a draft of which is provided to Congress in advance of re-authorization.

As a result of these commitments, since at least MDUFA 2, the FDA has published quarterly and annual “MDUFA Reports” that include performance metrics related to the FDA’s medical device regulatory activities with a particular focus on review times for premarket submissions for medical devices.

Definitions

The definitions below are based on the information in the MDUFA 5 Commitment Letter with some simplifications. [4]

Applicant
The person or organization providing a 510(k) or other submission to the FDA. [4, p. 32]
FDA Days
The number of calendar days that a submission was “under review” at the FDA, not including:
  • any time the FDA spent reviewing information that could not be accepted or filed because it was not administratively complete, or
  • any time the FDA spent waiting for an applicant to respond to a substantive request for additional information. [4, p. 32]
Fiscal Year (FY):
A period beginning on October 1 and ending on the following September 30. For example, the period between 2023-10-01 and 2024-09-30 is “FY 2024.”
Industry Days
The number of calendar days an administratively complete submission was “on hold” while the FDA waited for an applicant to respond to a substantive request for additional information.
MDUFA Decision
For 510(k)s, a decision that a device is substantially equivalent (“SE” or “cleared”) and can be marketed in the U.S. or a decision that a device is not substantially equivalent (“NSE” or “not cleared”) and cannot be marketed in the U.S. [4, pp. 32–33] Throughout this post, this is referred to simply as a “decision.”
MDUFA Performance Report Cutoff Date
The date as of which the performance metrics in a MDUFA performance report were calculated. For the data included in this post, those dates are 2018-09-30 for MDUFA 3 and 2023-09-30 for MDUFA 4 and MDUFA 5. [1, p. 1] [2, p. 1] [3, p. 1]
Review Cycle
A period of FDA Days during which an accepted submission is “under review” at the FDA that is terminated by either a substantive request for additional information or a decision. For example, a submission that is cleared by the FDA without going “on hold” has one review cycle. A submission that the FDA places “on hold” through a substantive request for additional information, is then supplemented by the applicant with additional information, and then goes “under review” for a second time at the FDA before a final decision is made has two review cycles.1
Total Days
The number of calendar days from the FDA’s receipt of an administratively complete submission to a MDUFA Decision. When calculating summary statistics on this metric, the FDA excludes certain outlier submissions. The criteria that determine which outliers should be excluded change from year-to-year. [4, p. 35]

Metrics based on receipt cohorts are lagging indicators

Performance metrics included in the MDUFA reports are calculated across submissions grouped by the fiscal year in which they were received. These groups of submissions are called “receipt cohorts.” Grouping submissions by the fiscal year in which the FDA made a decision on the submissions results in groups of submissions called “decision cohorts.”

As of this writing, the FY 2023 510(k) receipt cohort includes many submissions for which the FDA has not made a final decision. This means that the FY 2023 510(k) receipt cohort is “open.” See Figure 2, which shows that, as of the most recent MDUFA 5 report, as many as 40% or more of the 510(k)s received in FY 2023 by some of the FDA’s medical device review groups remain under review. As review days continue to accumulate for these 510(k)s, the performance metrics that are based on those review times will change as well. Performance metrics calculated across open receipt cohorts continue to change until those cohorts “close,” which happens when each submission in the receipt cohort has received a decision.

Recent receipt cohorts are typically open. This means that performance metrics associated with these receipt cohorts, on their own, do not provide a particularly accurate indicator of final performance on those cohorts. It takes a while for them to stabilize. Before they stabilize, performance metrics associated with open receipt cohorts provide overly optimistic estimates of performance.2

While this post can help you understand how the FDA performed on closed receipt cohorts from past years, the MDUFA reports and this post will not provide you with much useful information about how things are going at the FDA right now, and data from open receipt cohorts should be interpreted with caution.

Reorganizations impacted these datasets

MDUFA 3 data is included for some review groups in the figures below and is not included for others. This is a result of reorganizations at the FDA. A major reorganization occurred on September 30, 2019, and a subsequent, smaller reorganization occurred later.

In some cases, the reorganizations took premarket review groups that already existed and were included in the MDUFA 3 reports and renamed them.3 This is essentially the case for OHT2, OHT5, OHT6, and OHT8, meaning that data for those review groups from the MDUFA 5 and MDUFA 4 reports can be aligned with data from the MDUFA 3 reports for corresponding review groups.

In other cases, the reorganizations took review groups that were included in the MDUFA 3 reports and divided up and separated their subgroups. This is essentially the case for OHT1, OHT3, and OHT4, meaning that MDUFA 5 and MDUFA 4 report data for those review groups cannot be aligned with MDUFA 3 report data.

In the case of OHT7, the reorganizations combined several review groups that already existed and were included in the MDUFA 3 reports into one larger group that is referenced in the MDUFA 4 and MDUFA 5 reports. This means that some data, such as the number of submissions received, can be summed across the corresponding groups in the MDUFA 3 reports to align the data with the larger organization that is included in the MDUFA 4 and MDUFA 5 reports. See, for example, Figure 1. In other cases, aligning the data is more complex. In still other cases, it is not possible to align the data. Other than submission volume in Figure 1, no additional efforts were made to align OHT7 data in the MDUFA 5 and MDUFA 4 reports to data from the MDUFA 3 reports for this post.

While I was not able to identify a reference that maps the review groups included in the MDUFA 3 reports to those used in the MDUFA 4 and MDUFA 5 reports, Table 1 shows how I mapped the groups for the purposes of this post based on the names of the old review groups and the names of the new review groups.

Table 1: Mapping of review groups in the MDUFA 3 reports to review groups in the MDUFA 4 and MDUFA 5 reports.
MDUFA 3 report review group (Old) Corresponding MDUFA 4 and MDUFA 5 report review group(s) (New) Relationship between old and new review groups MDUFA 3 data available in figures
DAGRID: Division of Anesthesiology, General Hospital, Respiratory, Infection Control, and Dental Devices OHT1: Ophthalmic, Anesthesia, Respiratory, ENT, and Dental Devices
OHT3: GastroRenal, ObGyn, General Hospital, and Urology Devices
OHT4: Surgical and Infection Control Devices		 Old review group was split and is part of more than one new review group No
DCD: Division of Cardiovascular Devices OHT2: Cardiovascular Devices Old review group and new review group are equivalent Yes
DCTD: Division of Chemistry and Toxicology Devices OHT7: In Vitro Diagnostics Old review group is a subset of new review group Only in Figure 1
DIHD: Division of Immunology and Hematology Devices OHT7: In Vitro Diagnostics Old review group is a subset of new review group Only in Figure 1
DMD: Division of Microbiology Devices OHT7: In Vitro Diagnostics Old review group is a subset of new review group Only in Figure 1
DMGP: Division of Molecular Genetics and Pathology Devices OHT7: In Vitro Diagnostics Old review group is a subset of new review group Only in Figure 1
DNPMD: Division of Neurological and Physical Medicine Devices OHT5: Neurological and Physical Medicine Devices Old review group and new review group are equivalent Yes
DOD: Division of Orthopedic Devices OHT6: Orthopedic Devices Old review group and new review group are equivalent Yes
DOED: Division of Ophthalmic and Ear, Nose, and Throat Devices OHT1: Ophthalmic, Anesthesia, Respiratory, ENT, and Dental Devices Old review group was combined with parts of another old review group to form new review group No
DRGUD: Division of Reproductive, Gastro-Renal, and Urological Devices OHT3: GastroRenal, ObGyn, General Hospital, and Urology Devices Old review group was combined with part of another old review group to form new review group No
DRH: Division of Radiological Health OHT8: Radiological Health Old review group and new review group are equivalent Yes
DSD: Division of Surgical Devices OHT4: Surgical and Infection Control Devices Old review group was combined with part of another old review group to form new review group No

The COVID-19 Public Health Emergency impacted these datasets

A Public Health Emergency (PHE) was declared for COVID-19 beginning on 2020-01-27 (FY 2020 Q2) and was renewed until it expired on 2023-05-11 (FY 2023 Q3). Responding to the PHE had a major impact on the FDA’s work. Other related declarations and determinations also impacted and continue to impact the FDA’s work. To acknowledge the impact of the PHE on the FDA’s work, graphs in this post that plot metrics over time also show the PHE as a shaded region underlaying data points associated with FY 2020 through FY 2023.

Exemptions impacted these datasets

The FDA exempts lower risk and well characterized types of medical devices from 510(k) review from time to time. Examples include a large round of exemptions that occurred in 2017 and smaller rounds of exemptions in 2018 and 2020. [8] [9] [10]

Exemptions can decrease the volume of 510(k)s that a review group receives. 510(k)s for medical devices that are not exempted and remain in the 510(k) program may include more complex or less well-characterized technologies that are more challenging to review. The review process for these 510(k)s may take more time on average, and this may affect performance metrics over time.

These datasets focus on efficiency, not quality

The performance metrics in the MDUFA reports primarily focus on how long things take. Information about interim and final submission outcomes – such as the rate at which 510(k)s result in requests for additional information and the rate at which 510(k)s result in a medical device being cleared for market in the United States – are also included. More comprehensive information about submission and review quality, meaning how good a job applicants do in preparing submissions and how good a job the FDA does in reviewing them, is not included in the MDUFA reports or this post.

To provide the public with some additional insight into quality and other topics, the MDUFA commitments have included “independent assessments” by a government contractor. The most recent one resulted in a report published in 2021. [11] Historically, the FDA and the GAO have also performed assessments. [12] [6]

Therefore, efficiency metrics associated with the amount of time submissions spend going through the FDA review process, especially when well-established submission and review quality metrics are not readily available, provide an incomplete picture of the FDA’s medical device review programs. In addition, differences in submission or review quality over time or across product areas may affect these metrics.

510(k) program metrics

The FDA provides visualizations of many 510(k) program performance metrics for the Center for Devices and Radiological Health (CDRH) as a whole, but the FDA does not provide these for review groups within CDRH. [3, pp. 102–111] This post provides visualizations for OHT1 through OHT8.

Submission volume

The MDUFA reports include information about submissions received over time. This can be plotted, and future submission volume can be projected. The projections provided here are somewhat naive: They are linear models based on fiscal year and review organization. This means that, for example, the OHT4 projections do not account for any impact that the end of the PHE may have on submission volume, and the OHT7 projections do not account for any impact that the FDA’s proposed rule on Laboratory Developed Tests or Procedures may have on submission volume.

Figure 1: 510(k) submission volume by OHT and fiscal year received as of the corresponding MDUFA report cutoff dates. Raw data.

Cohort closure percentage

See this section above for a discussion of what it means for a receipt cohort to be open or closed. As expected, Figure 2 shows that the FY 2023 510(k) receipt cohort had many open 510(k)s as of the most recent MDUFA performance report, meaning that performance metrics calculated on that cohort and shown in the figures throughout this post are likely to change substantially in future reports.

Figure 2: 510(k) cohort closure percentage by OHT and fiscal year received as of the corresponding MDUFA report cutoff dates. Raw data.

FDA Days metrics

The MDUFA performance reports include summary statistics about how long reviews take across the population of 510(k)s received. In particular, the reports provide quintiles (the 20th, 40th, 60th, and 80th percentiles) of FDA Days, Industry Days, and Total Days. Quintiles are an unusual choice for summary statistics about a distribution, and I’m not aware of a standard approach to plotting them. They can’t be plotted with boxplots, for example, which use quartiles. The visualizations in this post are inspired by boxplots though: They use line segments to show the percentile metrics and points to show averages.

Figure 3: FDA Days performance metrics by OHT and fiscal year received as of the corresponding MDUFA report cutoff dates. Raw data.

The PHE had a major impact on the data in Figure 3 for OHT7. Figure 4 shows the same data within 0 through 90 FDA Days, making it easier to see the data associated with OHTs 1-6 and 8.

Figure 4: FDA Days performance metrics by OHT and fiscal year received as of the corresponding MDUFA report cutoff dates. Values above 90 FDA Days are not shown. Raw data.

FDA Days on-time performance

The FDA has 90 FDA Days to review a 510(k).4 The percentage of 510(k)s reviewed within this time frame can be thought of as “on-time performance,” similar to how the metric is used in transportation. The MDUFA reports include on-time performance metrics for the 510(k) program, shown in Figure 5.

Figure 5: FDA Days on-time performance for the 510(k) program by review group and fiscal year received as of the corresponding MDUFA report cutoff dates. Raw data.

Industry Days metrics

By regulation, applicants have 30 days to respond to a request for additional information from the FDA before the FDA may consider a 510(k) to have been “withdrawn” by the applicant.5 By policy, the FDA allows up to 180 days for an applicant to respond. [13, p. 9] Figure 6 provides information about how long these responses have taken by fiscal year and by the FDA review group with which an applicant is interacting.

Figure 6: Industry Days performance metrics by OHT and fiscal year received as of the corresponding MDUFA report cutoff dates. Raw data.

Total Days metrics

As described above, the FDA has 90 FDA Days to review a 510(k), and applicants have 180 Industry Days to respond to questions the FDA raises in the course of that review. This means that the total amount of time available for a 510(k) to go through the review process if these activities are completed on time is 270 Total Days. Figure 7 and Figure 8 show how long the process has taken by fiscal year and by FDA review group.

Figure 7: Total Days performance metrics by OHT and fiscal year received as of the corresponding MDUFA report cutoff dates. Raw data.

The PHE had a major impact on the data in Figure 7 for OHT7. Figure 8 shows the same data within 0 through 270 Total Days, making it easier to see the data associated with OHTs 1-6 and 8.

Figure 8: Total Days performance metrics by OHT and fiscal year received as of the corresponding MDUFA report cutoff dates. Values above 270 Total Days are not shown. Raw data.

Review cycle metrics

The FDA reports the average number of review cycles for 510(k)s, which is related to the rate at which the FDA places 510(k)s “on hold” to request additional information from applicants.

Figure 9: Number of review cycles for 510(k)s over time as of the corresponding MDUFA report cutoff dates. Raw data.

While the FDA occasionally places a single 510(k) on hold more than once, this has been uncommon since at least 2014. [14, p. 8] It is not quite correct to assume that 510(k)s cannot have more than 2 review cycles, but it is a useful approximation for the purpose of estimating the rate at which the FDA places 510(k)s “on hold” to request additional information. With this assumption, that proportion is the Average Number of Review Cycles minus one.

Figure 10: Estimated rate at which additional information is requested for 510(k)s over time based on the average number of review cycles and an assumption that 510(k)s do not go on hold more than once. Raw data.

Decision metrics

In the MDUFA performance reports, the rates at which submissions receive an “SE” decision or receive an “NSE” decision are reported as percentages of submissions that receive either an SE or an NSE decision. The rates at which submissions are withdrawn by applicants or deleted by the FDA because an applicant did not respond to a request for additional information in a timely manner are reported as percentages of all accepted submissions. Because two different denominators are used to calculate these percentages in the MDUFA reports, adding the four percentages together results in a total that exceeds 100%.

This makes it difficult to understand what percentage of 510(k)s received by the FDA were cleared for market over time based on the rates in the MDUFA reports. However, the MDUFA reports also include the counts of submissions that received each of these decisions. Figure 11 shows the rates of cleared and not cleared (including withdrawn and deleted) 510(k)s for each OHT.

Figure 11: The rate at which 510(k)s are cleared or not cleared for market over time as of the corresponding MDUFA report cutoff dates. Raw data.

Closing thoughts

The objective of this work was to make the information in the MDUFA reports easier to access, analyze, and use. This post provides novel visualizations of about 1,500 values from the FDA’s 510(k) program metrics over time and across review groups. These visualizations are intended to provide a more useful view of the tabular data provided throughout the FDA’s MDUFA reports while still presenting that data dispassionately and with appropriate context.

References

[1]
“MDUFA III (FY 2013-2017) performance report,” U.S. Food and Drug Administration, Report, Dec. 2018. Available: https://boleary-com-blog.s3.us-east-005.dream.io/mdufa-3_2018-12-10_quarterly-report.pdf
[2]
“MDUFA IV (FY 2018-2022) performance report,” U.S. Food and Drug Administration, Report, Nov. 2023. Available: https://boleary-com-blog.s3.us-east-005.dream.io/mdufa-4_2023-11-16_quarterly-report.pdf
[3]
“MDUFA V (FY 2023-2027) performance report,” U.S. Food and Drug Administration, Report, Nov. 2023. Available: https://boleary-com-blog.s3.us-east-005.dream.io/mdufa-5_2023-11-16_quarterly-report.pdf
[4]
“MDUFA performance goals and procedures, fiscal years 2023 through 2027,” U.S. Food and Drug Administration, User Fee Commitment Letter, Mar. 2022. Available: https://boleary-com-blog.s3.us-east-005.dream.io/mdufa-5_goals-and-procedures.pdf
[5]
“MDUFA performance goals and procedures, fiscal years 2018 through 2022,” U.S. Food and Drug Administration, User Fee Commitment Letter, Dec. 2016. Available: https://boleary-com-blog.s3.us-east-005.dream.io/mdufa-4_goals-and-procedures.pdf
[6]
M. Crosse et al., “Medical devices: FDA has met most performance goals but device reviews are taking longer,” U.S. Government Accountability Office, Report to Congressional Requesters GAO-12-418, Feb. 2012. Available: https://www.gao.gov/products/gao-12-418
[7]
“CDRH preliminary internal evaluations – volume I,” U.S. Food and Drug Administration, Report, 2010. Available: https://boleary-com-blog.s3.us-east-005.dream.io/fda-cdrh-510k-working-group-report-2010.pdf
[8]
“82 FR 31976: Medical devices; exemptions from premarket notification: Class II devices,” U.S. Food and Drug Administration, Federal Register Notice, Jul. 2017. Available: https://www.federalregister.gov/d/2017-14453
[9]
“83 FR 25910: Medical devices; exemptions from premarket notification: Class II devices,” U.S. Food and Drug Administration, Federal Register Notice, Jun. 2018. Available: https://www.federalregister.gov/d/2018-11879
[10]
“85 FR 44186: Medical devices; exemptions from premarket notification: Class II devices,” U.S. Food and Drug Administration, Federal Register Notice, Jul. 2020. Available: https://www.federalregister.gov/d/2020-15256
[11]
“MDUFA IV phase 2 independent assessment final report,” U.S. Food and Drug Administration, Report, 2021. Available: https://boleary-com-blog.s3.us-east-005.dream.io/mdufa-4_independent-assessment-2021.pdf
[12]
“Analysis of premarket review times under the 510(k) program,” U.S. Food and Drug Administration, Report, Sep. 2011. Available: https://boleary-com-blog.s3.us-east-005.dream.io/mdufa-data-requests-2011.pdf
[13]
“FDA and industry actions on premarket notification (510(k)) submissions: Effect on FDA review clock and goals,” U.S. Food and Drug Administration, Final Guidance Document FDA-2003-D-0033, 2022. Available: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/fda-and-industry-actions-premarket-notification-510k-submissions-effect-fda-review-clock-and-goals
[14]
“Types of communication during the review of medical device submissions,” U.S. Food and Drug Administration, Final Guidance Document FDA-2013-D-0147, Apr. 2014. Available: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/types-communication-during-review-medical-device-submissions

Footnotes

  1. This term is not defined in the MDUFA V Commitment Letter, but this definition is consistent with its use in other documents. [5, p. 28] [6, pp. 4, 7] [7, p. 39] While the MDUFA 3 reports are silent on the issue, the MDUFA 4 and MDUFA 5 reports confirm that “review cycle” refers to a substantive review of an “accepted” 510(k). [2, p. 257] [3, p. 238] The FDA’s approach to analyzing requests for additional information suggests that this has been the practice since FY 2013. [3, p. 103]↩︎

  2. The FDA has used a number of analysis techniques to correct for this and get insights into recent performance based on data from open receipt cohorts. One approach is to look at historical data from when previous receipt cohorts that are now closed were still open. That information is compared to the final calculated performance metrics for the corresponding cohorts and to the receipt cohorts that are currently open to infer whether performance is better or worse now than it was then and to estimate where the final performance metric calculations for open cohorts may converge. The MDUFA reports include several figures based on this approach. [3, pp. 106–109] Best- and worst-case outcomes can also be estimated for some metrics.
    These complex techniques can be difficult to interpret. I favor looking at recently closed decision cohorts instead. Metrics similar to the Total Days metrics in the MDUFA reports can be calculated for cleared 510(k) receipt and decision cohorts using the FDA’s public 510(k) database. You can access this from R using the fdadata package, and I’ve also built a free web app you can use to explore clearance timeframes using this dataset.↩︎

  3. The reorganizations had other impacts as well, but this post is focused on their implications for MDUFA performance metric data.↩︎

  4. In addition to being a MDUFA commitment, this is in 21 USC § 360(n)(1).↩︎

  5. See 21 CFR 807.87(m).↩︎

Reuse

Citation

BibTeX citation:
@online{o'leary2024,
  author = {O’Leary, Brendan},
  title = {Downloadable Spreadsheets and Graphs of the {FDA’s} Medical
    Device Review Performance Metrics},
  date = {2024-01-03},
  url = {https://www.boleary.com/blog/mdufa/202401/},
  langid = {en},
  abstract = {The FDA publishes quarterly and annual reports that
    include performance metrics for the medical device review process.
    These reports can include thousands of performance metrics in a
    series of tables, and they are published as PDFs that regularly
    exceed three hundred pages. This post makes many of these metrics
    available for download in spreadsheets and as a data package for the
    R programming language, making the data easier to analyze and use.
    This post also provides novel visualizations of 510(k) program
    performance metrics over time and by review group.}
}
For attribution, please cite this work as:
B. O’Leary, “Downloadable spreadsheets and graphs of the FDA’s medical device review performance metrics,” Jan. 03, 2024. https://www.boleary.com/blog/mdufa/202401/
]]> FDA 510(k) devices datasets MDUFA https://www.boleary.com/blog/mdufa/202401/ Wed, 03 Jan 2024 05:00:00 GMT Regulatory considerations for digital surrogate endpoint response biomarkers Brendan O'Leary https://www.boleary.com/blog/posts/202311-digital-biomarkers/ .img-fluid { border: 1px solid #00000080; } .anchored { font-size: 0px }

On July 24, 2023, I gave a “quick” talk on regulatory considerations at the EverythingALS Digital Biomarkers Summit. This post is adapted from that talk.

Regulatory Considerations for Digital Biomarkers

Slide 1: Regulatory Considerations for Digital Surrogate Endpoint Response Biomarkers. By Brendan O'Leary

My name is Brendan O’Leary, and, up until a few months ago, I was a federal regulator. I spent 14 years with the FDA’s Center for Devices and Radiological Health, or CDRH. For about 10 of those years, I worked in the diagnostics office, where my focus was on radiology devices, software, and digital health. Then, I helped launch the FDA’s Digital Health Center of Excellence as its founding deputy director, and I led the Center of Excellence throughout 2022.

My education is in engineering, and, over the years, I helped the FDA make somewhere on the order of a thousand decisions related to digital health, biomarkers, and diagnostics, including decisions on companion diagnostics, complementary diagnostics, and digital biomarkers used with drugs. I’ll approach this topic from that experience today.

  • I’ll start by covering definitions for terms like “digital biomarker,” “response biomarker,” and “surrogate endpoint response biomarker.”

  • From there, I’ll discuss the differences between a biomarker and a clinical outcome assessment, including clinical outcome assessments enabled by digital health technologies.

  • Then, I’ll cover what a useful biomarker or other measure should do.

  • And finally, I’ll end with a few thoughts about how to convince the clinical community, payors, and the FDA that your biomarker or other measure is actually useful.

Biomarker: The “Best” Definition

Slide 2: Biomarker: The 'Best' Definition. A defined characteristic that is measured as an indicator of normal biological processes, pathogenic processes, or biological responses to an exposure or intervention, including therapeutic interventions. Biomarkers may include molecular, histologic, radiographic, or physiologic characteristics. A biomarker is not a measure of how an individual feels, functions, or survives. Categories of biomarkers include: susceptibility/risk biomarker, diagnostic biomarker, monitoring biomarker, prognostic biomarker, predictive biomarker, response biomarker, and safety biomarker. Source: BEST Resource.

The most important thing to know about digital biomarkers is that they are biomarkers. “Digital” is not the most important part. “Digital” is a reference to enabling technology. It’s a description of how we measure a biomarker or how we transmit information about a biomarker. That enabling technology is often critical: Sometimes, it’s the only reason it’s possible to measure or use the biomarker at all! But it’s still a means to an end, and it’s the end that’s most important here: From a regulatory perspective, the most important thing about digital biomarkers is that they are biomarkers.

So, let’s focus on that for now: What is a biomarker?

Back in 2016, some folks from the FDA’s device and drug centers and from the NIH got together to define this and other terms. They published the cleverly-but-perhaps-not-so-humbly-named “BEST Resource,” [1] which you can find at the link in the QR code on this slide. The document is about 60 pages, and it’s an important read if you ever want to communicate clearly with the FDA on this topic.

Among other things, the BEST Resource defines the terms biomarker and digital health technology.

And, based on this, according to the FDA, a biomarker, including a digital biomarker, is a characteristic measured as an indicator of a biological process, whether that process is normal or affected by a disease.

Equally important is what a biomarker is not. A biomarker is not a measure of how someone feels, functions, or survives. We often call measures of how someone feels, functions, or survives “outcomes,” and I’ll say more about these in a few moments. But the thing to keep in mind is that a biomarker is not itself an outcome.

The BEST Resource lays out several categories of biomarkers. There are two important things to know about these categories:

  • First, the categories are defined based on their context of use – which is very similar to “intended use” if you’re familiar with that term in regulatory contexts – and,

  • Second, the categories are not mutually exclusive: a biomarker can be both a prognostic biomarker and a predictive biomarker, for example.

On that first point, when I say that the biomarker categories are defined based on the context of use of the biomarker, that means that the categories are more about how a biomarker is used than about what a biomarker is measuring. That’s why the same biomarker can belong in more than one category depending on how it is being used. For example, hemoglobin A1c (HbA1c) can be used as a diagnostic biomarker to identify patients with Type 2 diabetes. It can be measured repeatedly over time as a monitoring biomarker to assess disease progression. And, it can be used as a response biomarker to evaluate diabetes control following treatment with a drug. You are measuring the same thing in each case, but it is how you are using the information that drives what words you should use to describe the biomarker.

These different contexts of use for a biomarker are all relevant in medical product development, but today, we’ll focus on the use of biomarkers as response biomarkers in particular. This use has well-defined regulatory considerations laid out in the BEST Resource.

Response Biomarker

Slide 3: Response Biomarker: A biomarker used to show that a biological response, potentially beneficial or harmful, has occurred in an individual who has been exposed to a medical product or an environmental agent. Pharmacodynamic response biomarker: indicates biologic activity of a medical product without necessarily drawing conclusions about efficacy or disease outcome or necessarily linking this activity to an established mechanism of action. Useful for: establishing proof-of-concept, assisting in dose selection, and as potential secondary endpoints in clinical trials. Surrogate endpoint response biomarker: An endpoint used in clinical trials as a substitute for a direct measure of how a patient feels, functions, or survives. Does not measure the clinical benefit of primary interest in and of itself, but rather is expected to predict that clinical benefit or harm. Source: BEST Resource.

When we use a biomarker as a response biomarker, we’re using it to measure a response. Typically, we’re measuring a response to a drug or other medical product. If we’re only looking for evidence of biological activity and are not trying to use the biomarker to assess a disease outcome, we’re probably using the biomarker as a pharmacodynamic response biomarker. In contrast, when we use a biomarker as a measure of whether we can expect that a therapeutic is providing a clinical benefit, like whether a therapeutic is likely to have a positive impact on how a patient feels, functions, or survives, we’re using the biomarker as a surrogate endpoint response biomarker. Surrogate endpoint response biomarkers are not direct measures of the expected clinical benefit itself. They are measures that we hope or expect to correlate with that benefit.

Surrogate Endpoint Response Biomarkers

Slide 4: Surrogate Endpoint Response Biomarkers are characterized by the level of clinical validation. This slide shows that within the hierarchy of 'Biomarker', we have 'Response Biomarker.' Within that, we have 'Surrogate Endpoint Response Biomarker'. Within that, a pyramid graphic shows three levels, starting with 'Validated Surrogate Endpoint' at the top, 'Reasonably Likely Surrogate Endpoint' in the middle, and 'Candidate Surrogate Endpoint' at the base. Source: BEST Resource.

We’ve covered biomarkers and digital biomarkers. Within that category, we’ve covered a number of types of biomarkers and have focused on response biomarkers in particular. From there, we zoomed in on surrogate endpoint response biomarkers. And now, within that sub-sub-category, we’ll cover three different types of surrogate endpoint response biomarkers.

Surrogate endpoint response biomarkers are characterized by the level of clinical validation. At the top, with the most evidence, we have “Validated Surrogate Endpoints.” Moving down from there, we have “Reasonably Likely Surrogate Endpoints” and “Candidate Surrogate Endpoints.” Each is associated with different levels of evidence for their validity, and that has implications for using them in clinical studies to support FDA authorizations of therapeutic products.

Slide 5: Surrogate Endpoint Response Biomarkers are characterized by the level of clinical validation. The pyramid graphic appears again, with 'Validated Surrogate Endpoint' highlighted. A Validated Surrogate Endpoint has a clear mechanistic rationale, clinical data providing strong evidence that an effect on the endpoint predicts a specific clinical benefit, and can support marketing authorizations for therapeutics without additional studies to demonstrate clinical benefit directly. Source: BEST Resource.

Let’s start with validated surrogate endpoints. The BEST Resource identifies two characteristics of these surrogate endpoints.

First, we have a clear mechanistic rationale for validated surrogate endpoint response biomarkers. That means that we understand the link between the biomarker and the mechanisms of the disease that we’re studying. The link is known, and it’s clear.

Second, clinical data shows that an effect on a validated surrogate endpoint response biomarker predicts a clinical benefit, and we know from the clinical data what that clinical benefit will be.

If you have these two things, you have everything you need. A validated surrogate endpoint response biomarker can support FDA marketing authorizations for therapeutics without additional studies. Seeing an effect on a validated surrogate endpoint response biomarker is enough, and if you have that, you don’t need additional pre- or postmarket studies that directly demonstrate that the therapeutic has a clinical benefit.

This is the mountaintop. This is what we want to attain.

Slide 6: Surrogate Endpoint Response Biomarkers are characterized by the level of clinical validation. The pyramid graphic appears again, with 'Validated Surrogate Endpoint' highlighted. A 'tracked changes' style is used to compare and contrast Reasonably Likely Surrogate Endpoints with Validated Surrogate Endpoints. A Reasonably Likely Surrogate Endpoint has a strong mechanistic rationale (as opposed to a clear mechanistic rationale for Validated Surrogate Endpoints), an effect on the endpoint is expected to correlate with a specific clinical benefit (as opposed to an expectation that it will predict that endpoint based on clinical data that provides strong evidence for Validated Surrogate Endpoints), and can support accelerated approval for therapeutics, often with postmarketing confirmatory trials (as opposed to supporting marketing authorizations without additional studies to demonstrate clinical benefit directly for Validated Surrogate Endpoints). Source: BEST Resource.

It’s a long journey to get to a validated surrogate endpoint response biomarker, and on the way there, you can find yourself with some really good indications that you have a good biomarker on your hands, but without all of the evidence you would need to say that it’s validated. This is called a “reasonably likely surrogate endpoint response biomarker,” and it’s easiest to introduce these by highlighting how they differ from validated surrogate endpoint response biomarkers.

First, while for a validated surrogate endpoint response biomarker we have a clear understanding of the link between the biomarker and the mechanisms of the disease, for a reasonably likely surrogate endpoint response biomarker, we only have a strong rationale: We have a really good, really plausible theory, but not enough to convince a regulator.

Second, instead of having clinical data that shows that an effect on the surrogate endpoint response biomarker predicts a specific clinical benefit, for reasonably likely surrogate endpoint response biomarkers, we only have an expectation that the biomarker will correlate with a specific clinical benefit. Notably though, based on the strong mechanistic rationale, we should still have a specific expectation for how a specific clinical benefit will correlate with the biomarker.

As a result of the increased uncertainty associated with reasonably likely surrogate endpoint response biomarkers, they alone can’t support full FDA marketing authorizations for therapeutics without additional studies. But they are still incredibly powerful, and they can support accelerated approval of therapeutics, often with postmarketing confirmatory trials.

And, if you plan things right, and if the science works out the way you hope it will, those confirmatory trials can validate the surrogate endpoint response biomarker at the same time as they validate the therapeutic.

Slide 7: Surrogate Endpoint Response Biomarkers are characterized by the level of clinical validation. The pyramid graphic appears again, with 'Candidate Surrogate Endpoint' highlighted. A 'Candidate Surrogate Endpoint' is an endpoint still under evaluation for its ability to predict clinical benefit.

Finally, we have candidate surrogate endpoint response biomarkers. These are all of the things that we’re working to develop and hope will work, but for which we don’t yet have the data.

Clinical Outcome Assessment (COA)

Slide 8: Clinical Outcome Assessment (COA). This slide shows a table from a paper about clinical outcome assessments. The table illustrates how similar measurement technologies can be grouped into different 'types of measures' based on the 'concept being measured'.

At the beginning of this post, I said that part of understanding what a biomarker is involves understanding what it is not. A biomarker is an indicator of a biological process. A biomarker is not a measure of how someone feels, functions, or survives. How someone feels, functions, or survives is an outcome, and one way to measure an outcome is to use a Clinical Outcome Assessment, or COA.

Clinical Outcome Assessments are performance-based assessments or assessments made by a clinician, a patient, or another observer. Depending on who is doing the assessment, a COA may be a patient-reported outcome (PRO), a clinician-reported outcome, or an observer-reported outcome. Sometimes, like with biomarkers, COAs may be enabled or facilitated by digital technologies. When people want to emphasize this, they sometimes stick the word “electronic” to the front of the term, and that’s where you get acronyms like “ePRO.” It just keeps going: There is no shortage of more and more specific jargon and acronyms for all of these concepts.

But, for our purposes, just know that an outcome is not a biomarker, and so a digitally facilitated outcome assessment is not a digital biomarker. Some of my colleagues at the FDA put together a nice paper describing this and showing how, based on the context of use and the concept being measured, very similar measurement technologies can fall into many different categories. [2]

Step 1: Find something meaningful

Slide 11: Step 1: Find something meaningful: 'Do I need mortality, do I want to live longer and suffer longer? No,' Fernandes said patients told the company’s researchers. 'Do I want to improve my six-minute walk? No, I don’t want six minutes, I cannot even walk a minute. Let me at least be able to get up and brush my teeth or climb the stairs, get to the bedroom, get to the bathroom, do something simple.' -Peter Fernandes, CEO of Bellerophon Therapeutics, as quoted by M. Aguilar in STAT. Source: https://www.statnews.com/2023/05/17/clinical-trials-digital-biomarkers-fda-pharma-bellerophon/.

Find something meaningful to measure.

Identify the gaps between the measures we have today and the needs of patients, trial participants, clinicians, and therapeutic product developers. Find ways to fill those gaps.

Peter Fernandes put it well here. [3] He was talking about what he was hearing from people with pulmonary fibrosis. And the bottom line is, our traditional measures aren’t cutting it. We’ve absolutely got to do better.

There are legacy measures that have been used for a long time, like the six-minute walk test in this example or the ALS-FRS. But Fernandes points out the shortcomings of those measures from a patient perspective. And, to be blunt, those measures are a hot mess from a scientific and regulatory perspective as well. There is just so much noise in them. Any researcher who has used them will tell you their limitations, which are numerous and extensive. But we still use them.

And, as awful as these measures are, we have good reason to keep using them: In part, it’s because decades of experience have given us such a good sense for the particular limitations of these measures. We know a lot about their strengths and their weaknesses.

So, as you look for something new and more meaningful to measure, keep in mind that there is a lot to be said for planning to measure it alongside the measures that we’re already familiar with, at least for a while. Doing that is part of the process of confirming that the new measures we’re crafting really are meaningful and really are better.

Which brings us to Step 2.

Step 2: Measure it

Slide 12: Step 2: Measure it: A definition of measurement: An 'Information Theory' version. Measurement: A quantitatively expressed reduction of uncertainty based on one or more observations. Source: Hubbard, Douglas W. How to Measure Anything: Finding the Value of Intangibles in Business. John Wiley & Sons, Inc., 2014.

Measure it.

This is the part of the post where I make the “big reveal” and tell you that you don’t really need to know that much about digital biomarkers, clinical outcome assessments, or any of the rest of this. The bottom line is that they are all things you measure or measurements. So, let’s ignore the BEST Resource and all of its biomedical jargon and definitions and just focus on what a measurement is.

This definition comes from the field of information theory and is in a book by Douglas Hubbard. [4] The definition identifies three key features of a measurement:

First, a measurement is a reduction of uncertainty. This isn’t about knowing “for sure.” Few if any of our existing measures get us there, and our new measures don’t need to get us to certainty either. They just need to meaningfully reduce our uncertainty.

Second, a measurement is based on one or more observations. I don’t have much to say about this. It’s what science is all about. It’s empiricism in a nutshell.

Finally, with a measurement, we can quantitatively express the reduction in uncertainty that these observations provide. For example, both qualitative and quantitative measurements may produce confidence intervals that can be expressed quantitatively. Regulators love confidence intervals, and regulators especially love to see a confidence interval shrink.

Slide 13: Conclusion: If you can find something meaningful, if you can reduce the uncertainty around that meaningful something, and if you can express that reduction quantitatively, you’ve made progress.

So that’s the real message. Digital biomarkers are exciting. Digital Health Technologies that enable new Clinical Outcome Assessments are too. It’s all good stuff. But don’t get too caught up in all of that. At the end of the day, the real key here is to measure something meaningful.

If you can find something meaningful, if you can reduce the uncertainty around that meaningful something, and if you can express that reduction quantitatively, you’ve made progress.

And if you can do all of that, then you’ve got a good case to make to the FDA and others.

References

[1]
Food and Drug Administration, National Institutes of Health, et al., “BEST (Biomarkers, EndpointS, and other Tools) resource,” Silver Spring, MD: FDA-NIH Biomarker Working Group, 2016, Available: https://www.ncbi.nlm.nih.gov/books/NBK338449/
[2]
S. Vasudevan, A. Saha, M. E. Tarver, and B. Patel, “Digital biomarkers: Convergence of digital health technologies and biomarkers,” NPJ digital medicine, vol. 5, no. 1, p. 36, 2022, Available: https://doi.org/10.1038/s41746-022-00583-z
[3]
M. Aguilar, “Biotech and pharma execs wrestle with the ‘nightmare’ of developing digital biomarkers,” STAT, May 2023, Accessed: Nov. 10, 2023. [Online]. Available: https://www.statnews.com/2023/05/17/clinical-trials-digital-biomarkers-fda-pharma-bellerophon/
[4]
D. W. Hubbard, How to measure anything: Finding the value of intangibles in business. Wiley; 3rd edition, 2014. Available: https://www.amazon.com/How-Measure-Anything-Intangibles-Business/dp/1118539273

Reuse

Citation

BibTeX citation:
@online{o'leary2023,
  author = {O’Leary, Brendan},
  title = {Regulatory Considerations for Digital Surrogate Endpoint
    Response Biomarkers},
  date = {2023-11-14},
  url = {https://www.boleary.com/blog/posts/202311-digital-biomarkers/},
  langid = {en}
}
For attribution, please cite this work as:
B. O’Leary, “Regulatory considerations for digital surrogate endpoint response biomarkers,” Nov. 14, 2023. https://www.boleary.com/blog/posts/202311-digital-biomarkers/
]]> FDA digital health biomarkers https://www.boleary.com/blog/posts/202311-digital-biomarkers/ Tue, 14 Nov 2023 05:00:00 GMT Taxpayers should demand better Brendan O'Leary https://www.boleary.com/blog/posts/202309-shutdowns/ Shutdowns are among the biggest causes of government waste that I observed in my fourteen years as a federal employee. Shutdowns inflict irreparable harm on the federal government’s reputation as a good and stable employer, directly impacting the government’s ability to recruit and retain top talent. Shutdowns are an entirely avoidable self-inflicted scourge on the US federal government. Taxpayers should demand better.

This post originally appeared on LinkedIn.

]]> https://www.boleary.com/blog/posts/202309-shutdowns/ Fri, 29 Sep 2023 04:00:00 GMT “We are making critical clinical trial decisions on measures that we know are bad” Brendan O'Leary https://www.boleary.com/blog/posts/202308-bad-measures/ Christine Cong Guo of ActiGraph hit the nail on the head in a discussion with STAT’s Mario Aguilar:

In my mind, we are making critical clinical trial decisions on measures that we know are bad. It just shouldn’t be the state of the art for our industry. You know, we are continuing to rely on these measures that have been developed more than 50 years ago as the gold standard today. If it happened in any other industry, it would sound like a joke. So I think the wearable and digital measures, the fact that it is measuring patients directly in their home environment about something they’re doing in their everyday environment, this is completely aligned with what FDA wants you to do — to measure a patient’s function in their daily activity, in their daily life.

This post originally appeared on LinkedIn.

]]> news FDA digital health biomarkers drugs https://www.boleary.com/blog/posts/202308-bad-measures/ Thu, 17 Aug 2023 04:00:00 GMT Data for researchers: Extracted text from more than 89,000 FDA medical device 510(k) clearance packages Brendan O'Leary https://www.boleary.com/blog/posts/202307-pmn/
Important

510(k) summaries are written by device manufacturers, not the FDA. When the FDA requests edits to a 510(k) summary, it may be as a final step, under time pressure, when the FDA is ready to clear the device but has not yet issued its final decision. The quality and quantity of information varies widely, and even the best 510(k) summaries are summaries: They are not comprehensive.

Therefore, it is easy to draw the wrong conclusions when analyzing 510(k) summaries. For example, 510(k) summaries alone cannot be used to provide reliable estimates of the proportion of 510(k)s that included clinical data, a subgroup analysis, or other types of evidence. For those types of analyses, decision summaries are a better option when they are available, and other sources should also be included, such as device labeling, ClinicalTrials.gov, and scientific publications. Even if you use all of the best available public data, your estimates may be far from the truth.

Introduction

Every year, the FDA clears around 3,000 medical devices to enter the U.S. market through a review pathway known as the 510(k) program. Descriptions of the data and information that formed the basis for many of these decisions is available in 510(k) summaries, which are posted as PDFs on the FDA’s website after a decision is made. This makes it possible to find information about individual 510(k) clearance decisions, but analyzing the information across the 510(k) program over time has been more difficult because it is only made publicly available through tens of thousands of individual files – until now. Here, you’ll find a dataset with the full text contents of more than 91,000 510(k) clearance packages.1 This includes full embedded and OCR text from over 620,000 pages.2 It can be downloaded in CSV format.

If you use this, please cite this page.

Note

The information in this dataset is from sources in the public domain. It is provided here “as-is” without warranty of any kind. You may also want to refer to the FDA’s website.

Dataset description

Note

When originally published, this dataset focused on 510(k) summaries. It now also includes clearance packages for many submissions where a 510(k) statement was used instead of a 510(k) summary. For these submissions, the clearance letter and indications for use are included in this dataset.

I changed the OCR approach in October 2025 to improve quality. OCR text for clearance packages with a date_obtained of October 2025 or later should be a bit better as a result. I’m slowly refreshing the dataset and am prioritizing clearance packages that may have been missed in earlier versions of this dataset and clearance packages where no embedded text was available in earlier versions of this dataset.

Most 510(k) clearance packages include:

  • A clearance letter from the FDA

  • An “indications for use” form

  • A 510(k) summary

For submissions where a “510(k) Statement” is used, only the clearance letter and the indications for use form are present.3

This dataset provides one row per page for each 510(k) clearance package and includes the following fields:

submission_number
The 510(k) or De Novo number for the submission associated with the 510(k) clearance package.
date_obtained
The date the 510(k) clearance package was obtained from the FDA website. The date is formatted according to ISO 8601.
page_number
The PDF page index from which the text was obtained.
text_embedded
The contents of any text embedded in the PDF. This is obtained using pdftools::pdf_text.
text_ocr
The contents of any text found using optical character recognition (OCR). This is from Tesseract`.

Download the data

For finding predicate devices, a site-specific search will probably serve you better than this dataset will. For example, if you are looking for tumor segmentation algorithms:

FDA Regulatory Expertise:
I provide digital health and medical device regulatory strategy solutions to technology developers, healthcare organizations, trade and professional associations, and others. Book a meeting.

The dataset is available as a gzip-compressed CSV file: pmn_summary_text.csv.gz (Size: 285M, MD5: 0930763803888184ab407b24cc0dd009)

Example of how to access and use the dataset with the R programming language

Here is a sample script in R that downloads and reads the dataset:

# Load libraries -----
library(fs)
library(readr)
library(utils)

# Set up URLs and paths -----
url_download <- 
  "https://www.boleary.com/blog/posts/202307-pmn/data/pmn_summary_text.csv.gz"
path_download <- fs::dir_create("data-raw")
filename_download <- "pmn_summary_text.csv.gz"
filepath_download <- 
  fs::path_expand(paste(path_download, filename_download, sep = "/"))

# Download the data -----
utils::download.file(
  url = url_download,
  destfile = filepath_download
)

# Read in the data -----
# Naming this "pmn_summaries" because "premarket notification (PMN)" is 
# another name for 510(k)s that's a bit easier to use in code. 
pmn_summaries <- 
  readr::read_delim(
    file = filepath_download,
    delim = ";",
    col_types = 
      readr::cols(
        submission_number = readr::col_character(),
        date_obtained = readr::col_date(),
        page_number = readr::col_integer(),
        text_embedded = readr::col_character(),
        text_ocr = readr::col_character()
      )
  )

Expand the code block below for a sample script in R that identifies the 510(k)s that were referenced in the most 510(k) summaries for radiological image processing devices cleared between calendar years 2008 and 2018.

Show the code 
# Load and install additional libraries -----
# Install the fdadata package from GitHub if it's missing
if (!require("fdadata")) {
  if (!require("devtools")) install.packages("devtools")
  devtools::install_github("bjoleary/fdadata")
}
library(dplyr)
library(lubridate)
library(stringr)
library(testthat)
library(tidyr)

# Load 510(k) submission metadata and filter to image processing devices -----
submissions_of_interest <- 
  fdadata::pmn |> 
  dplyr::filter(
    # Looking for submissions in product code LLZ for "System, Image 
    # Processing, Radiological"
    .data$product_code == "LLZ",
    # Looking for submissions with a decision date on or after 2008-01-01: 
    .data$date_decision >= lubridate::ymd("2008-01-01"),
    # Looking for submissions with a decision date before 2019-01-01: 
    .data$date_decision < lubridate::ymd("2019-01-01"),
  ) |> 
  # Just keep the submission_number field for this analysis
  dplyr::select("submission_number")

# Filter the pmn_summaries data by joining the submissions_of_interest -----
summaries_to_search <- 
  dplyr::inner_join(
    x = submissions_of_interest, 
    y = pmn_summaries, 
    by = c("submission_number" = "submission_number")
  )

# Set up a search term -----
submission_number_pattern <- 
  stringr::regex(
    # Match the letter "K" followed by exactly 6 numeric digits
    pattern = "K[0-9]{6}",
    # If, instead, you wanted to find both 510(k)s and De Novos, you might 
    # start with a pattern like this: "(K|DEN)[0-9]{6}"
    # Accept either upper- or lower-case "K"s
    ignore_case = TRUE
  )

# Double check that the regular expression search term is behaving as expected
testthat::expect_equal(
  object = 
    stringr::str_extract_all(
      string = "Can we find the submission number for K000000?",
      pattern = submission_number_pattern
    ),
  expected = list(c("K000000"))
)
testthat::expect_equal(
  object = 
    stringr::str_extract_all(
      string = "What if we include a supplement number? K123456/S001",
      pattern = submission_number_pattern
    ),
  expected = list(c("K123456"))
)
testthat::expect_equal(
  object = 
    stringr::str_extract_all(
      string = "And if it's a lower case K? k180001",
      pattern = submission_number_pattern
    ),
  expected = list(c("k180001"))
)
testthat::expect_equal(
  object = 
    stringr::str_extract_all(
      string = "This time we will want to see both K123456 and k180001",
      pattern = submission_number_pattern
    ),
  expected = list(c("K123456", "k180001"))
)
testthat::expect_equal(
  object = 
    stringr::str_extract_all(
      string = "We don't expect it to match a q-submission number like Q123456",
      pattern = submission_number_pattern
    ),
  expected = list(character(0L))
)

# Search for submission numbers
search_results <- 
  summaries_to_search |> 
  # For each page, concatenate the embedded and OCR text so we can create one 
  # string where we have the best chance of finding submission numbers 
  # (we'll de-duplicate the numbers we find later)
  tidyr::unite(
    col = "combined_text",
    c("text_embedded", "text_ocr"),
    sep = " ",
    remove = TRUE,
    na.rm = TRUE
  ) |> 
  # Combine all clearance package pages from each submission into a single string
  dplyr::group_by(.data$submission_number) |> 
  dplyr::summarise(
    text = paste(.data$combined_text, collapse = "\\n")
  ) |> 
  # Extract 510(k) submission numbers
  dplyr::mutate(
    submission_referenced =
      stringr::str_extract_all(
        string = .data$text,
        pattern = submission_number_pattern
      ) 
  ) |> 
  # Keep only submission number and results
  dplyr::select(
    "submission_number",
    "submission_referenced"
  ) |> 
  # Make 1 row for each reference found
  tidyr::unnest(cols = c(submission_referenced)) |> 
  # Make sure they are all upper case
  dplyr::mutate(
    submission_referenced = stringr::str_to_upper(.data$submission_referenced)
  ) |> 
  # Remove results where the reference found is the same as the submission 
  # it was found in
  dplyr::filter(.data$submission_number != .data$submission_referenced) |> 
  # Don't double count a reference just because it may have been mentioned 
  # more than once
  dplyr::distinct() |> 
  # Tally it up
  dplyr::group_by(.data$submission_referenced) |> 
  dplyr::tally(name = "references") |> 
  # Put in order of frequency of appearance followed by submission number, 
  # placing more recent submission numbers first
  dplyr::arrange(
    dplyr::desc(.data$references), 
    dplyr::desc(.data$submission_referenced)
  ) |> 
  # Limit to the first five rows
  utils::head(5) |> 
  # Join in some metadata
  dplyr::left_join(
    y = 
      fdadata::pmn |> 
      dplyr::select(
        "submission_number",
        "date_decision",
        "sponsor",
        "device"
      ),
    by = c("submission_referenced" = "submission_number")
  )

This produces Table 1.

Table 1: Five submissions frequently referenced in 510(k) summaries for image processing devices cleared from 2008 - 2018
Submission Referenced References Date Decision Sponsor Device
1 K071331 14 2007-05-25 Vital Images, Inc. VITREA VERSION 4.0
2 K120361 12 2012-04-06 Fujifilm Medical Systems U.S.A, Inc. SYNAPSE 3D BASE TOOLS
3 K150843 11 2015-04-24 Siemens AG syngo.via (version VB10A)
4 K110300 11 2011-07-01 Materialise Dental NV SIMPLANT 2011
5 K073714 11 2008-03-19 Orthocrat, Ltd. TRAUMACAD VERSION 2.0

Additional considerations

  • PDF Portfolios are not included. A small number of 510(k) summaries are posted as PDF portfolios and may not have been processed correctly or included in this dataset. Based on manual spot-checks, I believe that problems are particularly common when a PDF portfolio includes a fillable version of the indications for use form.

  • Many 510(k) summaries do not include embedded text. Embedded text is not present in many of the 510(k) clearance packages, particularly for decisions made many years ago. Both embedded text, when available, and text from OCR should be included for each page in this dataset. Which you choose to use and when may depend on your specific use-case.

  • Many 510(k)s have a 510(k) statement instead of a summary. Not all cleared 510(k)s have 510(k) clearance packages on the FDA website. Some manufacturers use a 510(k) statement in lieu of a 510(k) summary, which means they promise to provide safety and effectiveness information within 30 days of a request from any person.4 In addition, the 510(k) Summary/Statement requirement did not exist until the 1990s, so earlier submissions do not have 510(k) clearance packages.5

  • 510(k) summaries are not written by the FDA. A 510(k) summary is written by the manufacturer of the device, not by the FDA. Sometimes, the FDA provides considerable input. Other times, the FDA may conduct only a cursory review of a 510(k) summary. Practice has varied over the decades. Sometimes, the manufacturer and the FDA may forget to update the contents of a 510(k) summary at the end of a review after additional information has been provided, and a 510(k) summary may only reflect what was initially provided to the FDA before all questions were resolved. Be cautious about drawing firm conclusions about what was included – or absent – from a 510(k) on the basis of a 510(k) summary.

FDA Regulatory Expertise:
I help technology developers put their best foot forward when they need to interact with the FDA.

Availability of clearance packages and summaries in this dataset

As noted above, the 510(k) summary regulation came into effect in the 1990s. While I believe this dataset includes all or nearly all of the 510(k) summaries the FDA has made available on its website, there are notable gaps. Figure 1 shows the percentage of expected 510(k) summaries that are available in this dataset by the fiscal year of decision.

Figure 1: 510(k) summary availability by fiscal year of decision. Data.

Clearance packages that include decision letters and indications for use statements may also be posted when a 510(k) statement is used, and correction letters may be posted for historical submissions when the FDA changes how it describes, identifies, groups, and tracks devices with product codes [1] or when the FDA makes other changes. Figure 2 shows the percentage of clearance packages that are available in this dataset based on the total number of clearances in each fiscal year.

Figure 2: Clearance package availability by fiscal year of decision. Data.

I don’t know why the availability of clearance packages in this dataset falls off so dramatically in 2000 and 2001, but I’m looking into it.

In addition, 27 submissions associated with clearance packages in this dataset no longer appear in the FDA’s database: K110981, K112204, K112294, K131823, K131831, K152517, K170014, K170024, K170025, K170027, K180177, K180178, K182156, K192179, K201601, K201706, K201929, K210496, K211351, K211727, K212972, K220183, K220185, K221416, K222647, K230447, and K230999.

Some of these submissions appear to be for Powdered Surgeon’s Gloves, Powdered Patient Examination Gloves, and Absorbable Powder for Lubricating a Surgeon’s Glove, which were banned under a regulation issued in 2016. [2] The FDA also issued several notifications on data integrity, and this may account for some of these submissions. The FDA does not appear to have published information on its criteria or process for removing submissions from its databases after a clearance decision. Other submissions may have been removed before I was able to access them and add them to this dataset.

Changelog

  • 2026-03-24:
    • Added 220 additional 510(k) clearance packages.
  • 2026-02-23:
    • Added 244 additional 510(k) clearance packages.
  • 2026-01-13:
    • Added 283 additional 510(k) clearance packages.
  • 2025-12-16:
    • Added 255 additional 510(k) clearance packages.
  • 2025-11-19:
    • Added 326 additional 510(k) clearance packages.
  • 2025-10-22:
    • Added 673 additional 510(k) clearance packages.
    • Added new note about the limitations of this dataset at the top of the post.
    • Added new section on the availability of clearance packages and summaries in this dataset.
    • Updated post title and graphic.
    • I changed the OCR approach in October 2025 to improve quality. OCR text for clearance packages with a date_obtained of October 2025 or later should be a bit better as a result. I’m slowly refreshing the dataset, prioritizing clearance packages that may have been missed in earlier versions of this dataset as well as those where no embedded text is available.
  • 2025-09-10:
    • Added additional 510(k) clearance packages.
  • 2025-08-03:
    • Added additional 510(k) clearance packages.
    • Revised disclaimer.
  • 2025-06-10:
    • Added additional 510(k) clearance packages.
  • 2025-05-22:
    • Added additional 510(k) clearance packages.
  • 2025-05-14:
    • Added additional 510(k) clearance packages.
  • 2025-04-15:
    • Added additional 510(k) clearance packages.
  • 2025-03-16:
    • Added additional 510(k) clearance packages.
  • 2025-02-15:
    • Removed “Known Issues” because I have not been keeping it up-to-date.
    • Updated ad.
    • Added additional 510(k) clearance packages.
  • 2025-01-22:
    • Added additional 510(k) clearance packages.
  • 2025-01-04:
    • Added additional 510(k) clearance packages.
  • 2024-12-18:
    • Added additional 510(k) clearance packages.
  • 2024-11-07:
    • Added additional 510(k) clearance packages.
  • 2024-10-17:
    • Added additional 510(k) clearance packages.
  • 2024-10-08:
    • Removed parquet file format option.
    • Added additional 510(k) clearance packages.
  • 2024-09-03:
    • Added additional 510(k) clearance packages.
  • 2024-08-19:
    • Added additional 510(k) clearance packages.
  • 2024-08-12:
    • Added additional 510(k) clearance packages.
  • 2024-08-06:
    • Added additional 510(k) clearance packages.
  • 2024-07-22:
    • Added additional 510(k) clearance packages.
  • 2024-07-08:
    • Added additional 510(k) clearance packages.
    • Changed parquet compression from gzip to snappy.
  • 2024-06-27:
    • Added additional 510(k) clearance packages.
  • 2024-06-05:
    • Added clearance packages for 510(k)s with 510(k) Statements instead of summaries. These include the clearance letter and the indications for use and do not include a 510(k) summary.
  • 2024-05-10:
    • Added additional 510(k) summaries.
  • 2024-02-29:
    • Added additional 510(k) summaries.
  • 2023-12-23:
    • Added additional 510(k) summaries.
  • 2023-10-25:
    • Added additional 510(k) summaries.
  • 2023-08-14:
    • Fixed an error in the sample script in R that identifies the 510(k)s that were referenced in the most 510(k) summaries for radiological image processing devices cleared between calendar years 2008 and 2018. After submission numbers are extracted, they are now all made upper case using stringr::str_to_upper() before they are counted. Before, for example, “K100001” and “k100001” would have been counted as different submissions because of the difference in case for the “K”. This fix did not change the results presented in Table 1.
    • Fixed a spelling mistake in a footnote.
    • Added Known issues section.
    • Clarified that De Novo reclassification orders are treated as 510(k) summaries for the purposes of this dataset.
    • Added new submissions recently posted to the FDA website.
    • Added old submissions that were previously missing from the dataset. I believe the dataset is now comprehensive as of 2023-08-14.
  • 2023-07-17: Initial publication.

References

[1]
“Medical device classification product codes,” U.S. Food and Drug Administration, Final Guidance Document. Docket Number FDA-2011-D-0916, Apr. 2013. Available: https://www.regulations.gov/document/FDA-2011-D-0916-0009
[2]
“81 FR 91722: Banned devices; powdered surgeon’s gloves, powdered patient examination gloves, and absorbable powder for lubricating a surgeon’s glove,” U.S. Food and Drug Administration, Final Rule. Federal Register Notice 81 FR 91722, Dec. 2016. Available: https://www.federalregister.gov/d/2016-30382

Footnotes

  1. Upon initial publication, this dataset included more than 72,000 510(k) summary packages. This includes De Novos, where the reclassification order was used as the summary. Earlier versions of this dataset did not include clearance packages for 510(k)s where a 510(k) Statement was used.↩︎

  2. Upon initial publication, this dataset included more than 494,000 pages.↩︎

  3. See: https://www.fda.gov/medical-devices/premarket-notification-510k/content-510k#link_7.↩︎

  4. See the FDA’s description of the necessary Content of a 510(k), which describes this in more depth.↩︎

  5. The requirement for a 510(k) summary or a 510(k) statement is from the Safe Medical Devices Act (SMDA) of 1990. The regulation, 21 CFR 807.92, was established through an interim rule with 57 FR 18066 on April 28, 1992 and was finalized with 59 FR 64295 on December 14, 1994.↩︎

Reuse

Citation

BibTeX citation:
@online{o'leary,
  author = {O’Leary, Brendan},
  title = {Data for Researchers: {Extracted} Text from More Than 89,000
    {FDA} Medical Device 510(k) Clearance Packages},
  date = {},
  url = {https://www.boleary.com/blog/posts/202307-pmn/},
  langid = {en}
}
For attribution, please cite this work as:
B. O’Leary, “Data for researchers: Extracted text from more than 89,000 FDA medical device 510(k) clearance packages.” https://www.boleary.com/blog/posts/202307-pmn/
]]> 510(k) datasets FDA medical devices R regulatory science https://www.boleary.com/blog/posts/202307-pmn/ Mon, 17 Jul 2023 17:00:00 GMT Plots for MDUFA wonks Brendan O'Leary https://www.boleary.com/blog/posts/202306-fda-calendar-days/ Plots for MDUFA wonks, with the caveat that efficiency/speed metrics only tell part of the story, especially when quality metrics are not readily available: https://www.fdadays.com/

These are based on the FDA’s publicly posted databases for cleared 510(k)s and granted De Novos, so they only represent the submissions that made it through.

Citation

BibTeX citation:
@online{o'leary2023,
  author = {O’Leary, Brendan},
  title = {Plots for {MDUFA} Wonks},
  date = {2023-06-25},
  url = {https://www.boleary.com/blog/posts/202306-fda-calendar-days/},
  langid = {en}
}
For attribution, please cite this work as:
B. O’Leary, “Plots for MDUFA wonks,” Jun. 25, 2023. https://www.boleary.com/blog/posts/202306-fda-calendar-days/
]]> 510(k) De Novo FDA Medical Devices https://www.boleary.com/blog/posts/202306-fda-calendar-days/ Sun, 25 Jun 2023 04:00:00 GMT A promise that should not have been made Brendan O'Leary https://www.boleary.com/blog/posts/202306-stat-data-integrity/ A good read from Allison DeAngelis at STAT: The inside story of how data integrity issues roiled a biotech seen as ‘Moderna 2.0’

Some years back, a mentor encouraged me to read a similar article about the Volkswagen emissions defeat device scandal (What Was Volkswagen Thinking?), and a sentence from that article has been ringing in my ears ever since: “It started, as Volkswagen’s problems apparently did, with a promise that should not have been made.”

]]> news https://www.boleary.com/blog/posts/202306-stat-data-integrity/ Mon, 12 Jun 2023 04:00:00 GMT AIM-HI: Supporting prospective evaluations of diagnostic AI/ML tools in real-world clinical settings https://www.boleary.com/blog/posts/202305-aim-hi-announcement/ AIM-HI: “…an effort to stimulate the generation of higher quality evidence of clinical AI safety, effectiveness and fairness.” Sign me up! I’m honored to join the National Advisory Committee for this effort, and I’m excited to support the work the Gordon and Betty Moore Foundation and Kaiser Permanente are doing to advance the science around the implementation of AI/ML diagnostic decision support tools. Don’t miss this opportunity and be sure to check out the Virtual Information Session on May 22!

]]> digital health AI/ML funding https://www.boleary.com/blog/posts/202305-aim-hi-announcement/ Thu, 11 May 2023 04:00:00 GMT Four things to know about the FDA’s new discussion paper on AI/ML in drug development https://www.boleary.com/blog/posts/202305-aiml-in-drug-development/ First, a big congratulations to the great folks at the FDA who got this out the door! Here are four key points from the FDA’s discussion paper on Using AI & ML in the Development of Drug & Biological Products [1]:

  1. The FDA is working across centers. The paper was authored by the FDA’s drug center (CDER) in collaboration with the biologics center (CBER) and the medical device center’s Digital Health Center of Excellence (CDRH/DHCoE). Scientists talking to scientists across the FDA is a good thing: It promotes more holistic, consistent, and efficient approaches agency wide. Reinforcing this, don’t miss footnote 4 on page 1, which talks about how the principles in this document can apply to combination products, companion devices, and complementary devices.

  2. The FDA is pulling back the curtain on how its own experts talk about AI/ML in drug development. Take section II.C, for example, where the FDA breaks down the use of AI/ML in clinical research into 8 broad categories. If you’re putting together a study that involves AI/ML, show the FDA that you’re a pro and describe your proposed study using the same categories the FDA’s own experts used here. And don’t miss the glossary!  Use these terms when you communicate with the FDA and reference the discussion paper when you do.

  3. The FDA wants more decentralized clinical trials. There are other documents out there about this right now, and the early mention on page 5 of this paper is notable. This topic just keeps coming up. It’s definitely a “tailwind topic” at the agency right now: If you have an opportunity to frame your next proposal to FDA in the context of driving towards the future of decentralized clinical trials, that could help push you ahead.

  4. And finally, the Big Takeaway: The FDA is inviting you to help drive a larger effort. This paper is generating a dialog before the FDA gets too far down the path of deciding whether or not to put formal policy language together. Yeah, a discussion paper is not guidance, but, at least in my experience, the FDA doesn’t go to this much trouble unless they’re testing the waters and seriously considering putting something like a guidance together down the road. If you have policy ideas you want to see the FDA implement in the future, This Is Your Chance. Get your comments into the Federal Register so the FDA can use them!

Oh – and don’t miss this tidbit from the announcement: “In 2021, more than 100 drug and biologic applications submitted to the FDA included AI/ML components.” [2]

References

[1]
U.S. Food and Drug Administration, “Using artificial intelligence and machine learning in the development of drug and biological products,” May 10, 2023. https://www.fda.gov/media/167973/download (accessed May 10, 2023).
[2]
P. Cavazzoni, “FDA releases two discussion papers to spur conversation about artificial intelligence and machine learning in drug development and manufacturing,” May 10, 2023. https://www.fda.gov/news-events/fda-voices/fda-releases-two-discussion-papers-spur-conversation-about-artificial-intelligence-and-machine (accessed May 10, 2023).

Citation

BibTeX citation:
@online{2023,
  author = {},
  title = {Four Things to Know about the {FDA’s} New Discussion Paper on
    {AI/ML} in Drug Development},
  date = {2023-05-10},
  url = {https://www.boleary.com/blog/posts/202305-aiml-in-drug-development/},
  langid = {en}
}
For attribution, please cite this work as:
“Four things to know about the FDA’s new discussion paper on AI/ML in drug development,” May 10, 2023. https://www.boleary.com/blog/posts/202305-aiml-in-drug-development/
]]> FDA digital health AI/ML drugs https://www.boleary.com/blog/posts/202305-aiml-in-drug-development/ Wed, 10 May 2023 04:00:00 GMT