Addressing Image-Based Compliance and Validation Issues

Review times for 510(k) submissions have increased by >55% since 2005 because of poor-quality submissions by medical device manufacturers, according to a US FDA July 2011 report (1). Such setbacks can debilitate research and development (R&D) budgets of medical device and pharmaceutical companies and significantly affect their return on investment. As the FDA increases scrutiny of submissions, organizations must ensure adequate controls in assessing drug and device efficacy for preclinical animal studies and clinical trials. Doing so lends validity to presented data in new product submissions (510k, premarket approval) and postmarket evaluations. It also assures reviewers that the necessary due diligence has been performed. Choosing proper controls for an FDA submission, however, can vary significantly depending on application and host.

In Vitro Imaging Studies

In vitro studies are commonly used to investigate the effects of drug treatments on specific cell types as a first-pass surrogate for in-vivo performance or for manufacturing quality assurance testing. Goals of such studies vary from assessing behavior of cell types in the presence of these drugs (e.g., migration, proliferation, apoptosis, and toxicity) to evaluating the presence or absence of particular proteins or evidence of genetic or phenotypic modifications. Before conducting those studies, however, carefully consider establishing appropriate controls that can both provide a “baseline” for measured outcomes and account for variables that could potentially invalidate study conclusions. Such flexibility is an attractive feature of noninvasive, in vitro analyses that require considerably less overhead than full-scale animal studies in which multiple iterations can be costly and time consuming.

To ensure that measured outcomes can be attributed to drug activity rather than carrier interaction or process artifacts, drug delivery controls are critical yet often overlooked in in vitro assays. For example, if a particular drug is applied to a cell culture with a specific vehicle for cell penetration, that vehicle alone must be tested to confirm that it or its contaminants are not responsible for observed effects present with the drug. In addition, physical processes for material delivery to cells (e.g., electroporation or microinjection) must also be replicated without the drug and material. Doing so confirms whether the process itself is not harmful to cells or produces an effect previously attributed to the drug.

A number of controls can be included to strengthen drug-specificity arguments. For example, if a drug interacts with a known cell surface receptor, that receptor may be genetically modified in a cell line to determine whether the drug can still elicit a downstream response. Similarly, if multiple proteins can bind with a drug’s targeted receptor, then that receptor may be preblocked with those proteins. This process may be performed before adding the drug or in conjunction with the drug to assess competitive binding efficiency. The drug itself also may be modified using known inhibitors that induce a conformational change or block its active binding sites (cell receptor fragment) before cell application.

Assessing presence of a particular protein (antigen) in cells as a result of genetic manipulation or drug treatment often involves antibody binding. That is followed by fluorescence secondary antibody targeting (immunohistochemistry) for subsequent in situ microscopic evaluation or flow cytometry. In such assays, it is imperative to evaluate the specificity of primary antibodies and examine negative controls such as nonspecific binding of secondary antibodies and natural cell fluorescence or autofluorescence. If possible, you should include positive controls known to elicit up-regulation or inhibition of a protein of interest.

Manufacturers of primary antibodies perform specificity analyses using immunoprecipitation and immunoblotting assays. Because those antibodies are generated using protein sequences derived from a specific species, it is often necessary to repeat those assays if the cells used in an in vitro experiment are acquired from an unrelated species. Although such assays provide specificity of an antibody to a particular epitope (antigen binding site), they do not assess the affinity of an antibody to other proteins containing similar epitopes (2). In that case, a positive control in which the target protein can be up-regulated or inhibited or a priori knowledge of expected protein location is invaluable.

To determine whether labeling can be attributed to a primary antibody specifically, it may be necessary to include a control. In that case, only serum from the species of origin without the antibody present is applied. Alternatively, a nonreactive antibody of the same isotype can be used, followed by secondary fluorescent staining. Depending on cell type, environment, or particular secondary used, nonspecific staining from “sticky” secondaries can result. They must be assessed separately from the primary antibody application. Nonspecific staining resulting from secondary antibody cross-reactivity can also be minimized by “preblocking” with serum from the host species before secondary antibody staining. For example, if a rabbit polyclonal primary antibody is used and the secondary consists of a goat anti-rabbit IgG, then goat serum should be used during preblocking. With all necessary controls in place, exposure times and optical settings used to acquire images for comparison and analysis must kept constant for all groups and optimized such that camera signal intensity is not saturated for any group (Figure 1).

Figure 1: ()

Preclinical Animal Imaging Studies

Animal imaging studies for evaluating drug or device safety and efficacy can be extremely effective in providing quantitative outcome measures that may not be plausibly or ethically acquired through human clinical trials. Unfortunately, without incorporating certain controls, interpretation of such analyses can be flawed — especially in light of the preexisting variability within an animal population.

First and foremost, it is absolutely essential that you validate animal models thoroughly and establish a baseline for comparison. For example, if a study uses ovariectomy as a means to induce osteoporosis (3), then bone-quality metrics obtained with microcomputed tomography (micro-CT) pre- and post-ovariectomy can provide population variability and establish a baseline for percent bone loss. Without that initial assessment, the effectiveness of subsequent treatments of a drug hypothesized to promote bone remodeling or regeneration would have no basis for comparison. In the previous example, treatments applied to an osteoporotic animal group also need to be extended to nonovariectomized animals to assess safety of a treatment regimen. In addition, a nontreated control group is necessary to determine the rate of “normal” bone loss due to aging.

As with in vitro analyses, physical manipulations required for drug delivery or device implantation necessitate procedural controls. For example, if a drug is delivered by injection to a site with a specific pat
hology, then the same procedure would need to be replicated either in a contralateral or nonaffected site with the drug delivery vehicle or placebo. Doing so can help determine whether the delivery procedure or vehicle itself produces an unanticipated response. Furthermore, if a surgical defect is generated for inserting biomaterial to produce a healing response, then scanning and analysis of de novo healing in a secondary defect site as well as the native implant before insertion are necessary for comparison (Figure 2). Because of variability in animal populations, such use of internal controls is recommended whenever possible. For example, in a micro-CT imaging study that examines femoral bone remodeling and necrosis resulting from injection of prostate cancer cell lines (4), the contralateral, unaffected femur can be mirrored, spatially coregistered (5), and superimposed to evaluate voxel by voxel changes in bone volume and density (Figure 3). This approach is particularly useful for longitudinal studies in which the scanning and analysis of unaffected contralateral sites can provide “normal”, nonpathologic changes for each time point.

Figure 2: ()

Figure 3: ()

Controls for implantable devices can also include insertion at multiple sites (e.g., subcutaneously) to assess biocompatibility and host response. The surgical procedure for implant placement may be performed at ancillary sites without actually implanting a device to ensure that effects of surgical manipulation are minimized and achieve a specific defect size. Do not overlook the effects of assessment techniques on typical progression of pathology in an animal model and on drug or device efficacy. For instance, irradiative effects of micro-CT imaging at longitudinal time points or effects of multiple blood draws for protein panel analyses should be evaluated independently in nonaffected and pathologic animal models.

Although not necessarily controls, use of multiple modalities for evaluating outcome measures can provide another means to ensure accurate assessment of drug or device performance. For example, bone implant in- and on-growth can be quantified comprehensively using micro-CT imaging and analysis and confirmed at higher resolutions using postsacrifice histological sectioning, staining, imaging, and analysis (6) (Figure 4).

Figure 4: ()

Clinical Imaging Studies

Obviously, many controls available for preclinical studies are impossible to generate for human clinical trials. Routine extraction of implants for histological examination and efficacy is not performed unless a device has been deemed to pose a significant risk to patients or has become nonfunctional. Performing sham operations on patients as controls in evaluating surgical procedures involved in implanting a particular device is generally not justifiable. In addition, using untreated, pathologic or nonpathologic patients as controls or baselines for treatment efficacy can be problematic because of the significant variability in the human population that can make disease progression unpredictable.

Fortunately researchers have options for controls in clinical trials. In the case of implantable devices, cadavers can be used as surrogates to determine surgical procedure reproducibility and implant placement as well as to assess feasibility of imaging techniques (e.g., CT and MRI) to extract quantitative outcome metrics. It is also possible to generate internal controls within a given patient. For example, if a patient has a pathologically altered shoulder imaged temporally following a particular treatment, then the opposite shoulder — if nonpathologic — can be mirrored and coregistered to serve as an internal control (Figure 5). Such a control not only serves as an indicator of systemic effects for a prescribed treatment, but it also provides a surrogate baseline to evaluate treatment efficacy at each imaging time point. Furthermore, this particular type of control is reasonable because of the symmetry in human anatomy.

Figure 5:

Additional controls are also necessary to validate customized algorithms developed to analyze imaging data for extraction of quantitative output metrics. For example, to assess the accuracy of in vivo, CT-based, automated volume analysis, implants may be scanned before insertion using different orientations to account for voxel resolution anisotropy. Furthermore, if feasible, those volumetric analysis algorithms may be applied to implant scans acquired using micro-CT imaging for higher resolution accuracy assessment (Figure 6A–C). For precision analysis, particularly important for temporal data, such algorithms may be applied to additional anatomical sites or implants. As an example, if a particular set of patients were implanted with metallic screws, volumetric analysis of their CT scans should reveal little or no variation across multiple time points (Figure 6B). Similarly, analysis of a bone cross-sectional area of multiple imaging planes in spatially coregistered temporal CT volumes can also be used to determine algorithm and registration precision.

Figure 6: ()

Although the controls presented here are generally applicable to a number of studies, the decision to use a particular control can be host, assay, and imaging modality specific. Our goal here is to convey the importance of establishing controls before conducting assays or trials to ensure that outcome metrics can be placed into proper context and correctly interpreted. Neglecting to include appropriate controls may significantly impact validity of results presented in FDA submissions.

About the Author

Author Details
Corresponding author Amit Vasanji, PhD, is chief technology officer at ImageIQ, Inc.,10000 Cedar Avenue, MS18, Cleveland, OH 44106; Carl Winalski, MD, is staff radiologist at Imaging Institute, Cleveland Clinic, 9500 Euclid Avenue, Cleveland OH 44195; Ronald J. Midura, PhD, is staff scientist and principal investigator at the Department of Biomedical Engineering at Lerner Research Institute, Cleveland Clinic.

REFERENCES

1.) Center for Devices and Radiological Health, US Food and Drug Administration 2011. Analysis of Premarket Review Times Under the 510(k) Program.

2.) Burry, RW. 2000. Specificity Controls for Immunocytochemical Methods. J. Histochem. Cytochem. 48:163-166.

3.) Yamazaki, I, and H. Yamaguchi. 1989. Characteristics of an Ovariectomized Osteopenic Rat Model. J. Bone Mineral Res. 4:13-22.

4.) Shekhar, R. 2003. High-Speed Registration of Three- and Four-Dimensional Medical Images by Using Voxel Similarity. Radiographics 23:1673-1681.

5.) McCabe, NP. 2008. Intraosseous Injection of R M1 Murine Prostate Cancer Cells Promotes Rapid Osteolysis and Periosteal Bone Deposition. Clin, Exp, Metastasis. 25:581-590.

6.) Takigami, H. 2007. Bone Formation Following OP-1 Implantation is Improved by Addition of Autogenous Bone Marrow Cells in a Canine Femur Defect Model. J. Orthop. Res. 25:1333-1342.