Two new User-Friendly Approaches to Assess Pharmacometric Model Identifiability

Introduction

In pharmacometric modelling, it is often important to know whether the data is sufficiently rich to identify the parameters of a proposed model. While it may be possible to assess this based on the results of a model fit, it may be difficult to disentangle identifiability issues from other model fitting and numerical problems. Furthermore, it can be of value to ascertain identifiability beforehand from study design.

Objectives

The aim of this work was to develop two new methods for identifiability analysis prior to model optimization that can be applied to a wide range of models and detect a variety of potential identifiability issues. Their use is illustrated with an example problem.

Software

An R package implementing these methods is available under the GPL license:
Download the package

Application

The methods were applied to a quasi-equilibrium (QE) approximation to a Target Mediated Drug Disposition (TMDD) model describing leukaemia inhibitory factor (LIF) data in sheep, with parameter values, sample times and dose levels obtained from Abraham et al. (Abraham, Krzyzanski, and Mager 2007). The output consisted of the PK concentration at the reported sample times. The methods were presented at the PAGE conference (Noort and DeJongh 2023).

Figure 1: Model structure

Table 1: Model parameters, from Abraham et al. (Abraham, Krzyzanski, and Mager 2007).

Methods

The parameters of a model are identifiable if two different parameter vectors always lead to two different model outputs. In practice one often focuses on local non-identifiability, characterized by a curve in parameter space of constant model output. The tangent to the curve is the unidentifiable direction. If a model is locally unidentifiable then its parameters cannot be determined uniquely from the data. Two methods for local identifiability were developed.

Method 1: The Sensitivity Matrix Method (SMM)

The sensitivity matrix (SM) is the matrix of derivatives of the model outputs with respect to the structural parameters. Unidentifiable directions in parameter space correspond to vectors in the null space of this matrix. As the null space is difficult to determine numerically, three proxy indicators were developed that identify near-singularities of the SM and the corresponding parameter vectors.

1. The Skewing Angle, which measures the angle between the images in the output space of the parameter vectors. A small angle indicates non-identifiability;

Figure 2: Skewing Angle

2. The Minimal Parameter Relations, listing the parameter directions closest to singularity. The vector norm (M-norm) indicates the level of identifiability, with small lengths corresponding to non-identifiability;

Figure 3: Minimal Parameter Relations

3. The Least Identifiable Parameter(s) (L-norms), indicating which parameters are closest to linear dependence on the others. In the figure below, the parameter e₃ is unidentifiable if the vector w₃ is small. The norm of the vector w₃ is the L-norm and is used as an identifiability indicator.

Figure 4: Least Identifiable Parameter(s)

Method 2: The Fisher Information Matrix Method (FIMM)

The Fisher Information Matrix (FIM) determines the approximate shape of the objective function value (OFV) with respect to the parameters. Unidentifiable directions in parameter space correspond to vectors in the null space of this matrix. As for the SMM, proxy indicators are determined.

1. Curvatures of the OFV surface. A low curvature corresponds to non-identifiability;

2. The maximum change in parameter values corresponding to a given OFV change. Large changes indicate non-identifiability;

3. Relative standard errors (RSE), where high values characterize non-identifiability.

Unlike the SM method, this method can handle random effect parameters. It assumes Gaussian distributions.

Figure 5: Visual representation of curvature and parameter changes

Results

General overview plot

This shows overall identifiability indicators from the two methods. For each one, a high value indicates identifiability. In this way, different scenarios can be compared in one view.

Figure 6: Composite model identifiability visualization from SMM (skewing angle, Mnorm, norm of the least identifiable parameter) and FIMM (curvature 1), on a log scale. All indicators have low values for the single dose level scenarios, indicating non-identifiability, and higher ones for all dose levels combined, showing identifiability.

SMM: Minimal parameter relations

For unidentifiable scenarios, the minimal parameter relation indicates the direction of unidentifiability, that is, the parameters that are badly identifiable. They correspond to high values.

Figure 7: SMM results: minimal parameter relation per unidentifiable scenario. High values indicate the parameter is badly identifiable. This shows that different combinations of parameters are badly identifiable for the individual dose levels

SMM: Least identifiable parameters

The L-norm of a parameter characterizes the distance of this parameter from all the others ones. If Lnorm is zero, this parameter is unidentifiable. The larger the L-norm, the more identifiable the parameter is.

Figure 8: L-norms per parameter

FIMM: Curvatures

Low curvatures correspond to a shallow OFV surface as function of the parameters. The number of low or zero curvatures is the number of unidentifiable parameter directions.

The plot shows a red “warning” zone where curvatures are low. In this case it is defined below the minimal Lmin*100, but could be optional for the user to choose.

Figure 9: FIMM results: Curvature 1-8 for scenarios for each single dose level and for a scenario with all three dose levels combined. These confirm the radar plot summary and demonstrate that the QE TMDD model becomes identifiable only in a scenario where the three dose levels are combined.

FIMM: Directions (interactive plot)

For each scenario and curvature, the following plot has a column showing the maximum relative parameter changes corresponding to a change in OFV that is not significant (at 95%). Different colors are used for relative changes above and below a threshold. Changes corresponding to non-positive curvatures (if any) are also colored differently. The dot size indicates the size of the change.

Figure 10: FIMM results: size of relative parameter change corresponding to significant change in OFV, per scenario and curvatures. Orange color and large sizes indicate bad identifiability, while green indicates parameter changes below 50%. This confirms the SMM results and shows identifiability for the combined scenario.

References

Abraham, Krzyzanski, and Mager. 2007. “Partial Derivative-Based Sensitivity Analysis of Models Describing Target-Mediated Drug Disposition.” AAPS 9 (2).

Noort, van, and DeJongh. 2023. “Two New User-Friendly Approaches to Assess Pharmacometric Model Identifiability.” PAGE, A Coruña, Spain, June 27-30 (poster). https://www.page-meeting.org/pdf_assets/10322-abstract.pdf?r=5275.

Colophon

Developed by: Richard Hooijmaijers, Rolien Bosch, Martijn van Noort
2023-07-04
Contact person: Martijn van Noort (m.vannoort@lapp.nl)