Post‑Market Surveillance Systems — More Than Just Complaints

Post‑market surveillance is often reduced to a mechanical exercise: count complaints, categorize them, “investigate” them, file the required reports, and move on. Some investigators may even consider this minimally compliant. But this narrow view does nothing to improve products, strengthen processes, or create value for patients, clinicians, or the organization.

What regulators actually require—and what organizations genuinely need—is a holistic, closed‑loop post‑market surveillance system. One that continuously monitors product performance, rapidly detects when products deviate from expectations, and actively drives improvements to quality, safety, and efficacy.

Yes, there are explicit regulatory requirements for complaint handling and adverse event reporting. These must be followed precisely. But compliance alone provides only a partial and often distorted picture of product performance. Many organizations become so focused on meeting the letter of the regulation that they lose sight of the broader purpose: early detection of issues and continuous improvement. When post‑market systems devolve into check‑the‑box activities, the organization forfeits the strategic value these systems can deliver.

What IS in Your Complaint Data

Complaint data is imperfect—too late, too sparse, and too underreported to stand alone. But when handled correctly, it contains meaningful signals.

Evidence of Design and Use Issues

Complaints often reveal mismatches between intended use and actual use. Patterns such as repeated user errors, confusion about instructions, or misuse that leads to harm can indicate design shortcomings. This information helps teams identify where labeling, instructions, user interface elements, or physical design features may not adequately support safe and effective use. Even a small cluster of similar complaints can highlight latent design vulnerabilities that were not fully captured during formative or summative usability testing.

Evidence of Systemic Manufacturing Issues

When evaluating a complaint, two questions should always be asked:

1. Is a product failure being reported?

2. Is that failure intermittent or systemic?

Intermittent Failures

Intermittent failures are inherent to processes due to ever present variation. Even highly capable processes produce occasional defects. For example, plastic components may occasionally crack, leading to a leaking infusion set. If the failure rate aligns with process capability and no trend is present, the event likely represents normal variation. The key characteristic: Units produced immediately before or after the defective unit are not more likely to share the defect.

No additional action is required beyond documenting the evaluation.

Systemic Failures

Systemic failures indicate a repeatable, assignable cause. For example, if a leaking infusion set is found to have no bonding solvent at a joint, an empty solvent well may be the root cause. In this case, every unit produced after the solvent ran out is likely affected. This requires deeper investigation, potential containment, and possibly field action. The key characteristic: Units produced immediately before or after the defective unit are more likely to share the defect and the manufacturing controls are not functioning as intended.

Even a single complaint can provide enough information to distinguish between intermittent and systemic issues—saving time, preventing unnecessary investigations, and ensuring real issues are not missed.

Unknown and Unexpected Failures

Complaints are often the first source of information about failure modes not identified during design verification, validation, or risk analysis. These may include rare environmental interactions, unexpected user behaviors, or degradation mechanisms that only emerge after long‑term field exposure. Each unexpected failure expands the organization’s knowledge space and should feed directly into risk management, design improvements, and future hazard analyses.

What IS NOT in Your Complaint Data

Because of how complaints are generated and reported, they are poor sources for certain types of information.

Usability Information

Usability is different from use error.  For example, Workday is a product with notoriously poor usability.  Users may be able to complete a task without error, but the experience, intuitiveness, and efficiency suffer.  Complaint data rarely captures the nuance of user experience. To understand usability, organizations must rely on proactive mechanisms such as:

• Regular field visits to observe real‑world use conditions

• Advisory boards composed of clinicians, patients, or caregivers

• Structured feedback sessions that explore workflow, pain points, and unmet needs

• Rapid prototyping of UI or labeling improvements to validate enhancements before implementation

These sources provide richer, more actionable insights than complaints ever can.

Determining Outgoing Product Quality

Outgoing quality cannot be deduced from complaint rates. Complaints may represent only a fraction of actual defects or may report defects in a way that cannot be directly tied to manufacturing defects. A far more reliable principle for determining outgoing quality is that “the more defects you make, the more defects you ship”.  Because of this, organizations should rely on:

• In‑process monitoring

• Inspection and test data

• Release metrics

• Process capability indices (Cp, Cpk)

These datasets reflect the true quality of manufactured product and should be the primary source for assessing outgoing quality and triggering action when necessary.

Get the Most Out of Your Complaint Trending

Complaint trending is required—so it should be designed to deliver maximum value.

Trend Patterns in the Product, Not Patterns in How People Contact You

Trending by complaint report date instead of manufacturing date obscures true signals. A spike in calls may reflect a marketing campaign, a desire for end of fiscal period reimbursement, a supply disruption, or a seasonal shift—not a product issue. Indexing complaint data to manufacturing date aligns events with the process conditions that created the defects being reported, enabling meaningful analysis.

Trend the Number of Actual Malfunctions, Not the Number of Calls

Customers often will report multiple incidents (defects, alarms, errors, etc.) in single call or email. Trending the number of calls distorts the signal. The correct denominator is the number of product malfunctions, normalized to the number of units produced or distributed.

Do Not Discard Complaints when the Product is Not Returned for Evaluation

If a customer calls you to tell you a product is malfunctioning, the overwhelming likelihood is that it is because the product is malfunctioning. Complaints that cannot be confirmed should still be trended both from a compliance standpoint and from a data completeness standpoint. If a sample is returned and the reported malfunction is refuted that may be taken into account, but simply not receiving a sample or photo is not a reason to ignore the report.

Achieving the Goldilocks Principle

Complaint data is noisy. Effective trending must minimize both false signals and missed signals.

Common pitfalls include:

1. Using traditional UCL methods on non‑normal data Complaint data is typically Poisson or binomial, not normal. Applying normal‑distribution control limits generates excessive false alarms.

2. Assigning statistical meaning to very small numbers Zero, one, or two events carry no statistical significance. Even three or four events provide limited insight. Systems that trigger on tiny counts create noise, not value.

3. Improper categorization of events Categories that are too broad dilute signals; categories that are too granular prevent aggregation. Both lead to missed trends.

Non‑Normalized Data Is Not Analysis—It’s Counting

Trending raw complaint counts without normalization is misleading. Normalization must match the population that generates the events—for example:

• Units produced

• Units distributed

• Patient‑days of use

• Cycles or activations

Without normalization, a product with high sales volume will always appear worse than a low‑volume product, regardless of true performance.

React to Unexpected Defect Levels in Individual Lots

Even with all the flaws in complaint data, one rule holds: If complaints for a specific lot exceed the number of defects expected based on process capability, a response is required.

Example

If a molding process has a demonstrated defect rate of 100 ppm, and a lot of 50,000 units generates 12 confirmed defects, this exceeds the expected 5 defects by a wide margin. Even if the absolute number seems small, the deviation from expected performance is significant and warrants investigation.

Capture Information About Performance After the Product Leaves Manufacturing

Complaints may be the only source of information about issues occurring during shipping, storage, or distribution.

Example

If multiple complaints report cracked housings, and investigation reveals impact damage consistent with rough handling, the issue may lie with a specific distributor or shipping lane. Separating shipping/distribution issues in trending allows targeted corrective actions such as packaging redesign or logistics changes.

Timeliness of Post‑Market Activities

Organizations that wait for annual PSURs or periodic risk reviews are missing critical opportunities and exposing themselves to compliance risk. Annual or semiannual analysis is not actionable and does not protect patients.

With modern analytics and AI tools, organizations can—and should—perform real‑time or near‑real‑time post‑market analysis. Automated dashboards, anomaly detection algorithms, and integrated data pipelines allow teams to identify emerging trends within days, not months. Timely detection enables timely action, which is the core purpose of post‑market surveillance.

Why Holistic Post‑Market Surveillance Matters

A robust post‑market surveillance system is far more than complaint processing—it is a strategic capability. Organizations that treat post‑market surveillance as a closed‑loop, real‑time system gain earlier detection of issues, stronger product quality, and deeper insight into design, manufacturing, and usability. By trending the right data, normalizing appropriately, distinguishing systemic from intermittent failures, and integrating field insights beyond complaints, companies protect patients, strengthen regulatory trust, and drive continuous improvement. The value is not in counting and reporting complaints—it is in transforming post‑market data into actionable intelligence that improves products and safeguards public health.

Looking for support in this area? A‑Z Continuous Compliance, LLC provides Post Market Surveillance process development, training, and creation of Risk Files which support Post Market Trending. We can also develop culture change strategies that engages the organization cross-functionally through Post Market value propositions.

Contact