Internal Quality Audit Scheduling
With 2021 around the corner, many quality organizations are beginning to plan their internal audit schedules for the upcoming year. Some will plan their audit schedule to follow the same sequence of audits they conducted this year. This unfortunately common practice is non-compliant with ISO13485, as audit schedules must be based on risk. Still, how to incorporate risk in audit scheduling is not often taught or discussed in depth.
In this short article I’ll review some of the techniques I have used with success. Much of what I will share is made easier with the use of a suitably designed eQMS.
Some priorities to consider:
- The audit program should cover the entire scope of the QMS on a regular basis. In most companies, re-audit of the full QMS is completed on an annual cadence.
- The audit program should give higher scrutiny to processes that have recently changed or are the subject of many NCRs or CAPAs. It goes without saying that CAPAs generally cause a process to change. Changed processes are more likely to have new non-conformities than well-established ones.
- The audit program should cover all product families. In small companies, there may not be more than one product. In larger companies with multiple products or sites it is necessary to cover all products and all sites on a regular basis. Typically with multi-site organizations, audits may be tiered with all sites being audited at least once per year by the parent organization (with a scope typically limited to a small sampling of processes or products) and all internal processes being audited by the site-level quality management once per year.
- The audit program should consider the business context. The Quality Management System may have adjacencies with an Information Security Management System or other management systems that themselves also have audit programs. These audit programs may have shared objectives with the quality audit program or may have subject matter experts that are shared. As a general principle, audits will be more successful and collect more useful audit information if they are conducted with minimal disruption to the auditee. Consider where there may be overlap with other management systems and allow one audit program to take the lead.
Given these priorities, there are a few techniques you can use to have a more successful risk-based audit schedule.
Audit criteria reviewed this year
Clearly spelling out the audit criteria covered in each audit can be a bit daunting but at the end of the year it provides data that can be used for the next audit schedule. If the audit schedule this year was risk-based and had criteria or entire audits that were skipped, these should be considered as high priority for the upcoming year.
Documents reviewed this year
Every QMS has a Quality Manual, but not every QMS has this information tabulated. A table with columns “SOP” and “applicable quality system element” can be very useful. By “quality system element” I mean, for example: complaints, non-conformances, or purchasing. This Quality System Structure is incredibly useful to have in a tabular form. Mapping of quality system elements may align with the subparts of 21CFR820 or the clauses of ISO 13485 or ISO 9001— frankly to any set of audit criteria you’d like.
With a Quality System Structure table in hand, you can confirm what has been well or poorly covered by tallying the number of instances each SOP has come up in each of the current year’s audits. For example, the Risk Management SOP may have been covered in the Purchasing Department audit as well as in the Engineering Department audit — two tallies for this SOP. Likewise, the Pest Control SOP might have been missed in the Logistics audit — zero tallies for this SOP. After reviewing all of the current year’s audit files, you’ll have a good idea what SOPs have been looked over a few times and which ones will need to be dusted off for next year.
A product vs quality system element matrix
Another use for the tabular Quality System Structure is to track which products have been audited for each of the different quality system elements. With each quality system element as rows and each product (or product family) as columns, identify which “product-quality system element” combinations have been covered in the current audit year. Coverage can be indicated in the matrix by placing the audit number where each “product-quality system element” was covered. Areas that have not been audited in the current year may pose a higher risk, since their compliance status is not known.
Change Control records from this year
Collect a report of product, process, and quality system changes made in the past year. Consult this report in conjunction with the “product-quality system element” matrix discussed above. Are there new changes to products that were not audited this year? Not all changes need to be audited in the next audit cycle, but preparing this information may illuminate coverage gaps.
Latent class analysis
Run a report of all the NCRs, CAPAs, and audit findings from the current year. These are quality events that could have been prevented if only they were caught by the audit program and resolved the prior year. With this in mind these are a great data source to consult to build a risk-based audit schedule.
Some of the issues raised may not be all too important — a quick 2-minute review of each of the problem statements should give you an idea on the relative risk level. I use a crude three-level risk assessment scheme to separate out bigger issues from smaller ones. I find that a quick and independent assessment of risk, even if crude, is better than relying on more sophisticated risk assessment tools that the company may employ for product risks.
Add a column for priority to each quality event row. If your report doesn’t already identify the owning department for each quality event, add this information as an additional column. Also add the quality system element that best describes the issue. Think of this as the answer to the hypothetical “I would have caught this auditing _____ quality system element.” Again, use the same quality system elements outlined in the Quality System Structure.
If you have enough quality events, clustering algorithms can be used to naively group similar issue areas together. For example, issues relating to the Customer Care and Product Development departments may be related in that they both pertain to Design Controls — this relationship or others may naturally bubble up from the data. There may be some interfaces that a joint audit of these departments together might expose where a department-by-department audit schedule might not. In this example, consider how customer feedback is incorporated into design and development — would this be as well scrutinized if the feedback loop was split over two audits, occurring perhaps months apart?
I have used latent class analysis in JMP for this clustering but there may be more appropriate tools.
Complaints and Adverse Events
If complaints are managed well, they will be escalated to NCRs or CAPAs based on risk. For this reason, these risk indicators are not orthogonal to other quality events, and too much focus could be to the detriment of other metrics, such as full-scope coverage. That said, any audit schedule should consider if known issue areas are appropriately covered.
Alternative styles of auditing
Some audits are best conducted in a mock external audit style with a front and back room and all-day structure with a high demand on the support team. However, you may find that holding longer, slower audits give you more time as an auditor to make requests for and receive information that might not be pursued in a shorter mock audit setting. Some audits can be conducted by placing requests in a Google Sheet and tagging an SME. This approach allows for deeper audits that are at the same time less disruptive to subject matter experts who may take a day to respond. Consider if a change to the style of auditing can be used as justification to also change the number of audits.
Conclusion
I hope you find some of these considerations and techniques helpful as you build your next quality audit schedule. If you have any other tips please let me know!
