The One Size Fits All - or Not - by Jeanne Moldenhauer, reproduced from the May RMUG (Rapid Microbiology Users Group) newsletter.

A common occurrence for many distributors of rapid microbiological systems and consultants dealing with rapid methods is to go to a client site and have the intended User ask, What system should I buy?

The typical response is, What do you intend to do? As you might guess, the response is, Everything.

Rapid micro systems have been in the market for over ten years, but there still is a belief that one should be able to buy a single system that will do all the microbiological testing one might ever conceive to do. Ideally, it won't cost much money. Companies have established corporate goals, PAT (Process Analytical Technology) teams, and performance objectives that indicate that the microbiologists should purchase and validate a rapid microbiological system this year. Sounds easy, right? If you don't want to waste a lot of time or money, the right way to approach this objective is to spend time and decide exactly what you want to do with the system. In validation speak: we call this writing a User's Requirements Specification or URS.

Unfortunately, many of the laboratory personnel who are responsible for selecting and purchasing the new system have limited experience in writing the URS. In many cases, this makes the task of writing a URS daunting. The following discussion includes some of the considerations that one may use in selecting a system and writing a URS for a rapid microbiology system.

One of the first things that is important is to determine which tests (e.g., microbiological tests) are currently performed, the effectiveness and the limitations of the tests. For example, due to the need to grow microorganisms, environmental monitoring results are typically evaluated retrospectively. Obtaining the results several days after the production batch has been completed limits the usefulness of the data to perform additional meaningful investigations and appropriate corrective actions. PDA Technical Report Number 33 includes a decision tree for determining whether a new method is needed or desired.[1]

Part of this process may include gaining experience on the types of systems available and their capabilities in order to determine some of the possibilities to be considered. Extensive overviews of some systems that are available are included in the Encyclopedia of Rapid Microbiological Methods published by PDA/DHI2. Additional information on these systems can be found in many Internet searches or at www.rapidmicrobiology.com.

For example, if none of the new systems can do the testing methodology you desire to replace, your options are quite limited. Selection of an instrument for purchase should not take place until after a URS has been written. Many of the following considerations for purchase should be included in the URS. Some of the typical purchase considerations for a rapid microbiological sample include: economic issues, technical issues, compliance concerns, company culture and regulatory climate.

The following questions indicate some of the types of economic considerations for selection of a system:[3]

• What is the cost for purchase of the initial system?

A perfect system that provides for the tests you desire may exist, but the real question is whether your company can afford the system. For example, one might purchase a system that automates the environmental monitoring process but the cost of the system is prohibitive based upon the number of samples tested and the cost of the system.

• What is the projected return on the investment?

Although this is not a typical concern of the microbiologist, most pharmaceutical companies have requirements to determine this return as part of the request for the money. Several different manufacturers of rapid systems have resources available to help the User determine the return on investment. Calculating this return changes depending upon the type and number of tests performed. In many cases, the actual savings is more accurately cost avoidance, e.g., reduction of inventory hold or work-in-process hold times.

• What is the cost per test of the conventional method used?

The cost of performing the conventional test methodology is frequently understood in terms of the cost of supplies. In reality, one should also consider the associated labor costs, including performance of the test, and reviewing/maintaining the data. In the case of environmental monitoring tests, companies frequently need to maintain this data and report the data at the time of product release, but they also need summaries of data that are submitted to the appropriate regulatory agencies over a two or three year timeframe.

• What is the proposed cost per test of the rapid method?

Like the conventional test method cost, the cost of the proposed test should also include the associated labor costs for the method. It is common for the actual cost of many rapid methods to exceed the cost of the conventional test method.

• What is the impact on test release time, e.g., does it save inventory storage time or expedite product release?

One needs to be realistic when performing this analysis. For example, one might want to implement a same day sterility test method eliminating the need for a 14-day hold time during the conventional incubation time. It might seem easy to say that the inventory hold time will be reduced or the product release will be expedited by 14-days. In reality, the microbiological testing time may be reduced, but the actual product release time may not show the same time period elimination due to other tests, e.g. chemistry may still take multiple days. Other considerations include the time it takes for the supporting quality assurance tasks like: investigation management, change control documentation, batch release processes. In many companies it typically takes at least a week after the last release test is performed to release the batch.

Some companies use metrics to show that new methods implemented achieve the savings/returns that are specified in the capital requests. In other cases, they implement budget cuts as indicated in the capital request based upon the savings.

• Has anyone else published on this system, i.e., will my company be the first to use the system and deal with the many extra validation costs associated with being the first user in an industry?

There may be great benefits in being the first User to be approved utilizing a rapid microbiological method. Unfortunately, the first User frequently has experienced a longer learning curve than someone who implements the system many years later. There are benefits you can utilize if there is published literature, or other supporting information to aid you during the validation process.

What is the labor impact of using this system? Many of the systems currently available are not highly automated. As such, the new method may be more labor intensive than using the conventional methodology.

• How many units will my company need to obtain the desired throughput?

System throughput is a concern for some of the systems currently marketed. Realistic throughput evaluations are necessary in determining both the cost and need for additional units. Part of this consideration should include validation times, handling system downtime, preventative maintenance concerns, whether outside laboratories could perform the testing for your laboratory if the system is out-of-service. In addition to the economic concerns, there are technical considerations that relate to the scientific issues with the new methodology or system.

Some of these considerations include: [3]
Feasibility or proof of concept that the method will work for your purposes. These studies may be performed at the User site, at the Vendor site, via a contract testing laboratory or via literature reference. Some of the types of considerations include: filterability (if required), product interference with the test method, evaluation of limits of detection or sensitivity and so forth.

Scientific literature for the system, e.g., is this a novel system or can it be compared to existing technologies? Published literature is useful in setting acceptance criteria for the subsequent validation protocols, understanding the limitations and strengths of a system, and appropriateness of the system for specific uses.

Number and type of samples that the system can handle e.g., throughput, Vendors frequently promise throughput values that are not confirmed by end users. It is important to really understand the throughput that can be achieved at your site and with your products.

Sensitivity (limit of detection) of the methodology, The term sensitivity is defined in PDA Technical Report No. 33 [1] and proposed chapter USP <1223>. [4]

Specificity (organism detection) of the methodology, The term specificity is defined in PDA Technical Report No. 33 [1] and proposed chapter USP <1223>.[4]

Ease of use , One should consider the difficulty in performing this type of test method.

What level of operator education/training is required for the system? Some of the complexities of the rapid systems may necessitate more technical experience than some of the conventional methods.

What is involved in qualifying operators? The expertise required for some rapid systems may necessitate longer training periods to be proficient on the system. A secondary concern is how to keep personnel trained on the system when perhaps they do not routinely do the testing.

How much data can the system manage? It is important to ensure that if the system stores data, that it can maintain the level of storage necessary for your facility.

What types of analysis, summarization, and presentation of data are available? If reports are available for the system, will they meet the requirements of your company?

What type of vendor support is available, e.g., technical support, validation support, regulatory filings, training materials, User's Groups? Networking and technical support is very useful in successful validation of these systems.

Regulatory Compliance is necessary and considerations should be taken to ensure that the necessary provisions are made. They include :[3]

• Is the system validatable? This consideration includes whether there is appropriate supporting data to successfully complete all validation tasks, e.g., error message listings, version control, change control, and so forth.
• Does the software use electronic signature/electronic records?
• If yes, can 21 CFR Part 11 requirements be met?
• What will be required to successfully meet the requirements for a GMP site inspection?
• Are there FDA-483s, Warning Letters or other types of regulatory, compendial, or industry guidance that may be applicable to this technology? If so, can these issues be addressed?

Company culture is critical to the purchase, validation, and implementation of these systems. In many cases, the management team is reluctant to use these systems, or is extremely concerned about the potential for differences in counts between systems. As such, one might prefer to purchase an instrument that minimizes the differences in counts. Understanding management concerns is helpful in selecting the instrument that can most easily be implemented and validated.[3]

The regulatory climate should be considered prior to purchase of an instrument. Regulatory climate refers to an understanding of what regulatory agency approvals will be required for the approval of the new test methodology; will it require a prior approval supplement, what type of validation information will be required, and so forth. For example, while one may use the comparability protocol approach for a rapid microbiological method implementation in the US Regulatory System, there is not a corresponding approach in Europe. Other considerations may include:[3]

What requirements, if any, must be met for a regulatory filing on the system method? Not all rapid microbiological methods require a regulatory filing prior to implementation. It is important to involve the regulatory affairs department in the selection process, in order to ensure that all necessary regulatory requirements may be addressed. Additionally, a significant time investment may be necessary on the part of regulatory affairs to determine all of the applicable regulatory agencies included by the change. It is important to note that while FDA has some very clear guidance on how to submit rapid microbiological methods, as well as PAT initiatives, this may not be true of all the other applicable regulatory agencies.

Will there be issues with the method being significantly better than the conventional method, e.g., cannot show equivalence because it is so much better? A common concern for industry is whether the specification limits will have to change based upon the sensitivity of the new method. When this question was posed to some FDA reviewers in an open meeting, they advocated reminding industry of the change from TLC to HPLC.[5] Not being a chemist, I needed some additional clarification. Basically, the purpose of their comments was to indicate that specifications are linked to the technology used to collect the information. As such, the limits changed when the methods changed from TLC to HPLC. If a change in specifications is likely, this might be an appropriate time for a pre-meeting with the appropriate regulatory agency to discuss the change and its potential impact.

Is this a technology the agency has seen before, or will you have to educate the regulators? With the rate at which new methods are being developed, it is impossible for any one person to know all of the methods that are available. It may be necessary for your company to provide background on the technology being selected and used. This can be accomplished in a variety of ways including published literature, providing vendor presentations on the system, or use of a Vendor Drug Master File Letter of Authorization.

Are there established guidelines or regulatory documents regarding this technology? The compendial for the USA and EU both have chapters on validation of alternative (rapid) microbiological methods.[6] The chapters for the EP and USP are projected to be implemented this coming summer. The selection criteria and typical company requirements for hardware and software systems for the laboratory are typically incorporated into the URS. Some of the specific sections of the URS include the following:[3]

What is the purpose of the system? How is testing currently performed? How or what should the new system do? Technologies desired/excluded (based upon the information gathering discussed earlier) Economic considerations Technical considerations Regulatory considerations Safety considerations Engineering considerations, e.g., wiring, size, power usage, and so forth Vendor support requirements Requirements based upon your site's policies and procedures, e.g., only PC-based computers are used, system must be compatible with the site network; configurations must be documented; and so forth.

The results of performing this analysis is that one typically will determine that a single rapid microbiology system is not available to do all of the types of testing desired. As such, one may need several different systems to perform all of the microbiological testing. As such the microbiology laboratory of the future is likely to look like the chemistry laboratory of today, with many different instruments required to perform all of the desired testing.

References:
[1] Bauer, B.; et al. Technical Report No. 33 Evaluation, Validation and Implementation of New Microbiological Testing Methods. PDA Journal of Pharmaceutical Science and Technology. 54(3) Supp. 2000.
[2] Miller, M., Encyclopedia of Rapid Microbiological Methods, Volume 1 - 3, DHI Publishing, River Grove, IL. 2006.
[3] Miller, M., Encyclopedia of Rapid Microbiological Methods, Volume 1, Chapter 15, DHI Publishing, River Grove, IL. 2006.
[4] Validation of Alternative Microbiological Methods <1223>, United States Pharmacopeial Forum vol. 29(1) 256 - 264 , United States Pharmacopeial Convention, Inc., Rockville, MD, 2003.
[5] Personal communication from FDA Reviewers at the OGD Training Session on Rapid Microbiological Methods, held in Rockville.
[6] Validation of Alternative Microbiological Methods <1223>, United States Pharmacopeial Forum vol. 29(1) 256 - 264 , United States Pharmacopeial Convention, Inc., Rockville, MD, 2003.
5.1.6. Alternative Methods for Control of Microbiological Quality, European Pharmacopoeia 5.5, July 2006 p. 4131 - 4142. Conseil de L'Europe 2005.

Click here for more information on the Rapid Microbiology Users Group (RMUG)