The Value of Plug-and-Play Automation in Single-Use Technology

Figure 1: Physical and procedural model implementation based on principle 88 of International Society for Automation (ISA) standards for batch process control

Automation can improve efficiency, track performance, adjust operations, and liberate operators from mundane routines. Automation requires a flexible set of tools that align well with the inherent flexibility of single-use technology (SUT). Although SUT flexibility enhances a biomanufacturer’s ability to modify operations to meet the needs of today’s dynamic industry, it also increases timelines and costs related to customizing and validating automated additions.

We present herein the findings of a team of industry automation experts who are sharing their experiences within the BioPhorum Operations Group’s Technology Roadmapping collaboration to test new automation methods (1). Our group’s vision is to move from the current state of unique and custom software development to a reusable, standardized approach that enables rapid integration of intelligent process skids (1).

The group includes representatives from end users, equipment manufacturers, and control systems suppliers. Input from those three major constituencies is providing a balanced assessment of industry standards and compelling the development of implementation options that can be widely adopted in the biopharmaceutical industry.

As detailed in the “Automated Facilities” section of the first edition of the BioPhorum Biomanufacturing Technology Roadmap (1), automation can provide significant benefits to biopharmaceutical manufacturing operations. The report highlights several ways to increase automation. In general, automating processes increases operational consistency and product quality. With the biopharmaceutical market’s need to reduce both cost of goods (CoG) and time to market, facility build time has become a critical reference point. Being able to use a facility during product changeover is equally important.

This is an area in which the concept of “plug-and-play” (P&P) can help. The industry needs P&P principles for production units: principles that can standardize communication protocols and reduce equipment startup or process changeover validation time in ways that comply with good manufacturing practices (GMPs).

In developing a P&P concept, we considered corresponding standards developments in both similar and disparate technology systems. Then we determined how those standards fit with the aims of global standardization activities including the work of the Open Group (www.opengroup.org), NAMUR (www.namur.net), and the Allotrope Foundation (www.allotrope.org). We believe the development of P&P complements and builds on the work done by such organizations. The group also is assessing the use of data integration services, such as MQ Telemetry Transport (MQTT), for facilitating interoperability. Evaluations performed by the Open Process Automation Forum (OPA) will be considered in the assessment. OPA is focused on developing standards-based, interoperable process control architecture.

Here we focus on standardization of software structures by using a common automation interface for unit operations. Having a common automation interface provides for equipment interchangeability, reduced facility build time, and greater flexibility in operations. Such efficiencies can drive down costs while maintaining consistent operations.

The Need for Plug-and-Play
The biopharmaceutical industry’s movement away from large-scale, fixed-tank facilities to flexible, SUT-based facilities has demonstrated the value of moving toward modular equipment and agile process design. Deloitte reports that such a strategy already has been valuable to biomanufacturers while they are being squeezed to do more with less (2). The added flexibility has proven to be a clear advantage to end users challenged with today’s production requirements: End users need to build quickly and reconfigure equipment with minimal time.

SUT addresses the mechanical aspects of flexibility. But linking the equipment with equally flexible automative technology has been difficult. A common approach is to remove all intelligence from automated single-use unit operations and retain only the inputs and outputs (I/O). This “remote I/O” approach requires significant coordination. End users must describe their needs, equipment manufacturers must communicate the capabilities of their systems, and supervisory system vendors must integrate all systems accordingly. In addition, all parties must validate the entire automation solution from scratch.

Several additional months may be needed to identify the control elements that must be brought into the supervisory system, especially for scenarios in which vendor programming is leveraged for only some aspects of the equipment. Development of custom interface software generally can take from two to eight weeks, depending on the complexity of the equipment:

Although custom software provides great flexibility, its development and use can inhibit projects. Custom code can complicate the patching and security elements of lifecycle management. Interface development needs to cover all aspects of process control, including definition of data that need to be transferred. That requires creation of data-exchange tables for process parameters, report values, and device and sensor status points as well as specification of how equipment will be operated.

Increased documentation, testing, and validation efforts require significant collaboration across all three parties. And long lead times (of several months) increase the overall time to operate new or expanding facilities Custom code also can lock end users into arrangements with original suppliers.

When considered against the design principles of Industry 4.0 (3) — interoperability, information transparency, technical assistance, and decentralized decision making — the P&P concept seems especially useful and sorely needed. The concept addresses three of those principles:

The potential benefits of applying a P&P structure to multiple unit operations — and to an entire biopharmaceutical manufacturing process — can decrease manufacturing costs, shorten manufacturing timelines, and improve product quality. Therapies ultimately could cost less and get to patients more quickly.

BioPhorum’s Plug-and-Play
Objectives: Numerous sources explain the value of “ballroom-style” manufacturing. “Smart Modular Package Units for Single-Use Manufacturing” (4) is one of the many that highlight the benefits of incorporating modular units. A ballroom provides space for equipment to be used and moved but still reduces the total space needed by 38% relative to a conventional factory (5). Thus, a ballroom scheme addresses physical requirements for bioprocessing equipment. But automation systems still are needed to orchestrate operations, communications, and interactions among the various processing units and supervisory control systems. A P&P approach can reduce substantially the time and effort to implement equipment in a flexible SUT environment by standardizing the communication framework. A ballroom provides the space; P&P would facilitate partner switching in the biomanufacturing dance.

The objective of BioPhorum’s P&P concept is seamless integration of intelligent unit operations in the ISA S88 procedural batch engine of the overlying supervisory automation system of a GMP-compliant facility. That objective can be achieved through a standardized interface and data model — which we introduce below.

Implementation Scenarios: P&P is intended for both existing and new facilities. The supervisory automation system can operate in both brownfield and greenfield single-use facilities. It will be programmed in compliance with BioPhorum’s communication specifications and unit operation data model, coupled with an original equipment manufacturer (OEM) control system with software that uses the same interface and data model.

Supervisory control system support of protocol and data model specifications is a fundamental functionality required to support the BioPhorum’s P&P interface standard. Brownfield facility project managers considering implementing this interface standard will need to replace the existing supervisory control system or upgrade to an open platform communications unified architecture (OPC UA)–compatible version. That would be a one-time cost to reconcile OPC UA protocol requirements and implement a unit operation data model. Alternatively, a facility that continues using a nonstandard custom interface will face high recurring costs with each subsequent equipment replacement. Actual costs of either scenario will vary depending on the existing facility.

In facilities where an existing supervisory control system is OPC UA–compatible, two results are possible depending on the architecture. In the case of conventional architecture, an aging asset that existed as a standalone skid or custom interface is replaced with a new skid that is BioPhorum P&P–compliant. The supervisory control system will need to adopt the data model for ISA S88 batch recipes and unit operations belonging to the aged asset. That would be a one-time cost to align the old system with current standards and enable upgrades and replacements to follow the workflow of the next case.

The second sceanrio occurs when the network architecture, supervisory control system, and equipment skid already are BioPhorum P&P–compliant. This might happen when a manufacturing facility scales up operations from laboratory to commercial volumes or when a company replaces one manufacturer’s skid equipment with that of another. In either case, existing ISA S88 batch recipes can be leveraged on the new equipment skid. That requires minimal changes on the supervisory control system to accommodate the new skid’s metadata and configuration or to incorporate any nonstandard extensions.

Table 1 lists the relative cost of various use cases to transition from an existing equipment asset to P&P-enabled equipment.

Equipment Systems Benefits: A communication protocol can be set up using a P&P approach and validated successfully between a supervisory system and an OEM local automation system, running a single-use unit operation in a matter of hours. Shifting the degree of validation from an end user to a supplier, as shown in Figure 1, contributes to reducing validation time at startup.

Using a standard communication interface and data model also reduces risks during new installations: Because the software has been used and tested in commercial operations, end users can trust its communicative reliability and do not need to inquire further into validation. This results in time and cost savings. Table 2 notes essential points of comparison between conventional and OPC UA—compliant architectures.

To summarize, the benefits of flexible and standardized automation based on modular methods derive primarily from having intelligent equipment integration preconfigured rather than developed for each device. Such an approach promises to

Communicative Benefits: Using standard architectures and interfaces tends to reduce or even eliminate the negotiation phase between end users and distributed control system vendors. In terms of overall timelines, vendor interface capabilities generally are not available until after equipment orders are placed. Such a delay puts additional strain on a project. By eliminating the need for custom development, end users can save time on design and testing, alleviating schedule constraints and potentially allowing them to develop recipes before equipment is purchased. Thus, standardized automation can be more cost- and time-effective.

Table 3 lists estimated P&P cost savings. Although the first two functions in the table are self-explanatory, the others might require some illustration. A function related to troubleshooting communications is one needing to wait two to three business days before initial communications work between a supervisory control system and an OEM system. Functions related to developing workarounds and diagnostic tools include development of strategies to assess the reliability of — and perhaps reestablish communications between — different interfaces. The category for using tools to detect, diagnose, or facilitate communications includes time spent engaging third-party OPC “tunnellers,” which standardize communications between interfaces. That final category is important for dynamic installations such as single-use skids. Tunnellers allow for flexible configurations and can be used to detect communicative failures that diminish data integrity.

Manufacturing Performance: A standardized software interface also can improve analyses of manufacturing performance. Benefits in this area stem from facilitating data transmission from process equipment to analytics platforms. Standardizing a data model for both operational parameters and report parameters also enforces standard naming conventions. Contextualizing data at their source simplifies integration of operational data into crossplatform analytical systems, allowing aggregation with data from other systems. With those data, process engineers and scientists can trend the performance of critical process parameters and investigate process deviations. Moreover, GMP-compliant batch reports encompassing all equipment can be generated relatively easily and used as a basis for release by exception.

The general impact of automation on overall operations is listed in Table 4, which is an excerpt from the Technology Roadmap’s automated facilities report (1). It classifies automation’s expected cost impacts at five and 10 years after implementation. The estimates provided in Tables 3 and 4 all are based on the experiences of BioPhorum group members. Although variations due to specific company operations are possible, the level of impact listed here is likely to align with that seen in many operations. That said, automation is typically on the critical path for a project, which magnifies its impact on overall time scales and costs.

Figure 2: Plug-and-play development schedule; green diamonds indicate current progress. PP02 refers to the automated facility team’s second face-to-face (F2F) meeting; PP03 refers to a third meeting scheduled for late 2019.

Progress Achieved So Far
The P&P vision started in 2017 with a proposal to BioPhorum’s Automated Facilities group to develop a standard data interface between supervisory control systems and single-use biopharmaceutical equipment skids. Once that proposal was accepted, a team started to review proposed data models for bioreactor, chromatography, and standard filtration skids. With end users, equipment manufacturers, and control system suppliers on board, an aggressive development schedule was created (Figure 2).

Another team was formed from the end-user community to develop a bioreactor data model, while a third team of equipment and control system suppliers worked on the control interface. The goal was to hold a proof-of-concept test before the end of 2018, at which point a decision would be made to carry on or abandon the project. To keep the test simple, two representative phases were created with a common data model. They were implemented by participating equipment suppliers. The test was performed in November 2018, with 30 people representing six end-user companies, three control system suppliers, and four equipment suppliers.

Figure 3: Interoperability equipment used during the “Plugfest” test

The test required three pairs of control systems and equipment skid controllers to coexist on a common network, with each pair executing both phases. Figure 3 diagrams the communication structure of the test program, named “Plugfest.”

The test was a success and showed that there are no technical showstoppers to realizing a BioPhorum P&P standard. The team will incorporate the lessons learned into future testing as well as the development of the standard.

Figure 4: Upcoming face-to-face (F2F) activities for the plug-and-play program; PP04 and PP05 refer, respectively, to the fourth and fifth plug-and-play team meetings.

Plugfest’s Proof of Concept: The success of Plugfest led the BioPhorum subgroup to continue with plans to automate the equipment. Figure 4 shows the current plan.
Although many unit operations comprise a biopharmaceutical manufacturing process, the current focus has been on automating bioreactors. Cell culture is one of the more difficult unit operations to automate, and if the proof is positive here, it will be easier to apply the interface to other unit operations. As summarized above, the proof has been positive thus far, and the team is converging on true P&P automation for a single-use bioreactor.

Savings can multiply when applying P&P to a large number of systems. After bioreactors, our group will develop a P&P structure for normal-flow filtration and chromatography systems. The goal will be to have the supervisory system pass control signals farther downstream and collect data from upstream unit operations with minimal configuration effort. We also hope that the system can do so independent of supervisory-system and OEM manufacturer. As listed in Table 5, several unit operations are good candidates for consideration.

Looking Ahead
Rough analysis of P&P implementation provided sufficient incentive for the BioPhorum team of end users, equipment manufacturers, and control system suppliers to initiate proof-of-concept tests. The proof of concept demonstrated that it is possible for equipment and control systems to communicate with a common interface. Enthusiasm generated by the results prompted the drafting of this report as well as the next step of the test program.

Photo 1: Proof-of-concept test team

A second publication will illustrate how the BioPhorum team has transformed the current bioreactor P&P specification to reflect NAMUR module type package (MTP) standards. Subsequent Plugfests will demonstrate that NAMUR MTP standards facilitated by formal collaborations with BioPhorum can meet the biopharmaceutical industry’s specific requirements. Once interface standards have been defined for each unit operation, a way to govern them also will be implemented.

The potential benefits of applying a plug-and-play structure to multiple unit operations — and to an entire biopharmaceutical manufacturing process — can decrease manufacturing costs and significantly improve timelines and product quality. Through the reduction of capital and operational expenses, therapies can be developed at reduced cost and get to patients faster.

References
1 Automated Facility: Biomanufacturing Technology Roadmap. BioPhorum Operations Group, 2017; https://www.biophorum.com/phorum/technology-roadmapping/overview/?doing_wp_cron=1568623665.4744229316711425781250.

2 Embracing the Future of Work to Unlock R&D Productivity: Measuring the Return from Pharmaceutical Innovation. Deloitte, 2018; https://www2.deloitte.com/us/en/pages/life-sciences-and-health-care/articles/measuring-return-from-pharmaceutical-innovation.html.

3 Hermann M, Pentek T, Otto B. Design Principles for Industrie 4.0 Scenarios. 49th Hawai’i International Conference on System Sciences. Koloa, HI, 2016: 3928-3937; doi: 10.1109/HICSS.2016.488.

4 Burkhard J, Tindal S. Smart Modular Package Units for Single-Use Processing. BioProcess Int. 17(3) 2019: 62–68; https://bioprocessintl.com/sponsored-content/smart-modular-package-units-for-single-use-processing-addressing-cost-speed-and-flexibility-challenges-in-biologics-manufacturing.

5 Hodge G. FlexFactory: The Economic and Strategic Value of Flexible Manufacturing Capacity. ISPE Strasbourg Conference, 28–29 September, 2009.

Acknowledgments
The writers thank the following reviewers for their careful consideration of this manuscript: Michel Claes (Siemens), Burkhard Joksch (Sartorius Stedim), Bruce Kane (Rockwell Automation), Uwe Kritzler (F. Hoffmann-La Roche), and Robin Payne (BioPhorum).

Chun Lai is an automation engineer at Merck. Steven Miller is director of software and automation at MilliporeSigma. Keith Morris is automation subject matter expert at the PM Group. Pietro Perrone is an automation process engineer at GE Healthcare. Rene Reinbigler is enterprise architect in the software and automation division of MilliporeSigma. And Gene Tung is executive director of manufacturing information technology at Merck. Please direct inquiries to Robin Payne of BioPhorum Operations Group, The Gridiron Building, 1 Pancras Square, London, N1C 4AG, [email protected].

The life science business of Merck KGaA, Darmstadt, Germany operates as MilliporeSigma in the United States and Canada.

Editorial Notes: BPI published this article first as an eBook in October 2019 and next as an article in the November–December 2019 print issue. Both editions referred erroneously to “NAMUR video desktop infrastructure media transfer protocol (VDI MTP) standards.” MTP should abbreviate “modular type package.”