Is MLOps different in Industry 4.0?

General and Speciﬁc Challenges

Leonhard Faubel, Klaus Schmid and Holger Eichelberger

University of Hildesheim, Germany

Keywords:

MLOps, Machine Learning, Industry 4.0, Challenges.

Abstract:

An important part of the Industry 4.0 vision is the use of machine learning (ML) techniques to create novel

capabilities and ﬂexibility in industrial production processes. Currently, there is a strong emphasis on MLOps

as an enabling collection of practices, techniques, and tools to integrate ML into industrial practice. However,

while MLOps is often discussed in the context of pure software systems, Industry 4.0 systems received much

less attention. So far, there is no specialized research for Industry 4.0 in this regard. In this position paper, we

discuss whether MLOps in Industry 4.0 leads to signiﬁcantly different challenges compared to typical Internet

systems. We identify both context-independent MLOps challenges (general challenges) as well as challenges

particular to Industry 4.0 (speciﬁc challenges) and conclude that MLOps works very similarly in Industry 4.0

systems to pure software systems. This indicates that existing tools and approaches are also mostly suited for

the Industry 4.0 context.

1 INTRODUCTION

Industry 4.0 is aimed at the next industrial evolution

in manufacturing, this time based on digital technolo-

gies. A core part of it is the use of machine learn-

ing (ML) to enable more intelligent, ﬂexible, and ef-

ﬁcient industrial processes. Scenarios like lot-size

one, predictive maintenance, or supply-chain opti-

mization can signiﬁcantly transform business models

in Industry 4.0 (Borgmeier et al., 2017). Currently,

Machine Learning Operations (MLOps) as a collec-

tion of methods, techniques, and tools for integrating

ML into software development practice is widely dis-

cussed as an enabler for large-scale ML applications,

however, this happens mostly in the context of pure

software systems.

As Industry 4.0 aims at applying ML to indus-

trial production, the need for MLOps in this con-

text is clear. Thus, an important question is whether

the speciﬁc context of Industry 4.0, i.e., complex,

large-scale Cyber-Physical Systems (CPS), changes

the challenges to the application of MLOps. The

aim of this paper is to discuss challenges of apply-

ing MLOps in an Industry 4.0 context. As a result

we identify challenges to MLOps that are speciﬁc to

the Industry 4.0 context or to speciﬁc scenarios within

Industry 4.0 (speciﬁc challenges) and challenges that

are roughly comparable to MLOps in other contexts

(general challenges). This serves as a basis for ﬁnding

solutions to address the identiﬁed novel challenges.

The next section introduces our understanding of

MLOps, which relies on existing models, but with an

adaptation to the Industry 4.0 context. Section 3 is the

core of the paper and presents the challenges we could

identify. We discuss these from a cross-sectional per-

spective in Section 4, while Section 5 concludes.

2 MLOps

MLOps aim to enhance the automation and quality

of intelligent systems (Meedeniya and Thennakoon,

2021). It combines principles from DevOps with ma-

chine learning. The ﬂexibility provided by DevOps

principles is beneﬁcial to machine learning (ML) as,

typically, several iterations need to occur to identify

well-working ML models and then adapt them over

time as the situation changes in the application.

The ﬁrst step in MLOps is always manual and is

performed in an analysis environment (Fig. 1). This

serves to ﬁrst understand the problem and possibili-

ties of ML. Based on the results of this step, a high-

level architecture in which MLOps operate must be

created. This takes the ML method, SE architecture,

hardware architecture, conﬁguration, and, if applica-

ble, even the architecture of the CPS into account and

Faubel, L., Schmid, K. and Eichelberger, H.

Is MLOps different in Industry 4.0? General and Speciﬁc Challenges.

DOI: 10.5220/0011589600003329

In Proceedings of the 3rd International Conference on Innovative Intelligent Industrial Production and Logistics (IN4PL 2022), pages 161-167

ISBN: 978-989-758-612-5; ISSN: 2184-9285

161

may even evolve them, if necessary. In particular, this

deﬁnes the deployment of the various MLOps compo-

nents and under which conditions they are triggered.

One input to this architecture deﬁnition is whether

automated retraining of models should be supported

(Step B). This is on the hand dependent on the out-

come of the model development and on the other it

depends on business decisions. Also the type of algo-

rithms that are identiﬁed in the model development

signiﬁcantly inﬂuence the high-level system archi-

tecture, e.g., neural networks have very different re-

source implications vs. random forest classiﬁcation.

Based on monitoring information, automated model

adaptations can be triggered (automated MLOps) or

it can be signaled that such adaptations may need to

be done manually, leading to a redeﬁnition of the ma-

chine learning approach. Finally, there is always a

model application stage (Step C), which will typi-

cally, but not always, be performed on edge devices.

The details of the architecture will have a strong in-

ﬂuence on the hardware resources as well as the ML

components used, as we will discuss in Section 3.

While Fig. 1 provides a high-level overview of

MLOps in Industry 4.0, we need a more detailed

MLOps model to deﬁne the individual challenges.

Various life-cycle models have been proposed for

MLOps (van der Goes, 2021). Two MLOps life cy-

cles are predominant in the literature. Symeonidis et

al. depict an MLOps life-cycle with three stages: ML,

development, and operations (G. Symeonidis et al.,

2022). Van der Goes describes a variant with four

stages (van der Goes, 2021). Here, the ML stage

is subdivided into data management and modeling.

Each stage consists of a cycle with tasks that connect

cycles to each other. These models do not address the

1. Manual

step

2. Autom./

MLOps

High

level

architecture

Deployed

model

3. Model

application

Monitoring

Feedback

Business

problem/

idea

Figure 1: High level structure of the MLOps cycle.

speciﬁc activities of MLOps, but make it clear that

ML is added to DevOps principles. Moreover, they

do not include cross-cutting activities. Therefore, we

propose a new life-cycle. Our life-cycle model de-

ﬁnes the phases (Fig. 2): Data Engineering, Model

Engineering, and Operations, each consisting of mul-

tiple activities; they are complemented by supporting

activities.

Data Engineering

Data Collection: This provides a basis for ma-

chine learning. The collected data can include

machine data, product information, customer

data, behavioral data, etc. The relevant data

needs to be determined based on the use case

and typically provides insights into the effec-

tiveness and quality of the production process.

Data Analysis: This step aims at understanding

of the data and its quality, e.g., identifying out-

liers. Typically, statistical methods are used.

Data Preparation: Transformations are applied,

e.g, for data cleaning or value imputation. Fea-

ture transformation can also be done here (Car-

doso Silva et al., 2020).

Model Engineering: In the structure given in Fig. 1

this can either be performed manually (especially

in the ﬁrst iteration) or in an automated way.

Model Building: Model building aims at creat-

ing the necessary machine learning models.

This includes the identiﬁcation of the relevant

approaches (e.g., neural networks vs. decision

trees), determining corresponding model struc-

tures, and potentially determining hyperparam-

eters.

Model Training: Candidate ML models are

trained and ﬁtted to the data.

Model Evaluation: ML models are evaluated on

test data (Sun et al., 2022).

Model Selection: The most appropriate (usually

the best best performing) ML model is se-

lected (or multiple models, if there are several

problems, which are addressed by ML tech-

niques). Potentially further ﬁne-tuning is per-

formed (Cardoso Silva et al., 2020).

Model Packaging: The ﬁnal ML models are

packaged as one or more application compo-

nents or as a ”model as a service” (Sato et al.,

2019).

Operations

CI/CD-testing: As part of continuous integration

and deployment, special tests for features and

data, for models, for ML infrastructure, and

IN4PL 2022 - 3rd International Conference on Innovative Intelligent Industrial Production and Logistics

162

High level architecture

Data Engineering

Model Engineering

Supporting activities

Versioning

Infrastructure

Tools

Automation

Operations

Data

collection

Data

analysis

Data

preparation

Model

building

Model

training

Model

evaluation

Model

selection

Model

packaging

Model

deployment

Monitoring

CI/CD

testing

Figure 2: MLOps activities.

monitoring for ML are run to ensure quality of

the deployed system.

Model Deployment: The production-ready ML

models are deployed as part of the production

system (Rahman and Kandogan, 2022).

Monitoring: The performance (quality) of the

ML models is continuously monitored in order

to determine whether a manual or automated

intervention is necessary (Rahman and Kando-

gan, 2022). Although ML methods are highly

appreciated in Industry 4.0 settings, according

to our experience, the application in production

settings is sometimes a bit conservative, i.e.,

automatically adjusting an existing or deploy-

ing a new version of a working ML model is

judged rather sceptically. Thus, at least the op-

tion to manually approve such an intervention

is often requested.

Supporting Activities

Infrastructure: The necessary infrastructure

components, including the relevant hardware,

required to develop, deploy, and run complex

ML systems must be selected, acquired,

installed and maintained. Often, this also

involves data integration activities, e.g., in

Industry 4.0 settings to obtain additional data

from ERP or MES systems.

Versioning: Versioning of code, data, and the

ML model as well as conﬁguration information

is needed (Oluyisola et al., 2022). Versioning

is often desirable to be able to go back to the

last working model to avoid or at least mini-

mize (factory) downtimes. Versioning is also

viewed as a ML safety mechanism, in particu-

lar if manual approvals of changed ML models

are demanded.

Automation: Various steps in the overall lifecy-

cle are often automated. This requires the im-

plementation of this steps, either using existing

automation capabilities or implementing them

in special ways.

Tools: Tools for developing of ML applications

are needed. This can include domain-speciﬁc

tools like domain-oriented modeling or simula-

tion (e.g., for factories or machines in an Indus-

try 4.0 scenario).

3 CHALLENGES

This section presents challenges encountered when

using MLOps in an Industry 4.0 environment. We

identiﬁed them based on 2.5 years of project experi-

ence as well as analysis of the involved tasks. The

challenges are organized based on the MLOps ac-

tivities model in Fig. 2. The focus of the discus-

Is MLOps different in Industry 4.0? General and Speciﬁc Challenges

163

sion is always: which activity does not cause addi-

tional difﬁculties and what additional difﬁculties ex-

ist in MLOps for Industry 4.0 over more traditional

MLOps scenarios. Of course, this may vary accord-

ing to project-speciﬁc requirements. The challenges

are divided into general and speciﬁc challenges. Gen-

eral challenges also exist in situations in other con-

texts, while speciﬁc challenges relate exclusively to

case-dependent speciﬁc situations in Industry 4.0 sce-

narios.

3.1 Data-related Tasks

Here, we describe the challenges in the data-related

MLOps tasks.

3.1.1 Data Collection

Depending on the use case speciﬁc substantial tech-

nical challenges exist here. Data acquisition requires

suitable sensors and data transmission in the factory

environment since data collection starts in the man-

ufacturing machine and ends in the software system.

Depending on the application, real-time requirements

exist and large amounts of data are transmitted or

stored.

If suitable technical conditions exist, i.e., ma-

chines with the appropriate equipment, in which

the sensors and the respective networking are ade-

quately dimensioned, the application of MLOps is

more straightforward, as one only has to think about

potential problems in storing the relevant amounts of

data.

If technical prerequisites are not met, data collec-

tion for MLOps becomes very difﬁcult as conversion

or adaptation of existing hardware and software may

involve a signiﬁcant effort, if possible at all. For ex-

ample, adding additional sensors to a machine may

require breaking guarantee seals or safety certiﬁca-

tions, which would render the machine unusable or

imply additional effort or costs. In the worst-case

scenario, whole machines, infrastructures, or manu-

facturing lines must be redeployed or exchanged.

3.1.2 Data Analysis

Typically, this is performed with a sample data set

outside the industry 4.0 environment. Thus, there is

no unusual complexity in this task.

3.1.3 Data Preparation

In data preparation, features are provided online for

inference and ofﬂine for experiments and training.

This is often implemented in the form of feature stores

to realize reproducibility and versioning using feature

and metadata registries (Kreuzberger et al., 2022).

Features usually describe characteristics of the pro-

duction process. The following speciﬁc challenges

arise in this regard. When data preparation is done

as a pre-processing step in model inference in pro-

duction real-time requirements typically apply. In this

case, the frequency and size of samples required for

further steps as well as the capabilities of existing

hardware inﬂuences how challenging this activity is.

With high real-time requirements, high-performance

hardware and software measures may be required in

the production environment. However, applicability

and availability of such hardware may be limited by

technical factory ﬂoor regulations, e.g., electrical or

mechanical norms, as well as by budget restrictions.

3.2 Model-related Tasks

Here, we describe the identiﬁed challenges in the

model-related MLOps activities.

Generally, if the model building, model training,

model evaluation and model selection activities are

part of an automated pipeline, there is a need for addi-

tional software, hardware, and infrastructure consid-

erations (see Section 3.4.1 below).

3.2.1 Model Building, Training, Evaluation, and

Selection

If these activities take place outside the Industry 4.0

environment, which is usually the case, there is no

impact on regular operation. However, there is a gen-

eral challenge in this respect. If the steps are per-

formed in an automated way within a factory envi-

ronment, e.g., as automated retraining, then additional

hardware resources and software infrastructures like

GPUs or ML implementations for embedded devices

are needed within the factory.

3.2.2 Model Packaging

In model packaging, the particular constraints of the

available infrastructure in the factory like hardware

resources (e.g., GPUs/TPUs/ FPGAs, special opera-

tion systems, and implementations, or available net-

work bandwidth and storage capacitites) must be

taken into account.

3.3 Operations-related Tasks

The following challenges address operations-related

tasks that relate to activities required for deployment

and during ongoing operations in the MLOps environ-

ment.

IN4PL 2022 - 3rd International Conference on Innovative Intelligent Industrial Production and Logistics

164

3.3.1 CI/CD Testing

As large parts of a CI/CD testing environment will

be carried out in a classical IT-environment, no chal-

lenges stem from this part. However, there is a gen-

eral challenge: the heterogeneity of hardware and

operating systems typical of Industry 4.0 can make

these tasks more complex than usual. Also, intelli-

gent methods require additional CI/CD tests. These

include tests for features and data, model develop-

ment, ML infrastructure, and monitoring tests for ML

serving, e.g., performance. A particular challenge is

that ML elements are typically analyzed on a statis-

tical basis, while traditional testing requires correct-

ness of each individual test case. Moreover, in au-

tomated adaptation scenarios even these tasks may

happen within the factory environment, making this

a rather complex task.

3.3.2 Model Deployment

MLOps activities can be performed on the factory

ﬂoor, in the cloud, or in corporate IT environments.

Speciﬁcally, in the case of the factory ﬂoor, edge re-

sources are needed, leading to the typical problems of

sufﬁcient and appropriate resources to ensure neces-

sary technical performance requirements.

3.3.3 Monitoring

In Industry 4.0, monitoring physical processes is an

integral part. Ideally, the existing monitoring solu-

tion is suitable for MLOps or can be extended easily.

If this is not the case, e.g., if physical separation of

groups or networks is required, a speciﬁc challenge

arises. Additional hardware requirements or develop-

ment efforts become necessary.

3.4 Support Activities

Support Activities are cross-cutting activities related

to infrastructure, tools, versioning and automation.

Here, we address challenges refering to support ac-

tivities.

3.4.1 Infrastructure

A major general challenge related to the infrastruc-

ture is the heterogeneity, which is even larger than for

MLOps in information systems. Some parts need to

run in an embedded context, some in corporate IT-

environments. In the case that model engineering ac-

tivities are also automated to some extend, there may

even exist the case that for the same steps multiple dif-

ferent infrastructures are needed (IT vs. embedded).

If frameworks like TensorFlow are used for ma-

chine learning in the manual environment, they are

typically not available as implementations on the fac-

tory ﬂoor. Rather this requires special frameworks

available for edge computing that may not bring the

same features, e.g., TensorFlow-light. The higher the

level of automation and the requirements for the prop-

erties associated with MLOps become, the more dif-

ﬁcult it is to deploy them in the environment of IoT

and edge devices.

Another general problem is the lack of widely and

homogeneously adopted standards; while there are

standards, many different standards are around and

applied in inhomogeneous fashion, partially also due

to (expensive) legacy machines or retroﬁtting of fac-

tory equipment.

This also leads to a lack of standardization of

tools, making tool selection in the context of Indus-

try 4.0 a major challenge.

The extent to which parts of the MLOps cycle

are automated varies signiﬁcantly among cases. Of

course, several steps, like deployment or productive

operation are usually automated. However, so far our

observation is that while many companies envision a

high degree of automation, and even are interested in

full automation of model adaptation, so far none, we

are aware of, implemented this degree of automation

in production.

We envision for the future that some degree of

automated adaptation will become standard practice

also in Industry 4.0. Nevertheless, difﬁcult ques-

tions about what may be changed independently in the

model and productive operation will remain. In par-

ticular, changes by self-adaptation may impact hard-

ware requirements and reliability.

MLOps emphasizes the management of inter-

dependencies among data, models, and code. Thus,

versioning this information is important. If these in-

formation should be available at edge nodes, e.g., as

feature stores, appropriate versioning infrastructure

must also be available there.

3.4.2 Tools

Here, we present a general challenge. Typically,

a more complex tool environment is required for

MLOps in Industry 4.0. This is due to the fact

that some steps will be done manually in an IT-

environment, but also corresponding tools in the em-

bedded environment are needed. Also tools that ad-

dress the speciﬁcs of the industry environment (e.g.,

cross-compilation) are required. Special tools like

simulation environments may also be needed in order

to study the impact of the ML solutions. In particular,

if they inﬂuence the factory behavior.

Is MLOps different in Industry 4.0? General and Speciﬁc Challenges

165

4 DISCUSSION

Typically, existing MLOps life-cycle models do not

describe that some activities can be either performed

manually or in an automated way at different points

in time. With our revised model of MLOps, we tried

to capture this.

In our view, an important part of MLOps in Indus-

try 4.0, which is typically not present in other MLOps

models, is the deﬁnition of a high-level architecture.

This is particularly relevant as a connection to soft-

ware engineering. Hence, we introduced this here.

The application of MLOps principles in an Indus-

try 4.0 context is not very special. Challenges exist

mainly for the reason that it is a heterogeneous envi-

ronment and many individual solutions are involved.

A full-scale architecture and implementation needs to

cover the whole environment of the cyber-physical

system or at least interface with relevant existing solu-

tions in this context. This is particularly challenging

due to severe resource constraints on the embedded

devices and the complexity that is introduced due to

the many different hardware platforms and operating

environment as well as due increasingly distributed

computing. If self-adaptation is introduced and cor-

responding model engineering happens on edge de-

vices, the complexity of the environment becomes

even more severe.

The majority of MLOps activities are of similar

complexity in an Industry 4.0 case as in information

systems. The main reason for this is that they are

typically performed outside the factory infrastructure,

especially, if they are not automated. This applies

to ”Data analysis” and the majority of model-related

tasks.

Challenges arise in other tasks under certain con-

ditions. It is highly case-dependent whether MLOps

is difﬁcult to implement in Industry 4.0. Also the

starting-level of the technical infrastructure as well

as the aimed at degree of automation inﬂuence very

signiﬁcantly the overall complexity of implementing

MLOps in Industry 4.0. For example, in data-related

tasks, technical challenges arise when already exist-

ing devices are not ready for data collection for the

speciﬁc use case of an ML model or for its execu-

tion. Operations are problematic when special imple-

mentations are required or no sufﬁcient resources are

available.

Today, some guidance, best practices, frame-

works, and platforms already exist to facilitate

MLOps in IoT environments (and thus Industry 4.0):

Ruf et al. discuss a selection of beneﬁts using MLOps

in industrial scenarios (Ruf et al., 2022). A digital

twin architecture with MLOps techniques is proposed

by Fujii et al. (Fujii et al., 2022). An MLOps frame-

work for automated ML at the edge is described by

Raj et al. (Raj, E et al., 2021). None of these takes

a broad view of problems in MLOps in Industry 4.0

as we do here. Hence, not only our analysis of chal-

lenges, but also the speciﬁc MLOps model can be re-

garded as a contribution of our work.

5 CONCLUSION

MLOps is an important set of practices and activities,

which are key to the implementation of modern ML-

based software solutions. In this paper, we discussed

challenges from the perspective of MLOps in Indus-

try 4.0 and discuss how they differ from MLOps chal-

lenges in other contexts.

Overall, we conclude that most Industry 4.0

MLOps challenges exist in a similar manner in the

more traditional software engineering context. Some

additional challenges exist, at least in some applica-

tion scenarios. In particular, we could identify signif-

icant (speciﬁc) challenges for four activities. Most

of the challenges are not unique to Industry 4.0, a

positive indicator for using existing technologies and

practices in this context as well. We plan to study

these and the corresponding ways to address them in

more detail in the future. For this purpose, we will

conduct industry studies around MLOps and apply

what we have learned to platforms and frameworks

that implement MLOps for Industry 4.0.

ACKNOWLEDGMENTS

This work was partially supported by HAISEM-Lab

(grant 01—S19063B) and EXPLAIN (01—S22030E)

funded by the Federal Ministry of Education and IIP-

Ecosphere funded by the German Ministry of Eco-

nomics and Climate Action (BMWK) under grant

number 01MK20006C.

REFERENCES

Borgmeier, A., Grohmann, A., and Gross, S. F.

(2017). Smart Services und Internet der Dinge:

Gesch

aftsmodelle, Umsetzung und Best Practices: In-

dustrie 4.0, Internet of Things (IoT), Machine-to-

Machine, Big Data, Augmented Reality Technologie.

Carl Hanser.

Cardoso Silva, L., Rezende Zagatti, F., Silva Sette, B.,

Nildaimon dos Santos Silva, L., Lucr

edio, D., Fur-

tado Silva, D., and de Medeiros Caseli, H. (2020).

IN4PL 2022 - 3rd International Conference on Innovative Intelligent Industrial Production and Logistics

166

Benchmarking machine learning solutions in produc-

tion. In International Conference on Machine Learn-

ing and Applications, pages 626–633.

Fujii, T. Y., Hayashi, V. T., Arakaki, R., Ruggiero, W. V.,

Bulla, R., Hayashi, F. H., and Khalil, K. A. (2022).

A Digital Twin Architecture Model Applied with

MLOps Techniques to Improve Short-Term Energy

Consumption Prediction. Machines, pages 455–462.

G. Symeonidis, E. Nerantzis, A. Kazakis, and G. A. Pa-

pakostas (2022). MLOps - Deﬁnitions, Tools and

Challenges. In Annual Computing and Communica-

tion Workshop and Conference, pages 453–460.

Kreuzberger, D., K

uhl, N., and Hirschl, S. (2022). Machine

Learning Operations (MLOps): Overview, Deﬁnition,

and Architecture. arXiv:2205.02302 [cs].

Meedeniya, D. and Thennakoon, H. (2021). Impact factors

and best practices to improve effort estimation strate-

gies and practices in DevOps. In International Con-

ference on Information Communication and Manage-

ment, pages 11–17.

Oluyisola, O. E., Bhalla, S., Sgarbossa, F., and Strandha-

gen, J. O. (2022). Designing and developing smart

production planning and control systems in the indus-

try 4.0 era: A methodology and case study. Journal of

Intelligent Manufacturing, pages 311–332.

Rahman, S. and Kandogan, E. (2022). Characterizing prac-

tices, limitations, and opportunities related to text in-

formation extraction workﬂows: A human-in-the-loop

perspective. In CHI Conference on Human Factors in

Computing Systems.

Raj, E, Buffoni, D, Westerlund, M, and Ahola, K (2021).

Edge MLOps: An Automation Framework for AIoT

Applications. In International Conference on Cloud

Engineering, pages 191–200.

Ruf, P., Reich, C., and Ould-Abdeslam, D. (2022). Aspects

of Module Placement in Machine Learning Opera-

tions for Cyber Physical Systems. In Mediterranean

Conference on Embedded Computing, pages 1–6.

Sato, D., Wider, A., and Windheuser, C. (2019). Continuous

delivery for machine learning. visited 2022-06-18.

Sun, Z., Sandoval, L., Crystal-Ornelas, R., Mousavi, S. M.,

Wang, J., Lin, C., Cristea, N., Tong, D., Carande,

W. H., Ma, X., Rao, Y., Bednar, J. A., Tan, A., Wang,

J., Purushotham, S., Gill, T. E., Chastang, J., Howard,

D., Holt, B., Gangodagamage, C., Zhao, P., Rivas, P.,

Chester, Z., Orduz, J., and John, A. (2022). A review

of earth artiﬁcial intelligence. Computational Geo-

science.

van der Goes, M. (2021). Scaling Enterprise Recommender

Systems for Decentralization. In Conference on Rec-

ommender Systems, pages 592–594.

Is MLOps different in Industry 4.0? General and Speciﬁc Challenges

167