52 ISE Magazine | www.iise.org/ISEmagazine
Inside IISE Journals
research
Should one go deep when using
Gaussian process models, and how?
Gaussian process (GP) models are popular in several major
applications, such as in machine learning, computer experi-
ments and spatial statistics. In many applications involving a
GP model, stationarity is assumed for making its covariance
function simple and easy to handle. But this assumption
may not be appropriate in practice; for example, functions
may vary more rapidly in some regions of the input space
than in other regions.
This problem can be solved using a deep Gaussian pro-
cess (DGP), which is formed by stacking multiple layers of
latent variables, where the mappings between the layers are
governed by GPs. DGPs can model nonstationary response
surfaces with a much greater flexibility via successive warp-
ing of latent variables through multiple layers of GPs.
An engineering application of DGPs to integrate physi-
cal and computer experiments with nonstationary relation-
ships was investigated in the paper, “Deep Gaussian Pro-
cess Models for Integrating Multifidelity Experiments with
Nonstationary Relationships,” by Jongwoo Ko, a Ph.D.
student at Korea Advanced Institute of Science and Tech-
nology (KAIST), and Heeyoung Kim, an associate profes-
sor at KAIST. Computer experiments are widely used in
many science and engineering fields to understand com-
plex real systems in which a larger amount of computer
experimental data can be easily collected, as compared to
relatively scarce physical experimental data. But there are
inevitable discrepancies between the outputs from the two
sources. The problem of integrating the data from the two
sources for more effective analysis has been extensively ad-
dressed in the literature using a linear model that performs
location and scale adjustments. So far, the treatment has a
catch, which is the assumption of stationary on the location
and scale adjustment parameters.
To allow greater flexibility in solving this broad class of
data integration problems, the authors develop DGP mod-
els that can handle nonstationary relationships between
the two data sources. More than this benefit, the proposed
DGP model is also highly interpretable as a linear model,
while providing high-expressive power using the deep ar-
chitecture. For the inference of the proposed DGP model,
the authors derived a doubly stochastic variational inference
algorithm, which allows scalability to large datasets. Several
numerical examples with complex relationships between
the two sources are used for demonstrating the benefit of
going deep when using GP models.
This month, we highlight two articles from the July 2022 issue of IISE Transactions (Volume 54, No. 7). Both papers are
about Gaussian process models, a method widely used in both engineering statistics and machine learning applications.
The first article addresses the issue of whether there is enough benet of going deep in creating Gaussian process
models, namely stacking multiple layers of latent variables where the mappings between the layers are governed by
Gaussian processes. The authors argue that a major benet in doing so is the capability and flexibility for a deep Gaussian
process model to handle nonstationary relationship in a dataset. They demonstrated this benefit in the development of
calibration models connecting computer and physical experiment responses. The second article looks into the issue of
feature selection using Gaussian process models. The vanilla version already embodies a feature relevance determination
mechanism using the bandwidth parameters associated with each input variable. The authors argue that using the
default mechanism is not sufcient and instead proposed a new formulation that incorporates explicitly and forcefully the
relevance of the input variables as auxiliary model parameters to the Gaussian process learning framework.
.
Jongwoo Ko Heeyoung Kim
June 2022 | ISE Magazine 53
CONTACT: Heeyoung Kim; heeyoungkim@kaist.ac.kr; Department of
Industrial and Systems Engineering, KAIST, 291 Daehak-ro, Yuseong-gu,
Daejeon 34141, Republic of Korea
How to stay focused on relevant features
When constructing machine learning models, one may be
easily tempted to include as many exploratory variables as
possible. That is because we typically do not know which
variables are important in the modeling stage. We also do
not want to take the risk of leaving out potentially relevant
variables.
However, taking the most inclusive approach would cre-
ate overcomplicated models, generating many potential is-
sues like increasing the risk of model overfitting, poor data
visualization and lack of model interpretability. It has been
long desired to limit the number of exploratory variables
by eliminating irrelevant or redundant variables. The main
question is how to determine the relevance of the explor-
atory variables and stay “focused” on only relevant vari-
ables?
In their paper, “Variable Selection for Gaussian Process
Regression Through a Sparse Projection,” professor Chi-
woo Park from Florida State University, David Borth for
University of Dayton Research Institute, and Nicholas
Wilson and Chad Hunter from Air Force Research Labora-
tory formulate an automatic relevance determination prob-
lem for a popular class of machine learning tasks under the
framework of Gaussian process regression. The new formu-
lation incorporates the relevance of the exploratory vari-
ables as auxiliary model parameters to the Gaussian process
learning framework.
The automatic relevance determination is carried out
through a joint optimization of these auxiliary relevance
parameters and the other parameters of Gaussian process
models. The new formulation also presents a computational
challenge where the existing techniques are either ineffec-
tive or too slow in solving. The authors further develop a
forward stagewise-type algorithm with convergence guar-
antee and demonstrate that the new algorithm has both se-
lection accuracy and computational efficiency superior to
the existing solution approaches.
The authors apply the
new method to identifying
important design factors in
the process of developing
real-world accelerated test
protocols for assessing en-
vironmental degradation
of aerospace alloys. Many
environmental factors such
as temperature, relative hu-
midity, salt, corrosion gases
and sunlight may affect metal corrosion of civil and defense
structures. Preselecting a few influential factors is highly
desirable for an efficient design of the accelerated corrosion
test protocol.
The proposed method successfully identified 10 influ-
ential factors out of 27 environmental candidates. Using
the smaller set of influential factors can reduce the required
number of accelerated testing experiments and thus lower
the test costs. The authors also expect their method to find
broad applications in engineering experiments and uncer-
tainty quantification, considering the popular use of Gauss-
ian process regression models in these fields.
CONTACT: Chiwoo Park, cpark5@fsu.edu; Department of Industrial and
Manufacturing Engineering, Florida State University, 2525 Pottsdamer
St., Tallahassee, FL 32310-6046
Effect of exoskeleton tested on back loading
and metabolic effort during repetitive lifting
Commercial exoskeletons for the lower back are designed
to reduce the risk of developing work-related musculoskel-
etal disorders (WMSDs). To examine their potential ben-
efit, a team recently investigated the benefits of a newly
introduced exoskeleton, Paexo Back by Ottobock SE & Co.
of Germany, designed to reduce low-back loading during
lifting tasks and to minimize interference with normal lo-
comotion.
The team included Thomas Schmalz, Malte Bellmann,
Samuel Reimer and Michael Ernst from Ottobock, along
with colleagues Anja Colienne from the Private University
of Applied Science in Göttingen, Germany; Emily Bywa-
ter of Purdue University, and Lars Fritzsche and Christian
Gärtner of IMK Automotive in Chemnitz, Germany.
To replicate a typical workplace situation, the team had
10 healthy subjects perform a repetitive lifting task with
and without the exoskeleton. Oxygen uptake and heart rate
This month we highlight two articles from IISE
Transactions on Occupational Ergonomics and
Human Factors (Volume 10, No. 1). In the first, a
team led by Thomas Schmalz found that a newly
developed exoskeleton design to reduce back
loading during lifting could reduce low-back
loading and metabolic demands by approximately
20% and 10%, respectively. In the second paper,
Kaitlin Gallagher and colleagues examined the
effects of sustained neck flexion when using
a smartphone, finding that that even less than
30 minutes of use could lead to many people
developing neck or upper-back pain.
Chiwoo Park
54 ISE Magazine | www.iise.org/ISEmagazine
research
were measured using a wireless spiroergometry system, and
activation levels of back, abdominal and thigh muscles were
measured using a wireless electromyographic system. Kine-
matic data were recorded using an optoelectronic device,
and ground reaction forces were measured with two force
plates. Joint compression forces in the lower spine were esti-
mated for the lifting portion of the task using the AnyBody
Modeling System.
As reported in their paper, “A Passive Back-Support
Exoskeleton for Manual Materials Handling: Reduction of
Low Back Loading and Metabolic Effort During Repeti-
tive Lifting,” they found that using the exoskeleton signi-
cantly reduced oxygen consumption rate by 10%, activa-
tion of back and thigh muscles by up to 14% and peak and
mean compression forces acting on the lower spine by 20%.
Overall, these results indicate the exoskeleton investigated
can be an effective device for reducing physical demands
during a symmetric lifting task. It may also be beneficial
in a variety of similar lifting tasks, and use of the exoskel-
eton could contribute to preventing the development and/
or progression of WMSDs.
CONTACT: Thomas Schmalz, Ph.D., Ottobock SE & Co. KGaA, Depart-
ment Clinical Research & Services, Göttingen, Germany
Can smartphone use lead to neck and back
pain? Study focuses on risk to workers
Ninety-six percent of Americans ages 18 to 29 say they
own a smartphone for work or leisure. Many people look
down while using their mobile devices and the resulting
flexed neck posture is linked with reports of neck pain.
Mobile computer use is also associated with neck pain de-
velopment. Since mobile technology permits an increased
connection at home and other nontraditional workplaces,
employers need to understand how device usage can impact
worker pain and discomfort.
A collaborative project between Kaitlin Gallagher, Ash-
ton Human and Caleb Burruss of the University of Arkan-
sas Exercise Science
program, along with
Jon Jefferson from
the physical therapy
program at the Uni-
versity of Arkansas
for Medical Sciences
Northwest Cam-
pus, resulted in the
paper titled “Acute
Pain, Neck Exten-
sor Endurance and
Kinematic Changes
Resulting from Sus-
tained Neck Flexion
During Smartphone
Use.”
In this study, 40
participants ages 18
to 29 with no pre-
vious neck or upper
back injuries sat for
30 minutes while
In the researchers’ study, 40
participants ages 18 to 29 used
a smartphone for 30 minutes to
determine the effect of such posture
on neck and upper back injuries.
Malte Bellmann
Samuel Reimer Michael Ernst
Anja Colienne Emily Bywater
Lars Fritzsche Christian Gärtner
Thomas Schmalz
June 2022 | ISE Magazine 55
IIISE Transactions (link.iise.org/iisetransactions) is IISE’s flagship
research journal and is published monthly. It aims to foster
exchange among researchers and practitioners in the industrial
engineering community by publishing papers that are grounded
in science and mathematics and motivated by engineering
applications.
IISE Transactions on Occupational Ergonomics and Human Factors
(link.iise.org/iisetransactions_ergonomics) is devoted to compiling and
disseminating knowledge on occupational ergonomics and human
factors theory, technology, application and practice across diverse
areas. You can follow on Twitter at twitter.com/iisetoehf or @iisetoehf.
To subscribe, call (800) 494-0460 or (770) 449-0460.
About the journals
using a smartphone in their lap and performing controlled
tasks. This conguration resulted in participants looking
down with a flexed neck to use their phones. Motion cap-
ture was used to track neck posture, and neck muscle en-
durance was tested before and after the 30-minute periods.
Participants were divided into those who reported neck and
upper back pain during smartphone use and those who did
not. Participants came to the lab twice to ensure their pain
reporting was consistent.
Half of their sample, 20 participants, reported consistent
neck or upper back pain symptoms, typically described as
aching.” Those who developed pain had lower endurance
after smartphone use in the muscles required to hold their
neck flexed to view the phone. This study emphasizes that
employers should communicate with their employees who
depend on mobile technology for work that neck pain can
develop in less than 30 minutes even without a previous in-
jury history. Researchers and practitioners should develop
ways to help workers mitigate this pain.
CONTACT: Kaitlin M. Gallagher, University of Arkansas
Yu Ding is the Mike and Sugar Barnes Professor of Industrial
and Systems Engineering at Texas A&M University and Associate
Director for Research Engagement at the Texas A&M Institute of
Data Science. He is editor-in-chief of IISE Transactions and a
fellow of IISE.
Maury Nussbaum is HG Prillaman professor at Virginia Tech in
the Department of Industrial and Systems Engineering, editor-in-
chief of the IISE Transactions on Occupational Ergonomics
and Human Factors, and a fellow of IISE.
Share & Like IISE
Follow IISE pages, like posts and share them with
your network of contacts on social media. Watch
and enjoy IISE videos at www.youtube.com/c/
iisechannel. Go to www.iise.org/social for details on
IISE’s social networks. If youre a member of IISE,
chat with fellow members on IISE Connect (connect.
iise.org) and its communities specific to your area of
interest.
On-campus, online
and in Manhattan
binghamton.edu/ssie
Systems Science and
Industrial Engineering at
Binghamton University
WATSON COLLEGE
Systems Science and Industrial Engineering
4
SUNY Distinguished
Professors
26
tenured/tenure-
track faculty
9
SUNY Chancellors
Award recipients
$10
million
in research
expenditures
in 2020–21
Degree
programs
in Industrial and
Systems Engineering
(BS, MS, PhD), Systems
Science (MS, PhD) and
Healthcare Systems
Engineering (MS)
280
undergraduates
210
master’s
180
doctoral students