46 ISE Magazine | www.iise.org/ISEmagazine

Financial markets are inherently chaotic. For more than a century, the fren-

zy took physical form as orders were shouted from the trading pits of the

major exchanges. Transactions were completed with hand signals that al-

lowed for rapid-ﬁre deal-making in a process where speed was everything.

Today, the majority of trades are negotiated from behind a computer

screen in a quiet ofﬁce building, often without human intervention. None-

theless, what’s happening remains as fast-paced as it is unpredictable.

A good buy can still turn into a loser in a matter of moments. If the system were

fully predictable in a way that could be replicated, an algorithm could guarantee the

success of every transaction. Not only would this magic algorithm achieve “alpha”

by beating the market in the long and short terms, losing would also be impossible.

Such an algorithm would ensure high returns with zero risk, which is the sort of a

claim that can only be made by charlatans or the extremely lucky.

Contrast ﬁnancial markets’ ups and downs with the predictability of a factory,

where items are manufactured in an orderly, step-by-step process. Factories are con-

trolled environments speciﬁcally designed to be free from randomness and variation

to the greatest extent possible. Luck is not a factor in a controlled environment.

Finance is anything but controlled, though its complexity is traditionally tamed

with statistics. One can measure the various performance characteristics of a set of

companies, properties or commodities. By comparing ﬁgures from a set of compa-

nies, an investor can spot the outliers, ﬁrms that, by the numbers, should be per-

forming better on the market and might represent a bargain. This works because a

Using AI

to ﬁnd ﬁnancial

opportunity in chaos

Intelligence technology may offer a model for dealing

with markets’ complexity and uncertainty

By Joseph Byrum

August 2019 | ISE Magazine 47

48 ISE Magazine | www.iise.org/ISEmagazine

Using AI to find financial opportunity in chaos

set of common criteria can be applied to compare across

wholly different ﬁelds. To take a simple example, you can

compare the earnings per share of a media company like

Disney against that of an airplane manufacturer like Boe-

ing and decide that one makes for a better investment than

the other.

Despite today’s highly sophisticated statistical formulas

and automated trading algorithms, luck continues to play

a major role in how that investment plays out (Michael L.

Hartzmark, “Luck Versus Forecast Ability: Determinants

of Trader Performance in Futures Markets” The Journal of

Business, January 1991).

What AI does differently

Artiﬁcial intelligence (AI) has the potential to change ev-

erything by minimizing the impact of luck. AI brings caus-

al analysis to the table, which when implemented properly

has the potential to replace luck with insight, at least to

some measurable degree.

To do this right, the AI system must go beyond simple

statistical analysis to fashion a more complete picture of

business performance that takes into account the inherent

market complexity and individual characteristics of each

particular company of interest. It would build models and

conduct simulations far more complex and complete than

has ever been possible.

This approach borrows from the philosophy of Wilhelm

Windelband, the 19th century follower of Immanuel Kant,

who laid out two ways of describing reality. The ﬁrst, the

“nomothetic,” seeks to ﬁnd what different items have in

common so that they can be analyzed and compared. This

technique relies upon generalization, which naturally lends

itself to statistical analysis. The second method, the “idio-

graphic” or “idiosyncratic,” examines each item as an indi-

vidual within a speciﬁc context. Instead of statistics, idio-

syncratic analysis has relied on pen and paper. Until now.

Mathematics is the ideal example of a nomothetic dis-

cipline. Numbers are abstractions, generalizations that are

everywhere and always the same. Two plus two is always

equal to four. When we need to apply mathematics to the

real world, we can use modern statistical theory, which ac-

counts for what appears to be random variation through

the concept of population. All members of a population

share a set of general properties; the larger the population,

the more exact the estimates that statistics can make of the

population’s properties.

Population models and their generalizations are less use-

ful in describing things that are inherently unique. For

instance, statistics and population models are less accurate

in discussing history. Generalizations can only provide so

much insight into the events of a given day or a given year,

as these are populations of one. A full understanding of

what happened on an important date can’t be reduced to

statistics or numbers.

Mathematical abstractions also can fall short when ap-

plied to messy, chaotic circumstances. Say you’ve decided

to grow corn crops on a 1-acre ﬁeld. You plant seeds, which

are known to have an average yield of 176 bushels per acre.

Do you end up with 176 bushels of corn at harvest time?

Maybe. It depends on the weather, how hungry the in-

sects happen to be, whether there are any disease outbreaks,

the prevalence of weeds, soil quality levels and hundreds of

other individual factors that apply not just from one ﬁeld to

the next, but from one part of one ﬁeld to another part of

the same ﬁeld. Simple statistical analysis is not the best way

to deliver results.

So it is in ﬁnance, where the urge to generalize business

performance is strong and is often reduced to a set of basic

A blast of success from the past

The top 25 companies listed in the ﬁrst Fortune 500 list of top

performers from 1955. Of these, ﬁve appeared on the 2019 list

(Exxon Mobil at No. 2, Chevron at No. 11, General Motors at No.

13, AT&T Technologies at No. 15 and General Electric at No. 21).

See the full list at link.iise.org/1955Fortune500.

1. General Motors

2. Exxon Mobil

3. U.S. Steel

4. General Electric

5. Esmark

6. Chrysler

7. Armour

8. Gulf Oil

9. Mobil

10. DuPont

11. Amoco

12. Bethlehem Steel

13. CBS

14. Texaco

15. AT&T Technologies

16. Shell Oil

17. Kraft

18. ChevronTexaco

19. Goodyear Tire & Rubber

20. Boeing

21. Sinclair Oil

22. Navistar International

23. RCA

24. Union Carbide

25. Firestone Tire & Rubber

August 2019 | ISE Magazine 49

statistical calculations. While appropriate statistical models

can offer a great deal of insight that can only go so far. As

the standard ﬁnancial disclaimer goes, past performance is

no guarantee of future results. What AI can do differently

is create simulations based on a population of one to pro-

vide information on present and future performance under

present and expected conditions, rather than just through a

statistical analysis that extrapolates from past performance.

Markets are not the same from one day to the next, from

one year to the next or from one decade to the next. Half

of the companies on the S&P 500 are expected to be re-

placed within the next 10 years, (Scott D. Anthony, S. Pat-

rick Viguerie and Andrew Waldeck, “Corporate Longev-

ity: Turbulence Ahead for Large Organizations,” Innosight).

Fortune published its ﬁrst-ever list of the top 500 companies

in the country in 1955 (see a list on page 48). Since then,

most of the list’s most storied names have found themselves

absorbed by other corporations or have dropped off the list

entirely. Only 60 of the original companies remain today.

Companies must change to survive. Eastman Kodak, No.

43 on Fortune’s ﬁrst list, has tumbled in the ranks as photo-

chemical ﬁlm grew irrelevant in a digital age. The company

was ofﬁcially booted from the list in 2013, tumbling to a

ranking of 966 by 2015. Idiosyncratic models are better able

to capture the day-to-day changes that are ignored when

generalizing.

The AI advantage: Causal analysis

Generalized population models force assumptions about

individual companies. Apple and Microsoft are both high-

tech industry giants that sell hardware, software and cloud

services, but they are fundamentally different. They have

different outlooks that appeal to different consumer seg-

ments. Instead of treating them as the same, AI can embrace

what makes companies like these unique by using an idio-

syncratic model.

Idiosyncratic models dig in deeper to account for a larger

set of observable behaviors. Think of it as detective work.

You can try to solve a murder by reading the latest crime

statistics, which will tell you the hot spots where felonies

are most often committed. You can ﬁnd the other com-

mon elements of each crime: the time of day, the number

of accomplices and even the clearance rate (how likely it is

that the crime will be solved). These facts tell you what hap-

pened. Yet you still have no idea who the culprit is or why

the crime was committed.

In the classic TV drama formula, the alternative is to

seek the motive, means and opportunity. Round up ev-

eryone who was in the neighborhood at the time of the

crime (opportunity), then investigate each one and deter-

mine whether they had both an incentive to commit the

crime (motive) and the ability to do so (means). That is to

say, you need to conduct an idiosyncratic analysis to zero

in on the answer that tells you why something happened.

This is fundamentally different from what you get from the

nomothetic and statistical approach.

By creating an artiﬁcial population of several unique

entities, the idiosyncratic model can simulate hypothetical

causal relationships among the entities. This can be used to

identify the causes of observed events and produce a theory

that can be tested and veriﬁed.

For instance, Colonel Mustard was in the billiard room

with a revolver when the crime was committed and he

hated the victim. You can lay out the motive, means and

opportunity. At the same time, the system has conﬁrmed

the other characters’ alibis. Colonel Mustard becomes the

prime suspect through a process of elimination.

Unlike the sort of analysis provided by artiﬁcial intel-

ligence based on mysterious deep learning models, knowl-

edge-based idiosyncratic models can explain their predic-

tions. There is no guesswork involved. The idiosyncratic

model builds a case so that an action can be taken and the

subsequent results evaluated. It creates a paper trail based

on causality – a trail that can be followed and, more impor-

tantly, veriﬁed.

Idiosyncratic models are likely to be the next frontier of

ﬁnancial analysis. It would be foolish not to develop the

competitive edge possible from capturing the “why” of in-

dividual company performance, or individual commodity

values or any other ﬁnancial instrument. AI offers better

information that can increase the accuracy of analysis as

well as the efﬁcacy of ﬁnancial decisions. Such a system

will cut through the chaos with a fact-based assessment of

likely future performance based on causal factors for each

individual company, commodity, property or other ﬁnan-

cial instrument. Winning in a market where such systems

exist would depend more on the skill in the application of

this insight than on luck.

Considering the massive advantage that such a system of-

fers, it would only be a matter of time before having this

level of insight is a prerequisite for anyone hoping to suc-

ceed in the ﬁnancial world. 

Joseph Byrum is the chief data scientist at Principal and an IISE

member. He was previously senior R&D and strategic marketing

executive in life sciences-global product development, innovation

and delivery at Syngenta. In that role, he was chief architect of

initiatives that won Syngenta the 2016 ANA Genius Award in

Marketing Analytics and 2015 Franz Edelman prize for contribu-

tions in operations research and the management sciences. He holds

a bachelor’s degree in crop and soil science and a master’s in genetics

from Michigan State University, an MBA from the University of

Michigan Stephen M. Ross School of Business and a doctorate in

quantitative genetics from Iowa State University.