When Businesses Should Calculate Vs Judge
The Importance Of Understanding Risk and Uncertainty
Note: This is part 2 of understanding the economics behind Growth Isn’t One Sided.
One Takeaway: Growth Isn’t One Sided established that managers and entrepreneurs think differently. But how do you know when a problem requires calculation versus judgment? Luckily, economists have done a lot to help provide some ideas on this. Their research and ideas can explain why many companies trying to scale can get this balance wrong.
The Big Picture. Businesses can benefit both managerial thinking (optimization) and entrepreneurial thinking (innovation). Companies that excel at both outperform those that choose one or the other.
This point raises critical questions for companies trying to scale.
How do you know when to calculate versus when to judge?
Why do organizations default to one approach even when it’s failing?
How can you build systems that apply the right approach to the right challenges?
Economists have spent decades studying the fundamental difference between risk and uncertainty. Their ideas can help explain why managers and entrepreneurs operate from different directions. But they also can help us understand when each approach may work best. And why mixing them up destroys both efficiency and innovation.
The Two Approaches to the Same Challenge
Consider how Netflix approached two growth problems that required completely different methods. One growth challenge was how to get their current users to watch more programs. The other growth problem was how to gain new users in new regions. To increase current user usage, they turned to recommendation algorithms. To gain new users they had to take a different approach.
For recommendation algorithms, Netflix could use calculation. They had millions of user ratings. They had data on viewing patterns and completion rates. They ran A/B test results to get reliable signals about what content to suggest. More data usually improved the system. They optimized piece by piece. They used statistical models to predict what users would enjoy. They based their models on viewing history and similarity to other users. They also refined these models and made updates to make them better over time.
For international market entry, Netflix had to use judgment instead of calculation. Would Korean audiences respond like American audiences? How would local competitors react? What content would work in markets with different cultural contexts? What regulatory challenges would emerge? No amount of US data answered these questions. They had to experiment, adapt, and learn through actual market presence.
Same company, same leadership, same commitment to being data-driven. Yet each action took different approaches. Why?
Because the information needed for each problem was different. Understanding this separates companies that grow effectively from those that optimize into irrelevance.
Two Different Decision-Making Environments
Frank Knight and other economists have explained the difference between risk and uncertainty. Their work boils down to the fact that our actions, including all our business decisions, fall into one of two categories: risk or uncertainty
Risk-based problems have measurable probabilities and known variables. You can calculate optimal inventory levels based on things like demand patterns, lead times, and storage costs. You can optimize pricing because you have data on competitor prices and customer price sensitivity. Supply chain efficiency. Operational processes. Resource allocation. These questions and decisions fit within established systems. They generally all involve stable relationships that analytical approaches can identify and improve.
Think of operating under risk like a well-shuffled deck of cards. You don’t know which card comes next, but you know the probability distribution exactly.
Uncertainty-based problems involve unknown possibilities and unmeasurable variables. You can’t calculate whether customers will want a product that doesn’t exist yet. You can’t optimize for market segments you haven’t discovered. You can’t measure the success of innovations before they’re tested. New market entry. Competitive differentiation. Product development. These all involve changing relationships where historical data provides limited guidance.
Think of operating under uncertainty like taking a card from a well shuffled deck of cards. The problem is you don’t know how many cards there are. You don’t know whether the cards are playing cards, Pokemon cards, or business cards. There’s no probability to calculate. The outcomes themselves are being discovered through your actions.
Why This Difference Is About the Nature of Knowledge, Not More Information
The distinction between risk and uncertainty isn’t about how much data you have. It’s about whether probability theory can even apply to your situation.
When you’re optimizing inventory, more data helps. Demand patterns tend to be a product of stable underlying relationships. Customer behavior, seasonal patterns, and supply chain dynamics create variation. Because of this, statistical analysis can capture and leverage.
When you’re deciding whether to enter a new market or develop a new product, more historical data doesn’t help in the same way. You’re operating in new territory. You can’t predict the future through analysis of past patterns. The relevant relationships don’t yet exist.
No amount of data about horse-drawn carriage demand would have told you the market size for automobiles. No analysis of letter-writing patterns would have predicted email adoption curves. No study of video rental behavior fully predicted streaming service success.
This is why even sophisticated companies with massive data capabilities still need judgment. The limitation isn’t technological or informational. It’s fundamental to the nature of creating new knowledge and markets. When you’re creating new product categories you’re actively bringing new realities into existence. These don’t have probability distributions yet.
Many economists (such as Frank Knight, Israel Kirzner, Armen Alchian, Peter Klein, and Nicolai Foss) emphasize this point. Entrepreneurship is more than better decision-making under uncertainty. It’s a completely different activity than optimization under risk. Entrepreneurs do more than divide resources efficiently. They take action and discover new possibilities that weren’t knowable in advance.
Most businesses need both types of decision-making, but in different rates depending on the situation.
Why Organizations Default to Calculation (And Why This Kills Growth)
Most companies try to apply analytical approaches to everything. They do this because calculation feels more professional and defensible. (Also, people HATE uncertainty). Data-driven decisions sound smarter than judgment-based decisions. Spreadsheet projections look more serious than experimental budgets. Optimization metrics appear more scientific than adaptation strategies.
There are good reasons for this bias. Calculation is easier to justify to boards and investors. It’s easier to scale through organizational systems. It’s also easier to “run the numbers.” Business schools teach analytical frameworks. Consulting firms sell optimization methodologies. Performance reviews reward measurable outcomes.
But this preference for calculation creates hidden costs that compound as organizations grow.
1. Missed opportunities accumulate. The opportunities that demand judgment get overlooked. This happens because you can’t measure them with existing tools. Economists call this opportunity cost; the value of the road not taken. Using analytical resources on problems that don’t have calculable answers leads to waste. It also leads to missing out on the judgment-based opportunities you could have pursued instead.
2. Bureaucratic costs multiply. Requiring ROI projections for experimental projects kills experimentation. The doesn’t happen because the projects are bad. It happens because genuine uncertainty means ROI projections are simple fiction. Demanding data justification for market adaptation slows response to changing conditions. The time spent analyzing often exceeds the window for effective action. Bureaucratic processes optimized for control can create “trained incapacity.” Playbooks can kill adaptation.
3. Competitive disadvantage grows. While you’re analyzing market data, adaptive competitors are testing solutions with customers. While you’re optimizing existing processes, they’re discovering new possibilities. Their advancements could make your optimized processes irrelevant. Companies that over-rely on calculation excel at static efficiency. They are great at optimizing the now. But, they can do this at the expense of dynamic efficiency (changing under new situations).
When Calculation Works, When Judgment Works
Clear patterns emerge for when each approach creates the most value:
Use calculation for stable processes with measurable relationships. Operator work (maintaining inventory, optimizing staffing, etc.) responds well to analytical approaches. You have historical data, known variables, and measurable outcomes. More data helps. Statistical analysis reveals patterns. Optimization creates value.
Use judgment for uncertain markets and novel situations. Creator work (identifying market opportunities and adapting to competitive changes) requires entrepreneurial thinking. In these situations you’re working with incomplete information and shifting variables. The goal isn’t optimization but discovery of what’s possible and valuable. Israel Kirzner called this “entrepreneurial alertness.” This alertness has to do with the ability to notice opportunities that aren’t yet reflected in current market prices or metrics. As Peter Klein and Nicolai Foss point out, this “alertness” is based on judgment and the decision to act. The opportunities themselves are being created by discovery.
Use systematic improvement for optimization within known systems. Refiner work to an extent combines both approaches. Refiners use analytical tools to identify improvement opportunities. They also apply some level of judgment to rank which improvements are worth pursuing as well as how to best bring them to life.
The Rideshare Example: When Each Approach Creates Value (And When It Destroys Value)
Uber and Lyft’s expansion provides clear examples of both approaches working well. They also highlight the costs of applying the wrong approach.
When Calculation Worked. Uber’s surge pricing algorithm usually works great because it operates in more of a calculable/risk environment. They have tons of historical supply and demand data. This allows them to make predictions about how riders and drivers may respond to price changes under normal conditions. The calculation approach enabled dynamic optimization that would be impossible through judgment alone.
When Calculation Failed. The same approach can fail when applied to situations out of the norm. The centralized algorithm does poorly when calculating surge pricing during unprecedented events. During events like festivals or weather emergencies rider behavior doesn’t follow normal patterns. Letting the algorithms run wild has led to some serious PR disasters in the past. 5x surge pricing might be economically optimal during rush hour. But it may be socially catastrophic when people are trying to evacuate in an emergency.
Rideshare companies ended up adopting manual processes to cap price increases in these times. As such, they moved to using judgment-based interventions for unusual situations. This illustrates a critical point. Using calculation for uncertainty problems can produce worse outcomes than using judgment-based rules. If the algorithm gets this wrong, it can create disaster that human judgment would have avoided.
When Judgment Enabled Market Entry. Uber and Lyfts market entry decisions required judgment from the start. Local teams discovered that Las Vegas tourists had different needs than San Francisco commuters. That college towns required different supply strategies than business districts. That regulations varied in ways that affected success far more than simple demographic data suggested.
Calculation helped optimize operations within markets they understood. But judgment was required to discover how to succeed in markets with different characteristics. The most successful local teams combined both. Teams used calculation for operational planning. And the relied on judgment for local strategy (that is, if central teams allowed it).
The Information Challenge in Growing Organizations
As organizations scale, they encounter a version of what Nobel-winning economist Friedrich Hayek called “the knowledge problem.” No single person has perfect knowledge. No one has all the knowledge needed for every decision a scaling business needs to make. This knowledge is distributed across teams and people. As a result it is difficult, if not impossible, to centralize all this knowledge.
Calculation approaches assume you can combine relevant knowledge and information into analytical tools. But much of the knowledge needed for effective decision-making exists in local contexts. In the minds of those closest to the problem. It can’t be easily transmitted through formal reporting systems.
Hayek’s had a key insight. Prices help send the knowledge of millions of people around the world. Prices allow markets to coordinate efforts across millions of people. As a result, central planners don’t need to know everything for the world to run smoothly. Inside organizations, you don’t have price signals. Because of this, you need other mechanisms to handle distributed knowledge.
Judgment approaches assume decision-makers have access to relevant local knowledge and context. But growing organizations create distance between decision-makers and front-line information. This makes judgment-based decisions less reliable because they’re based on incomplete knowledge, determined by people too far away from the problem.
This means organizations must match decision-making authority to information access. People with the best information about specific situations should have authority to make decisions about those situations. This should be the case whether those decisions need calculation or judgment.
This is why Uber and Lyft empowered local teams with decision authority. These teams had contextual knowledge that headquarters couldn’t access through data systems. It’s why Netflix empowers regional content teams to greenlight local productions.
What About AI?
Artificial intelligence can improve calculation capabilities while highlighting the continued importance of judgment.
AI excels at pattern recognition and optimization in data-rich, stable environments. These are exactly the conditions where calculation approaches work best. AI can handle things like supply chain optimization, pricing analysis, and fraud detection. Generally AI will likely be more effective than human analysis in these cases. Modern machine learning attempts to automate the discovery of patterns in data. It accomplishes this goal very well. But not all patterns are causal or meaningful.
AI struggles with novel situations and contextual judgment. These situations rely on understanding of values, culture, and unprecedented circumstances. Market entry decisions, innovation strategy, and organizational design still need human judgment. These situations involve factors that you can not reduce to historical patterns.
AI systems trained on past data don’t develop alertness to future possibilities that don’t yet exist in the training data. They optimize based on patterns. They don’t discover novel opportunities that break existing patterns.
Companies that use AI to automate calculation tasks can free up human capacity. This allows them to focus on judgment-intensive challenges where context creates the most value. But this only works if organizations understand which problems demand calculation versus judgment. If you use AI to automate the wrong things, you can wind up making mistakes faster and at greater scale.
Why Dual-Method Organizations Are Hard (And Worth Building)
Successful organizations need both calculation and judgment. More important, they need to know how to use them for the right problems at the right times. This fitness of “approach to situation” is based on the information environment of different decisions. But building these organizations is difficult.
The challenge is, calculation and judgment rely on incompatible evaluation systems.
Calculation demands measurable metrics, short feedback loops, and clear success criteria. You can gauge analytical decisions quickly. Outcomes are observable and counterfactuals are estimable.
Judgment requires tolerance for failure, long time horizons, and portfolio-based evaluation. You can’t assess individual entrepreneurial decisions in isolation. Many judgment-based initiatives fail. Value comes from the distribution of outcomes across repeated attempts.
Most organizations solve this tension by choosing one approach and pushing the other into a corner. Mature companies choose calculation, creating sophisticated systems that kill innovative capacity. Startups choose judgment, maintaining flexibility but possibly limiting optimization.
A possible solution is to focus on “ambidexterity.” This means separate out the calculation and judgment activities, but coordinate between them.
On one hand you can do this by having different evaluation systems for different teams. Operators measured on efficiency, creators measured on discovery, refiners using hybrid approaches. This is why the operators/refiners/creators distinction matters. It creates organizational space for dual methods to coexist.
Or you can use different evaluation periods. You could use quarterly reviews for operational metrics. This would give calculation approaches the short feedback loops they need. You could then use annual or longer periods for innovation initiatives. This would give judgment approaches the time they need to see results or learnings. Many effective organizations have dual budget processes to do this. They usually manage operational budgets quarterly with tight ROI requirements. For innovation budgets, they stick to annual evaluations with a focus on learning-based outcomes.
Coordination requires people or leaders who can translate between calculation and judgment logics. This helps the organization make decisions that balance both ideas rather than letting one dominate.
This is coordination is where the Harmonizer concept becomes essential. Harmonizers solve the fundamental problem of these double-method organizations. Harmonizers can assess when to use calculation and when to use judgment. This can help prevent either approach from taking over areas where it doesn’t apply.
Without this coordination function, organizations either:
Let calculation logic dominate everything, killing judgment-based work, or
Let judgment logic dominate everything, undermining analytical based efficiency.
A Practical Test: Is This a Calculation or Judgment Problem?
When facing a specific business decision, ask these questions:
Can I identify the probability of different possible outcomes?
Yes, based on historical patterns → Calculation likely works
No, this is a novel situation → Judgment likely needed
Would running an experiment give me more useful information than more analysis?
No, analysis would clarify the situation → Calculation likely works
Yes, only market testing will reveal the answer → Judgment likely needed
Is the situation novel enough that historical patterns may not apply?
No, past patterns should hold → Calculation likely works
Yes, new circumstances → Judgment likely needed
Can I wait for enough data to make the decision analytically?
Yes, without significant opportunity cost → Consider calculation
No, market conditions need faster action → Use judgment
Are the key variables measurable and stable?
Yes, clear metrics exist and relationships are consistent → Calculation likely works
No, important factors resist quantification → Judgment likely needed
If 3+ questions point toward judgment, treat it as an uncertainty problem. Use experimental approaches and tolerate failure (within bounds). Judge on learning rather than immediate results. Give decision-making authority to people with contextual knowledge.
If 3+ point toward calculation, treat it as a risk problem. Use analytical approaches and optimize. Assess on measurable outcomes, and centralize decision-making where you can combine information.
If they’re split, you’re dealing with a mixed problem. This will requires using both approaches either in sequence or in parallel. If in sequence, use judgment to discover possibilities and then turn to calculation to operate. If in parallel, use judgment for the novel aspects and calculation for established aspects.
Building Organizational Systems That Apply the Right Approach
Create different evaluation systems for management and entrepreneurship roles. Using management metrics for entrepreneurship kills experimentation. Using entrepreneurship metrics for management creates operational chaos.
Establish different time horizons for different types of decisions. Calculation-based decisions can have shorter feedback loops because results are measurable quickly. Judgment-based decisions need longer evaluation periods. Discovery takes time and requires portfolio thinking. A quarterly review cycle works well for operational optimization. It can fails for innovation initiatives where learning compounds over years.
Design different information flows for different decision types. Calculation decisions need aggregated data and systematic analysis. Judgment decisions need access to local information and knowledge that often can’t be centralized.
Match coordination mechanisms to decision types. Standardized processes work well for calculation-based decisions. In these situations, consistency and optimization are the goal. Experimental processes work better for judgment-based decisions. In these situations adaptation and discovery are the goal.
The Competitive Advantage of Getting This Right
Companies that master the calculation-judgment distinction gain competitive advantages that compound over time:
Less waste because you invest resources appropriately.
Faster decision-making by using the right approach rather than universal processes.
Better adaptation to changing market conditions. You can quickly shift to judgment-based approaches when calculation becomes inadequate
More effective innovation because experimental work gets tolerance for failure that discovery requires. Operational work gets the systematic improvement focus that efficiency requires
Enhanced learning because you’re applying the right method to extract value from experience
Where We’re Going Next
Understanding when to calculate versus judge is foundational for growing organizations. But there’s a related distinction that scaling companies struggle with. Some problems respond to systematic, scalable approaches, while others need adaptive, non-scalable thinking.
Next week, we’ll explore how information economics explains this distinction. The most valuable opportunities often exist in the non-scalable category. It just happens that most measurement systems happen to ignore these options.
The Bottom Line
Economists over the last century have provided insights that help explain when to rely on calculation versus judgment hat apply directly to the management versus entrepreneurship distinction from Growth Isn’t One Sided.
Frank Knight’s risk versus uncertainty framework provides the foundation. Calculation works when you’re operating under risk with knowable probability distributions. Judgment works when you’re operating under uncertainty where the future is new and unknowable.
But the distinction is deeper than having more or less data. It’s based on the nature of knowledge in our world (a.k.a epistemological, for the philosophy nerds out there). Probability theory either applies to your situation or it doesn’t. No amount of analysis converts genuine uncertainty into calculable risk.
Companies that match their decision-making methods to their information excel at both efficiency and adaptation. Those that apply calculation logic across every problem miss opportunities that need judgment. Those that rely only on judgment miss efficiencies that calculation could provide.
The competitive advantage goes to organizations that apply the right approach to the right problems. The alternative is to defaulting to one method for all decisions. To do this well, organizations must develop some ambidexterity. They must separate calculation and judgment activities while maintaining coordination between them.
This requires Harmonizers who can translate between these different approaches. They can shift between both approaches rather than letting one dominate areas where it doesn’t apply.
Understanding the economics underlying these ideas helps you make better individual decisions. It also helps you design organizational systems that enable both optimization and discovery. You need both to create advantages as markets evolve.