Statistics vs Physics for Long-Range Weather Forecasting
Business
Long-range weather forecasting can be achieved via two methods: statistics-based and physics-based. But only one can deliver accurate results consistently. In this post, we explore both methods and look at which one is better for business.
Basic Meteorology
Physics-based weather forecasting is more commonly known as meteorology. It refers to the meteorological forecasts the average person encounters every day through TV, websites, and smart phone apps. Meteorology is often criticized for being inaccurate and too dependent on guesswork. And while the criticisms of its accuracy are warranted, there is still a lot of science that goes into it.
In fact, physics-based weather forecasting is actually quite complicated, utilizing physics and fluid dynamics to predict the movements and patterns that dictate weather. In traditional, physics-based models, scientists make calculations using complex formulas that have been around for a hundred years.
One analogy for understanding meteorology is that it's like trying to predict the behavior of ripples in a body of water.
If you throw one pebble in a lake, it creates a set of ring-like waves that are pretty easy to predict. But meteorology is more akin to dropping millions of pebbles, all of different sizes, and trying to predict how the waves collide with and absorb one another.
Physics-based forecasting necessitates the observation of millions of interactions, collisions and absorptions. And you have to predict all of these interactions, as they happen, a hundred thousand feet in the air. Needless to say, it is a forecasting model defined by uncertainty - and this causes a few problems.
Problems with Physics-Based Forecasting
It's no secret physics-based models struggle with accuracy. And this is a problem because there are consequences when weather forecasts are inaccurate.
You only have to go back to 2012 to Hurricane Sandy. Meteorologists expected it to be a minor storm with minimal impact on the Caribbean and East Coast of the U.S. However, it became much bigger than the models predicted, resulting in $70 billion in damages and 100+ deaths that no one saw coming.
The issue with these physics models is they have more rounding errors with each iteration of time. The errors build and compound as meteorologists predict further and further into the future. So, while hourly and daily forecasts are fairly accurate, as you move into the two-week range, the predictions start to fall apart.
Another challenge of physics-based meteorology is that most models change two to four times a day. This makes long-range meteorological forecasting a moving target that's nearly impossible to hit. It's also the reason forecasts change constantly.
Meteorology: Not Good for Business
So, what does this mean if you run a business? A lot, actually.
There is a huge correlation between weather and sales. A slight change in temperature or rainfall can mean a big change, in either direction, for how much a given product will sell. Businesses that understand this relationship pay close attention to weather forecasts. And the further into the future they can predict, the better.
But this is where physics-based models fall flat - their accuracy drops beyond a week or two out. Most businesses can't effectively adjust their operations to the weather in that time. Retailers, for example, need 6-12 months to plan inventory, pricing, markdown and logistics strategies.
Additionally, forecasts that change multiple times are damaging for businesses because they're forced to hit a moving and unpredictable target. By the time the business makes a decision, the weather forecast changes again.
So, if physics-based forecasting won't work, what will?
If you want to produce long-range weather forecasts that don't change, it means you have to be as accurate as possible. And you also need to be able to demonstrate your accuracy over time. To do this you must embrace statistics.
But this is far from easy. So, to illustrate the point, we're going to look at the only statistics-based weather forecasting model we know - our own.
Historical Trends
To begin our process, we review historical and current weather data - a lot of it.
We employ a trillion times trillion times billion points of data. That's 10 to 61st factorial. This utilizes massive amounts of computing power. We then use these historical patterns to predict future weather. Here's an example of what that looks like in practice.
In 2016, the Northeast United States experienced a record hot March. The month was 7.1 degrees Fahrenheit above average - one standard deviation. It was blistering hot. Not the hottest month in that region's history, but top ten.
If you run a business in the Northeast and just experienced the hottest March in decades, how do you plan for next year? Most businesses plan for the same March weather to happen the following year. But in almost every case, that's the opposite of what happens.
In the Northeast, after a warm March, there's an 86% chance the next March will be cold-below normal temperature-and a 94% chance it will be snowy. Then, there's an 80-90% chance of colder Marches for two to three years.
We know this because of historical data. We can see that over the last 125 years, anytime there was a really hot March in the Northeast, there was usually a cold and snowy March the following year.
After the hot spell in March 2016, we were able to use these statistics to predict the cold spells the region has experienced for the past two to three years.
To put this level of accuracy in perspective, 86-94% odds are far beyond what is possible in other areas of business, like investing.
A good statistics-based forecasting method is about calculating, identifying and modeling scenarios with extremely favorable odds. What is different about this methodology is that meteorologists don't think this way. It's against their training. NOAA uses traditional methodologies to predict next month but our statistical model has outperformed their monthly outlook over the past 11 years despite being a YEAR-AHEAD.
Retailers and businesses, however, do think this way, which is why statistics-based forecasting is great for business applications. It helps identify opportunities with tremendous odds a year ahead.
Another advantage of this method is that the prediction doesn't change. The forecast we make a year out for a specific month stays the same as we get closer. This has tremendous benefits for business applications (more on that in a bit).
Climate Cycles
After considering historical trends, the next step is to look at climate cycles.
What are the oceans doing? Are they getting colder or hotter in these regions? To answer these questions, we study ocean and current cycles. For example, the Atlantic goes through 20-year cycles while the Pacific cycle lasts 30 years. Another important consideration is the El Niño and La Niña ocean phenomena that tend to go through 3- to 7-year cycles.
We also evaluate 11-year solar cycles and solar activity levels to determine if the sun is getting more or less active.
All told, our statistics-based weather forecasting model utilizes 24 climate cycles. Most physics-based models won't use more than 6-10.
After studying these cycles, we combine our findings with historical data. The climate cycles work to verify or modify the statistical trends we've identified. Typically, we'll readjust projections based on climate cycle data to provide more accurate insights.
Accuracy
The most attractive thing about statistics-based weather forecasting is its accuracy, particularly at long range.
Our statistics-based forecasts are 85% accurate.
Typical short-range, physics-based models will have 4-10 degrees of error in the seven to fourteen-day timeframe. Our statistics-based year-ahead forecast has between 2.5-6 degrees of error. Because of this, our year-ahead forecast is actually more accurate than the majority of week two meteorological forecasts for the same time period.
This degree of long-range accuracy is possible because of trillions of statistics and calculations coupled with advanced computing technology.
Once you have a reliable way to predict temperature and precipitation, you can forecast things like product sales for retail businesses or crop yields for agricultural ones.
For example, we've quantified that for every 1 degree colder, coat sales increase 7%. And that every 1 degree hotter means a 15% increase in air conditioner sales.
Business intelligence of this kind helps retailers optimize inventory and operations a year ahead. But it's not just for retail. Businesses in finance, agriculture, airline, supply chain and other industries can benefit from long-range weather forecasts as well.
Wait, Why Isn't Anyone Else Doing This?
So, if statistics-based forecasting is so accurate, why isn't everybody using it?
The biggest reason is training. Anyone with a background or interest in weather forecasting is trained through the physics-based meteorological model. This training is great for analyzing current, short-term weather patterns and phenomena. But it doesn't provide the tools or mindset to think outside the box and look beyond a two-week projection. Year-ahead forecasting requires expertise in meteorology, statistics and software engineering.
Conclusion
Weather impacts every business. If you're looking to plan your business around the weather, statistics-based weather forecasting provides a more accurate method for long-range predictions than traditional meteorology.
Our proprietary year-ahead forecasting model gives you 85% weather accuracy and lets you predict sales and optimize your business. That's how we've delivered more than $2 billion in ROI for our customers. Let's talk.
Physics-Based Weather Forecasting
Basic Meteorology
Physics-based weather forecasting is more commonly known as meteorology. It refers to the meteorological forecasts the average person encounters every day through TV, websites, and smart phone apps. Meteorology is often criticized for being inaccurate and too dependent on guesswork. And while the criticisms of its accuracy are warranted, there is still a lot of science that goes into it.
In fact, physics-based weather forecasting is actually quite complicated, utilizing physics and fluid dynamics to predict the movements and patterns that dictate weather. In traditional, physics-based models, scientists make calculations using complex formulas that have been around for a hundred years.
One analogy for understanding meteorology is that it's like trying to predict the behavior of ripples in a body of water.
If you throw one pebble in a lake, it creates a set of ring-like waves that are pretty easy to predict. But meteorology is more akin to dropping millions of pebbles, all of different sizes, and trying to predict how the waves collide with and absorb one another.
Physics-based forecasting necessitates the observation of millions of interactions, collisions and absorptions. And you have to predict all of these interactions, as they happen, a hundred thousand feet in the air. Needless to say, it is a forecasting model defined by uncertainty - and this causes a few problems.
Problems with Physics-Based Forecasting
It's no secret physics-based models struggle with accuracy. And this is a problem because there are consequences when weather forecasts are inaccurate.
You only have to go back to 2012 to Hurricane Sandy. Meteorologists expected it to be a minor storm with minimal impact on the Caribbean and East Coast of the U.S. However, it became much bigger than the models predicted, resulting in $70 billion in damages and 100+ deaths that no one saw coming.
The issue with these physics models is they have more rounding errors with each iteration of time. The errors build and compound as meteorologists predict further and further into the future. So, while hourly and daily forecasts are fairly accurate, as you move into the two-week range, the predictions start to fall apart.
Another challenge of physics-based meteorology is that most models change two to four times a day. This makes long-range meteorological forecasting a moving target that's nearly impossible to hit. It's also the reason forecasts change constantly.
Meteorology: Not Good for Business
So, what does this mean if you run a business? A lot, actually.
There is a huge correlation between weather and sales. A slight change in temperature or rainfall can mean a big change, in either direction, for how much a given product will sell. Businesses that understand this relationship pay close attention to weather forecasts. And the further into the future they can predict, the better.
But this is where physics-based models fall flat - their accuracy drops beyond a week or two out. Most businesses can't effectively adjust their operations to the weather in that time. Retailers, for example, need 6-12 months to plan inventory, pricing, markdown and logistics strategies.
Additionally, forecasts that change multiple times are damaging for businesses because they're forced to hit a moving and unpredictable target. By the time the business makes a decision, the weather forecast changes again.
So, if physics-based forecasting won't work, what will?
Statistics-Based Weather Forecasting
If you want to produce long-range weather forecasts that don't change, it means you have to be as accurate as possible. And you also need to be able to demonstrate your accuracy over time. To do this you must embrace statistics.
But this is far from easy. So, to illustrate the point, we're going to look at the only statistics-based weather forecasting model we know - our own.
Historical Trends
To begin our process, we review historical and current weather data - a lot of it.
We employ a trillion times trillion times billion points of data. That's 10 to 61st factorial. This utilizes massive amounts of computing power. We then use these historical patterns to predict future weather. Here's an example of what that looks like in practice.
In 2016, the Northeast United States experienced a record hot March. The month was 7.1 degrees Fahrenheit above average - one standard deviation. It was blistering hot. Not the hottest month in that region's history, but top ten.
If you run a business in the Northeast and just experienced the hottest March in decades, how do you plan for next year? Most businesses plan for the same March weather to happen the following year. But in almost every case, that's the opposite of what happens.
In the Northeast, after a warm March, there's an 86% chance the next March will be cold-below normal temperature-and a 94% chance it will be snowy. Then, there's an 80-90% chance of colder Marches for two to three years.
We know this because of historical data. We can see that over the last 125 years, anytime there was a really hot March in the Northeast, there was usually a cold and snowy March the following year.
After the hot spell in March 2016, we were able to use these statistics to predict the cold spells the region has experienced for the past two to three years.
To put this level of accuracy in perspective, 86-94% odds are far beyond what is possible in other areas of business, like investing.
A good statistics-based forecasting method is about calculating, identifying and modeling scenarios with extremely favorable odds. What is different about this methodology is that meteorologists don't think this way. It's against their training. NOAA uses traditional methodologies to predict next month but our statistical model has outperformed their monthly outlook over the past 11 years despite being a YEAR-AHEAD.
Retailers and businesses, however, do think this way, which is why statistics-based forecasting is great for business applications. It helps identify opportunities with tremendous odds a year ahead.
Another advantage of this method is that the prediction doesn't change. The forecast we make a year out for a specific month stays the same as we get closer. This has tremendous benefits for business applications (more on that in a bit).
Climate Cycles
After considering historical trends, the next step is to look at climate cycles.
What are the oceans doing? Are they getting colder or hotter in these regions? To answer these questions, we study ocean and current cycles. For example, the Atlantic goes through 20-year cycles while the Pacific cycle lasts 30 years. Another important consideration is the El Niño and La Niña ocean phenomena that tend to go through 3- to 7-year cycles.
We also evaluate 11-year solar cycles and solar activity levels to determine if the sun is getting more or less active.
All told, our statistics-based weather forecasting model utilizes 24 climate cycles. Most physics-based models won't use more than 6-10.
After studying these cycles, we combine our findings with historical data. The climate cycles work to verify or modify the statistical trends we've identified. Typically, we'll readjust projections based on climate cycle data to provide more accurate insights.
Accuracy
The most attractive thing about statistics-based weather forecasting is its accuracy, particularly at long range.
Our statistics-based forecasts are 85% accurate.
Typical short-range, physics-based models will have 4-10 degrees of error in the seven to fourteen-day timeframe. Our statistics-based year-ahead forecast has between 2.5-6 degrees of error. Because of this, our year-ahead forecast is actually more accurate than the majority of week two meteorological forecasts for the same time period.
This degree of long-range accuracy is possible because of trillions of statistics and calculations coupled with advanced computing technology.
Why Statistics-Based Forecasting is Better for Business
Once you have a reliable way to predict temperature and precipitation, you can forecast things like product sales for retail businesses or crop yields for agricultural ones.
For example, we've quantified that for every 1 degree colder, coat sales increase 7%. And that every 1 degree hotter means a 15% increase in air conditioner sales.
Business intelligence of this kind helps retailers optimize inventory and operations a year ahead. But it's not just for retail. Businesses in finance, agriculture, airline, supply chain and other industries can benefit from long-range weather forecasts as well.
Wait, Why Isn't Anyone Else Doing This?
So, if statistics-based forecasting is so accurate, why isn't everybody using it?
The biggest reason is training. Anyone with a background or interest in weather forecasting is trained through the physics-based meteorological model. This training is great for analyzing current, short-term weather patterns and phenomena. But it doesn't provide the tools or mindset to think outside the box and look beyond a two-week projection. Year-ahead forecasting requires expertise in meteorology, statistics and software engineering.
Conclusion
Weather impacts every business. If you're looking to plan your business around the weather, statistics-based weather forecasting provides a more accurate method for long-range predictions than traditional meteorology.
Our proprietary year-ahead forecasting model gives you 85% weather accuracy and lets you predict sales and optimize your business. That's how we've delivered more than $2 billion in ROI for our customers. Let's talk.