The Benefits of Big Data and Forecasting in Today's Economy

Forecasting is a prediction of some future event. It is used as a mechanism for strategic planning in most industries but is pronounced in marketing, economics, human resources and investing that rely on future predictions to determine the best courses of actions. Forecasting affords the ability to understand market risks as companies seek to stay ahead of changes and adjust their processes to meet market challenges.

Forecasting can be simple or it can be complex. Typically methodologies use formulas and strict processes to ensure they are giving fair weight and evaluation to the necessary factors used in making that prediction. The process is so important that people have made their living simply off of analyzing information and making usable intelligence for others.

Predictions are as much likely to be wrong as they are to be correct. The problem with predictions is that no one can truly know the exact possibility of the future or all of the outlier events that can change a course of action. However, it is possible that people can determine within a range the most likely events based upon today's influencing factors.

 How and what we see in big data makes a big difference on the quality of the prediction. Some methodologies funnel certain information into the prediction model leaving out clues that may also have relevance. Other times big data can confuse practitioners and they are unable to find any meaningful patterns in the information.

San Diego is getting up to speed on using big data to not only solve problems and make better scientific predictions. According to an article in UT San Diego biomedical community is hiring big data and software gurus to work on problems that range from cancer to Alzheimer. The collection and analysis of information can find statistical significant among events leading to better prevention of future health problems. 

Forecasting and predictions rely on the analysis of big data. Without the ability to understand past data and put it within an appropriate mental framework the patterns of past behavior cannot be used to predict future behavior. Ensuring that you have the capacity to collect information, analyze that information, and put it to good use is furthered through scientific thinking and better data management systems.

Events in Isolation: One way to evaluate likely actions is to assess the probability of events occurring not in a sequence but as stand alone events. The history of the particular problem doesn't matter as long as you have the data to understand how each factor influences the changes one route will happen over another.

Consider how the flip of a coin will either land on a head or tail regardless of previous spins. We know this is 50%/50%. Other events may be 30%/70% among two possible outcomes or 10%, 40%, 50% among three possible outcomes. Each possible outcome has an associated probability of it occurring and can be used in the prediction model.

The problem is that many people do not always see all of the outcomes or know the possible probable outcomes. Their mind and perspective skips over important data that is useful in the analysis. For example, one person may see two possible outcomes and another three. If you cannot see all the possible outcomes then your percentages are likely to be skewed making an assessment incorrect.

Events Based on History: Events don't always work in isolation and have historical outcomes. Understanding history and they way in which things turned out in the past helps to determine your current probabilities. Those probabilities are used to make an evaluation of the current situation to determine whether or not a particular outcome is likely.

For example, if we looked back at the history of a dog sniffing the grass in front of your house you may need to watch what other dogs, as well as that dog, have done over the course of a period of time. If the dog sniffs the same place 70% of the time it walks past the chances are it will do it again. This determination would not be possible without a historical evaluation.

Events Based on Trends and Momentum: Sometimes events are already in a sequence of actions and simply taking a snap shop of it like that used in a Markov chain will not produce an accurate forecast. Evaluating the trend of an action will help determine what influences are likely to either maintain its current trajectory or influence a changes in that trajectory. A ball in motion may stay in motion.

Finding trends means looking at information not only from its current place but also its longitudinal history. This often requires multiple methods of making measurements to see how things are changing. For example, over the past two years unemployment is dropping and assuming nothing in substantial in the market is changing then that trajectory will continue to move forward until other economic factors slow it down.

Events Based on Human Goal Directed Behavior: Humans have animal spirits that are led by logic, emotion, and social expectations to come to conclusion about certain events. Their natural selection and backgrounds will determine what they see in their environment and how they will evaluate that information. Their behaviors are a result of their conclusions and choices will likely be directed toward their goals.

Understanding the history of the person, what type of information they have, the medium they received it in, and the messages create a better interpretation of the individual and their likely conclusions. Such behavior is often calculated through polls, sociological/psychological studies, focus groups and marketing analysis.

Robbins, G. (2015). UCSD hiring 'big data' stars. UT San Diego. Retrieved
http://www.utsandiego.com/news/2015/apr/15/bigdata-stars-ucsd/