3. Techniques
3.14 Optimization
Guide to Business Data Analytics
3.14.1 Purpose
Business decisions are often based on some amount of uncertainty regarding the outcome. Optimization can be described as choosing the best possible option among multiple available options under some constraints.
Optimization as a technique that allows decision-makers to make an informed decision based on available information while acknowledging specific constraints.
Business decisions are often based on some amount of uncertainty regarding the outcome. Optimization can be described as choosing the best possible option among multiple available options under some constraints.
Optimization as a technique that allows decision-makers to make an informed decision based on available information while acknowledging specific constraints.
3.14.2 Description
Optimization is used differently based on the type of analytics problem or research question. While most descriptive analysis provides the necessary background for Optimization, it is most useful for problems that are predictive or prescriptive in nature. Optimization can be applied differently to both a low- and a high-uncertainty environment.
Consider an organization trying to create a production plan for two products that generate different levels of profit per unit of sale. There are additional constraints which specify limits on resources and time taken to produce each type of product. An optimization scheme can be used to maximize the profits given all the constraints for each product. Such a problem has low uncertainty because the profit is a direct function of the number of units produced for each product, and the resources and time required to produce them is known with a high degree of certainty. Additionally, the profit is linearly dependent on the number of units of production. These types of decision-making problems can be solved using linear programming, which is an optimization technique that is extensively used in logistics, manufacturing, and project management.
There are other types of problems where the result is not related linearly. For example, overall volatility of a stock portfolio is modelled non-linearly based on the volatility of individual stocks. The decision problem in this scenario is to find the best mix of individual stocks (for example, stock weights in a portfolio) with the lowest volatility (for example, risk).
Most machine learning and deep learning applications have a high degree of uncertainty and utilize optimization of some kind. These models try to discover an approximate solution (formally called functional approximation), that best match the available data. The “best match” means that the error is minimized to as low as possible, which indicates an optimization problem.
The difference in application of optimization in such scenarios is that an optimum value is found iteratively based on an optimization scheme, such as Laplace's method or gradient descent method, that are very popular in methods for neural networks.
When using Optimization, analysts consider:
Optimization is used differently based on the type of analytics problem or research question. While most descriptive analysis provides the necessary background for Optimization, it is most useful for problems that are predictive or prescriptive in nature. Optimization can be applied differently to both a low- and a high-uncertainty environment.
Consider an organization trying to create a production plan for two products that generate different levels of profit per unit of sale. There are additional constraints which specify limits on resources and time taken to produce each type of product. An optimization scheme can be used to maximize the profits given all the constraints for each product. Such a problem has low uncertainty because the profit is a direct function of the number of units produced for each product, and the resources and time required to produce them is known with a high degree of certainty. Additionally, the profit is linearly dependent on the number of units of production. These types of decision-making problems can be solved using linear programming, which is an optimization technique that is extensively used in logistics, manufacturing, and project management.
There are other types of problems where the result is not related linearly. For example, overall volatility of a stock portfolio is modelled non-linearly based on the volatility of individual stocks. The decision problem in this scenario is to find the best mix of individual stocks (for example, stock weights in a portfolio) with the lowest volatility (for example, risk).
Most machine learning and deep learning applications have a high degree of uncertainty and utilize optimization of some kind. These models try to discover an approximate solution (formally called functional approximation), that best match the available data. The “best match” means that the error is minimized to as low as possible, which indicates an optimization problem.
The difference in application of optimization in such scenarios is that an optimum value is found iteratively based on an optimization scheme, such as Laplace's method or gradient descent method, that are very popular in methods for neural networks.
When using Optimization, analysts consider:
| Observations regarding optimization | Business analysis perspective |
| Most analytics problems use optimization in the context of predictive or prescriptive analytics. There are different conventions used by data science professionals to refer to this process, for example, parameter and hyper-parameter tuning, weights optimization, cost/reward function optimization, and so on. | Knowledge of how optimization is used in different problem types help analysts relate to how analytics models are developed. |
| Most optimization problems lead to an approximate solution or prediction. | Analysts ground the expectations of stakeholders to a reasonable level by explaining how optimization works. |
| The process of optimization takes time; this is the phase where data science professionals spend most of their time improving analytics models. | Analysts gain a basic understanding of what is being optimized and how, so that the process can be clearly communicated to stakeholders. |
| Simple problems, such as optimization under low uncertainty, can be quickly computed and analyzed even through basic tools. | Using simple demonstrations tools, such as Excel Solver, help explain and build stakeholder confidence on how optimization can be used. Simple optimization methods are often the most effective supplement the decision-making process. |
3.14.3 Elements
.1 Decision Variables
Decision variables are the elements in a decision process that the decision-maker controls. To explain, assume a company produces two garment cleaning products: soap pods and detergent packs. The basic ingredients for the soap pod and the detergent pack are the same but used in different quantities. The cost per unit, as well as time taken to produce, is different for both the products. The decision for decision-makers is to create a production plan which maximizes profit, assuming all produced units are going to be sold.
The elements in control of the decision-makers are how many units of soap pods and detergent packs to produce. These two variables are the decision variables.
When machine learning and related fields are used to generate an analytical model, it is important to note that the predictor variables represent decision variables. The influence of the predictor variables over the outcome (for example, the coefficients/weights in a regression equation) is optimized so that the resulting error is reduced.
.2 Decision/Objective/Cost/Error Function
The decision function represents the goal for the organization and decision-makers. Based on its application, it is referred to as objective, cost, or error function. It can be stated as a mathematical function to succinctly define the relationship between the decision variables and the objective.
Expanding the example of the production plan:
Decision variables are:
Then, the objective function that needs to be maximized is:
This represents a simple linear programing decision function. Depending on the context, decision functions can be quite complex. Analysts collaborate with the data team to articulate the decision functions clearly to stakeholders and explain how an analytics model works in simple terms.
.3 Constraints
Constraints are the limitations placed on the decision. Analysts help in identifying constraints through a discovery process such as stakeholder elicitation or business rules.
For example, in the production plan problem, the constraints can be that the available labour for the organization is limited to 800 hours. The ingredient for production for both soap pods and detergent packs requires an enzyme and an ethanol, with 20 liters of each available. One detergent pack uses 0.0008 liters of enzyme and 0.0005 of ethanol whereas one soap pod packet uses 0.00001 and 0.0001 liters, respectively. The other constraints to note could be that soap pods and detergent packs can be produced only in whole numbers. There may be constraints based on time of manufacturing as well, which may be discovered through further analysis.
.4 Optimization Model
The optimization model is the formulation of decision variables, objective function, and the constraints to a form (usually a mathematical model) so that tools can utilize this information to determine the optimum parameters. The optimization model for the production plan example could be as follows:
Maximize Profit ($) = 0.2 * S + 0.5 * D
Constraints:
Such a problem can be easily solved using any optimization tool, and complex analytics models use significantly more involved procedures. Analysts collaborate with data science professionals and articulate the analytics model and optimization process to stakeholders.
.1 Decision Variables
Decision variables are the elements in a decision process that the decision-maker controls. To explain, assume a company produces two garment cleaning products: soap pods and detergent packs. The basic ingredients for the soap pod and the detergent pack are the same but used in different quantities. The cost per unit, as well as time taken to produce, is different for both the products. The decision for decision-makers is to create a production plan which maximizes profit, assuming all produced units are going to be sold.
The elements in control of the decision-makers are how many units of soap pods and detergent packs to produce. These two variables are the decision variables.
When machine learning and related fields are used to generate an analytical model, it is important to note that the predictor variables represent decision variables. The influence of the predictor variables over the outcome (for example, the coefficients/weights in a regression equation) is optimized so that the resulting error is reduced.
.2 Decision/Objective/Cost/Error Function
The decision function represents the goal for the organization and decision-makers. Based on its application, it is referred to as objective, cost, or error function. It can be stated as a mathematical function to succinctly define the relationship between the decision variables and the objective.
Expanding the example of the production plan:
Profit per unit for the soap pod: 20 cents.
Profit per unit for the detergent pack: 50 cents.
Decision variables are:
Number of soap pods to produce: S.
Number detergent packs to produce: D.
Then, the objective function that needs to be maximized is:
Profit ($) = 0.2xS + 0.5xD.
This represents a simple linear programing decision function. Depending on the context, decision functions can be quite complex. Analysts collaborate with the data team to articulate the decision functions clearly to stakeholders and explain how an analytics model works in simple terms.
.3 Constraints
Constraints are the limitations placed on the decision. Analysts help in identifying constraints through a discovery process such as stakeholder elicitation or business rules.
For example, in the production plan problem, the constraints can be that the available labour for the organization is limited to 800 hours. The ingredient for production for both soap pods and detergent packs requires an enzyme and an ethanol, with 20 liters of each available. One detergent pack uses 0.0008 liters of enzyme and 0.0005 of ethanol whereas one soap pod packet uses 0.00001 and 0.0001 liters, respectively. The other constraints to note could be that soap pods and detergent packs can be produced only in whole numbers. There may be constraints based on time of manufacturing as well, which may be discovered through further analysis.
.4 Optimization Model
The optimization model is the formulation of decision variables, objective function, and the constraints to a form (usually a mathematical model) so that tools can utilize this information to determine the optimum parameters. The optimization model for the production plan example could be as follows:
Maximize Profit ($) = 0.2 * S + 0.5 * D
Constraints:
0.0008 * D + 0.00001 * S <= 20 liters of enzyme
0.0005 * D + 0.0001 * S <=20 liters of ethanol
S, D are non-negative integer valued
Such a problem can be easily solved using any optimization tool, and complex analytics models use significantly more involved procedures. Analysts collaborate with data science professionals and articulate the analytics model and optimization process to stakeholders.
3.14.4 Usage Considerations
.1 Strengths
.1 Strengths
- Optimization is the mathematical basis of most of the predictive, prescriptive, and operation research analytical models. Analysts utilize Optimization for a variety of use cases by formulating decision variables, objective function, and the constraints correctly. Some techniques such as linear programming have been in use for a long time and are well accepted for business decision-making.
- Optimization methods converge rapidly (equating to finding the optimum solutions faster) when applied to large scale and complex problems using many variables.
- The optimized solution may not be the best solution available (for example, the exact solution).
- More complex formulations are difficult to explain to the stakeholders.
- The process requires very accurate formulation of the constraints. Analysts follow good discovery analysis techniques to uncover any implicit constraints.
- The optimization process in large scale neural networks or large data sets requires processing power and time. It is an iterative process; incorrect formulations often result in time and resource wastage. The data team must collaborate closely to mitigate such risks.
