The term experiment is defined as the systematic procedure carried out under controlled conditions in order to discover an unknown effect, to test or establish a hypothesis, or to illustrate a known effect. When analyzing a process, experiments are often used to evaluate which process inputs have a significant impact on the process output, and what the target level of those inputs should be to achieve a desired result (output). Experiments can be designed in many different ways to collect this information. Design of Experiments (DOE) is also referred to as Designed Experiments or Experimental Design – all of the terms have the same meaning.
Experimental design can be used at the point of greatest leverage to reduce design costs by speeding up the design process, reducing late engineering design changes, and reducing product material and labor complexity. Designed Experiments are also powerful tools to achieve manufacturing cost savings by minimizing process variation and reducing rework, scrap, and the need for inspection.
This Toolbox module includes a general overview of Experimental Design and links and other resources to assist you in conducting designed experiments. A glossary of terms is also available at any time through the Help function, and we recommend that you read through it to familiarize yourself with any unfamiliar terms.
Design of Experiments (DOE)
Design of Experiments (DOE) is a branch of applied statistics focused on using the scientific method for planning, conducting, analyzing and interpreting data from controlled tests or experiments. DOE is a mathematical methodology used to effectively plan and conduct scientific studies that change input variables (X) together to reveal their effect on a given response or the output variable (Y). In plain, non-statistical language, the DOE allows you to evaluate multiple variables or inputs to a process or design, their interactions with each other and their impact on the output. In addition, if performed and analyzed properly you should be able to determine which variables have the most and least impact on the output. By knowing this you can design a product or process that meets or exceeds quality requirements and satisfies customer needs.
Why Utilize Design of Experiments (DOE)?
DOE allows the experimenter to manipulate multiple inputs to determine their effect on the output of the experiment or process. By performing a multi-factorial or “full-factorial” experiment, DOE can reveal critical interactions that are often missed when performing a single or “fractional factorial” experiment. By properly utilizing DOE methodology, the number of trial builds or test runs can be greatly reduced. A robust Design of Experiments can save project time and uncover hidden issues in the process. The hidden issues are generally associated with the interactions of the various factors. In the end, teams will be able to identify which factors impact the process the most and which ones have the least influence on the process output.
When to Utilize Design of Experiments (DOE)?
Experimental design or Design of Experiments can be used during a New Product / Process Introduction (NPI) project or during a Kaizen or process improvement exercise. DOE is generally used in two different stages of process improvement projects.
- During the “Analyze” phase of a project, DOE can be used to help identify the Root Cause of a problem. With DOE the team can examine the effects of the various inputs (X) on the output (Y). DOE enables the team to determine which of the Xs impact the Y and which one(s) have the most impact.
- During the “Improve” phase of a project, DOE can be used in the development of a predictive equation, enabling the performance of what-if analysis. The team can then test different ideas to assist in determining the optimum settings for the Xs to achieve the best Y output.
Some knowledge of statistical tools and experimental planning is required to fully understand DOE methodology. While there are several software programs available for DOE analysis, to properly apply DOE you need to possess an understanding of basic statistical concepts.
Components of Experimental Design
Consider the following diagram of a cake-baking process. There are three aspects of the process that are analyzed by a designed experiment:
- Factors, or inputs to the process
Factors can be classified as either controllable or uncontrollable variables. In this case, the controllable factors are the ingredients for the cake and the oven that the cake is baked in. The controllable variables will be referred to throughout the material as factors. Note that the ingredients list was shortened for this example – there could be many other ingredients that have a significant bearing on the end result (oil, water, flavoring, etc). Likewise, there could be other types of factors, such as the mixing method or tools, the sequence of mixing, or even the people involved. People are generally considered a Noise Factor (see the glossary) – an uncontrollable factor that causes variability under normal operating conditions, but we can control it during the experiment using blocking and randomization. Potential factors can be categorized using the Fishbone Chart (Cause & Effect Diagram) available from the Toolbox.
- Levels, or settings of each factor in the study
Examples include the oven temperature setting and the particular amounts of sugar, flour, and eggs chosen for evaluation.
- Response, or output of the experiment
In the case of cake baking, the taste, consistency, and appearance of the cake are measurable outcomes potentially influenced by the factors and their respective levels. Experimenters often desire to avoid optimizing the process for one response at the expense of another. For this reason, important outcomes are measured and analyzed to determine the factors and their settings that will provide the best overall outcome for the critical-to-quality characteristics – both measurable variables and assessable attributes.