Evaluation of output images carrying visual semantics is a challenging task which is carried out by domain experts through visual inspection. Automatic test oracle is required to augment the test oracle problem and to eliminate the manual efforts. Metamorphic testing is an effective technique to...

Comparison with a standard is a general multiple comparison problem, where each system is required to be compared to a single system, referred to as a “standard”, as well as to other alternative systems. The goal is to determine the best system among a number of systems that are better than...

Data truncation is a commonly accepted method of dealing with initialization bias in discrete-event simulation. An algorithm for determining the appropriate initial-data truncation point for univariate output is proposed. The technique entails averaging across independent replications and...

This paper presents a procedure for determining the number of simulation observations required to achieve a preassigned confidence interval for means estimated by simulation. This procedure, which is simple to implement and efficient to use, is compared with two other methods for determining the...

We consider the problem of comparing a finite number of stochastic systems with respect to a single system (designated as the "standard") via simulation experiments. The comparison is based on expected performance, and the goal is to determine if any system has larger expected performance than...

We propose some new two-stage stopping procedures to construct absolute-width and relative-width confidence intervals for a simulation estimator of the steady-state mean of a stochastic process. The procedures are based on the method of standardized time series proposed by Schruben and on...

When the simulation department of Management Science was created in 1978 it ushered in an era of significant methodological advances in stochastic simulation. However, the foundation for the field---not just the work that has been published in Management Science---was provided by two papers...

We present a general recipe for constructing experiment design and analysis procedures that simultaneously provide indifference-zone selection and multiple-comparison inference for choosing the best among k simulated systems. We then exhibit two such procedures that exploit the...

A supersaturated design is a factorial design in which the number of factors to be estimated is larger than the available number of experimental runs. The cost and time required for many industrial experimentations can be reduced by using the class of supersaturated designs, since the main goal...