Setting process control chart limits for rounded-off measurements

Ran Etgar, Sarit Freund

Research output: Contribution to journalArticlepeer-review


Measurements can often be imprecise and subjected to rounding-off. Typically, this rounding-off is ignored and assumed to have little to no effect. However, when the measuring scale step is not negligible, it may affect statistical control tools such as X‾-chart. Designing statistical process controls without considering the effects of rounding leads to high exposure to false negative results. This study illustrates the effects of rounding on the X-chart and shows that the result may further deteriorate due to asymmetry (incompatibility of the process and the measuring device parameters). A new simple method to design control limits is proposed, based on maintaining the original characteristics of the chart as devised by Shewhart.

Original languageEnglish
Article numbere13655
Issue number3
StatePublished - Mar 2023
Externally publishedYes

Bibliographical note

Publisher Copyright:
© 2023 The Authors


  • Measurement
  • Quality control
  • Round-off
  • Rounding error
  • Statistical process control


Dive into the research topics of 'Setting process control chart limits for rounded-off measurements'. Together they form a unique fingerprint.

Cite this