Menu
Accuracy vs Resolution: Being Correct vs Showing Fine Detail

Accuracy vs Resolution: Being Correct vs Showing Fine Detail

Accuracy is how close a measurement is to the true value; resolution is the smallest change an instrument can display. See why high resolution does not mean high accuracy.
Accuracy vs Resolution: Being Correct vs Showing Fine Detail
Accuracy vs Resolution: Being Correct vs Showing Fine Detail

Key takeaways

  • Accuracy is how close a measurement is to the true value.
  • Resolution is the smallest change or increment an instrument can detect or display.
  • High resolution does not mean high accuracy — an instrument can show many decimal places yet be wrong.
  • Resolution is about fineness of display; accuracy is about correctness of the reading.
  • An instrument needs both adequate resolution and accuracy to be trustworthy.

Short answer: Accuracy and resolution are two different qualities of a measuring instrument that are easily confused. Accuracy is how close a measurement is to the true value — is the reading correct? Resolution is the smallest increment the instrument can detect or display — how fine is the detail it shows? The trap is assuming that an instrument showing many decimal places is therefore accurate: high resolution and high accuracy are independent. A gauge can display to four decimals and still be wrong. For the related quality pair, see precision vs accuracy.

What accuracy means

Accuracy is how close a measurement is to the actual true value of what is being measured. An accurate instrument reads 100.0 when the true value is 100.0; an inaccurate one reads something else, off by some error. Accuracy is about correctness — does the reading reflect reality? It is a property of how well the instrument is calibrated and free of bias, not of how finely it displays. An accurate instrument might display only to one decimal place and still be trustworthy, because the value it shows is genuinely close to the truth. Accuracy is ultimately what you care about when a measurement matters: a reading that is finely displayed but wrong is worse than useless, because it is wrong with an air of authority.

What resolution means

Resolution is the smallest change or increment an instrument can detect and display — the fineness of its scale. A scale with a resolution of 1 gram can only show whole grams; one with a resolution of 0.01 grams can show hundredths. A digital caliper reading to 0.01 mm has finer resolution than one reading to 0.1 mm. Resolution is about detail, not correctness: it tells you how small a difference the instrument can represent, regardless of whether what it represents is true. High resolution is useful — you cannot detect a 0.05 gram change with a 1 gram resolution scale — but it is a separate quality from accuracy. An instrument can have very fine resolution, displaying many decimal places, while being badly inaccurate, showing those fine decimals around a value that is simply wrong.

Correctness versus fineness

The clean distinction: accuracy is correctness, resolution is fineness. Accuracy asks is the reading close to the truth; resolution asks how small a change can the instrument show. They are independent — an instrument can be accurate but coarse (correct, but only to the nearest gram), or fine but inaccurate (showing hundredths of a gram, but all of them wrong by ten grams). The dangerous confusion is reading high resolution as high accuracy: a display full of decimal places looks authoritative and precise, and it is easy to trust it as correct. But those extra digits only tell you how finely the instrument can show a value, not whether that value is right. A four-decimal display around a biased reading is four decimals of wrongness.

A worked example

Two scales weigh a true 50.000 gram mass. Scale A has a resolution of 1 gram and reads 50 grams — coarse, but correct: accurate, low resolution. Scale B has a resolution of 0.001 grams and reads 53.247 grams — beautifully fine, but wrong by over three grams: high resolution, poor accuracy. An operator glancing at Scale B might trust it precisely because of all those decimal places, and be confidently wrong by three grams; Scale A, showing a plain 50, is actually telling the truth. The lesson is that the number of digits an instrument displays says nothing about whether they are right. Scale B needs calibration to fix its accuracy; its resolution was never the problem. Always ask whether a finely-displayed reading is actually correct, not just how many decimals it shows.

Why you need both

A trustworthy measurement needs adequate resolution and adequate accuracy — they are complementary, not interchangeable. Resolution must be fine enough for the job: you cannot measure a tolerance of plus or minus 0.05 mm with an instrument whose resolution is 0.1 mm, because it cannot even display the differences that matter. But fine resolution is worthless without accuracy: an instrument that resolves to 0.001 mm but is biased by 0.5 mm gives you precise-looking nonsense. The two set different requirements — resolution must match the smallest difference you need to see, accuracy must ensure what you see is true — and both have to be satisfied. Specifying an instrument means checking both: is it fine enough, and is it correct? Neither alone makes a measurement reliable.

Common mistakes

  • Reading resolution as accuracy. More decimal places do not mean a more correct reading.
  • Trusting the digits. A finely-displayed value can be confidently, authoritatively wrong.
  • Resolution too coarse for the tolerance. An instrument that cannot display the differences that matter cannot judge them.
  • Ignoring calibration. Resolution is built in, but accuracy drifts and must be maintained through calibration.

How it shows up in OEE

Like precision and accuracy, accuracy and resolution sit beneath the trustworthiness of the data feeding the quality factor of OEE. The quality factor counts good versus defective units, and that judgement depends on instruments that are both fine enough to see the relevant differences and accurate enough to read them truly. An instrument with coarse resolution might miss out-of-tolerance parts because it cannot show the difference; an inaccurate one might wrongly pass or scrap parts despite a fine display. Either way the OEE quality number becomes unreliable. Trustworthy measurement — adequate resolution and maintained accuracy through calibration — is the unglamorous foundation beneath honest OEE.

How Fabrico fits

Fabrico consumes the quality data your instruments produce, so their resolution and accuracy flow into the reliability of its OEE. By trending good-versus-defective results over time, it can help surface the symptoms of a measurement problem — reject rates that shift oddly around an instrument change, or scrap that does not match downstream reality — prompting a check of whether the gauge is both fine enough and correct. Reliable measurement upstream and honest OEE downstream go together. Book a demo to see trustworthy quality data drive trustworthy OEE.

Related reading

Frequently asked questions

What is the difference between accuracy and resolution?

Accuracy is how close a measurement is to the true value — is the reading correct? Resolution is the smallest change an instrument can detect or display — how fine is the detail? They are independent: an instrument can have high resolution (many decimals) yet poor accuracy (wrong values).

Does high resolution mean high accuracy?

No. High resolution means the instrument can display fine increments, but those increments can be centred on a wrong value. A gauge showing four decimal places can be badly inaccurate. Resolution is about fineness of display; accuracy is about correctness of the reading.

Can an instrument be accurate but have low resolution?

Yes. A scale with a resolution of 1 gram that reads the true value correctly is accurate but coarse. It tells the truth, just not in fine increments. Accuracy and resolution are separate qualities, and an instrument can have one without the other.

Why do you need both accuracy and resolution?

Resolution must be fine enough to show the differences that matter (you cannot measure a 0.05 mm tolerance with 0.1 mm resolution), and accuracy must ensure what you see is true. Fine resolution with poor accuracy gives precise-looking nonsense; both are required for a trustworthy measurement.

How do accuracy and resolution affect OEE?

The OEE quality factor counts good versus defective units, which depends on instruments fine enough to see relevant differences and accurate enough to read them truly. Coarse resolution can miss out-of-tolerance parts, and poor accuracy can wrongly pass or scrap them, making the quality number unreliable.

Latest from our blog

Încă te întrebi?
Verificați singuri!
Încă te întrebi?

Programați o întâlnire individuală cu experții noștri sau înscrieți-vă direct în planul nostru gratuit.
Nu este nevoie de card de credit!

By clicking the Accept button, you are giving your consent to the use of cookies when accessing this website and utilizing our services. To learn more about how cookies are used and managed, please refer to our Privacy Policy și Cookies Declaration