Sensors & Measurement · Study deck

Sensor Calibration: Math and Validation

A calibration equation is a rule for correcting a sensor reading.

Physics Phoebe is your guide for this deck.

sensorcalibration

After studying this chapter

Learning objectives

You will be able to:

Apply a calibration equation to raw sensor readings.
Validate corrected results with error and quality metrics.
Explain: With a 12-bit ADC (4096 levels) spanning the full 50 kg range, theoretical resolution would be 50/4096 = 12.2 g.
Explain: The window size (N) controls the tradeoff between smoothing and responsiveness.

Major section

Putting Numbers to It

With a 12-bit ADC (4096 levels) spanning the full 50 kg range, theoretical resolution would be 50/4096 = 12.2 g.

Adjusting amplification to use more of the ADC range would improve resolution.
Explore how ADC bit depth and measurement range affect the smallest detectable change (resolution per count).

Major section

Checkpoint: Calibration Math

The window size (N) controls the tradeoff between smoothing and responsiveness.

Larger windows produce smoother output but react more slowly to real changes.
Background: Sensors drift over time due to aging, temperature changes, and contamination.
Automatic Baseline Correction (ABC) can compensate by assuming the sensor occasionally sees a known reference (e.g., CO2 sensors assume 400ppm outdoor air).

Major section

Key Calibration Concepts Summary

To consolidate the lab into a repeatable workflow, inspect Figure: Mind map of key sensor calibration concepts covered before moving from calculation to field use.

The map connects error types, calibration methods, validation, and maintenance so none is mistaken for the whole job.
That order shows why a fitted gain and offset are only useful when their reference conditions, residual error, and recalibration trigger remain recorded.

Mind map of key sensor calibration concepts covered in this lab

Major section

Calibration Formulas

The worked load-cell example in Part 4 shows the same calculation with real numbers.

If the release test also checks just beyond the operating endpoints, include those limits in the reference plan.
The core lab above shows how to build and store a two-point calibration.
The companion page focuses on the failure modes that determine whether those coefficients are trustworthy in production.

Real photograph of sensor calibration laboratory

Deck summary

Key takeaways

With a 12-bit ADC (4096 levels) spanning the full 50 kg range, theoretical resolution would be 50/4096 = 12.2 g.

The window size (N) controls the tradeoff between smoothing and responsiveness.
To consolidate the lab into a repeatable workflow, inspect Figure: Mind map of key sensor calibration concepts covered before moving from calculation to field use.
The worked load-cell example in Part 4 shows the same calculation with real numbers.

Retrieval practice

Recall check 1 of 5

Physics Phoebe says: answer from memory, then check your reasoning.

Q1A corrected mass reading is 22 kg against a 20 kg reference. The allowed relative error is 5%. Does it pass?

ANo; 2 kg over the 20 kg reference is 10% relative error.

BYes; calibration makes every corrected reading pass tolerance.

CYes; a 2 kg difference is small compared with the reference mass.

DNo; any reading other than exactly 20 kg fails calibration.

Show answer

Answer: A |22 kg − 20 kg| = 2 kg; 2 kg / 20 kg × 100% = 10%, above the 5% limit.

Q2A sensor reads 20% when the true value is 10%, and reads 80% when the true value is 90%. What is the gain (slope) coefficient?

A0.75

B1.0

C4.0

D1.33

Show answer

Answer: D see answers page

Retrieval practice

Recall check 2 of 5

Physics Phoebe says: answer from memory, then check your reasoning.

Q3In the lab code, why is the moving average filter applied BEFORE calibration, not after?

AFiltering first makes every calibration operation execute faster.

BCalibration cannot operate on readings that have passed through a filter.

CFiltering first stabilizes the reference readings captured for calibration.

DThe order of filtering and calibration has no effect on reference values.

Show answer

Answer: C Filtering removes noise that would cause incorrect calibration captures When capturing reference points (pressing ‘l’ or ‘h’), you want a stable, accurate reading - not a noisy instantaneous sample.

Retrieval practice

Recall check 3 of 5

Physics Phoebe says: answer from memory, then check your reasoning.

Q4Why does automatic baseline correction (ABC) track the MINIMUM reading over 24 hours, rather than the average or maximum?

AMinimum values are more accurate

BMaximum values consume too much memory

CIt expects periodic exposure to the baseline

DAverages can't be stored in EEPROM

Show answer

Answer: C see answers page

Retrieval practice

Recall check 4 of 5

Physics Phoebe says: answer from memory, then check your reasoning.

Q5You need to calibrate a soil moisture sensor that will measure between 20% and 80% moisture in a greenhouse. Which reference points should you use?

AUse the 0% and 100% values as exact moisture standards because their endpoints define the calibration scale.

B50% and 60% standards concentrated in the middle of the expected range.

C25% and 75% standards, both inside rather than beyond the operating range.

D15% and 85% standards that bracket the 20–80% operating range.

Show answer

Answer: D 15% and 85% (bracketing the expected range) You want reference points that bracket (surround) your expected measurement range, but with some margin.

Retrieval practice

Recall check 5 of 5

Physics Phoebe says: answer from memory, then check your reasoning.

Q6A deployed outdoor temperature sensor gradually reads 1.5C above a traceable reference after 6 months. Checks with a known electrical input show the ADC and wiring path remain stable. What is the most likely cause and next step?

AThe ADC resolution degraded over time and needs replacement

BThe sensor wiring corroded and needs resoldering

CThe sensor has drifted due to aging and environmental exposure.

DThe microcontroller firmware has a memory leak causing numerical errors

Show answer

Answer: C A stable electrical path plus a verified reference makes sensor-element drift the leading explanation for a growing offset.

Print reference

Answers 1 of 3

Answer key.

A · |22 kg − 20 kg| = 2 kg; 2 kg / 20 kg × 100% = 10%, above the 5% limit.
D · 1.33 Using the formula: gain = (actual_high - actual_low) / (raw_high - raw_low) gain = (90 - 10) / (80 - 20) = 80 / 60 = 1.333. The sensor’s sensitivity is lower than expected (it only spans 60% when it should span 80%), so we multiply readings by 1.33 to “stretch” them to the correct range.

Print reference

Answers 2 of 3

Answer key.

C · Filtering removes noise that would cause incorrect calibration captures When capturing reference points (pressing ‘l’ or ‘h’), you want a stable, accurate reading - not a noisy instantaneous sample.
C · The sensor assumes it occasionally sees the baseline/reference condition For sensors like CO2 monitors, ABC assumes the sensor is exposed to “fresh air” (baseline ~400ppm) at least once during the 24-hour period - typically overnight when the room is empty.

Print reference

Answers 3 of 3

Answer key.

D · 15% and 85% (bracketing the expected range) You want reference points that bracket (surround) your expected measurement range, but with some margin.
C · A stable electrical path plus a verified reference makes sensor-element drift the leading explanation for a growing offset.