Phillip Stanley-Marbell

Foundations of Embedded Systems

Department of Engineering, University of Cambridge

http://physcomp.eng.cam.ac.uk

Topic 02: Precision, Accuracy, and Sensor Measurement Uncertainty

(~45 minutes)

Version 0.2020

(Video)

26

Intended Learning Outcomes for This Topic

2

➌ Deﬁne the principal kinds of measurement uncertainty

By the end of this topic, you will be able to:

➋ Deﬁne the essential components of a measurement

➍ Demonstrate how uncertainty in sensor data aﬀects embedded software

➊ Deﬁne precision, accuracy, and reliability

➎ Derive / propagate uncertainties through arithmetic operations on sensor data

26

Three Key Components of a Measurement

3

The measurement

instrument or sensor

The phenomenon being

measured: the measurand

The environment

(e.g., ambient vibrations, temperature, etc.)

The measurement is what

comes out of the

measurement instrument

Environment

➊

Measurement

Instrument

➍

➌

➋

26

Precision

4

Repeatability or ﬁneness of control

More preciseLess precise

26

Accuracy

5

Diﬀerence from correct value

Example: A sensor value can be measured with great precision (repeatability and

resolution), but the sensed value may differ from the actual value of the signal

More accurate

Less accurate

More precise

Less precise

26

Faults, Masking, Errors, Erasures

6

Hardware or software defect.

A fault that is not masked (and is hence visible)

Example: a signal shorted to ground when it should not be

When a value of a signal is the same as the value induced by

a fault, the fault is said to be masked

A fault whose value is different from any valid signal value

Fault or failure:

Masking:

Error:

Erasure:

26

Sources of Errors and Erasures

7

Microprocessor

LD @(R4), R2

ADD R5, R6

SHRL, R4, #8

Program:

λx.+2x

Temperature

Fluctuations

Electrical Noise

Circuit state disturbance inducement

26

Sources of Errors and Erasures: Soft Errors

8

Radioactive Decay of

238

U and

232

Th from

device packaging mold resin,

210

Po from

PbSn solder (and Al wire)

12

C

α-particles

γ- rays

Lithium

Cosmic rays Thermal neutrons

High energy neutron

(can penetrate up to 5

ft of concrete)

Neutron capture within Si

and B in integrated circuits

Unstable isotope

Magnesium

or

Possible interaction paths

Circuit state disturbance inducement

Microprocessor

–

+

–

+

Temperature

Fluctuations

}

LD @(R4), R2

ADD R5, R6

SHRL, R4, #8

Program:

λx.+2x

?

Electrical Noise

High-Energy Particles

26

Sources of Errors and Erasures: Noise

9

Thermal / Johnson-Nyquist

Noise

Possible interaction paths

Circuit state disturbance inducement

Microprocessor

Temperature

Fluctuations

LD @(R4), R2

ADD R5, R6

SHRL, R4, #8

Program:

λx.+2x

?

Shot Noise

“Flicker” / 1/f Noise

Random Telegraph Noise

26

Some Common Classiﬁcations of Errors

10

Random Errors

Systematic Errors

Epistemic Uncertainty

Aleatoric Uncertainty

Type A Uncertainty

Type B Uncertainty

Errors that vary over time (e.g., due to noise)

Errors that are ﬁxed over time (e.g., an offset)

Uncertainty in a measurements due to insuﬃcient (or no) information

Uncertainty in a measurement due to random errors

Uncertainty quantiﬁed by (statistical) analysis of measurand

Uncertainty speciﬁed as properties of sensor, independent of measurand

▶︎

26

Measurement Distributions for an Actual Sensor (Bosch BMX055)

11

   









-  ()



Mean: 135.421, Stdev.: 2.4002, Significant Digits: 1

       















-  ()



- - - - -











-  ()



       













-  ()



       















  



1800 mV

1900 mV

2000 mV

2100 mV

2200 mV

2300 mV

2400 mV

2500 mV

Histogram of 100

readings of x-axis

acceleration data

from BMX055

operating at 2.5V

Accelerometer

Gyroscope

Magnetometer

Source: BMX055 datasheet

26

x-component analysis

y-component analysis

z-component analysis

Pedometer / Step Counting System

Low-

Pass

Filter

Maximum-

Activity

Axis

Selection

Step

Count

Extremal

Value

Marking

Processor

Accelerometer

Eﬀect of Noise in End-to-End Applications: Pedometer

12

(Source: iﬁtxit.com)

ARM Cortex M3 Microcontroller

Bluetooth Low-Energy IC

Accelerometer IC

(Source: Fitbit)

‡

Pedometer algorithm: N. Zhao. “Full-Featured Pedometer Design Realized with 3-Axis Digital Accelerometer”. Analog Dialogue, 44(06), June 2010.

➊ ➋ ➌

‡

26

Maximal activity axis

     









 ()



([- ]   )

Example 1: Add Noise By Zeroing 5% of Samples

13

    









 ()



([- ]   )

Unperturbed accelerometer data

‡

for one axis

Maximal activity axis

➊

Data with induced errors for 5% of samples

0, 2.03, -2.72, -5.41, -3.72, 3.11, …

9.81, 15.51, 14.75, 8.28, 7.55, 8.08, …

-0.84, 0.04, -3.17, -3.72, 0.27, 0, …

x

y

z

1.54, 2.03, -2.72, -5.41, -3.72, 3.11, …

9.81, 15.51, 14.75, 8.28, 7.55, 8.08, …

-0.84, 0.04, -3.17, -3.72, 0.27, 0.8, …

x

y

z

Unperturbed accelerometer data

‡

for all three axes

Data with induced errors for 5% of samples

‡

3-axis accelerometer data from WISDM activity recognition dataset: J. R. Kwapisz, et al. “Activity recognition using cell phone accelerometers”. SIGKDD Explor. Newsl., 12(2):74– 82, Mar. 2011.

26

Maximal activity axis

     









 ()



([- ]   )

     

















 ()



([- ]   )

Example 1: Add Noise By Zeroing 5% of Samples

14

    









 ()



([- ]   )

     

















 ()



([- ]   )

     

















 ()



([- ]   )

Unperturbed accelerometer data

‡

for one axis

Reports 19 steps

Reports 19 steps!

Maximal activity axis

Low-pass ﬁlter

Extremal-value marking

➊

➋

➌

Low-pass ﬁlter

Extremal-value marking

➊

➋

➌

Data with induced errors for 5% of samples

     

















 ()



([- ]   )

‡

3-axis accelerometer data from WISDM activity recognition dataset: J. R. Kwapisz, G. M. Weiss, and S. A. Moore. “Activity recognition using cell phone accelerometers”. SIGKDD Explor. Newsl., 12(2):74– 82, Mar. 2011.

26

15

What else can we do

to estimate eﬀect of noise/uncertainty,

other than just “run noise through programs” ?

26

Quantifying Effect of Noise: Propagating Uncertainties

1 of 5

16

y = f(x

1

,...,x

n

).

y = f(x

1

,...,x

n

),

First, we assume that if

then

26

Quantifying Effect of Noise: Propagating Uncertainties

2 of 5

17

f(x

1

,...,x

n

)=f(x

1

,...,x

n

)+

∂f(

x

1

,...,x

n

)

∂x

1

(x

1

−

x

1

)+...+

∂f(

x

The Taylor series expansion of

f(x)=f(a)+

f

0

(a)(x − a)

1!

+

f

00

(a)(x − a)

2

2!

+ ...

Next, take Taylor series expansion of about

Recall, that the Taylor series expansion of

f(x) about a point a is

= f(x

1

,...,x

n

).

f(x

1

,...,x

n

)

= f(x

1

,...,x

n

).

f(x

1

,...,x

n

),

about :

Note: see what happens to this term on next slide

26

Quantifying Effect of Noise: Propagating Uncertainties

3 of 5

18

Substituting ➊ into ➋:

(y − y)=

∂f(

x

1

,...,x

n

)

∂x

1

(x

1

−

x

1

)+...+

∂f(

x

1

,...,x

n

)

∂x

n

(x

n

−

x

n

)+....

σ

2

y

=lim

N→∞

2

4

1

N

X

j=1

(y

j

−

y)

2

3

5

From the assumption

y = f(x

1

,...,x

n

),

, we have

Now, from the deﬁnition of variance,

➊

➋

Simplifying:

σ

2

y

' lim

N→∞

1

N

X

j=1

n

X

i=1

∂f(

x

1

,...,x

n

)

∂x

i

(x

i



x

i

)

!

2

.

<latexit sha1_base64="LPIIx3wLU92a+H2RXhD+af6e7+c=">AAACq3icdVFdb9MwFHXC11a+CjzyYlEhtRKL4jLYeECa4IWnMQTdCnUXOa7Tmvkj2A4sivzr+AU88m9wmiI2BFeyfXTuubrH9+al4Nal6c8ovnL12vUbW9u9m7du37nbv3f/2OrKUDahWmgzzYllgis2cdwJNi0NIzIX7CQ/e93mT74yY7lWH1xdsrkkS8ULTokLVNb/vo0tX0qS1adjGKBkXyAWXGbNIcSGL1eOGKO/QcxV4WqPC0Nog3xz6LGtgurzS+RPg1Swwg07irdUozZaXBLjOBGwGGIdnLRGm/MM+SehaKGdDe8FXvmR/1NznnE/DBfcuSTiftR5GwXTSS/rD9Ikfb6LxgimybMU7Y/TDrzYewpRkq5jADZxlPV/4IWmlWTKUUGsnaG0dPOmbUoF8z1cWVYSekaWbBagIpLZebMetoePA7OAhTbhKAfX7MWKhkhra5kHpSRuZf/OteS/crPKFfvzhquyckzRrlFRCeg0bDcHF9ww6kQdAKGGB6+QrkiYsQv7DTP4/VH4f3A8TlDA73YHB68209gCD8EjMAQI7IED8AYcgQmg0Sh6G02jj/FO/D7+FONOGkebmgfgUsTsF4xY1K4=</latexit>

σ

2

y

'

✓

∂f(

x

1

,...,x

n

)

∂x

1

◆

2

σ

2

x

1

+

✓

∂f(

x

1

,...,x

n

)

∂x

2

◆

2

σ

2

x

2

+ ...+2σ

2

x

1

x

2

✓

∂f(

x

1

,...,x

n

)

∂x

1

◆✓

∂f(

x

1

∂

<latexit sha1_base64="Cn7mORBm37cU4c3tXmzTCCNDiTk=">AAADo3ictVJbb9MwFHYTLlu5dfDIi6Ga1GkocsJg422CB5B46dC6TWpC5LhOa82xg+0gqih/jH/BI/+Gk6ZoBY03dqQ4n79z+XyOTlZKYR0hP3uef+v2nbtb2/179x88fDTYeXxmdWUYnzAttbnIqOVSKD5xwkl+URpOi0zy8+zyXes//8qNFVqdumXJk4LOlcgFow6odKf3fTu2Yl7QdPk5wgAL/gXHkuduFOeGsjouqXGCSpyPYg2VWqH6Wxo2LyBspp2F/wavmr3mKqeNi42YL9xeV70VWmXDFWy/Pf6jWnStWtSp7a9LAIg2ntL5b6Llm2jsqougnw6GJCCvD8IoxCR4RcKjiHTgzeFLHAZkZUO0tnE6+BHPNKsKrhyT1NppSEqX1K0Kk7zpx5XlJWWXdM6nABUtuE3q1ZI1eBeYGc61gU85vGI3M2paWLssMogsqFvYv30teZ1vWrn8KKmFKivHFeuE8kpip3G7sXgmDGdOLgFQZgS8FbMFhaE62GuYwe9G8b/BWRSEgE8Ohsdv19PYQk/RczRCITpEx+gDGqMJYt4z77039k78Xf+j/8k/7UK93jrnCfrD/OQXY8ctHQ==</latexit>

26

Quantifying Effect of Noise: Propagating Uncertainties

4 of 5

19

σ

2

y

'

✓

∂y

∂x

1

◆

2

σ

2

x

1

+

✓

∂y

∂x

2

◆

2

σ

2

x

2

+ ...+2σ

2

x

1

x

2

∂y

∂x

1

∂y

∂x

2

+ ....

So we have

When the parameters are uncorrelated:

f(x

1

,...,x

n

)

σ

2

y

'

n

X

i=1

✓

∂f(x

1

,...,x

n

)

∂x

i

◆

2

σ

2

x

i

.

<latexit sha1_base64="kuzvNPSQsCgPeVwC7+T2mGbdEis=">AAACcXicdVHLbhMxFPUMrza8AogFQgKrEVKqopGdtrQskCrYsCwSaSvF6cjjeBKrfkxtT9XImg/g87rsV7DgB/AkQRQEV7Lu8bn3+tjHRSWF8whdJ+mt23fu3ltb79x/8PDR4+6Tp0fO1JbxITPS2JOCOi6F5kMvvOQnleVUFZIfF2ef2vrxBbdOGP3Vzys+VnSqRSkY9ZHKu9/WiRNTRfP56QBGqPh5TLXKg/iAm9OgGyJ56fuktJQFUlHrBZWw7BMTj21Vw2WOm7eQyInxLuYbvG42m98zl7loiBXTmd9carWyoWXjNuvk3R7K0LsdPMAQZbsI7w/QErzf24Y4Q4vogVUc5t0rMjGsVlx7JqlzI4wqPw6tGpO86ZDa8YqyMzrlowg1VdyNw8KyBr6JzASWxsalPVywNycCVc7NVRE7FfUz93etJf9VG9W+3B8Hoavac82WQmUtoTew9R9OhOXMy3kElFkR7wrZjEZzffyl6MGvh8L/g6NBhiP+stM7+LhyYw28BBugDzDYAwfgMzgEQ8DA9+R58ip5nfxIX6Qw3Vi2pslq5hn4I9Ktn5g+vmU=</latexit>

26

Quantifying Effect of Noise: Propagating Uncertainties

5 of 5

20

If y = ax

1

+ bx

2

,then

σ

2

y

' a

2

σ

2

x

1

+ b

2

σ

2

x

2

+2abσ

2

x

1

x

2

.

If y = ax

1

x

2

,then

σ

2

y

2

'

σ

2

x

1

x

2

1

+

σ

2

x

2

x

2

+2

σ

2

x

1

x

2

x

1

x

2

.

Some special cases:

If y = a

bx

1

then

σ

y

' (b ln a)σ

x

1

26

x-component analysis

y-component analysis

z-component analysis

Pedometer / Step Counting System

Low-

Pass

Filter

Maximum-

Activity

Axis

Selection

Step

Count

Extremal

Value

Marking

Processor

Accelerometer

Eﬀect of Noise in End-to-End Applications: Pedometer

21

(Source: iﬁtxit.com)

ARM Cortex M3 Microcontroller

Bluetooth Low-Energy IC

Accelerometer IC

(Source: Fitbit)

‡

Pedometer algorithm: N. Zhao. “Full-Featured Pedometer Design Realized with 3-Axis Digital Accelerometer”. Analog Dialogue, 44(06), June 2010.

➊ ➋ ➌

‡

Recap

26

Maximal activity axis

     









 ()



([- ]   )

Example 1: Add Noise By Zeroing 5% of Samples

22

    









 ()



([- ]   )

Unperturbed accelerometer data

‡

for one axis

Maximal activity axis

➊

Data with induced errors for 5% of samples

0, 2.03, -2.72, -5.41, -3.72, 3.11, …

9.81, 15.51, 14.75, 8.28, 7.55, 8.08, …

-0.84, 0.04, -3.17, -3.72, 0.27, 0, …

x

y

z

1.54, 2.03, -2.72, -5.41, -3.72, 3.11, …

9.81, 15.51, 14.75, 8.28, 7.55, 8.08, …

-0.84, 0.04, -3.17, -3.72, 0.27, 0.8, …

x

y

z

Unperturbed accelerometer data

‡

for all three axes

Data with induced errors for 5% of samples

‡

3-axis accelerometer data from WISDM activity recognition dataset: J. R. Kwapisz, et al. “Activity recognition using cell phone accelerometers”. SIGKDD Explor. Newsl., 12(2):74– 82, Mar. 2011.

Recap

26

Quantifying Effect of Noise: Propagating Uncertainties

1 of 5

23

y = f(x

1

,...,x

n

).

y = f(x

1

,...,x

n

),

First, we assume that if

then

Recap

26

Quantifying Effect of Noise: Propagating Uncertainties

4 of 5

24

σ

2

y

'

✓

∂y

∂x

1

◆

2

σ

2

x

1

+

✓

∂y

∂x

2

◆

2

σ

2

x

2

+ ...+2σ

2

x

1

x

2

∂y

∂x

1

∂y

∂x

2

+ ....

So we have

When the parameters are uncorrelated:

f(x

1

,...,x

n

)

Recap

σ

2

y

'

n

X

i=1

✓

∂f(x

1

,...,x

n

)

∂x

i

◆

2

σ

2

x

i

.

<latexit sha1_base64="kuzvNPSQsCgPeVwC7+T2mGbdEis=">AAACcXicdVHLbhMxFPUMrza8AogFQgKrEVKqopGdtrQskCrYsCwSaSvF6cjjeBKrfkxtT9XImg/g87rsV7DgB/AkQRQEV7Lu8bn3+tjHRSWF8whdJ+mt23fu3ltb79x/8PDR4+6Tp0fO1JbxITPS2JOCOi6F5kMvvOQnleVUFZIfF2ef2vrxBbdOGP3Vzys+VnSqRSkY9ZHKu9/WiRNTRfP56QBGqPh5TLXKg/iAm9OgGyJ56fuktJQFUlHrBZWw7BMTj21Vw2WOm7eQyInxLuYbvG42m98zl7loiBXTmd9carWyoWXjNuvk3R7K0LsdPMAQZbsI7w/QErzf24Y4Q4vogVUc5t0rMjGsVlx7JqlzI4wqPw6tGpO86ZDa8YqyMzrlowg1VdyNw8KyBr6JzASWxsalPVywNycCVc7NVRE7FfUz93etJf9VG9W+3B8Hoavac82WQmUtoTew9R9OhOXMy3kElFkR7wrZjEZzffyl6MGvh8L/g6NBhiP+stM7+LhyYw28BBugDzDYAwfgMzgEQ8DA9+R58ip5nfxIX6Qw3Vi2pslq5hn4I9Ktn5g+vmU=</latexit>

26