A First Course in Electrical and Computer Engineering

By: Louis Scharf

Online: <http://cnx.org/content/col10685/1.2>

This selection and arrangement of content as a collection is copyrighted by Louis Scharf.

It is licensed under the Creative Commons Attribution License: http://creativecommons.org/licenses/by/3.0/

Collection structure revised: 2009/09/14

For copyright and attribution information for the modules contained in this collection, see the "Attributions" section at the end of the collection.

Dedication of "A First Course in Electrical and Computer Engineering"

         Louis Scharf dedicates this book to
    his wife Carol, son Greg, and daughter Heidi,
              for love and inspiration;
          his parents Louis Sr. and Ann,
 in celebration of their 50th wedding anniversary.

        Richard Behrens dedicates this book to
       his wife Debbie, and child as yet unborn,
             for love and encouragement;
            his parents Richard and Elsie,
          in gratitude for a good education.

Preface to "A First Course in Electrical and Computer Engineering"

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

This book was written for an experimental freshman course at the University of Colorado. The course is now an elective that the majority of our electrical and computer engineering students take in the second semester of their freshman year, just before their first circuits course. Our department decided to offer this course for several reasons:

we wanted to pique student' interest in engineering by acquainting them with engineering teachers early in their university careers and by providing with exposure to the types of problems that electrical and computer engineers are asked to solve;

we wanted students entering the electrical and computer engineering programs to be prepared in complex analysis, phasors, and linear algebra, topics that are of fundamental importance in our discipline;

we wanted students to have an introduction to a software application tool, such as MATLAB, to complete their preparation for practical and efficient computing in their subsequent courses and in their professional careers;

we wanted students to make early contact with advanced topics like vector graphics, filtering, and binary coding so that they would gain a more rounded picture of modern electrical and computer engineering.

In order to introduce this course, we had to sacrifice a second semester of Pascal programming. We concluded that the sacrifice was worth making because we found that most of our students were prepared for high-level language computing after just one semester of programming.

We believe engineering educators elsewhere are reaching similar conclusions about their own students and curriculums. We hope this book helps create a much needed dialogue about curriculum revision and that it leads to the development of similar introductory courses that encourage students to enter and practice our craft.

Students electing to take this course have completed one semester of calculus, computer programming, chemistry, and humanities. Concurrently with this course, students take physics and a second semester of calculus, as well as a second semester in the humanities. By omitting the advanced topics marked by asterisks, we are able to cover Complex Numbers through Linear Algebra, plus two of the three remaining chapters. The book is organized so that the instructor can select any two of the three. If every chapter of this book is covered, including the advanced topics, then enough material exists for a two-semester course.

The first three chapters of this book provide a fairly complete coverage of complex numbers, the functions ex and e jθ , and phasors. Our department philosophy is that these topics must be understood if a student is to succeed in electrical and computer engineering. These three chapters may also be used as a supplement to a circuits course. A measured pace of presentation, taking between sixteen and eighteen lectures, is sufficient to cover all but the advanced sections in Complex Numbers through Phasors.

The chapter on "linear algebra" is prerequisite for all subsequent chapters. We use eight to ten lectures to cover it. We devote twelve to sixteen lectures to cover topics from Vector Graphics through Binary Codes. (We assume a semester consisting of 42 lectures and three exams.) The chapter on vector graphics applies the linear algebra learned in the previous chapter to the problem of translating, scaling, and rotating images. "Filtering" introduces the student to basic ideas in averaging and filtering. The chapter on "Binary Codes" covers the rudiments of binary coding, including Huffman codes and Hamming codes.

If the users of this book find "Vector Graphics" through "Binary Codes" too confining, we encourage them to supplement the essential material in "Complex Numbers" through "Linear Algebra" with their own course notes on additional topics. Within electrical and computer engineering there are endless possibilities. Practically any set of topics that can be taught with conviction and enthusiasm will whet the student's appetite. We encourage you to write to us or to our editor, Tom Robbins, about your ideas for additional topics. We would like to think that our book and its subsequent editions will have an open architecture that enables us to accommodate a wide range of student and faculty interests.

Throughout this book we have used MATLAB programs to illustrate key ideas. MATLAB is an interactive, matrix-oriented language that is ideally suited to circuit analysis, linear systems, control theory, communications, linear algebra, and numerical analysis. MATLAB is rapidly becoming a standard software tool in universities and engineering companies. (For more information about MATLAB, return the attached card in the back of this book to The MathWorks, Inc.) MATLAB programs are designed to develop the student's ability to solve meaningful problems, compute, and plot in a high-level applications language. Our students get started in MATLAB by working through “An Introduction to MATLAB,” while seated at an IBM PC (or look-alike) or an Apple Macintosh. We also have them run through the demonstration programs in "Complex Numbers". Each week we give three classroom lectures and conduct a one-hour computer lab session. Students use this lab session to hone MATLAB skills, to write programs, or to conduct the numerical experiments that are given at the end of each chapter. We require that these experiments be carried out and then reported in a short lab report that contains (i) introduction, (ii) analytical computations, (iii) computer code, (iv) experimental results, and (v) conclusions. The quality of the numerical results and the computer graphics astonishes students. Solutions to the chapter problems are available from the publisher for instructors who adopt this text for classroom use.

We wish to acknowledge our late colleague Richard Roberts, who encouraged us to publish this book, and Michael Lightner and Ruth Ravenel, who taught "Linear Algebra" and "Vector Graphics" and offered helpful suggestions on the manuscript. We thank C. T. Mullis for allowing us to use his notes on binary codes to guide our writing of "Binary Codes". We thank Cédric Demeure and Peter Massey for their contributions to the writing of "An Introduction to MATLAB" and "The Edix Editor". We thank Tom Robbins, our editor at Addison-Wesley, for his encouragement, patience, and many suggestions. We are especially grateful to Julie Fredlund, who composed this text through many drafts and improved it in many ways. We thank her for preparing an excellent manuscript for production.

L. L. Scharf
R. T. Behrens
Boulder, Colorado

1. To the Teacher:

An incomplete understanding of complex numbers and phasors handicaps students in circuits and electronics courses, and even more so in advanced courses such as electromagnetics. optics, linear systems, control, and communication systems. Our faculty has decided to address this problem as early as possible in the curriculum by designing a course that drills complex numbers and phasors into the minds of beginning engineering students. We have used power signals, musical tones, Lissajous figures, light scattering, and RLC circuits to illustrate the usefulness of phasor calculus. "Linear Algebra" through "Binary Codes" introduce students to a handful of modern ideas in electrical and computer engineering. The motivation is to whet students' appetites for more advanced problems. The topics we have chosen – linear algebra, vector graphics, filtering, and binary codes – are only representative.

At first glance, many of the equations in this book look intimidating to beginning students. For this reason, we proceed at a very measured pace. In our lectures, we write out in agonizing detail every equation that involves a sequence or series. For example, the sum is written out as

(1) 1 + z + z ² + ⋯ + z N – 1 ,

and then it is evaluated for some specific value of z before we derive the analytical result Similarly, an infinite sequence like is written out as

(2)

and then it is evaluated for some specific x and for several values of n before the limit is derived. We try to preserve this practice of pedantic excess until it is clear that every student is comfortable with an idea and the notation for coding the idea.

2. To the Student:

These are exciting times for electrical and computer engineering. To celebrate its silver anniversary, the National Academy of Engineering announced in February of 1990 the top ten engineering feats of the previous twenty-five years. The Apollo moon landing, a truly Olympian and protean achievement, ranked number one. However, a number of other achievements in the top ten were also readily identifiable as the products of electrical and computer engineers:

communication and remote sensing satellites,

the microprocessor,

computer-aided design and manufacturing (CADCAM),

computerized axial tomography (CAT scan),

lasers, and

fiber optic communication.

As engineering students, you recognize these achievements to be important milestones for humanity; you take pride in the role that engineers have played in the technological revolution of the twentieth century.

So how do we harness your enthusiasm for the grand enterprise of engineering? Historically, we have enrolled you in a freshman curriculum of mathematics, science, and humanities. If you succeeded, we enrolled you in an engineering curriculum. We then taught you the details of your profession and encouraged your faith that what you were studying is what you must study to be creative and productive engineers. The longer your faith held, the more likely you were to complete your studies. This seems like an imperious approach to engineering education, even though mathematics, physics, and the humanities are the foundation of engineering, and details are what form the structure of engineering. It seems to us that a better way to stimulate your enthusiasm and encourage your faith is to introduce you early in your studies to engineering teachers who will share their insights about some of the fascinating advanced topics in engineering, while teaching you the mathematical and physical principles of engineering. But you must match the teacher's commitment with your own commitment to study. This means that you must attend lectures, read texts, and work problems. You must be inquisitive and skeptical. Ask yourself how an idea is limited in scope and how it might be extended to apply to a wider range of problems. For, after all, one of the great themes of engineering is that a few fundamental ideas from mathematics and science, coupled with a few principles of design, may be applied to a wide range of engineering problems. Good luck with your studies.

Chapter 1. Complex Numbers

1.1. Complex Numbers: Introduction*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

Notes to Teachers and Students:

When we teach complex numbers to beginning engineering students, we encourage a geometrical picture supported by an algebraic structure. Every algebraic manipulation carried out in a lecture is accompanied by a care-fully drawn picture in order to fix the idea that geometry and algebra go hand-in-glove to complete our understanding of complex numbers. We assign essentially every problem for homework.

We use the MATLAB programs in this chapter to illustrate the theory of complex numbers and to develop skill with the MATLAB language. The numerical experiment introduces students to the basic quadratic equation of electrical and computer engineering and shows how the roots of this quadratic equation depend on the coefficients of the equation.

“Representing Complex Numbers in a Vector Space,” is a little demanding for freshmen but easily accessible to sophomores. It may be covered for additional insight, skipped without consequence, or covered after Chapter 4. “An Electric Field Computation,” is well beyond most freshmen, and it is demanding for sophomores. Nonetheless, an expert in electromagnetics might want to cover the section "An electric Field Computation" for the insight it brings to the use of complex numbers for representing two-dimensional real quantities.

Introduction

It is hard to overestimate the value of complex numbers. They first arose in the study of roots for quadratic equations. But, as with so many other great discoveries, complex numbers have found widespread application well outside their original domain of discovery. They are now used throughout mathematics, applied science, and engineering to represent the harmonic nature of vibrating systems and oscillating fields. For example, complex numbers may be used to study

traveling waves on a sea surface;

standing waves on a violin string;

the pure tone of a Kurzweil piano;

the acoustic field in a concert hall;

the light of a He-Ne laser;

the electromagnetic field in a light show;

the vibrations in a robot arm;

the oscillations of a suspension system;

the carrier signal used to transmit AM or FM radio;

the carrier signal used to transmit digital data over telephone lines; and

the 60 Hz signal used to deliver power to a home.

In this chapter we develop the algebra and geometry of complex numbers. In Chapter 3 we will show how complex numbers are used to build phasor representations of power and communication signals.

1.2. Complex Numbers: Geometry of Complex Numbers*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

The most fundamental new idea in the study of complex numbers is the “imaginary number” j . This imaginary number is defined to be the square root of – 1 :

(1.1)

(1.2) j ² = – 1 .

The imaginary number j is used to build complex numbers x and y in the following way:

(1.3) z = x + j y .

We say that the complex number z has “real part” x and “imaginary part” y :

(1.4) z = Re [ z ] + j Im [ z ]

(1.5)

In MATLAB, the variable x is denoted by real(z), and the variable y is denoted by imag(z). In communication theory, x is called the “in-phase” component of z , and y is called the “quadrature” component. We call z = x + j y the Cartesian representation of z , with real component x and imaginary component y . We say that the Cartesian pair (x,y) codes the complex number z .

We may plot the complex number z on the plane as in Figure 1.1. We call the horizontal axis the “real axis” and the vertical axis the “imaginary axis.” The plane is called the “complex plane.” The radius and angle of the line to the point z = x + j y are

(1.6)

(1.7)

See Figure 1.1. In MATLAB, r is denoted by abs(z), and θ is denoted by angle(z).

Figure 1.1. 

Cartesian and Polar Representations of the Complex Number z

The original Cartesian representation is obtained from the radius r and angle θ as follows:

(1.8) x = rcos θ

(1.9)

The complex number z may therefore be written as

(1.10)

The complex number cosθ + jsinθ is, itself, a number that may be represented on the complex plane and coded with the Cartesian pair (cosθ,sin θ) . This is illustrated in Figure 1.2. The radius and angle to the point z = cosθ + jsin θ are 1 and θ . Can you see why?

Figure 1.2. 

The Complex Number

The complex number cosθ + jsinθ is of such fundamental importance to our study of complex numbers that we give it the special symbol e jθ :

(1.11) e jθ = cosθ + jsinθ.

As illustrated in Figure 1.2, the complex number e jθ has radius 1 and angle θ . With the symbol e jθ , we may write the complex number z as

(1.12) z = r e jθ .

We call z = r e jθ a polar representation for the complex number z . We say that the polar pair r∠θ codes the complex number z . In this polar representation, we define |z| = r to be the magnitude of z and arg(z) = θ to be the angle, or phase, of z :

(1.13) | z | = r

(1.14)

With these definitions of magnitude and phase, we can write the complex number z as

(1.15) z = |z|e jarg(z).

Let's summarize our ways of writing the complex number z and record the corresponding geometric codes:

(1.16)

In "Roots of Quadratic Equations" we show that the definition e jθ = cosθ + jsinθ is more than symbolic. We show, in fact, that e jθ is just the familiar function e x evaluated at the imaginary argument x = j θ . We call e jθ a “complex exponential,” meaning that it is an exponential with an imaginary argument.

Exercise 1.2.1.

Prove (j)2n = ( – 1) n and (j)2n + 1 = ( – 1) n j . Evaluate j ³,j ⁴,j ⁵ .

Exercise 1.2.2.

Prove e j[(π / 2) + m2π] = j,e j[(3π / 2) + m2π] = – j,e j(0 + m2π) = 1, and e j(π + m2π) = – 1. Plot these identities on the complex plane. (Assume m is an integer.)

Exercise 1.2.3.

Find the polar representation z = r e jθ for each of the following complex numbers:

z = 1 + j0;

z = 0 + j1;

z = 1 + j1;

z = – 1 – j1.

Plot the points on the complex plane.

Exercise 1.2.4.

Find the Cartesian representation z = x + j y for each of the following complex numbers:

;

;

z = e j3π / 4 ;

.

Plot the points on the complex plane.

Exercise 1.2.5.

The following geometric codes represent complex numbers. Decode each by writing down the corresponding complex number z:

?

?

?

?

Exercise 1.2.6.

Show that Im[j z] = Re [z] and Re [ – j z] = Im [z]. Demo 1.1 (MATLAB). Run the following MATLAB program in order to compute and plot the complex number e jθ for θ = i2π / 360,i = 1,2,...,360:

          j=sqrt(-1)
          n=360
          for i=1:n,circle(i)=exp(j*2*pi*i/n);end;
          axis('square')
          plot(circle)

Replace the explicit for loop of line 3 by the implicit loop

          circle=exp(j*2*pi*[1:n]/n);

to speed up the calculation. You can see from Figure 1.3 that the complex number e jθ ,

evaluated at angles θ = 2π / 360,2(2π / 360),..., turns out complex numbers that lie at angle θ and radius 1. We say that e jθ is a complex number that lies on the unit circle. We will have much more to say about the unit circle in Chapter 2.

Figure 1.3. 

The Complex Numbers e jθ for 0 ≤ θ ≤ 2π (Demo 1.1)

1.3. Complex Numbers: Algebra of Complex Numbers*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

The complex numbers form a mathematical “field” on which the usual operations of addition and multiplication are defined. Each of these operations has a simple geometric interpretation.

Addition and Multiplication.

The complex numbers z ₁ and z ₂ are

added according to the rule

(1.17)

We say that the real parts add and the imaginary parts add. As illustrated in Figure 1.4, the complex number z ₁ + z ₂ is computed from a “parallelogram rule,” wherein z ₁ + z ₂ lies on the node of a parallelogram formed from z ₁ and z ₂ .

Exercise 1.3.1.

Let z ₁ = r ₁ e j θ ₁ and z ₂ = r ₂ e j θ ₂ . Find a polar formula z ₃ = r ₃ e j θ ₃ for z ₃ = z ₁ + z ₂ that involves only the variables r ₁,r ₂,θ ₁ , and θ ₂ . The formula for r ₃ is the “law of cosines.”

The product of z₁ and z₂ is

(1.18)

Figure 1.4. 

Adding Complex Numbers

If the polar representations for z ₁ and z ₂ are used, then the product may be written as [1]

(1.19)

We say that the magnitudes multiply and the angles add. As illustrated in Figure 1.5, the product z ₁ z ₂ lies at the angle .

Figure 1.5. 

Multiplying Complex Numbers

Rotation. There is a special case of complex multiplication that will become very important in our study of phasors in the chapter on Phasors. When z ₁ is the complex number z ₁ = r ₁ e j θ ₁ and z ₂ is the complex number z ₂ = e j θ ₂ , then the product of z ₁ and z ₂ is

(1.20)

As illustrated in Figure 1.6, z ₁ z ₂ is just a rotation of z ₁ through the angle θ ₂ .

Figure 1.6. 

Rotation of Complex Numbers

Exercise 1.3.2.

Begin with the complex number z ₁ = x + j y = r e jθ . Compute the complex number z ₂ = j z ₁ in its Cartesian and polar forms. The complex number z ₂ is sometimes called perp. Explain why by writing perp as z ₁ e j θ ₂ . What is θ ₂ ? Repeat this problem for z ₃ = – j z ₁ .

Powers. If the complex number z ₁ multiplies itself N times, then the result is

(1.21)

This result may be proved with a simple induction argument. Assume z k ₁ = r k ₁ e j k θ ₁ . (The assumption is true for k = 1.) Then use the recursion . Iterate this recursion (or induction) until k + 1 = N . Can you see that, as n ranges from n = 1,...,N , the angle of z§ranges from θ₁ to 2θ ₁,..., to N θ ₁ and the radius ranges from r₁ to r ² ₁,..., to r N ₁ ? This result is explored more fully in Problem 1.19.

Complex Conjugate. Corresponding to every complex number z = x + j y = r e jθ is the complex conjugate

(1.22) z * = x – j y = r e – j θ .

The complex number z and its complex conjugate are illustrated in Figure 1.7. The recipe for finding complex conjugates is to “change jto – j . This changes the sign of the imaginary part of the complex number.

Figure 1.7. 

A Complex Variable and Its Complex Conjugate

Magnitude Squared. The product of z and its complex conjugate is called the magnitude squared of z and is denoted by |z|² :

(1.23)|z|² = z * z = (x – j y)(x + j y) = x ² + y ² = r e – j θ r e jθ = r ².

Note that |z| = r is the radius, or magnitude, that we defined in "Geometry of Complex Numbers".

Exercise 1.3.3.

Write z * as z * = z w . Find w in its Cartesian and polar forms.

Exercise 1.3.4.

Prove that angle .

Exercise 1.3.5.

Show that the real and imaginary parts of z = x + j y may be written as

Commutativity, Associativity, and Distributivity. The complex numbers commute, associate, and distribute under addition and multiplication as follows:

(1.24)

(1.25)

Identities and Inverses. In the field of complex numbers, the complex number 0 + j0 (denoted by 0) plays the role of an additive identity, and the complex number 1 + j0 (denoted by 1) plays the role of a multiplicative identity:

(1.26)

In this field, the complex number – z = – x + j( – y) is the additive inverse of z, and the complex number is the multiplicative inverse:

(1.27)

Exercise 1.3.6.

Show that the additive inverse of z = r e jθ may be written as r e j(θ + π).

Exercise 1.3.7.

Show that the multiplicative inverse of z may be written as Show that z * z is real. Show that z – 1 may also be written as z – 1 = r – 1 e – j θ . Plot z and z – 1 for a representative z .

Exercise 1.3.8.

Prove (j) – 1 = – j .

Exercise 1.3.9.

Find z – 1 when z = 1 + j1.

Exercise 1.3.10.

Prove . Plot z and for a representative z .

Exercise 1.3.11.

Find all of the complex numbers z with the property that j z = – z * . Illustrate these complex numbers on the complex plane.

Demo 1.2 (MATLAB). Create and run the following script file (name it Complex Numbers)[2]

clear, clg
j=sqrt(-1)
z1=1+j*.5,z2=2+j*1.5
z3=z1+z2,z4=z1*z2
z5=conj(z1),z6=j*z2
avis([-4 4 -4 4]),axis('square'),plot(z1,'0')
hold on
plot(z2,'0'),plot(z3,'+'),plot(z4,'*'),
plot(z2,'0'),plot(z3,'+'),plot(z4,'*'),
plot(z5,'x'),plot(z6,'x')

Figure 1.8. 

Complex Numbers (Demo 1.2)

With the help of Appendix 1, you should be able to annotate each line of this program. View your graphics display to verify the rules for add, multiply, conjugate, and perp. See Figure 1.8.

Exercise 1.3.12.

Prove that z ⁰ = 1 .

Exercise 1.3.13.

(MATLAB) Choose z ₁ = 1.05 e j 2 π / 16 and z ₂ = 0.95 e j 2 π / 16 . Write a MATLAB program to compute and plot and for n = 1 , 2 , ... , 32 . You should observe a figure like Figure 1.9.

Figure 1.9. 

Powers of z

1.4. Complex Numbers: Roots of Quadratic Equations*

You probably first encountered complex numbers when you studied values of z (called roots or zeros) for which the following equation is satisfied:

(1.28) a z ² + b z + c = 0.

For a ≠ 0 (as we will assume), this equation may be written as

(1.29)

Let's denote the second-degree polynomial on the left-hand side of this equation by p(z):

(1.30)

This is called a monic polynomial because the coefficient of the highest-power term is 1. When looking for solutions to the quadratic equation , we are really looking for roots (or zeros) of the polynomial p(z). The fundamental theorem of algebra says that there are two such roots. When we have found them, we may factor the polynomial p(z) as follows:

(1.31)

In this equation, z₁ and z₂ are the roots we seek. The factored form p(z) = shows clearly that , meaning that the quadratic equation p(z) = 0 is solved for z = z ₁ and z = z ₂ . In the process of factoring the polynomial p(z), we solve the quadratic equation and vice versa.

By equating the coefficients of z ²,z ¹ , and z⁰ on the left-and right-hand sides of Equation 1.31, we find that the sum and the product of the roots z ₁ and z ₂ obey the equations

(1.32)

You should always check your solutions with these equations.

Completing the Square. In order to solve the quadratic equation (or, equivalently, to find the roots of the polynomial z ² + , we “complete the square” on the left-hand side of Equation 1.29:

(1.33)

This equation may be rewritten as

(1.34)

We may take the square root of each side to find the solutions

(1.35)

Exercise 1.4.1.

With the roots z ₁ and z ₂ defined in Equation 1.29, prove that is, indeed, equal to the polynomial . Check that and

In the equation that defines the roots z ₁ and z ₂ , the term b ² – 4 a c is critical because it determines the nature of the solutions for z ₁ and z ₂ . In fact, we may define three classes of solutions depending on b ² – 4a c .

(i) Overdamped . In this case, the roots z ₁ and z ₂ are

(1.36)

These two roots are real, and they are located symmetrically about the point . When b = 0, they are located symmetrically about 0 at the points . (In this case, – 4a c > 0.) Typical solutions are illustrated in Figure 1.10.

Figure 1.10. 

(a)

(b)

(c)

Typical Roots in the Overdamped Case; (a) , (b) , and (c)

Exercise 1.4.2.

Compute and plot the roots of the following quadratic equations:

;

;

.

For each equation, check that and

(ii) Critically Damped . In this case, the roots z ₁ and z ₂ are equal (we say they are repeated):

(1.37)

These solutions are illustrated in Figure 1.11.

Figure 1.11. 

(a)

(b)

Roots in the Critically Damped Case; (a) b / 2a > 0, and (b) b / 2a < 0

Exercise 1.4.3.

Compute and plot the roots of the following quadratic equations:

z ² + 2z + 1 = 0;

z ² – 2z + 1 = 0;

z ² = 0.

For each equation, check that and

(iii) Underdamped . The underdamped case is, by far, the most fascinating case. When b ² – 4a c < 0, then the square root in the solutions for z ₁ and z ₂ (Equation 1.35) produces an imaginary number. We may write b ² – 4a c as and write z 1,2 as

(1.38)

These complex roots are illustrated in Figure 1.12. Note that the roots are

purely imaginary when b = 0, producing the result

(1.39)

Figure 1.12. 

(a)

(b)

(c)

Figure 1.12: Typical Roots in the Underdamped Case; (a) b / 2a > 0,(b) b / 2a < 0, and (c) b / 2a = 0

In this underdamped case, the roots z ₁ and z ₂ are complex conjugates:

(1.40) z ₂ = z * 1 .

Thus the polynomial also takes the form

(1.41)

and |z ₁|² are related to the original coefficients of the polynomial as follows:

(1.42)

(1.43)

Always check these equations.

Let's explore these connections further by using the polar representations for z ₁ and z ₂ :

(1.44) z 1 , 2 = r e ± j θ .

Then Equation 1.41 for the polynomial p(z) may be written in the “standard form”

(1.45)

Equation 1.42 is now

(1.46)

These equations may be used to locate z 1,2 = r e ± j θ

(1.47)

Exercise 1.4.4.

Prove that p(z) may be written as p(z) = z ² – 2r cos θ z + r ² in the underdamped case.

Exercise 1.4.5.

Prove the relations in Equation 1.47. Outline a graphical procedure for locating z ₁ = r e jθ and z ₂ = r e – j θ from the polynomial z ² + .

Exercise 1.4.6.

Compute and plot the roots of the following quadratic equations:

z ² + 2z + 2 = 0;

z ² – 2z + 2 = 0;

z ² + 2 = 0.

For each equation, check that and

1.5. Complex Numbers: Representing Complex Numbers in a Vector Space*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

So far we have coded the complex number z = x + j y with the Cartesian pair (x,y) and with the polar pair (r ∠ θ). We now show how the complex number z may be coded with a two-dimensional vector z and show how this new code may be used to gain insight about complex numbers.

Coding a Complex Number as a Vector. We code the complex number z = x + j y with the two-dimensional vector :

(1.48)

We plot this vector as in Figure 1.13. We say that the vector z belongs to a “vector space.” This means that vectors may be added and scaled according to the rules

(1.49)

(1.50)

Figure 1.13. 

The Vector z Coding the Complex Number z

Furthermore, it means that an additive inverse – z , an additive identity 0, and a multiplicative identity 1 all exist:

(1.51) z + ( – z ) = 0

(1.52)

The vector 0 is .

Prove that vector addition and scalar multiplication satisfy these properties of commutation, association, and distribution:

(1.53) z ₁ + z ₂ = z₂ + z ₁

(1.54)

(1.55) a ( b z ) = ( a b ) z

(1.56)

Inner Product and Norm. The inner product between two vectors z ₁ and z ₂ is defined to be the real number

(1.57)

We sometimes write this inner product as the vector product (more on this in Linear Algebra)

(1.58)

Exercise 1.5.1.

Prove

When z ₁ = z ₂ = z , then the inner product between z and itself is the norm squared of z :

(1.59) | | z | | ² =

( z , z )

= x ² + y ² .

These properties of vectors seem abstract. However, as we now show, they may be used to develop a vector calculus for doing complex arithmetic.

A Vector Calculus for Complex Arithmetic. The addition of two complex numbers z ₁ and z ₂ corresponds to the addition of the vectors z ₁ and z ₂ :

(1.60)

The scalar multiplication of the complex number z ₂ by the real number x ₁ corresponds to scalar multiplication of the vector z ₂ by x ₁ :

(1.61)

Similarly, the multiplication of the complex number z ₂ by the real number y ₁ is

(1.62)

The complex product is therefore represented as

(1.63)

This representation may be written as the inner product

(1.64)

where v and w are the vectors and . By defining the matrix

(1.65)

we can represent the complex product z ₁ z ₂ as a matrix-vector multiply (more on this in Linear Algebra):

(1.66)

With this representation, we can represent rotation as

(1.67)

We call the matrix a “rotation matrix.”

Exercise 1.5.2.

Call R(θ) the rotation matrix:

(1.68)

Show that R( – θ) rotates by ( – θ). What can you say about R( – θ)w when w = R(θ)z ?

Exercise 1.5.3.

Represent the complex conjugate of z as

(1.69)

and find the elements a,b,c , and d of the matrix.

Inner Product and Polar Representation. From the norm of a vector, we derive a formula for the magnitude of z in the polar representation z = r e jθ :

(1.70)

If we define the coordinate vectors and , then we can represent the vector z as

(1.71)

See Figure 1.14. From the figure it is clear that the cosine and sine of the angle θ are

(1.72)

Figure 1.14. 

Representation of z in its Natural Basis

This gives us another representation for any vector z :

(1.73) z =

| | z | | cos θ e ₁ +

| | z | | sin θ e ₂ .

The inner product between two vectors z ₁ and z ₂ is now

(1.74)

It follows that cos θ ₁ + sinθ ₁sin θ ₂ may be written as

(1.75)

This formula shows that the cosine of the angle between two vectors z ₁ and z ₂ , which is, of course, the cosine of the angle of z ₂ z * ₁ , is the ratio of the inner product to the norms.

Exercise 1.5.4.

Prove the Schwarz and triangle inequalities and interpret them:

(1.76)

(1.77)

1.6. Complex Numbers: An Electric Field Computation*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

We have established that vectors may be used to code complex numbers. Conversely, complex numbers may be used to code or represent the orthogonal components of any two-dimensional vector. This makes them invaluable in electromagnetic field theory, where they are used to represent the components of electric and magnetic fields.

The basic problem in electromagnetic field theory is to determine the electric or magnetic field that is generated by a static or dynamic distribution of charge. The key idea is to isolate an infinitesimal charge, determine the field set up by this charge, and then to sum the fields contributed by all such infinitesimal charges. This idea is illustrated in Figure 1.15, where the charge λ , uniformly distributed over a line segment of length dx at point – x , produces a field d E(x) at the test point (0,h). The field d E(x) is a “vector” field (as opposed to a “scalar” field), with components E ₁(x) and E ₂(x). The intensity or field strength of the field d E(x) is

(1.78)

Figure 1.15. 

Infinitesimal Charge λdx Producing Field d E(x)

But the field strength is directed at angle θ(x), as illustrated in Figure 1.15. The field d E(x) is real with components d E ₁(x) and d E ₂(x), but we code it as a complex field. We say that the “complex” field at test point (0,h) is

(1.79)

with components d E ₁(x) and d E ₂(x). That is,

(1.80) d E ( x )

= d E ₁ ( x )

+ j d E ₂ ( x )

(1.81)

(1.82)

For charge uniformally distributed with density λ along the x-axis, the total field at the test point (0,h) is obtained by integrating dE :

(1.83)

The functions cos θ(x) and sinθ(x) are

(1.84)

We leave it as a problem to show that the real component E ₁ of the field is zero. The imaginary component E ₂ is

(1.85)

(1.86)

(1.87)

(1.88)

We emphasize that the field at (0,h) is a real field. Our imaginary answer simply says that the real field is oriented in the vertical direction because we have used the imaginary part of the complex field to code the vertical component of the real field.

Exercise 1.6.1.

Show that the horizontal component of the field E is zero. Interpret this finding physically.

From the symmetry of this problem, we conclude that the field around the infinitely long wire of Figure 1.15 is radially symmetric. So, in polar coordinates, we could say

(1.89)

which is independent of θ . If we integrated the field along a radial line perpendicular to the wire, we would measure the voltage difference

(1.90)

An electric field has units of volts/meter, a charge density λ has units of coulombs/meter, and ϵ ₀ has units of coulombs/volt-meter; voltage has units of volts (of course).

1.7. Complex Numbers: Numerical Experiment (Quadratic Roots)*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

There is a version of the quadratic equation that will arise over and over again in your study of electrical and mechanical systems:

(1.91) s ² + 2ξ ω ₀ s + ω ² ₀ = 0.

For reasons that can only become clear as you continue your study of engineering, the parameter ω ₀ is called a resonant frequency, and the parameter ξ ≥ 0 is called a damping factor. In this experiment, you will begin by

finding the “underdamped” range of values ξ ≥ 0 for which the roots s₁ and s₂ are complex;

finding the “critically damped” value of ξ ≥ 0 that makes the roots s₁ and s₂ equal; and

finding the “overdamped” range of values ξ ≥ 0 for which s₁ and s₂ are real.

For each of these ranges, find the analytical solution for s 1,2 as a function of ω₀ and ξ; write your solutions in Cartesian and polar forms and present your results as

(1.92)

where ξc is the critically damped value of ξ. Write a MATLAB program that computes and plots s 1,2 for ω₀ fixed at ω ₀ = 1 and ξ variable between 0.0 and 2.0 in steps of 0.1. Interpret all of your findings.

Now organize the coefficients of the polynomial s ² + 2ξ s + 1 into the array [12ξ1]. Imbed the MATLAB instructions

r=roots([1 2*e 1]);
plot(real(r(1)),imag(r(1)),'o')
plot(real(r(2)),imag(r(2)),'o')

in a for loop to compute and plot the roots of s ² + 2ξ s + 1 as ξ ranges from 0.0 to 2.0. Note that r is a 1 × 2 array of complex numbers. You should observe Figure 1.16. We call this “half circle and line” the locus of roots for the quadratic equation or the “root locus” in shorthand.

Figure 1.16. 

Roots of Quadratic Equation

[1] We have used the trigonometric identities cos θ ₂ – sin θ₁ sin θ₂ and cos θ ₂ + cosθ ₁ sin θ₂ to derive this result.

[2] If you are using PC-MATLAB, you will need to name your file cmplxnos.m.

Chapter 2. The Functions e^x and e^jΘ

2.1. The Functions e^x and e^jθ: Introduction*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

Notes to Teachers and Students

It is essential to write out, term-by-term, every sequence and sum in this chapter. This demystifies the seemingly mysterious notation. The example on compound interest shows the value of limiting arguments in everyday life and gives e x some real meaning. The function e jθ , covered in the section "The Function of ejθ and the Unit Circle and "Numerical Experiment (Approximating ejθ , must be understood by all students before proceeding to "Phasors" . The Euler and De Moivre identities provide every tool that students need to derive trigonometric formulas. The properties of roots of unity are invaluable for the study of phasors in "Phasors" .

The MATLAB programs in this chapter are used to illustrate sequences and series and to explore approximations to sin θ and cos θ . The numerical experiment in "Numerical Experiment (Approximating ejθ illustrates, geometrically and algebraically, how approximations to e jθ converge.

“Second-Order Differential and Difference Equations” is a little demanding for freshmen, but we give it a once-over-lightly to illustrate the power of quadratic equations and the functions ex and e jθ . This section also gives a sneak preview of more advanced courses in circuits and systems.

Introduction

It is probably not too strong a statement to say that the function e x is the most important function in engineering and applied science. In this chapter we study the function e x and extend its definition to the function e jθ . This study clarifies our definition of e jθ from "Complex Numbers" and leads us to an investigation of sequences and series. We use the function e jθ to derive the Euler and De Moivre identities and to produce a number of important trigonometric identities. We define the complex roots of unity and study their partial sums. The results of this chapter will be used in "Phasors" when we study the phasor representation of sinusoidal signals.

2.2. The Functions e^x and e^jθ: The Function e^x*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

Many of you know the number e as the base of the natural logarithm, which has the value 2.718281828459045. . . . What you may not know is that this number is actually defined as the limit of a sequence of approximating numbers. That is,

(2.1)

(2.2)

This means, simply, that the sequence of numbers , . . . , gets arbitrarily close to 2.718281828459045. . . . But why should such a sequence of numbers be so important? In the next several paragraphs we answer this question.

Exercise 2.2.1.

(MATLAB) Write a MATLAB program to evaluate the expression for n = 1,2,4,8,16,32,64 to show that f n ≈ e for large n .

Derivatives and the Number e . The number arises in the study of derivatives in the following way. Consider the function

(2.3)

and ask yourself when the derivative of f(x) equals f(x). The function f(x) is plotted in Figure 2.1 for a > 1. The slope of the function at point x is

(2.4)

Figure 2.1. 

The Function f(x) = a x

If there is a special value for a such that

(2.5)

then would equal f(x). We call this value of a the special (or exceptional) number e and write

(2.6)

The number e would then be e = f(1). Let's write our condition that converges to 1 as

(2.7)

or as

(2.8) e ≅ (1 + Δ x)1 / Δ x .

Our definition of amounts to defining and allowing n → ∞ in order to make Δ x → 0. With this definition for e , it is clear that the function e x is defined to be (e) x :

(2.9)

By letting we can write this definition in the more familiar form

(2.10)

This is our fundamental definition for the function e x . When evaluated at x = 1, it produces the definition of e given in Equation 2.1.

The derivative of e x is, of course,

(2.11)

This means that Taylor's theorem[3] may be used to find another characterization for e x :

(2.12)

When this series expansion for e x is evaluated at x = 1, it produces the following series for e :

(2.13)

In this formula, n ! is the product n(n – 1)(n – 2)⋯( 2 ) 1 . Read n ! as " n factorial.”

Exercise 2.2.2.

(MATLAB) Write a MATLAB program to evaluate the sum

(2.14)

for N = 1,2,4,8,16,32,64 to show that S N ≅ e for large N . Compare S ₆₄ with f ₆₄ from Exercise 2.2.1.. Which approximation do you prefer?

Compound Interest and the Function . There is an example from your everyday life that shows even more dramatically how the function e x arises. Suppose you invest V ₀ dollars in a savings account that offers 100 x % annual interest. (When x = 0.01, this is 1%; when x = 0.10, this is 10% interest.) If interest is compounded only once per year, you have the simple interest formula for V ₁ , the value of your savings account after 1 compound (in this case, 1 year):

V ₁ = (1 + x)V ₀ . This result is illustrated in the block diagram of Figure 2.2(a). In this diagram, your input fortune V₀ is processed by the “interest block” to produce your output fortune V₁ . If interest is compounded monthly, then the annual interest is divided into 12 equal parts and applied 12 times. The compounding formula for V₁₂ , the value of your savings after 12 compounds (also 1 year) is

(2.15)

This result is illustrated in Figure 2.2. Can you read the block diagram? The general formula for the value of an account that is compounded n times per year is

(2.16)

V n is the value of your account after n compounds in a year, when the annual interest rate is 100x%.

Figure 2.2. 

(a)

(b)

Block Diagram for Interest Computations; (a) Simple Annual Interest, and (b) Monthly Compounding

Exercise 2.2.3.

Verify in Equation 2.16 that a recursion is at work that terminates at V n . That is, show that for i = 0,1,...,n – 1 produces the result .

Bankers have discovered the (apparent) appeal of infinite, or continuous, compounding:

(2.17)

We know that this is just

(2.18) V ∞ = e x V ₀.

So, when deciding between 100x ₁ % interest compounded daily and 100x ₂% interest compounded continuously, we need only compare

(2.19)

We suggest that daily compounding is about as good as continuous compounding. What do you think? How about monthly compounding?

Exercise 2.2.4.

(MATLAB) Write a MATLAB program to compute and plot simple interest, monthly interest, daily interest, and continuous interest versus interest rate 100x . Use the curves to develop a strategy for saving money.

2.3. The Functions e^x and e^jθ: The Function e^jθ and the Unit Circle*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

Let's try to extend our definitions of the function e x to the argument x = j Θ . Then e jΘ is the function

(2.20)

The complex number is illustrated in Figure 2.3. The radius to the point is and the angle is This means that the n th power of has radius and angle (Recall our study of powers of z.) Therefore the complex number may be written as

(2.21)

For n large, , and . Therefore is approximately

(2.22)

( cosθ + j sin θ).

This finding is consistent with our previous definition of e jθ !

Figure 2.3. 

The Complex Number

The series expansion for e jθ is obtained by evaluating Taylor's formula at x = j θ :

(2.23)

When this series expansion for e jθ is written out, we have the formula

(2.24)

It is now clear that cosθ and sinθ have the series expansions

(2.25)

(2.26)

When these infinite sums are truncated at N – 1, then we say that we have N-term approximations for cosθ and sinθ :

(2.27)

(2.28)

The ten-term approximations to cosθ and sinθ are plotted over exact expressions for cosθ and sinθ in Figure 2.4. The approximations are very good over one period (0 ≤ θ ≤ 2π), but they diverge outside this interval. For more accurate approximations over a larger range of θ ' s, we would need to use more terms. Or, better yet, we could use the fact that cosθ and sinθ are periodic in θ . Then we could subtract as many multiples of 2π as we needed from θ to bring the result into the range [0,2π] and use the ten-term approximations on this new variable. The new variable is called θ -modulo 2π .

Figure 2.4. 

Ten-Term Approximations to cosθ and sinθ

Exercise 2.3.1.

Write out the first several terms in the series expansions for cosθ and sinθ .

Demo 2.1 (MATLAB). Create a MATLAB file containing the following demo MATLAB program that computes and plots two cycles of cosθ and sin θ versus θ. You should observe Figure 2.5. Note that two cycles take in 2(2π) radians, which is approximately 12 radians.

clg;
j = sqrt(-1);
theta = 0:2*pi/50:4*pi;
s = sin(theta);
c = cos(theta);
plot(theta,s);
elabel('theta in radians');
ylabel('sine and cosine');
hold on
plot(theta,c);
hold off

Figure 2.5. 

The Functions cosθ and sinθ

Exercise 2.3.2.

(MATLAB) Write a MATLAB program to compute and plot the ten-term approximations to cos θ and sin θ for θ running from 0 to 2 ( 2 π ) in steps of 2 π / 50 . Compute and overplot exact expressions for cos θ and sin θ . You should observe a result like Figure 2.4.

The Unit Circle. The unit circle is defined to be the set of all complex numbers z whose magnitudes are 1. This means that all the numbers on the unit circle may be written as z = e jθ . We say that the unit circle consists of all numbers generated by the function z = e jθ as θ varies from 0 to 2π . See Figure 2.6.

A Fundamental Symmetry. Let's consider the two complex numbers z ₁ and , illustrated in Figure 2.6. We call “reflection of z through the unit circle” (and vice versa). Note that z ₁ = r ₁ e j θ ₁ and . The complex numbers z ₁ – e jθ and are illustrated in Figure 2.6. The magnitude squared of each is

(2.29)

(2.30)

The ratio of these magnitudes squared is

(2.31)

This ratio may be manipulated to show that it is independent of θ , meaning that the points z ₁ and maintain a constant relative distance from every point on the unit circle:

(2.32)

This result will be of paramount importance to you when you study digital filtering, antenna design, and communication theory.

Figure 2.6. 

The Unit Circle

Exercise 2.3.3.

Write the complex number z – e jθ as r e jφ . What are r and φ ?

2.4. The Functions e^x and e^jθ: The Euler and De Moivre Identities*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

The Euler and De Moivre identities are the fundamental identities for deriving trigonometric formulas. From the identity e jθ = cosθ + j sinθ and the conjugate identity sinθ , we have the Euler identities for cosθ and sinθ :

(2.33)

These identities are illustrated in Figure 2.7.

Figure 2.7. 

Euler's Identities

The identity e jθ = cosθ + jsin θ also produces the De Moivre identity:

(2.34)

When the left-hand side of this equation is expanded with the binomial expansion, we obtain the identity

(2.35)

Binomial Coefficients and Pascal's Triangle. The binomial coefficients in Equation 2.35 are shorthand for the number

(2.36)

This number gives the coefficient of x n – k y k in the expansion of (x + y) n . How do we know that there are terms of the form x n – k y k ? One way to answer this question is to use Pascal's triangle, illustrated in Figure 2.8. Each node on Pascal's triangle shows the number of routes that terminate at that node. This number is always the sum of the number of routes that terminate at the nodes just above the node in question. If we think of a left-hand path as an occurrence of an x and a right-hand path as an occurrence of a y , then we see that Pascal's triangle keeps track of the number of occurrences of x n – k y k .

Figure 2.8. 

Pascal's Triangle and the Binomial Coefficients

Exercise 2.4.1.

Prove .

Exercise 2.4.2.

Find an identity for .

Exercise 2.4.3.

Find “half-angle” formulas for cos2θ and sin2θ .

Exercise 2.4.4.

Show that

cos3θ = cos² θ – 3 cosθ sin² θ ;

sin3θ = 34cos² θ sinθ – sin³ θ .

Exercise 2.4.5.

Use to prove

;

.

2.5. The Functions e^x and e^jθ: Roots of Unity and Related Topics*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

The complex number z = e j2π / N is illustrated in Figure 2.9. It lies on the unit circle at angle θ = 2π / N . When this number is raised to the n th power, the result is z n = e j2π n / N . This number is also illustrated in Figure 2.9. When one of the complex numbers e j2π n / N is raised to the N th power, the result is

(2.37)

Figure 2.9. 

The Complex Numbers e j2π / N and e j2π n / N

We say that e j2π n / N is one of the N th roots of unity, meaning that e j2π n / N is one of the values of z for which

(2.38) z N – 1 = 0 .

There are N such roots, namely,

(2.39)

As illustrated in Figure 2.10, the 12 th roots of unity are uniformly distributed around the unit circle at angles 2π n / 12. The sum of all of the N th roots of unity is zero:

(2.40)

This property, which is obvious from Figure 2.10, is illustrated in Figure 2.11, where the partial sums are plotted for k = 1,2,...,N .

Figure 2.10. 

Roots of Unity

These partial sums will become important to us in our study of phasors and light diffraction in "Phasors" and in our discussion of filters in "Filtering".

Figure 2.11. 

Partial Sums of the Roots of Unity

Geometric Sum Formula. It is natural to ask whether there is an analytical expression for the partial sums of roots of unity:

(2.41)

We can imbed this question in the more general question, is there an analytical solution for the “geometric sum”

(2.42)

The answer is yes, and here is how we find it. If z = 1, the answer is S k = k . If z ≠ 1, we can premultiply S k by z and proceed as follows:

(2.43)

From this formula we solve for the geometric sum:

(2.44)

This basic formula for the geometric sum Sk is used throughout electromagnetic theory and system theory to solve problems in antenna design and spectrum analysis. Never forget it.

Exercise 2.5.1.

Find formulas for and for

Exercise 2.5.2.

Prove .

Exercise 2.5.3.

Find formulas for the magnitude and phase of the partial sum .

Exercise 2.5.4.

(MATLAB) Write a MATLAB program to compute and plot the partial sum for k = 1,2,...,N . You should observe Figure 2.11.

Exercise 2.5.5.

Solve the equation (z + 1)³ = z ³ .

Exercise 2.5.6.

Find all roots of the equation z ³ + z ² + 3z – 15 = 0.

Exercise 2.5.7.

Find c so that (1 + j) is a root of the equation z ¹⁷ + 2z ¹⁵ – c = 0.

2.6. The Functions e^x and e^jθ: Second-Order Differential and Difference Equations*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

With our understanding of the functions e x , e jΘ , and the quadratic equation , we can undertake a rudimentary study of differential and difference equations.

Differential Equations. In your study of circuits and systems you will encounter the homogeneous differential equation

(2.45)

Because the function e st reproduces itself under differentiation, it is plausible to assume that x(t) = e st is a solution to the differential equation. Let's try it:

(2.46)

If this equation is to be satisfied for all t , then the polynomial in s must be zero. Therefore we require

(2.47) s ² + a ₁ s + a ₂ = 0.

As we know from our study of this quadratic equation, the solutions are

(2.48)

This means that our assumed solution works, provided s = s ₁ or s ₂ . It is a fundamental result from the theory of differential equations that the most general solution for x(t) is a linear combination of these assumed solutions:

(2.49) x(t) = A ₁ e s ₁ t + A ₂ e s ₂ t .

If a ² ₁ – 4a ₂ is less than zero, then the roots s ₁ and s ₂ are complex:

(2.50)

Let's rewrite this solution as

(2.51) s 1 , 2 = σ ± j ω

where σ and ω are the constants

(2.52)

(2.53)

With this notation, the solution for x(t) is

(2.54) x(t) = A ₁ e σt e jωt + A ₂ e σt e – jtω t .

If this solution is to be real, then the two terms on the right-hand side must be complex conjugates. This means that A ₂ = A * ₁ and the solution for x(t) is

(2.55)

The constant A ₁ may be written as A ₁ = |A|e jφ . Then the solution for x(t) is

(2.56) x(t) = 2|A|e σt cos(ω t + φ).

This “damped cosinusoidal solution” is illustrated in Figure 2.12.

Figure 2.12. 

The Solution to a Second-Order Differential Equation

Exercise 2.6.1.

Find the general solutions to the following differential equations:

;

;

.

Difference Equations. In your study of digital filters you will encounter homogeneous difference equations of the form

(2.57) x n + a ₁ x n – 1 + a ₂ x n – 2 = 0.

What this means is that the sequence obeys a homogeneous recursion:

(2.58) x n = – a ₁ x n – 1 – a ₂ x n – 2 .

A plausible guess at a solution is the geometric sequence x n = z n . With this guess, the difference equation produces the result

(2.59)

If this guess is to work, then the second-order polynomial on the left-hand side must equal zero:

1 + a ₁ z – 1 + a ₂ z – 2 = 0

(2.60)

The solutions are

(2.61)

The general solution to the difference equation is a linear combination of the assumed solutions:

(2.62)

This general solution is illustrated in Figure 2.13.

Figure 2.13. 

The Solution to a Second-Order Difference Equation

Exercise 2.6.2.

Find the general solutions to the following difference equations:

x n + 2x n – 1 + 2 = 0;

x n – 2x n – 1 + 2 = 0;

x n + 2x n – 2 = 0.

2.7. The Functions e^x and e^jθ: Numerical Experiment (Approximating e^jθ)*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

We have demonstrated that the function e jθ has two representations:

; and

.

In this experiment, you will write a MATLAB program to evaluate the two functions f n and S n for twenty values of n :

; and

.

Choose θ = π / 4( = pi / 4). Use an implicit for loop to draw and plot a circle of radius 1. Then use an implicit for loop to compute and plot f n and an explicit for loop to compute and plot S n for n = 1,2,...,100. You should observe plots like those illustrated in Figure 2.14. Interpret them.

Figure 2.14. 

(a)

(b)

Plots for Convergence;(a) of fn , and (b) of Sn

[3] Taylor's theorem says that a function may be completely characterized by all of its derivatives (provided they all exist).

Chapter 3. Phasors

3.1. Phasors: Introduction*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

Notes to Teachers and Students:

Phasors! For those who understand them, they are of incomparable value for the study of elementary and advanced topics. For those who misunderstand them, they are a constant source of confusion and are of no apparent use. So, let's understand them.

The conceptual leap from the complex number e jθ to the phasor e j(ω t + θ) comes in "Phasor Representation of Signals". Take as long as necessary to understand every geometrical and algebraic nuance. Write the MATLAB program in "Exercise 6" to fix the key ideas once and for all. Then use phasors to study beating between tones, multiphase power, and Lissajous figures in "Beating between Tones" through "Lissajous Figures". We usually conduct a classroom demonstration of beating between tones using two phase-locked sources, an oscilloscope, and a speaker. We also demonstrate Lissajous figures with this hardware.

"Sinusoidal Steady State and the Series RLC Circuit" and Light Scattering by a Slit" on sinusoidal steady state and light scattering are too demanding for freshmen but are right on target for sophomores. These sections may be covered in a sophomore course (or a supplement to a sophomore course) or skipped in a freshman course without consequence.

In the numerical experiment in "Numerical Experiment (Interference Patterns)", students compute and plot interference patterns for two sinusoids that are out of phase.

Introduction

In engineering and applied science, three test signals form the basis for our study of electrical and mechanical systems. The impulse is an idealized signal that models very short excitations (like current pulses, hammer blows, pile drives, and light flashes). The step is an idealized signal that models excitations that are switched on and stay on (like current in a relay that closes or a transistor that switches). The sinusoid is an idealized signal that models excitations that oscillate with a regular frequency (like AC power, AM radio, pure musical tones, and harmonic vibrations). All three signals are used in the laboratory to design and analyze electrical and mechanical circuits, control systems, radio antennas, and the like. The sinusoidal signal is particularly important because it may be used to determine the frequency selectivity of a circuit (like a superheterodyne radio receiver) to excitations of different frequencies. For this reason, every manufacturer of electronics test equipment builds sinusoidal oscillators that may be swept through many octaves of Orequency. (Hewlett-Packard was started in 1940 with the famous HP audio oscillator.)

In this chapter we use what we have learned about complex numbers and the function e jθ to develop a phasor calculus for representing and ma- nipulating sinusoids. This calculus operates very much like the calculus we developed in "Complex Numbers" and "The Functions ex and ejθ" for manipulating complex numbers. We ap- ply our calculus to the study of beating phenomena, multiphase power, series RLC circuits, and light scattering by a slit.

3.2. Phasors: Phasor Representation of Signals*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

There are two key ideas behind the phasor representation of a signal:

a real, time-varying signal may be represented by a complex, time-varying signal; and

a complex, time-varying signal may be represented as the product of a complex number that is independent of time and a complex signal that is dependent on time.

Let's be concrete. The signal

(3.1) x(t) = Acos(ω t + φ),

illustrated in Figure 3.1, is a cosinusoidal signal with amplitude A, frequency ω, and phase φ. The amplitude A characterizes the peak-to-peak swing of 2A , the angular frequency ω characterizes the period between negative- to-positive zero crossings (or positive peaks or negative peaks), and the phase φ characterizes the time when the signal reaches its first peak. With τ so defined, the signal x(t) may also be written as

(3.2) x(t) = Acos ω(t – τ).

Figure 3.1. 

A Cosinusoidal Signal

When τ is positive, then τ is a “time delay” that describes the time (greater than zero) when the first peak is achieved. When τ is negative, then τ is a “time advance” that describes the time (less than zero) when the last peak was achieved. With the substitution we obtain a third way of writing x(t):

(3.3)

In this form the signal is easy to plot. Simply draw a cosinusoidal wave with amplitude A and period T; then strike the origin (t = 0) so that the signal reaches its peak at τ. In summary, the parameters that determine a cosinusoidal signal have the following units:

A, arbitrary (e.g., volts or meters/sec, depending upon the application)

ω, in radians/sec (rad/sec)

T, in seconds (sec)

φ, in radians (rad)

τ, in seconds (sec)

Exercise 3.2.1.

Show that is “periodic with period T ," meaning that x(t + m T) = x(t) for all integer m .

Exercise 3.2.2.

The inverse of the period T is called the “temporal frequency” of the cosinusoidal signal and is given the symbol f ; the units of are (s e c o n d s) – 1 or hertz (Hz). Write x(t) in terms of f . How is f related to ω ? Explain why f gives the number of cycles of x(t) per second.

Exercise 3.2.3.

Sketch the function versus t . Repeat for and . For each function, determine A,ω,T,f,φ , and τ . Label your sketches carefully.

The signal x(t) = Acos(ω t + φ) can be represented as the real part of a complex number:

(3.4)

We call A e jφ e jωt the complex representation of x(t) and write

(3.5) x(t) ↔ A e jφ e jωt ,

meaning that the signal x(t) may be reconstructed by taking the real part of A e jφ e jωt . In this representation, we call A e jφ the phasor or complex amplitude representation of x(t) and write

(3.6) x ( t )

↔ A e jφ ,

meaning that the signal x(t) may be reconstructed from A e jφ by multiplying with e jωt and taking the real part. In communication theory, we call A e jφ the baseband representation of the signal x(t).

Exercise 3.2.4.

For each of the signals in Problem 3.3, give the corresponding phasor representation A e jφ .

Geometric Interpretation. Let's call

(3.7) A e jφ e jωt

the complex representation of the real signal Acos(ω t + φ). At t = 0, the complex representation produces the phasor

(3.8) A e jφ .

This phasor is illustrated in Figure 3.2. In the figure, φ is approximately If we let t increase to time t ₁ , then the complex representation produces the phasor

Figure 3.2. 

Rotating Phasor

We know from our study of complex numbers that e j ω t ₁ just rotates the phasor A e jφ through an angle of ω t ₁ ! See Figure 3.2. Therefore, as we run t from 0, indefinitely, we rotate the phasor A e jφ indefinitely, turning out the circular trajectory of Figure 3.2. When then e jωt = e j2π = 1. Therefore, every seconds, the phasor revisits any given position on the circle of radius A. We sometimes call A e jφ e jωt a rotating phasor whose rotation rate is the frequency ω :

(3.9)

This rotation rate is also the frequency of the cosinusoidal signal Acos(ω t + φ).

In summary, A e jφ e jωt is the complex, or rotating phasor, representation of the signal Acos(ω t + φ). In this representation, e jωt rotates the phasor A e jφ through angles ωt at the rate ω . The real part of the complex representation is the desired signal A cos(ω t + φ). This real part is read off the rotating phasor diagram as illustrated in Figure 3.3. In the figure, the angle φ is about . As we become more facile with phasor representations, we will write and call X e jωt the complex representation and X the phasor representation. The phasor X is, of course, just the phasor A e jφ .

Figure 3.3. 

Reading a Real Signal from a Complex, Rotating Phasor

Exercise 3.2.5.

Sketch the imaginary part of A e jφ e jωt to show that this is Asin(ω t + φ). What do we mean when we say that the real and imaginary parts of A e jφ e jωt are "90 ∘ out of phase"?

Exercise 3.2.6.

(MATLAB) Modify Demo 2.1 in "The Function ex and ejθ" so that θ = ω t , with ω an input frequency variable and t a time variable that ranges from in steps of 0.02 . In your modified program, compute and plot , and for in steps of 0.02 . Plot e jωt in a two-dimensional plot to get a picture like Figure 3.2 and plot and versus t to get signals like those of Figure 3.1. You should observe something like Figure 3.4 using the subplot features discussed in An Introduction to MATLAB. (In the figure, w represents Greek ω.)

Figure 3.4. 

The Functions e jωt , , and

Positive and Negative Frequencies. There is an alternative phasor representation for the signal x(t) = Acos(ω t + φ). We obtain it by using the Euler formula of "The Function ex sjθ", namely, cos . When this formula is applied to x(t), we obtain the result

(3.10)

In this formula, the term is a rotating phasor that begins at the phasor value A e jφ (for t = 0) and rotates counterclockwise with frequency ω. The term is a rotating phasor that begins at the (complex conjugate) phasor value (for t = 0) and rotates clockwise with (negative) ffequency ω. The physically meaningful frequency for a cosine is ω, a positive number like 2π(60) for 60 Hz power. There is no such thing as a negative frequency. The so-called negative frequency of the term just indicates that the direction of rotation for the rotating phasor is clock-wise and not counterclockwise. The notion of a negative frequency is just an artifact of the two-phasor representation of A cos(ω t + φ). In the one-phasor representation, when we take the “real part,” the artifact does not arise. In your study of circuits, systems theory, electromagnetics, solid-state devices, signal processing, control, and communications, you will encounter both the one- and two-phasor representations. Become facile with them.

Exercise 3.2.7.

Sketch the two-phasor representation of Acos(ω t + φ). Show clearly how this representation works by discussing the counterclockwise rotation of the positive frequency part and the clockwise rotation of the negative frequency part.

Adding Phasors. The sum of two signals with common frequencies but different amplitudes and phases is

(3.11)

The rotating phasor representation for this sum is

(3.12)

The new phasor is A ₁ e j φ ₁ + A ₂ e j φ ₂ , and the corresponding real signal is . The new phasor is illustrated in Figure 3.5.

Figure 3.5. 

Adding Phasors

Exercise 3.2.8.

Write the phasor A ₁ e j φ ₁ + A ₂ e j φ ₂ as A ₃ e j φ ₃ ; determine A ₃ and φ ₃ in terms of A ₁,A ₂,φ ₁ , and φ ₂ . What is the corresponding real signal?

Differentiating and Integrating Phasors. The derivative of the signal Acos(ω t + φ) is the signal

(3.13)

This finding is very important. It says that the derivative of Acos(ω t + φ) has the phasor representation

(3.14)

These two phasor representations are entirely equivalent. The first says that the phasor A e jφ is complex scaled by jω to produce the phasor for φ), and the second says that it is scaled by ω and phased by + π / 2. The phasor representations of Acos(ω t + φ) and are illustrated in Figure 3.6. Note that the derivative “leads by π / 2 radians (90)."

Figure 3.6. 

Differentiating and Integrating Phasors

The integral of Acos(ω t + φ) is

(3.15)

This finding shows that the integral of Acos(ω t + φ) has the phasor representation

(3.16)

The phasor A e jφ is complex scaled by or scaled by and phased by e – j π / 2 to produce the phasor for . This is illustrated in Figure 3.6. Note that the integral “lags by π / 2 radians (90). Keep these geometrical pictures of leading and lagging by π / 2 in your mind at all times as you continue your more advanced study of engineering.

An Aside: The Harmonic Oscillator. The signal Acos(ω t + φ) stands on its own as an interesting signal. But the fact that it reproduces itself (with scaling and phasing) under differentiation means that it obeys the second-order differential equation of the simple harmonic oscillator.[4] That is, the differential equation

(3.17)

has the solution

(3.18) x ( t ) = A cos ( ω t + φ ) .

Try it:

(3.19)

The constants A and φ are determined from the initial conditions

(3.20)

Exercise 3.2.9.

Show how to compute A and φ in the equation x(t) = Acos(ω t + φ) from the initial conditions x(O) and .

3.3. Phasors: Beating between Tones*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

Perhaps you have heard two slightly mistuned musical instruments play pure tones whose frequencies are close but not equal. If so, you have sensed a beating phenomenon wherein a pure tone seems to wax and wane. This waxing and waning tone is, in fact, a tone whose frequency is the average of the two mismatched frequencies, amplitude modulated by a tone whose “beat” frequency is half the difference between the two mismatched frequencies. The effect is illustrated in Figure 3.7. Let's see if we can derive a mathematical model for the beating of tones.

We begin with two pure tones whose frequencies are ω ₀ + ν and ω ₀ – ν (for example, ω ₀ = 2π × 10³rad / sec and ν = 2πrad / sec). The average frequency is ω₀ , and the difference frequency is 2ν . What you hear is the sum of the two tones:

(3.21)

The first tone has amplitude A ₁ and phase φ ₁ ; the second has amplitude A ₂ and phase φ ₂ . We will assume that the two amplitudes are equal to A . Furthermore, whatever the phases, we may write them as

(3.22)

Figure 3.7. 

Beating between Tones

Recall our trick for representing x(t) as a complex phasor:

(3.23)

This is an amplitude modulated wave, wherein a low frequency signal with beat frequency ν rad/sec modulates a high frequency signal with carrier frequency ω ₀ rad/sec. Over short periods of time, the modulating term cos(ν t + ψ) remains essentially constant while the carrier term turns out many cycles of its tone. For example, if t runs from 0 to (about 0.1 seconds in our example), then the modulating wave turns out just 1/10 cycle while the carrier turns out cycles (about 100 in our example). Every time νt changes by 2π radians, then the modulating term goes from a maximum (a wax) through a minimum (a wane) and back to a maximum. This cycle takes

(3.24)

which is 1 second in our example. In this 1 second the carrier turns out 1000 cycles.

Exercise 3.3.1.

Find out the frequency of A above middle C on a piano. Assume two pianos are mistuned by ± 1Hz( ± 2π rad/sec ). Find their beat frequency ν and their carrier frequency ω ₀ .

Exercise 3.3.2.

(MATLAB) Write a MATLAB program to compute and plot

cos , and their sum. Then compute and plot .

Verify that the sum equals this latter signal.

3.4. Phasors: Multiphase Power*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

The electrical service to your home is a two-phase service.[5] This means that two 110 volt, 60 Hz lines, plus neutral, terminate in the panel. The lines are π radians (180) out of phase, so we can write them as

(3.25)

These two voltages are illustrated as the phasors X₁ and X₂ in Figure 3.8.

Figure 3.8. 

Phasors in Two-Phase Power

You may use x ₁(t) to drive your clock radio or your toaster and the difference between x ₁(t) and x ₂(t) to drive your range or dryer:

(3.26) x ₁ ( t )

– x ₂ ( t )

= 220

cos

[ 2 π ( 60 ) t + φ ] .

The phasor representation of this difference is

(3.27) X ₁ – X ₂ = 220 e jφ .

The breakers in a breaker box span the x 1 – to-neutral bus for 110 volts and the x ₁ -to- x ₂ buses for 220 volts.

Exercise 3.4.1.

Sketch the phasor X ₁ – X ₂ on Figure 3.8.

Most industrial installations use a three-phase service consisting of the signals x ₁(t),x ₂(t), and x ₃(t):

(3.28)

The phasors for three-phase power are illustrated in Figure 3.9.

Exercise 3.4.2.

Sketch the phasor X ₂ – X ₁ corresponding to x ₂(t) – x ₁(t) on Exercise 3.4.3.. Compute the voltage you can get with x ₂(t) – x ₁(t). This answer explains why you do not get 220 volts in three-phase circuits. What do you get?

Figure 3.9. 

Three-Phase Power

Constant Power. Two- and three-phase power generalizes in an obvious way to N-phase power. In such a scheme, the N signals x n (n = 0,1,...,N – 1) are

(3.29)

The phasors X n are A e j2π(n / N) . The sum of all N signals is zero:

(3.30)

But what about the sum of the instantaneous powers? Define the instantaneous power of the n th signal to be

(3.31)

The sum of all instantaneous powers is (see Exercise 3.4.3.)

(3.32)

and this is independent of time!

Exercise 3.4.3.

Carry out the computations of Equation 3.33 to prove that instantaneous power P is constant in the N-phase power scheme.

3.5. Phasors: Lissajous Figures*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

Lissajous figures are figures that are turned out on the face of an oscilloscope when sinusoidal signals with different amplitudes and different phases are applied to the time base (real axis) and deflection plate (imaginary axis) of the scope. The electron beam that strikes the phosphorous face then had position

(3.33)

In this representation, is the “x-coordinate of the point,” and A y cos(ω t + φ) is the “y-coordinate of the point.” As time runs from 0 to infinity, the point z(t) turns out a trajectory like that of Figure 3.10. The figure keeps overwriting itself because z(t) repeats itself every seconds. Do you see why?

Figure 3.10. 

Lissajous Figure on Oscilloscope Screen

Exercise 3.5.1.

Find the intercepts that the Lissajous figure makes with the real and imaginary axes in Figure 3.10. At what values of time are these intercepts made?

Exercise 3.5.2.

Show that the Lissajous figure is just the rotating phasor A e j(ω t + φ) when A x = A y = A , φ x = φ , and .

Two-Phasor Representation. We gain insight into the shape of the Lissajous figure if we use Euler's formulas to write z(t) as follows:

(3.34)

This representation is illustrated in Figure 3.11. It consists of two rotating phasors, with respective phasors B ₁ and B ₂ :

(3.35)

Figure 3.11. 

Two-Phasor Representation of a Lissajous Figure

As t increases, the phasors rotate past each other where they constructively add to produce large excursions of z(t) from the origin, and then they rotate to antipodal positions where they destructively add to produce near approaches of z(t) to the origin.

In electromagnetics and optics, the representations of z(t) given in Equation 3.33 and Equation 3.35 are called, respectively, linear and circular representations of elliptical polarization. In the linear representation, the x - and y -components of z vary along the horizontal and vertical lines. In the circular representation, two phasors rotate in opposite directions to turn out circular trajectories whose sum produces the same effect.

Exercise 3.5.3.

(MATLAB) Write a MATLAB program to compute and plot the Lissajous figure z(t) when A x = 1 / 2,A y = 1,φ x = 0, and φ y = π / 6. Discretize t appropriately and choose an appropriate range of values for t .

3.6. Phasors: Sinusoidal Steady State and the Series RLC Circuit*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

Phasors may be used to analyze the behavior of electrical and mechanical systems that have reached a kind of equilibrium called sinusoidal steady state. In the sinusoidal steady state, every voltage and current (or force and velocity) in a system is sinusoidal with angular frequency ω . However, the amplitudes and phases of these sinusoidal voltages and currents are all different. For example, the voltage across a resistor might lead the voltage across a capacitor by 90 ∘ ( radians) and lag the voltage across an inductor by .

In order to make our application of phasors to electrical systems concrete, we consider the series RLC circuit illustrated in Figure 3.12. The arrow labeled i(t) denotes a current that flows in response to the voltage applied,and the + and - on the voltage source indicate that the polarity of the applied voltage is positive on the top and negative on the bottom. Our convention is that current flows from positive to negative, in this case clockwise in the circuit.

Figure 3.12. 

Series RLC Circuit

We will assume that the voltage source is an audio oscillator that pro- duces the voltage

(3.36) V(t) = Acos(ω t + φ).

We represent this voltage as the complex signal

(3.37) V ( t )

↔ A e jφ e jωt

and give it the phasor representation

(3.38) V(t) ↔ V;V = A e jφ .

We then describe the voltage source by the phasor V and remember that we can always compute the actual voltage by multiplying by e jωt and taking the real part:

(3.39)

Exercise 3.6.1.

Show that when V = A e jφ .

Circuit Laws. In your circuits classes you will study the Kirchhoff laws that govern the low frequency behavior of circuits built from resistors (R), inductors (L), and capacitors (C). In your study you will learn that the voltage dropped across a resistor is related to the current that flows through it by the equation

(3.40) V R ( t )

= R i ( t )

.

You will learn that the voltage dropped across an inductor is proportional to the derivative of the current that flows through it, and the voltage dropped across a capacitor is proportional to the integral of the current that flows through it:

(3.41)

Phasors and Complex Impedance. Now suppose that the current in the preceding equations is sinusoidal, of the form

(3.42)

We may rewrite i(t) as

(3.43)

where I is the phasor representation of i(t).

Exercise 3.6.2.

Find the phasor I in terms of B and θ in Equation 3.43.

The voltage dropped across the resistor is

(3.44)

Thus the phasor representation for V R (t) is

(3.45) V R (t) ↔ V R ;V R = R I.

We call R the impedance of the resistor because R is the scale constant that relates the “phasor voltage VR ' to the “phasor current I.”

The voltage dropped across the inductor is

(3.46)

The derivative may be moved through the operator (see Exercise 3.6.3.) to produce the result

(3.47)

Thus the phasor representation of

(3.48) V L (t) ↔ V L ;V L = j ω L I.

We call jωL the impedance of the inductor because jωL is the complex scale constant that relates “phasor voltage V L ' to “phasor current I .”

Exercise 3.6.3.

Prove that the operators and Re [] commute:

(3.49)

The voltage dropped across the capacitor is

(3.50)

The integral may be moved through the operator to produce the result

(3.51)

Thus the phasor representation of is

(3.52)

We call the impedance of the capacitor because is the complex scale constant that relates “phasor voltage V C " to “phasor current I .”

Kirchhoff's Voltage Law. Kirchhoff's voltage law says that the voltage dropped in the series combination of R , L , and C illustrated in Figure 3.12 equals the voltage generated by the source (this is one of two fundamental conservation laws in circuit theory, the other being a conservation law for current):

(3.53) V ( t )

= V R ( t )

+ V L ( t )

+ V C ( t )

.

If we replace all of these voltages by their complex representations, we have

(3.54)

An obvious solution is

(3.55)

where I is the phasor representation for the current that flows in the circuit. This solution is illustrated in Figure 3.13, where the phasor voltages R I,j ω L I , and are forced to add up to the phasor voltage V .

Figure 3.13. 

Phasor Addition to Satisfy Kirchhoff's Law

Exercise 3.6.4.

Redraw Figure 3.13 for .

Impedance. We call the complex number the complex impedance for the series RLC network because it is the complex number that relates the phasor voltage V to the phasor current I :

(3.56)

The complex number Z depends on the numerical values of resistance (R), inductance (L), and capacitance (C), but it also depends on the angular frequency (ω) used for the sinusoidal source. This impedance may be manipulated as follows to put it into an illuminating form:

(3.57)

The parameter is a parameter that you will learn to call an "undamped natural frequency" in your more advanced circuits courses. With it, we may write the impedance as

(3.58)

The frequency is a normalized frequency that we denote by ν . Then the impedence, as a function of normalized frequency, is

(3.59)

When the normalized frequency equals one ( ν = 1 ) , then the impedance is entirely real and Z = R . The circuit looks like it is a single resistor.

(3.60)

The impedance obeys the following symmetries around ν = 1:

(3.61)

In the next paragraph we show how this impedance function influences the current that flows in the circuit.

Resonance. The phasor representation for the current that flows the current that flows in the series RLC circuit is

(3.62)

The function displays a "resonance phenomenon." that is, | H ( ν ) | peaks at ν = 1 and decreases to zero and ν = 0 and ν = ∞ :

(3.63)

When | H ( ν ) | = 0 , no current flows.

The function |H(ν)| is plotted against the normalized frequency ν = in Figure 3.14. The resonance peak occurs at ν = 1, where |H(ν)| = meaning that the circuit looks purely resistive. Resonance phenomena underlie the frequency selectivity of all electrical and mechanical networks.

Figure 3.14. 

Resonance in a Series RLC Circuit

Exercise 3.6.5.

(MATLAB) Write a MATLAB program to compute and plot |H(ν)| and argH(ν) versus ν for ν ranging from 0.1 to 10 in steps of 0.1. Carry out your computations for , and 0.01, and overplot your results.

Circle Criterion and Power Factor. Our study of the impedance Z(ν) and the function brings insight into the resonance of an RLC circuit and illustrates the ffequency selectivity of the circuit. But there is more that we can do to illuminate the behavior of the circuit.

(3.64)

This equation shows how voltage is divided between resistor voltage RI and inductor-capacitor voltage

(3.65)

or

(3.66)

In order to simplify our notation, we can write this equation as

(3.67) V = V R + j k ( ν ) V R

where V R is the phasor voltage RI and k(ν) is the real variable

(3.68)

Equation 3.67 brings very important geometrical insights. First, even though the phasor voltage V R in the RLC circuit is complex, the terms V R and j k(ν)V R are out of phase by radians. This means that, for every allowable value of V R , the corresponding j k(ν)V R must add in a right triangle to produce the source voltage V . This is illustrated in Figure 3.15. As the frequency ν changes, then k(ν) changes, producing other values of V R and j k(ν)V R that sum to V . Several such solutions for V R and j k(ν)V R are illustrated in Figure 3.15(b). From the figure we gain the clear impression that the phasor voltage V >R lies on a circle of radius centered at Let's try this solution,

(3.69)

and explore its consequences. When this solution is substituted into Equation 3.67, the result is

(3.70)

or

(3.71)

Figure 3.15. 

(a)

(b)

The Components of V;(a) Addition of VR and j k(ν)V R to Produce V, and (b) Several Values of VR and j k(ν)V R that Produce V

If we multiply the left-hand side by its complex conjugate and the right-hand side by its complex conjugate, we obtain the identity

(3.72)

This equation tells us how the angle ψ depends on k(ν) and, conversely, how k(ν) depends on ψ :

(3.73)

(3.74)

The number cosψ lies between – 1 and + 1, so a circular solution does indeed work.

Exercise 3.6.6.

Check – 1 ≤ cosψ ≤ 1 for – ∞ < k < ∞ and – ∞ < k < ∞ for – π ≤ ψ ≤ π . Sketch k versus ψ and ψ versus k .

The equation is illustrated in Figure 3.16. The angle that V R makes with V is determined from the equation

(3.75)

Figure 3.16. 

The Voltages V and VR , and the Power Factor cosφ

In the study of power systems, cos φ is a "power factor" that determines how much power is delivered to the resistor. We may denote the power factor as

(3.76)

But cosψ may be written as

(3.77)

But cosψ may be written as

(3.78)

Therefore the square of the power factor η is

(3.79)

The power factor is a maximum of 1 for k(ν) = 0, corresponding to ν = 1 . It is a minimum of 0 for k(ν) = ± ∞, corresponding to ν = 0,∞ (ω = 0,∞).

Exercise 3.6.7.

With k defined as , plot k ²(ν), cosψ , and η ² versus ν .

Exercise 3.6.8.

Find the value of ν that makes the power factor η = 0.707.

3.7. Phasors: Light Scattering by a Slit*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

One of the most spectacular successes for phasor analysis arises in the study of light diffraction by a narrow slit. The experiment is to shine laser light through a slit in an otherwise opaque sheet and observe the pattern of light that falls on a distant screen. When the slit is very narrow compared with the wavelength of the light, then the light that falls on the screen is nearly uniform in intensity. However, when the width of the slit is comparable to the wavelength of the light, the pattern of light that falls on the screen is scalloped in intensity, showing alternating light and dark bands. The experiment, and the observed results, are illustrated in Figure 3.17.

Figure 3.17. 

Light Diffraction by a Slit

Why should this experiment produce this result? Phasor analysis illuminates the question and produces an elegant mathematical description of a profoundly important optical experiment.

Huygens's Principle. We will assume, as Christiaan Huygens did, that the light incident on the slit sets up a light field in the slit that may be modeled by N discrete sources, each of which radiates a “spherical wave of light.” This model is illustrated in Figure 3.18. The distance between sources is d , and N d = L is the width of the slit. Each source is indexed by n , and n runs from 0 to N – 1. The 0 th source is located at the origin of our coordinate system.

The spherical wave radiated by the n th source is described by the equation

(3.80)

The function describes the “electric field” at time t and distance rn from the n th source. The field is constant as long as the variable ω t – (2π / λ)r n is constant. Therefore, if we freeze time at t = t ₀ , the field will be constant on a sphere of radius r n . This is illustrated in Figure 3.18.

Figure 3.18. 

Huygens's Model for Light Diffraction

Exercise 3.7.1.

Fix r n in and show that varies cosinusoidally with time t . Sketch the function and interpret it. What is its period?

Exercise 3.7.2.

Fix t in and show that varies cosinusoidally with radius r n . Sketch the function and interpret it. Call the “period in r n " the “wavelength.” Show that the wavelength is λ .

Exercise 3.7.3.

The “crest of the wave " occurs when ω t – (2π / λ)r n = 0. Show that the crest moves through space at velocity v = ω λ / 2π .

Geometry. If we now pick a point P on a distant screen, that point will be at distance r ₀ from source 0,...,r n from source n,..., and so on. If we isolate the sources 0 and n , then we have the geometric picture of Figure 3.19. The angle φ is the angle that point P makes with the horizontal axis. The Pythagorean theorem says that the connection between distances r ₀ and r n is

(3.81)

Figure 3.19. 

Geometry of the Experiment

Let's try the solution

(3.82) r n = r ₀ + n dsinφ.

This solution produces the approximate identity

r ² ₀ + (n d cos φ)² ≅ r ² ₀

(3.83)

This will be close for . We will assume that the slit width L is small compared to the distance to any point on the screen. Then , in which case the approximate solution for r n is valid for all n . This means that, for any point P on the distant screen, the light contributed by the n th source is approximately

(3.84)

The phasor representation for this function is just

(3.85)

Note that E ₀(φ), the phasor associated with the 0 th source, is . Therefore we may write the phasor representation for the light contributed by the n th source to be

(3.86) E n (φ) = E ₀(φ)e – j(2π / λ)n dsinφ .

This result is very important because it shows the light arriving at point P from different sources to be “out of phase” by an amount that depends on the ratio

Phasors and Interference. The phasor representation for the field observed at point P on the screen is the sum of the phasors contributed by each source:

(3.87)

This is a sum of the form

(3.88)

where the angle θ is (2π / λ)dsinφ . This sum is illustrated in Figure 3.20 for several representative values of θ . Note that for small θ , meaning small φ , the sum has large magnitude, whereas for θ on the order of 2π / N , the sum is small. This simple geometric interpretation shows that for some values of φ , corresponding to some points P on the screen, there will be constructive interference between the phasors, while for other values of φ there will be destructive interference. Constructive interference produces bright light, and destructive interference produces darkness.

Figure 3.20. 

Phasor Sums for Diffraction

The geometry of Figure 3.20 is illuminating. However, we already know from our study of complex numbers and geometric sums that the phasor sum of Equation 3.85 may be written as

(3.89)

This result may be manipulated to produce the form

(3.90)

The magnitude is the intensity of the light at angle φ from horizontal:

(3.91)

Exercise 3.7.4.

Derive Equation 3.90 from Equation 3.89.

Limiting Form. Huygens's model is exact when d shrinks to 0 and N increases to infinity in such a way that N d → L , the slit width. Then

(3.92)

This function is plotted in Figure 3.21 for two values of the width of the slit measured in wavelengths. When (i.e., ), then the light is uniformly distributed on the screen. However, when , then the function has many zeros for | sin φ| < 1, as illustrated in the figure. These zeros correspond to dark spots on the screen where the fields radiated from the infinity of points within the slit interfere destructively.

Figure 3.21. 

Interference Pattern for a Slit

Exercise 3.7.5.

Derive Equation 3.92 from Equation 3.91.

Exercise 3.7.6.

(MATLAB) Plot the discrete approximation

(3.93)

versus sin φ for and N = 2,4,8,16,32. Compare with the continuous, limiting form

(3.94)

3.8. Phasors: Numerical Experiment (Interference Patterns)*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

Let's add two sinusoids whose amplitudes and frequencies are identical and whose phases are different:

(3.95) x ( t ) = A cos ( ω t + φ ) + A cos ( ω t + φ + ψ ) .

Show analytically that this sum has the phasor representation

(3.96)

Interpret this finding. Then write a MATLAB program that computes and plots complex X on the complex plane as ψ varies from 0 to 2π and that plots magnitude, |X|, and phase, arg X , versus the phase angle ψ . (You will have to choose , for a suitable N.) When do you get constructive interfelence and when do you get destructive interference? Now compute and plot x(t) versus t (you will need to discretize t ) for several interesting values of ψ . Explain your interference results in terms of the amplitude and phase of x(t) and the magnitude and phase of X . Use the subplots discussed in "An Introduction to MATLAB" to plot all of your results together.

[4] This means, also, that we have an easy way to synthesize cosines with circuits that obey the equation of a simple harmonic oscillator!

[5] It really is, although it is said to be “single phase” because of the way it is picked off a single phase of a primary source. You will hear more about this in circuits and power courses.

Chapter 4. Linear Algebra

4.1. Linear Algebra: Introduction*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

We use this chapter to introduce students to the algebraic structure of vectors and matrices and to introduce them to matrix computations. These matrix computations are used in the chapters "Vector Graphics", "Filtering", and "Binary Codes" to solve problems in vector graphics, filtering, and binary coding.

Vectors are introduced in the section "Vectors", along with algebraic and geometric interpretations of some fundamental vector operations and properties. The section "Inner Product and Euclidean Norm", the section "Direction Cosines", and the section "Projections" introduce inner products and their applications, including norm, direction cosines, orthogonality, and projections. Some important alternatives to the Euclidean norm are introduced in the section "Other Norms". Matrices are motivated and introduced in this section. The notation in these sections can be daunting to the beginner, so we proceed very carefully, using example after example. In the section "Solving Linear Systems of Equations" we codify the elimination procedures that students have used in high school to solve linear systems of equations. The MATLAB demonstration in Demo 2 shows how to use MATLAB to solve linear equations. The section "Circuit Analysis" shows how linear algebra and MATLAB can be used to analyze dc circuits. The numerical experiment "Circuit Design" gives students practice in building function files in MATLAB and shows how to solve a sequence of linear equations in order to design a circuit with desired properties.

Occasionally we have placed important results in the problems. We feel that students should not miss the material in Exercise 3 in "Vectors", Exercise 3 in "Inner Product and Euclidean Norm", Exercise 3 in "Projections", Exercise 1 in "Matrices", and Exercise 4 in "Matrices".

Introduction

Linear algebra is a branch of mathematics that is used by engineers and applied scientists to design and analyze complex systems. Civil engineers use linear algebra to design and analyze load-bearing structures such as bridges. Mechanical engineers use linear algebra to design and analyze suspension systems, and electrical engineers use it to design and analyze electrical circuits. Electrical, biomedical, and aerospace engineers use linear algebra to enhance X rays, tomographs, and images from space. In this chapter and the next we study two common problems from electrical engineering and use linear algebra to solve them. The two problems are (i) electrical circuit analysis and (ii) coordinate transformations for computer graphics. The first of these applications requires us to understand the solution of linear systems of equations, and the second requires us to understand the representation of mathematical operators with matrices.

Much of linear algebra is concerned with systematic techniques for organizing and solving simultaneous linear equations by elimination and substitution. The following example illustrates the basic ideas that we intend to develop.

Example 4.1. 

A woman steps onto a moving sidewalk at a large airport and stands while the moving sidewalk moves her forward at 1.2 meters/seconds. At the same time, a man begins walking against the motion of the sidewalk from the opposite end at 1.5 meters/second (relative to the sidewalk). If the moving sidewalk is 85 meters long, how far does each person travel (relative to the ground) before they pass each other?

To solve this problem, we first assign a variable to each unknown quantity. Let x ₁ be the distance traveled by the woman, and let x ₂ be the distance traveled by the man. The sum of the two distances is 85 meters, giving us one equation:

(4.1) x ₁ + x ₂ = 85 .

Our second equation is based on the time required before they pass. Time equals distance divided by rate, and the time is the same for both people:

(4.2)

We may substitute Equation 4.2 into Equation 4.1 to obtain the result , or

(4.3) 5 x ₂ = 85 ⇒ x ₂ = 17 .

Combining the result from Equation 4.3 with that of Equation 4.1, we find that

(4.4) x ₁ = 68 .

So the man travels 17 meters, and the woman travels 68 meters.

Equation 4.1 and Equation 4.2 are the key equations of Equation 4.1. They may be organized into the “matrix equation”

(4.5)

The rules for matrix-vector multiplication are evidently

(4.6)

(4.7) ( 0 . 3 ) x ₁ +

( – 1 . 2 ) x ₂ = 0 .

Equation 4.2 and Equation 4.3 may be organized into the matrix equation

(4.8)

This equation represents one partially solved form of Equation 4.5, wherein we have used the so-called Gauss elimination procedure to introduce a zero into the matrix equation in order to isolate one variable. The MATLAB software contains built-in procedures to implement Gauss elimination on much larger matrices. Thus MATLAB may be used to solve large systems of linear equations.

Before we can apply linear algebra to more interesting physical problems, we need to introduce the mathematical tools we will use.

4.2. Linear Algebra: Vectors*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

For our purposes, a vector is a collection of real numbers in a one- dimensional array.[6] We usually think of the array as being arranged in a column and write

.

Notice that we indicate a vector with boldface and the constituent elements with subscripts. A real number by itself is called a scalar, in distinction from a vector or a matrix. We say that x is an n-vector, meaning that x has n elements. To indicate that x ₁ is a real number, we write

(4.9) x ₁ ∈ R ,

meaning that x ₁ is contained in R, the set of real numbers. To indicate that x is a vector of n real numbers, we write

(4.10) x ∈ R n ,

meaning that x is contained in R n , the set of real n-tuples. Geometrically, R n is n-dimensional space, and the notation x ∈ R n means that x is a point in that space, specified by the n coordinates x ₁,x ₂,...,x n . Figure 4.1 shows a vector in R ³ , drawn as an arrow from the origin to the point x . Our geometric intuition begins to fail above three dimensions, but the linear algebra is completely general.

Figure 4.1. 

A Vector in R³

We sometimes find it useful to sketch vectors with more than three dimensions in the same way as the three-dimensional vector of Figure 4.1. We then consider each axis to represent more than one dimension, a hyperplane, in our n-dimensional space. We cannot show all the details of what is happening in n-space on a three-dimensional figure, but we can often show important features and gain geometrical insight.

Vector Addition. Vectors with the same number of elements can be added and subtracted in a very natural way:

(4.11)

Example 4.2. 

The difference between the vector and the vector is the vector . These vectors are illustrated in Figure 4.2. You can see that this result is consistent with the definition of vector subtraction in Equation 4.11. You can also picture the subtraction in Figure 4.2 by mentally reversing the direction of vector y to get – y and then adding it to x by sliding it to the position where its tail coincides with the head of vector x . (The head is the end with the arrow.) When you slide a vector to a new position for adding to another vector, you must not change its length or direction.

Figure 4.2. 

Subtraction of Vectors

Exercise 4.2.1.

Compute and plot x + y and x – y for each of the following cases:

;

;

.

Scalar Product. Several different kinds of vector multiplication are defined.[7] We begin with the scalar product. Scalar multiplication is defined for scalar a and vector x as

(4.12)

If |a| < 1, then the vector ax is "shorter" than the vector x; if |a| > 1, then the vector ax is ‚"longer" than x. This is illustrated for a 2-vector in Figure 4.3.

Figure 4.3. 

The Scalar Product ax

Exercise 4.2.2.

Compute and plot the scalar product ax when for each of the following scalars:

a = 1;

a = – 1;

a = – 1 / 4;

a = 2.

Exercise 4.2.3.

Given vectors x,y,z ∈ R n and the scalar a ∈ R, prove the following identities:

x + y = y + x. Is vector addition commutative?

(x + y) + z = x + (y + z). Is vector addition associative?

a(x + y) = ax + ay. Is scalar multiplication distributive over vector addition?

4.3. Linear Algebra: Inner Product and Euclidean Norm*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

The inner product (x,y) between vectors x and y is a scalar consisting of the following sum of products:

(4.13)(x,y) = x ₁ y ₁ + x ₂ y ₂ + x ₃ y ₃ + ⋯ + x n y n .

This definition seems so arbitrary that we wonder what uses it could possibly have. We will show that the inner product has three main uses:

computing length or “norm”,

finding angles between vectors and checking for “orthogonality”, and

computing the “component of one vector along another” (projection).

Since the inner product is so useful, we need to know what algebraic operations are permitted when we are working with inner products. The following are some properties of the inner product. Given x,y,z ∈ R n and a ∈ R,

(x,y) = (y,x);

(ax,y) = a(x,y) = (x,ay); and

(x,y + z) = (x,y) + (x,z).

Exercise 4.3.1.

Prove the three preceding properties by using the definition of inner product. Is the equation x(y,z) = (x,y)z also a true property? Prove or give a counterexample.

Euclidean Norm. Sometimes we want to measure the length of a vector, namely, the distance from the origin to the point specified by the vector's coordinates. A vector's length is called the norm of the vector. Recall from Euclidean geometry that the distance between two points is the square root of the sum of the squares of the distances in each dimension. Since we are measuring from the origin, this implies that the norm of the vector x is

(4.14)

Notice the use of the double vertical bars to indicate the norm. An equivalent definition of the norm, and of the norm squared, can be given in terms of the inner product:

(4.15)

or

(4.16) ∥ x ∥ ² =

( x , x )

.

Example 4.3. 

The Euclidean norm of the vector

(4.17)

is

An important property of the norm and scalar product is that, for any x ∈ R n and a ∈ R,

(4.18)||ax|| = |a|||x||.

So we can take a scalar multiplier outside of the norm if we take its absolute value.

Exercise 4.3.2.

Prove Equation 4.18.

Cauchy-Schwarz Inequality. Inequalities can be useful engineering tools. They can often be used to find the best possible performance of a system, thereby telling you when to quit trying to make improvements (or proving to your boss that it can't be done any better). The most fundamental inequality in linear algebra is the Cauchy-Schwarz inequality. This inequality says that the inner product between two vectors x and y is less than or equal (in absolute value) to the norm of x times the norm of y , with equality if and only if y = αx:

(4.19)|(x,y)| ≤ ||x||||y||.

To prove this theorem, we construct a third vector z = λx – y and measure its norm squared:

(4.20) | | λ x

–

y | | ² =

( λ x – y , λ x – y )

= λ ² | | x | | ² – 2 λ ( x , y )

+

| | y | | ² ≥ 0 .

So we have a polynomial in λ that is always greater than or equal to 0 (because every norm squared is greater than or equal to 0). Let's assume that x and y are given and minimize this norm squared with respect to λ . To do this, we take the derivative with respect to λ and equate it to 0:

(4.21)

When this solution is substituted into the formula for the norm squared in Equation 4.20, we obtain

(4.22)

which simplifies to

(4.23)

The proof of the Cauchy-Schwarz inequality is completed by taking the positive square root on both sides of Equation 4.23. When y = αx, then

(4.24)

which shows that equality holds in Equation 4.19 when y is a scalar multiple of x .

Exercise 4.3.3.

Use the Cauchy-Schwarz inequality to prove the triangle inequality, which states

(4.25) | | x + y | | ≤ | | x | | + | | y | | .

Explain why this is called the triangle inequality.

4.4. Linear Algebra: Direction Cosines*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

Unit Vectors. Corresponding to every vector x is a unit vector u x pointing in the same direction as x . The term unit vector means that the norm of the vector is 1:

(4.26) | |

ux | |

= 1 .

The question is, given x , how can we find u x ? The first part of the answer is that u x will have to be a positive scalar multiple of x in order to point in the same direction as x , as shown in Figure 4.4. Thus

(4.27) ux = α x .

Figure 4.4. 

The Unit Vector ux

But what is α ? We need to choose α so that the norm of u x will be 1:

(4.28) | |

ux | |

= 1

(4.29) | α x | | = 1

(4.30) | α | | | x | | = 1

(4.31)

We have dropped the absolute value bars on α because ||x|| is positive. The α that does the job is 1 over the norm of x . Now we can write formulas for u x in terms of x and x in terms of u x :

(4.32)

(4.33) x =

| | x | |

ux .

So the vector x is its direction vector u x , scaled by its Euclidean norm.

Unit Coordinate Vectors. There is a special set of unit vectors called the unit coordinate vectors. The unit coordinate vector e i is a unit vector in ∇ n that points in the positive direction of the i th coordinate axis. Figure 4.5 shows the three unit coordinate vectors in ∇³ .

Figure 4.5. 

Unit Coordinate Vectors in R³

For three-dimensional space, R ³ , the unit coordinate vectors are

(4.34)

More generally, in n-dimensional space, there are n unit coordinate vectors given by

Figure 4.6. 

You should satisfy yourself that this definition results in vectors with a norm of l.

Exercise 4.4.1.

Find the norm of the vector au where u is a unit vector.

Exercise 4.4.2.

Find the unit vector u x in the direction of

;

;

Direction Cosines. We often say that vectors “have magnitude and direction.” This is more or less obvious from "Linear Algebra: Vectors", where the three-dimensional vector x has length and points in a direction defined by the components x ₁,x ₂ , and x ₃ . It is perfectly obvious from Equation 4.33 where x is written as x = ||x||ux . But perhaps there is another representation for a vector that places the notion of magnitude and direction in even clearer evidence.

Figure 4.7 shows an arbitrary vector x ∈ R³ and the three-dimensional unit coordinate vectors e₁,e₂,e₃ . The inner product between the vector x and the unit vector e k just reads out the k th component of x :

(4.35)

Since this is true even in R n , any vector x ∈ R n has the following representation in terms of unit vectors:

(4.36)

Figure 4.7. 

The Three-Dimensional Unit Vectors

Let us now generalize our notion of an angle θ between two vectors to R n as follows:

(4.37)

The celebrated Cauchy-Schwarz inequality establishes that – 1 – ≤ cos θ ≤ 1. With this definition of angle, we may define the angle θ k that a vector makes with the unit vector e k to be

(4.38)

But the norm of e k is 1, so

(4.39)

When this result is substituted into the representation of x in Equation 4.36, we obtain the formula

(4.40)

This formula really shows that the vector x has “magnitude” ||x|| and direction and that the magnitude and direction are sufficient to determine x . We call ( cosθ ₁ , cos θ ₂,..., cos θ n ) the direction cosines for the vector x . In the three-dimensional case, they are illustrated in Figure 4.7.

Exercise 4.4.3.

Show that Equation 4.37 agrees with the usual definition of an angle in two dimensions.

Exercise 4.4.4.

Find the cosine of the angle between x and y where

;

;

If we compare Equation 4.33 and Equation 4.40 we see that the direction vector u x is composed of direction cosines:

(4.41)

With this definition we can write Equation 4.40 compactly as

Here x is written as the product of its magnitude ||x|| and its direction vector u x . Now we can give an easy procedure to find a vector's direction angles:

(i) find ||x||;

(ii) calculate ; and

(iii) take the arc cosines of the elements of u x .

Step (iii) is often unnecessary; we are usually more interested in the direction vector (unit vector) ux . Direction vectors are used in materials science in order to study the orientation of crystal lattices and in electromagnetic field theory to characterize the direction of propagation for radar and microwaves. You will find them of inestimable value in your courses on electromagnetic fields and antenna design.

Exercise 4.4.5.

Sketch an arbitrary unit vector u ∈ R³ . Label the direction cosines and the components of

Exercise 4.4.6.

Compute the norm and the direction cosines for the vector

Exercise 4.4.7.

Prove that the direction cosines for any vector satisfy the equality

(4.42)

What does this imply about the number of scalars needed to determine a vector x ∈ R n ?

Exercise 4.4.8.

Astronomers use right ascension, declination, and distance to locate stars. On Figure 4.7 these are, respectively, , and ||x||. Represent in terms of φ,θ ₃ , and ||x|| only. (That is, find equations that give φ,θ ₃ , and ||x|| in terms of x ₁,x ₂ , and x₃ , and find equations that give x ₁,x ₂ , and x₃ in terms of φ,θ ₃ , and ||x||.)

Exercise 4.4.9.

(MATLAB) Write a MATLAB function that accepts any vector x ∈ R n and computes its norm and direction cosines. Make x an input variable, and make ||x|| and u x output variables.

Exercise 4.4.10.

Let x and y denote two vectors in R n with respective direction cosines ( cos θ ₁ , cos θ ₂ , ...

, cos θ n ) and ( cos φ ₁ , cos φ ₂ , ...

, cos φ n ). Prove that ψ , the angle between x and y , is

(4.43)

Specialize this result to R ² for insight.

Exercise 4.4.11.

Compute the angle between the vectors and . Sketch the result.

4.5. Linear Algebra: Projections*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

Orthogonality. When the angle between two vectors is , we say that the vectors are orthogonal. A quick look at the definition of angle (Equation 12 from "Linear Algebra: Direction Cosines") leads to this equivalent definition for orthogonality:

(4.44)

For example, in Figure 1(a), the vectors and are clearly orthogonal, and their inner product is zero:

(4.45) ( x , y ) = 3 ( – 2 ) + 1 ( 6 ) = 0 .

In Figure 1(b), the vectors , and are mutually orthogonal, and the inner product between each pair is zero:

(4.46) ( x , y ) = 3 ( – 2 ) + 1 ( 6 ) + 0 ( 0 ) = 0

(4.47) ( x , z ) = 3 ( 0 ) + 1 ( 0 ) + 0 ( 4 ) = 0

(4.48) ( y , z ) = – 2 ( 0 ) + 6 ( 0 ) + 0 ( 4 ) = 0 .

Figure 4.8. 

(a)

(b)

Orthogonality of Vectors

Exercise 4.5.1.

Which of the following pairs of vectors is orthogonal:

;

;

x = e₁,y = e₃ ;

,

We can use the inner product to find the projection of one vector onto another as illustrated in Figure 2. Geometrically we find the projection of x onto y by dropping a perpendicular from the head of x onto the line containing y . The perpendicular is the dashed line in the figure. The point where the perpendicular intersects y (or an extension of y ) is the projection of x onto y , or the component of x along y . Let's call it z .

Figure 4.9. 

Component of One Vector along Another

Exercise 4.5.2.

Draw a figure like Figure 2 showing the projection of y onto x.

The vector z lies along y , so we may write it as the product of its norm ||z|| and its direction vector u y :

(4.49)

But what is norm ||z||? From Figure 2 we see that the vector x is just z , plus a vector v that is orthogonal to y :

(4.50)

Therefore we may write the inner product between x and y as

(4.51)(x,y) = (z + v,y) = (z,y) + (v,y) = (z,y).

But because z and y both lie along y , we may write the inner product (x,y) as

(4.52)

From this equation we may solve for and substitute ||z|| into Equation 6 to write z as

(4.53)

Equation 10 is what we wanted–an expression for the projection of x onto y in terms of x and y .

Exercise 4.5.3.

Show that ||z|| and z may be written in terms of cosθ for θ as illustrated in Figure 2:

(4.54)

(4.55)

Orthogonal Decomposition. You already know how to decompose a vector in terms of the unit coordinate vectors,

(4.56)

In this equation, is the component of x along e k , or the projection of x onto e k , but the set of unit coordinate vectors is not the only possible basis for decomposing a vector. Let's consider an arbitrary pair of orthogonal vectors x and y :

(4.57) ( x , y ) = 0 .

The sum of x and y produces a new vector w , illustrated in Figure 3, where we have used a two-dimensional drawing to represent n dimensions. The norm squared of w is

This is the Pythagorean theorem in n dimensions! The length squared of w is just the sum of the squares of the lengths of its two orthogonal components.

Figure 4.10. 

Sum of Orthogonal Vectors

The projection of w onto x is x , and the projection of w onto y is y :

(4.58) w = ( 1 ) x + ( 1 ) y .

If we scale w by a to produce the vector z = aw, the orthogonal decomposition of z is

(4.59) z = a w = ( a ) x + ( a ) y .

Let's turn this argument around. Instead of building w from orthogonal vectors x and y , let's begin with arbitrary w and x and see whether we can compute an orthogonal decomposition. The projection of w onto x is found from Equation 10:

(4.60)

But there must be another component of w such that w is equal to the sum of the components. Let's call the unknown component w y . Then

(4.61) w = wx + wy .

Now, since we know w and w x already, we find w y to be

(4.62)

Interestingly, the way we have decomposed w will always produce w x and w >y orthogonal to each other. Let's check this:

(4.63)

To summarize, we have taken two arbitrary vectors, w and x , and decomposed w into a component in the direction of x and a component orthogonal to x .

4.6. Linear Algebra: Other Norms*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

Sometimes we find it useful to use a different definition of distance, corresponding to an alternate norm for vectors. For example, consider the l-norm defined as

(4.64) | | x | | ₁ =

( | x ₁ | + | x ₂ | + ⋯ + | x n | )

,

where |x i | is the magnitude of component xi . There is also the sup-norm, the “supremum” or maximum of the components x ₁,...,x n :

(4.65)

The following examples illustrate what the Euclidean norm, the l-norm, and the sup-norm look like for typical vectors.

Example 4.4. 

Consider the vector . Then

;

||x||₁ = (| – 3| + |1| + |2|) = 6; and

Example 4.5. 

Figure 1 shows the locus of two-component vectors with the property that ||x|| = 1,||x||₁ = 1, or

Figure 4.11. 

Locus of Two-Dimensional Vectors Whose Various Norms Are 1

The next example shows how the l-norm is an important part of city life.

Example 4.6. 

The city of Metroville was laid out by mathematicians as shown in Figure 2. A person at the intersection of Avenue 0 and Street – 2 (point A) is clearly two blocks from the center of town (point C). This is consistent with both the Euclidean norm

(4.66)

and the l-norm

(4.67) | | A | | ₁ =

( | 0 | + | – 2 | )

= 2 .

But how far from the center of town is point B at the intersection of Avenue-2 and Street 1? According to the Euclidean norm, the distance is

(4.68)

Figure 4.12. 

Metroville, U.S.A.

While it is true that point B is blocks from C , it is also clear that the trip would be three blocks by any of the three shortest routes on roads. The appropriate norm is the l-norm:

(4.69)

Even more generally, we can define a norm for each value of p from 1 to infinity. The so-called p-norm is

(4.70)

Exercise 4.6.1.

Show that the Euclidean norm is the same as the p-norm with p = 2 and that the 1-norm is the p-norm with p = 1. (It can also be shown that the sup-norm is like a p-norm with p = ∞.)

DEMO 4.1 (MATLAB). From the command level of MATLAB, type the following lines:

>> x = [1;3;-2;4]
>> y = [0;1;2;-0.5]
>> x - y

Check to see whether the answer agrees with the definition of vector subtraction. Now type

>> a = -1.5
>> a * x

Check the answer to see whether it agrees with the definition of scalar multiplication. Now type

>> x' * y

This is how MATLAB does the inner product. Check the result. Type

>> norm(y)
>> sqrt(y' * y)

>> norm(y,1)
>> norm(y' * y)

Now type your own MATLAB expression to find the cosine of the angle between vectors x and y . Put the result in variable t . Then find the angle θ by typing

>> theta = acos(t)

The angle θ is in radians. You may convert it to degrees if you wish by multiplying it by 180 / π :

>> theta = theta * (180/pi)

4.7. Linear Algebra: Matrices*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

The word matrix dates at least to the thirteenth century, when it was used to describe the rectangular copper tray, or matrix, that held individual leaden letters that were packed into the matrix to form a page of composed text. Each letter in the matrix, call it a ij , occupied a unique position (i j) in the matrix. In modern day mathematical parlance, a matrix is a collection of numbers arranged in a two-dimensional array (a rectangle). We indicate a matrix with a boldfaced capital letter and the constituent elements with double subscripts for the row and column:

Figure 4.13. 

Matrix

In this equation A is an mxn matrix, meaning that A has m horizontal rows and n vertical columns. As an extension of the previously used notation, we write A ∈ R m x n to show that A is a matrix of size m x n with a ij ∈ R . The scalar element a ij is located in the matrix at the i th row and the j th column. For example, a ₂₃ is located in the second row and the third column as illustrated in Figure 4.13.

The main diagonal of any matrix consists of the elements a ii . (The two subscripts are equal.) The main diagonal runs southeast from the top left corner of the matrix, but it does not end in the lower right corner unless the matrix is square .

The transpose of a matrix A ∈ R m × n is another matrix B whose element in row j and column i is b ji = a ij for 1 ≤ i ≤ m and 1 ≤ j ≤ n . We write B = A T to indicate that B is the transpose of A . In MATLAB, transpose is denoted by A'. A more intuitive way of describing the transpose operation is to say that it flips the matrix about its main diagonal so that rows become columns and columns become rows.

Exercise 4.7.1.

If A ∈ R m × n , then . Find the transpose of the matrix

Matrix Addition and Scalar Multiplication. Two matrices of the same size (in both dimensions) may be added or subtracted in the same way as vectors, by adding or subtracting the corresponding elements. The equation C = A ± B means that for each i and j,c ij = a ij ± b ij . Scalar multiplication of a matrix multiplies each element of the matrix by the scalar:

(4.71)

Matrix Multiplication. A vector can be considered a matrix with only one column. Thus we intentionally blur the distinction between R n × 1 and Rn . Also a matrix can be viewed as a collection of vectors, each column of the matrix being a vector:

(4.72)

In the transpose operation, columns become rows and vice versa. The transpose of an n × 1 matrix, a column vector, is a 1 × n matrix, a row vector:

(4.73)

Now we can define matrix-matrix multiplication in terms of inner products of vectors. Let's begin with matrices A ∈ R m × n and B ∈ R n × p . To find the product AB, first divide each matrix into column vectors and row vectors as follows:

Figure 4.14. 

Thus a i is the i th column of A and α T j is the j th row of A. For matrix multiplication to be defined, the width of the first matrix must match the length of the second one so that all rows α T i and columns b i have the same number of elements n . The matrix product, C = AB, is an m × p matrix defined by its elements as . In words, each element of the product matrix, c ij , is the inner product of the i th row of the first matrix and the j th column of the second matrix.

For n-vectors x and y , the matrix product x T y takes on a special significance. The product is, of course, a 1 × 1 matrix (a scalar). The special significance is that x T y is the inner product of x and y :

Figure 4.15. 

Thus the notation x T y is often used in place of (x,y). Recall from Demo 1 from "Linear Algebra: Other Norms" that MATLAB uses x'*y to denote inner product.

Another special case of matrix multiplication is the outer product. Like the inner product, it involves two vectors, but this time the result is a matrix:

Figure 4.16. 

In the outer product, the inner products that define its elements are between one-dimensional row vectors of x and one-dimensional column vectors of y T , meaning the (i,j) element of A is x i y j .

Exercise 4.7.2.

Find C = AB where A and B are given by

;

.

There are several other equivalent ways to define matrix multiplication, and a careful study of the following discussion should improve your understanding of matrix multiplication. Consider A ∈ R m × n ,B ∈ R n × p , and C = AB so that C ∈ R m × p . In pictures, we have

(4.74)

In our definition, we represent C on an entry-by-entry basis as

(4.75)

In pictures,

Figure 4.17. 

You will prove in Exercise 3 that we can also represent C on a column basis:

(4.76)

Figure 4.18. 

Finally, C can be represented as a sum of matrices, each matrix being an outer product:

(4.77)

Figure 4.19. 

A numerical example should help clarify these three methods.

Example 4.7. 

Let

(4.78)

Using the first method of matrix multiplication, on an entry-by-entry basis, we have

(4.79)

or

(4.80)

or

(4.81)

On a column basis,

(4.82)

Collecting terms together, we have

(4.83)

On a matrix-by-matrix basis,

(4.84)

(4.85)

(4.86)

(4.87)

as we had in each of the other cases. Thus we see that the methods are equivalent-simply different ways of organizing the same computation!

Exercise 4.7.3.

Prove that Equations 9, 11, and 13 are equivalent definitions of matrix multiplication. That is, if C = AB where A ∈ R m × n and B ∈ R n × p , show that the matrix-matrix product can also be defined by

(4.88)

and, if c k is the k th column of C and a k is the k th column of A , then

(4.89)

Show that the matrix C may also be written as the ‚ "sum of outer products"

(4.90)

Write out the elements in a typical outer product a k β T k .

Exercise 4.7.4.

Given A ∈ R m × n ,B ∈ R p × q , and C ∈ R r × s , for each of the following postulates, either prove that it is true or give a counterexample showing that it is false:

.

A B = BA when n = p and m = q . Is matrix multiplication commutative?

A(B + C) = A B + AC when n = p = r and q = s . Is matrix multiplication distributive over addition?

(AB ) T = B T A T when n = p .

( A B ) C = A ( B C ) when n = p and q = r . Is matrix multiplication associative?

Example 4.8. Rotation

We know from the chapter on complex numbers that a complex number z ₁ = x ₁ + j y ₁ may be rotated by angle θ in the complex plane by forming the product

(4.91) z ₂ = e jθ z ₁ .

When written out, the real and imaginary parts of z ₂ are

(4.92)

If the real and imaginary parts of z ₁ and z ₂ are organized into vectors z ₁ and z ₂ as in the chapter on complex numbers, then rotation may be carried out with the matrix-vector multiply

(4.93)

We call the matrix a rotation matrix.

Exercise 4.7.5.

Let R(θ) denote a 2 × 2 rotation matrix. Prove and interpret the following two properties:

R T (θ) = R( – θ);

4.8. Linear Algebra: Solving Linear Systems of Equations*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

We are now equipped to set up systems of linear equations as matrix- vector equations so that they can be solved in a standard way on a computer. Suppose we want to solve the equations from Example 1 from "Linear Algebra: Introduction" for x ₁ and x ₂ using a computer. Recall that Equations 1 and 2 from Linear Algebra: Introduction are

(4.94)

The first step is to arrange each equation with all references to x ₁ in the first column, all references to x ₂ in the second column, etc., and all constants on the right-hand side:

(4.95)

Then the equations can be converted to a single matrix-vector equation. The coefficients form a matrix, keeping their same relative positions, and the variables and constants each form a vector:

(4.96)

Exercise 4.8.1.

Verify by the rules of matrix multiplication that the system of equations in 2 in equivalent to the matrix equation in 3.

Equation 3 is of the general form

(4.97) A x = b

where in this case

(4.98)

Given any A ∈ R n × n and b ∈ R n , MATLAB can solve Equation 4 for x (as long as a solution exists). Key ideas in the solution process are the identity matrix and the inverse of a matrix.

When the matrix A is the 1 × 1 matrix a , the vector x is the l-vector x , and the vector b is the l-vector b , then the matrix equation A x = b becomes the scalar equation

(4.99) a x = b.

The scalar a – 1 is the inverse of the scalar a , so we may multiply on both sides of Equation 6 to obtain the result

(4.100)

We would like to generalize this simple idea to the matrix equation A x = b so that we can find an inverse of the matrix A , called A – 1 , and write

(4.101)

In this equation the matrix I is the identity matrix

(4.102)

It is clear that the identity matrix I and the inverse of a matrix, A – 1 , play a key role in the solution of linear equations. Let's study them in more detail.

The Matrix Identity. When we multiply a scalar by 1, we get back that same scalar. For this reason, the number 1 is called the multiplicative identity element. This may seem trivial, but the generalization to matrices is more interesting. The question is, is there a matrix that, when it multiplies another matrix, does not change the other matrix? The answer is yes. The matrix is called the identity matrix and is denoted by I . The identity matrix is always square, with whatever size is appropriate for the matrix multiplication. The identity matrix has l's on its main diagonal and 0 ' s everywhere else. For example,

(4.103)

The subscript 5 indicates the size. In terms of the unit coordinate vectors e i , we can also write the n × n identity matrix as

(4.104)

For any matrix A ∈ R n × p , we have

(4.105)A = I n A.

For p = 1, we obtain the following special form for any vector x ∈ R n :

(4.106)x = I n x.

This last equation can be written as the sum

(4.107)

This is a special case of one of the results you proved in Exercise 3 from "Linear Algebra: Matrices". Figure 1 illustrates Equation 14 and shows how the vector x can be broken into component vectors x i e i , each of which lies in the direction of one coordinate axis. The values of the x ' i s are the coordinates of x , and their magnitudes are also the lengths of the component vectors.

Figure 4.20. 

Breaking a Vector into Components

Exercise 4.8.2.

Use Equation 13 and the rules for matrix multiplication to show that x ∈ R n may also be written as

(4.108)

This verifies Equation 11 from "Linear Algebra: Direction Cosines".

When the product of two numbers is 1 (the identity element), we say that they are inverses of each other, like 2 and 0.5. Likewise, we say that two square matrices are inverses of each other if their product is the identity matrix:

(4.109) A B = I ⇔ B = A – 1 .

An interesting and useful result is implied by this definition. Take the first form of Equation 16 and multiply by B from the left:

(4.110)

We emphasize that the inference made in the last step here is only valid when B and A are square matrices. The result means that, even though matrix multiplication is not commutative in general, we have a special case where it is commutative. If A and B are inverses of each other, then

(4.111) A B = B A = I .

Exercise 4.8.3.

Prove that the inverse of the 2 × 2 rotation matrix R(θ) is R T (θ).

Matrices that are not square do not have inverses. In fact, not all square matrices have inverses. So it becomes an important issue to determine which matrices do have inverses. If a matrix has an inverse, the inverse is unique. If a square matrix has no inverse, it is called a singular matrix. The determinant of a square matrix is a scalar computed from the numbers in the matrix. It tells us whether a matrix will have an inverse or not. A matrix is singular if and only if its determinant is zero.[8] In MATLAB, the notation det(A) is used to compute the determinant. Whenever the matrix A in Equation 4 is singular, it means one of two things about the system of equations represented: either the equations are inconsistent and there is no solution, or the equations are dependent and there are infinitely many solutions.

Solving A x = b. Let's now study ways to solve the matrix equation A x = b. We will assume that a unique solution for x exists. Thus a unique matrix A – 1 exists with the property A – 1A = I. The trick is to find it. Here is one procedure.

For convenience, rewrite the matrix equation Ax = b as

(4.112)

The matrix is called the augmented matrix for the system of equations. The augmented matrix may be viewed as a systematic way of writing all the information necessary to solve the equations.

The advantage of Equation 19 is that we may premultiply both sides by any matrix C ₁ (of compatible size), and the right-hand side remains zero (although this is equivalent to multiplying on both sides of Equation 4, which some may prefer). We can repeat this operation as often as we like with matrices C₂,C₃ , etc. The general result is

(4.113)

Now suppose that we have found a sequence of matrices C₁,... , C m that transforms the matrix A to the identity matrix:

C m ⋯C₂C₁A = I. The augmented matrix equation in 20 simplifies to

(4.114)

which can be multiplied out to reveal that the solution for x is

(4.115) x = C m ⋯ C₂ C₁ b .

The method may be summarized as follows:

form the augmented matrix [ A b ] for the system;

premultiply [ A b ] by a sequence of selected matrices C i , designed to take A to I ; and

when A is transformed to I , b will be transformed to x , so the solution will appear as the last column of the transformed augmented matrix.

We may also conclude that the product of the matrices Ci must be the inverse of A since A – 1 is the unique matrix for which A – 1A = I. In solving for x by this method, we have found A – 1 implicitly as the product C m ⋯C₂C₁ .

Example 4.9. 

Consider the equation

(4.116)

(4.117)

The augmented matrix for this equation is

(4.118)

Now if we could add – 2 / 3 times the first row to the second row, we would get 0 in the lower left corner. This is the first step in transforming A to the identity I. We can accomplish this row operation with the matrix

(4.119)

(4.120)

Now adding – 3 / 10 times the new second row to the first row will introduce 0 in the (1,2) position, bringing us closer still to the identity. Thus

(4.121)

(4.122)

We now complete the transformation to identity by normalizing each row to get the needed l's:

(4.123)

(4.124)

According to the last column, the solution is

(4.125)

We note in passing that the inverse of A is the product of the C ' s, so

The method we have just used, combined with a particular way of choosing the C i matrices, is called Gauss elimination. Gauss elimination requires less computation than finding the inverse of A because only the effect of A – 1 on the specific vector b is computed. MATLAB can solve for x by either method, as shown in Demo 4.2. For hand computations, we suggest choosing the C i matrices so that C ₁ produces O ' s everywhere below the diagonal in the first column, C ₂ produces O ' s below the diagonal in the second column, and so on up to C n – 1 . Then C n produces O ' s above the diagonal in the n th column, C n + 1 produces O ' s above the diagonal in column n – 1, etc. The last one, C2n – 1 , normalizes the diagonal elements to l's. We have assumed for simplicity that no O ' s on the diagonal will be encountered in hand computations.

Exercise 4.8.4.

Check that A – 1A = I in Example 1

Exercise 4.8.5.

Augment Equation 3 as in Equation 19 and use the technique of Gauss elimination to solve for x .

Demo 1 (MATLAB). From the command level of MATLAB, solve the matrix equation of Example 1 from "Linear Algebra: Introduction" by typing

>> A = [1 1;0.3 -1.2]
>> b = [85;0]

You have entered the matrices A and b, which describe the problem. You can now solve for x by finding the inverse of A and multiplying b:

>> x = inv(A) * b

In this example the inverse is computed quickly because A is a small matrix. For a large (say, 30 × 30) matrix, the answer would take longer to compute, and the more efficient method of Gauss elimination would reduce waiting time. You can use Gauss elimination in MATLAB by typing

>> x = A \ b

You should get the same answer. Now type the MATLAB code required to compute Ax and to show A x – b = 0.

Exercise 4.8.6.

(MATLAB) Write the following system of equations as A x = b and solve using MATLAB:

(4.126)

(4.127) 4 x ₃ = 7 x ₂ – 5

(4.128)

4.9. Linear Algebra: Circuit Analysis*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

In this section we use the linear algebra we have developed to find the voltages and currents in a simple electrical circuit, such as the one shown in Figure 1. There are many reasons why this might be necessary; in this example we need to know the current flowing through the lamp to tell whether or not it will glow. Too little current will have no visible effect; too much current will cause the lamp to burn out. We will apply a few physical laws relating the voltages and currents in a circuit, turn these laws into systems of linear equations, and solve the equations for the voltages and currents.

Figure 4.21. 

A Simple Electrical Circuit

Current, Voltage, and Resistance. We will use three physical quantities in our analysis of electrical circuits: current, voltage, and resistance. Current is the flow of electrical charge from one place to another. Electrons flowing through a wire or through some other electronic device comprise a current. Voltage is a difference in electric potential that makes electrons flow. Voltage is sometimes called electromotive force because it is like a “force” that moves electrons. Resistance is a property of the device through which the electron current flows. The lower the resistance of a device, the more easily current can flow through the device.

The analogy of water flowing through pipes can help you develop intuition about electrical circuits. In this analogy, electrical current corresponds to the flow rate of water. Voltage corresponds to the pressure that forces the water to flow, and resistance is the friction of flow. A small pipe would impede the flow of water more than would a large pipe, so the small pipe would correspond to a higher resistance. While this analogy can be helpful, keep in mind that electricity is not water. All analogies break down at some point.

We measure electrical current in amperes. The standard symbol for current is i , and the direction of positive flow is indicated by an arrow on the circuit diagram. The arrow is for reference only; if the true current is in the opposite direction, we get negative values for i . Because electrons are negatively charged, current is defined as flowing in the opposite direction as electron motion. But to reduce confusion, you should learn to think in terms of current rather than electron motion.

A point in a circuit where several devices are connected together is called a node. The conservation law for current says that “what flows in must flow out of a node,” a principle known as Kirchhoff's current law. Kirchhoff's current law states that the sum of all currents leaving a node is zero. In this law, a current entering the node is considered to be a negative current leaving the node.

Voltage is measured in volts and is usually written as v (or e ). Since voltage is a difference in potential between two points (nodes), we can show it on a circuit diagram with a + and a –sign to indicate which two nodes we are comparing and which one of the nodes is considered negative. As with current, the markings are for reference only and we may end up with a negative value of v .

In an electrical circuit, one node is usually chosen as a reference node and is considered to have a voltage of zero. Then the voltage at every other node is measured with respect to the reference node. This saves us the trouble of always specifying pairs of nodes for voltage measurements and marking + and – signs for each voltage. Other names for the reference node are common and ground.

A constant voltage source is a device that always forces the voltage between its two terminals to be a constant value. In Figure 1 the circle at the left represents a constant voltage source of 5 volts, so that the voltage at the upper ( + ) end is always exactly 5 volts higher than the voltage at the lower (-) end. A voltage source is something like a battery, but idealized. Real batteries do not maintain a constant output voltage under all conditions.

Resistance is measured in ohms and is denoted by R . A resistor is shown as a zig-zag line in circuit diagrams and labeled with the value of its resistance in ohms. In this chapter we will consider only devices whose resistance is positive and the same in both directions. Ohm's law, also called the resistor law, relates the voltage and current in a resistor. For the resistor shown in Figure 2, with reference directions assigned to v and i as shown, Ohm's law is

(4.129) v = i R .

Note that current flows from + to - through the resistor.

Figure 4.22. 

Ohm's Law

Example 4.10. 

Ohm's law and Kirchhoff's current law are the only principles we need to write equations that will allow us to find the voltages and currents in the resistive circuit of Figure 1. We begin by choosing a reference node and assigning variables to the voltages at every other node (with respect to the reference node). These choices are shown in Figure 3.

Figure 4.23. 

Assigning Node Voltages

The constant voltage source forces v ₁ to be exactly 5 volts higher than the reference node. Thus

(4.130) v ₁ = 5 .

Next we write equations by applying Kirchhoff's current law to each node in the circuit (except the reference node and v ₁ , whose voltages we already know). At the node labeled v ₂ are three paths for leaving current. The current leaving through the 50 ohm resistor can be found by Ohm's law, where the voltage across that resistor is v ₂ – v ₁ :

(4.131)

For current leaving through the 300 ohm resistor, the voltage is v ₂ . Pay careful attention to the sign; since we are interested in the current leaving the node labeled v ₂ , Figure 4.14 indicates that to apply Ohm's law we should take the voltage as + v ₂ – reference = v ₂ – 0 = v ₂ . So

(4.132)

For the 100 ohm resistor, we can write

(4.133)

According to Kirchhoff's current law, the sum of these three leaving currents is zero:

(4.134)

Notice that when we wrote the equation for the node labeled v ₂ , the variable v ₂ had a + sign each time it occurred in the equation, while the others had a -sign. This is always the case, and watching for it can help you avoid sign errors. Now we apply Kirchhoff's current law at the node labeled v ₃ to get the equation

(4.135)

Note that this time it is v ₃ that always shows up with a + sign.

Equations 2, 6, and 7 give us a system of three equations in the three unknown variables v ₁,v ₂ , and v ₃ . We now write them in matrix form as

(4.136)

Exercise 4.9.1.

(MATLAB) Use MATLAB to solve Equation 8. You should find

(4.137)

(4.138)

(4.139)

What is the determinant of the coefficient matrix A ? Is the solution unique?

We can determine the current flowing through the lamp from v ₃ to ground in Example 1 by Ohm's law:

(4.140)

The visible effect will, of course, depend on the lamp. Let us assume that the specifications for our lamp indicate that 0.05 ampere or more is required before it will glow, and more than 0.075 ampere will cause it to burn out. In this case, our circuit would not make the lamp glow.

Exercise 4.9.2.

(MATLAB) Write and solve equations for the circuit in Figure 4. What are the voltages at the nodes labeled v ₁ through v ₄ ? What is the current labeled i ₁ ? And i ₂ ?

Figure 4.24. 

A Resistive Network

4.10. Linear Algebra: Numerical Experiment (Circuit Design)*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

Our analysis in Example 1 from "Linear Algebra: Circuit Analysis" and Problem 1 from "Linear Algebra: Circuit Analysis" indicates that not enough current will flow through the lamp to make it glow. We now wish to change the resistance of the 100 ohm resistor to a new value so that the lamp will glow. We replace 100 in the equations with an unknown resistance R . Equation 2 is unchanged, but Equation 6 becomes

(4.141)

Equation 7 from "Linear Algebra: Circuit Analysis" becomes

(4.142)

The matrix form of these equations is

(4.143)

Write a MATLAB function file called builda to accept R as an input and return the matrix A in Equation 3 as an output. The first line of your function file should be

function A = builda(R);

Now choose several values for R . For each choice, use your function builda and solve the resulting matrix equation A v = b for the voltages. Each time you choose a different R to build a different matrix A , check the determinant of A to be sure the equations have a unique solution:

>> det(A)

Make a table of R and the corresponding values of v ₃ :

Figure 4.25. 

Now add a column to your table for the current through the lamp i = v ₃ / 2. Add rows to your table until you have found a value of R for which the lamp will glow. ( i needs to be between 0.05 and 0.075 ampere.)

[6] In a formal development of linear algebra, the abstract concept of a vector space plays a fundamental role. We will leave such concepts to a complete course in linear algebra and introduce only the functional techniques necessary to solve the problems at hand.

[7] The division of two vectors is undefined, although three different ‚Äúdivisions‚Äù are defined in MATLAB.

[8] It is not important now to understand how the determinant is defined and computed from the elements of a matrix. In your more advanced courses you will study the determinant in some detail.

Chapter 5. Vector Graphics

5.1. Vector Graphics: Introduction*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

Acknowledgements

Fundamentals of Interactive Computer Graphics by J. D. Foley and A. Van Dam, ©1982 Addison-Wesley Publishing Company, Inc., Reading, Massachusetts, was used extensively as a reference book during development of this chapter. Star locations were obtained from the share- ware program “Deep Space” by David Chandler, who obtained them from the “Skymap” database of the National Space Science Data Center.

Notes to Teachers and Students:

In this chapter we introduce matrix data structures that may be used to represent two- and three-dimensional images. The demonstration program shows students how to create a function file for creating images from these data structures. We then show how to use matrix transformations for translating, scaling, and rotating images. Projections are used to project three-dimensional images onto two-dimensional planes placed at arbitrary locations. It is precisely such projections that we use to get perspective drawings on a two-dimensional surface of three-dimensional objects. The numerical experiment encourages students to manipulate a star field and view it from several points in space.

Once again we consider certain problems essential to the chapter development. For this chapter be sure not to miss the following exercises: Exercise 2 in "Two-Dimensional Image Transformations", Exercise 1 in "Homogeneous Coordinates", Exercise 2 in "Homogeneous Coordinates", Exercise 5 in "Three-Dimensional Homogeneous Coordinates", and Exercise 2 in "Projections".

Introduction

Pictures play a vital role in human communication, in robotic manufacturing, and in digital imaging. In a typical application of digital imaging, a CCD camera records a digital picture frame that is read into the memory of a digital computer. The digital computer then manipulates this frame (or array) of data in order to crop, enlarge or reduce, enhance or smooth, translateor rotate the original picture. These procedures are called digital picture processing or computer graphics. When a sequence of picture frames is processed and displayed at video frame rates (30 frames per second), then we have an animated picture.

In this chapter we use the linear algebra we developed in The chapter on Linear Algebra to develop a rudimentary set of tools for doing computer graphics on line drawings. We begin with an example: the rotation of a single point in the (x,y) plane.

Example 5.1. 

Point P has coordinates (3,1) in the (x,y) plane as shown in Figure 1. Find the coordinates of the point P ' , which is rotated radians from P .

Figure 5.1. 

Rotating a Single Point in the (x,y) Plane

To solve this problem, we can begin by converting the point P from rectangular coordinates to polar coordinates. We have

(5.1)

The rotated point P ' has the same radius r , and its angle is . We now convert back to rectangular coordinates to find x ' and y ' for point P ' :

(5.2)

So the rotated point P ' has coordinates (2.10, 2.37).

Now imagine trying to rotate the graphical image of some complex object like an airplane. You could try to rotate all 10,000 (or so) points in the same way as the single point was just rotated. However, a much easier way to rotate all the points together is provided by linear algebra. In fact, with a single linear algebraic operation we can rotate and scale an entire object and project it from three dimensions to two for display on a flat screen or sheet of paper.

In this chapter we study vector graphics, a linear algebraic method of storing and manipulating computer images. Vector graphics is especially suited to moving, rotating, and scaling (enlarging and reducing) images and objects within images. Cropping is often necessary too, although it is a little more difficult with vector graphics. Vector graphics also allows us to store objects in three dimensions and then view the objects from various locations in space by using projections.

In vector graphics, pictures are drawn from straight lines.[9] A curve can be approximated as closely as desired by a series of short, straight lines. Clearly some pictures are better suited to representation by straight lines than are others. For example, we can achieve a fairly good representation of a building or an airplane in vector graphics, while a photograph of a forest would be extremely difficult to convert to straight lines. Many computer- aided design (CAD) programs use vector graphics to manipulate mechanical drawings.

When the time comes to actually display a vector graphics image, it may be necessary to alter the representation to match the display device. Personal computer display screens are divided into thousands of tiny rectangles called picture elements, or pixels. Each pixel is either off (black) or on (perhaps with variable intensity and/or color). With a CRT display, the electron beam scans the rows of pixels in a raster pattern. To draw a line on a pixel display device, we must first convert the line into a list of pixels to be illuminated. Dot matrix and laser printers are also pixel display devices, while pen plotters and a few specialized CRT devices can display vector graphics directly. We will let MATLAB do the conversion to pixels and automatically handle cropping when necessary.

We begin our study of vector graphics by representing each point in an image by a vector. These vectors are arranged side-by-side into a matrix G containing all the points in the image. Other matrices will be used as operators to perform the desired transformations on the image points. For example, we will find a matrix R , which functions as a rotation: the matrix product RG represents a rotated version of the original image G .

5.2. Vector Graphics: Two-Dimensional Image Representation*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

Point Matrix. To represent a straight-line image in computer memory, we must store a list of all the endpoints of the line segments that comprise the image. If the point is such an endpoint, we write it as the column vector

(5.3)

Suppose there are n such endpoints in the entire image. Each point is included only once, even if several lines end at the same point. We can arrange the vectors P i into a point matrix:

(5.4)

We then store the point matrix G ∈ R2xn as a two-dimensional array in computer memory.

Example 5.2. 

Consider the list of points

(5.5)

The corresponding point matrix is

(5.6)

Line Matrix. The next thing we need to know is which pairs of points to connect with lines. To store this information for m lines, we will use a line matrix, H ∈ R2 × m . The line matrix does not store line locations directly. Rather, it contains references to the points stored in G . To indicate a line between points p i and p j , we store the indices i and j as a pair. For the k th line in the image, we have the pair

(5.7)

The order of i and j does not really matter since a line from p i to p j is the same as a line from p j to p i . Next we collect all the lines h k into a line matrix H :

(5.8)

All the numbers in the line matrix H will be positive integers since they point to columns of G . To find the actual endpoints of a line, we look at columns i and j of the point matrix G .

Example 5.3. 

To specify line segments connecting the four points of Example 1 into a quadrilateral, we use the line matrix

(5.9)

Alternatively, we can specify line segments to form a triangle from the first three points plus a line from P ₃ to P ₄ :

(5.10)

Figure 1 shows the points G connected first by H ₁ and then by H₂.

Figure 5.2. 

Two Sets of Lines

Demo 1 (MATLAB). Use your editor to enter the following MATLAB function file. Save it as vgraphl.m.

function vgraphl (points, lines);
% vgraphl (points, lines) plots the points as *'s and
% connects the points with specified lines. The points
% matrix should be 2xN, and the lines matrix should be 2xM.
% The field of view is preset to (-50,50) on both axes.
%
% Written by Richard T. Behrens, October 1989
%
m=length(lines);                    % find the number of
                                    % lines.
axis([-50 50 -50 50])               % set the axis scales
axis('square')
plot(points(1,:),points(2,:),'*')   % plot the points as *
hold on                             % keep the points...
for i=i:m                           % while plotting the
                                    % lines
    plot([points(1,lines(1,i)) points(1,lines(2,i))],..
        [points(2,lines(2,lines(1,i)) points(2,lines(2,i))],'-')

    end

    hold off

After you have saved the function file, run MATLAB and type the following to enter the point and line matrices. (We begin with the transposes of the matrices to make them easier to enter.)

>> G = [
    0.6052 -0.4728;
   -0.4366  3.5555;
   -2.6644  7.9629;
   -7.2541 10.7547;
  -12.5091 11.5633;
  -12.5895 15.1372;
   -6.5602 13.7536;
  -31.2815 -7.7994;
  -38.8314 -9.9874;
  -44.0593 -1.1537;
  -38.8314  2.5453;
  -39.4017  9.4594;
  -39.3192 15.0932;
  -45.9561 23.4158]

>> G = G'

>> H = [
   1   2;
   2   3;
   3   4;
   4   5;
   4   7;
   5   6;
   8   9;
   9   10;
   10  11;
   11  12;
   12  13;
   12  14]

>> H = H'

At this point you should use MATLAB's “save” command to save these matrices to a disk file. Type

>> save dippers

After you have saved the matrices, use the function VGRAPH1 to draw the image by typing

≫ vgraph1(G,H)

The advantage of storing points and lines separately is that an object can be moved and scaled by operating only on the point matrix G . The line information in H remains the same since the same pairs of points are connected no matter where we put the points themselves.

Surfaces and Objects. To describe a surface in three dimensions is a fairly complex task, especially if the surface is curved. For this reason, we will be satisfied with points and lines, sometimes visualizing flat surfaces based on the lines. On the other hand, it is a fairly simple matter to group the points and lines into distinct objects. We can define an object matrix K with one column for each object giving the ranges of points and lines associated with that object. Each column is defined as

(5.11)

As with the line matrix H , the elements of K are integers.

Example 5.4. 

Consider again Demo 1. We could group the points in G and the lines in H into two objects with the matrix

(5.12)

The first column of K specifies that the first object (Ursa Minor) is made up of points 1 through 7 and lines 1 through 6, and the second column of K defines the second object (Ursa Major) as points 8 through 14 and lines 7 through 12.

5.3. Vector Graphics: Two-Dimensional Image Transformations*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

We now turn our attention to operating on the point matrix G to produce the desired transformations. We will consider rotation; scaling; and translation (moving) of objects. Rotation and scaling are done by matrix multiplication with a square transformation matrix A. If we call the transformed point matrix G new , we have

(5.13)

We call A a matrix operator because it “operates” on G through matrix multiplication. In contrast, translation must be done by matrix addition.

In a later section you will see that it is advantageous to perform all operations by matrix operators and that we can modify our image representation to allow translation to be done with a matrix operator like rotation and scaling. We will call the modified representation homogeneous coordinates.

Rotation. We saw in the chapter on linear algebra that the matrix that rotates points by an angle θ is

(5.14)

When applied to the point matrix G, this matrix operator rotates each point by the angle θ, regardless of the number of points.

Example 5.5. 

We can use the rotation matrix to do the single point rotation of the example from "Vector Graphics: Introduction". We have a point matrix consisting of only the point (3,1):

(5.15)

The necessary transformation matrix is R(θ) with Then the rotated point is given by

(5.16)

Scaling. An object can be enlarged or reduced in each dimension inde- pendently. The matrix operator that scales an image by a factor of s x along the x-axis and s y along the y-axis is

(5.17)

Most often we take s x = s y to scale an image by the same amount in both dimensions.

Exercise 5.3.1.

Write out the following matrices. Simplify and give numerical answers to two decimal places:

;

S(3,2);

;

S( – 1,1).

Exercise 5.3.2. Reflections

What does S( – 1,1) do? S(1, – 1) ? S( – 1, – 1)? S(1,1)?

Exercise 5.3.3.

Given and find G new = R(θ)G. Give numerical answers to two decimal places.

Exercise 5.3.4.

Apply each of the transformations in Exercise 1 and 2 to the image

(5.18)

Sketch the original image and each transformation of it.

Translation. An object can be moved by adding a constant vector b to every point in the object. For example, will move an object 20 units to the right and 5 units down. We can write this in terms of the point matrix as

(5.19) G new = G + b 1 T

where 1 (read “the one-vector”) is a vector of n l's:

(5.20)

In MATLAB, 1 may be obtained by ones(n,1). The outer product of b with 1 in Equation 7 simply serves to make n copies of b so that one copy can be added to each point in G .

5.4. Vector Graphics: Composition of Transformations*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

Often we will want to perform several operations on an object before we display the result. For example, suppose we want to rotate by and reduce to size in each dimension:

(5.21)

If there are n points in the matrix G , it will require 4n multiplications to perform each of these operations, for a total of 8n multiplications. However, we can save some multiplications by noting that

(5.22)

where

(5.23)

In other words, we take advantage of the fact that matrix multiplication is associative to combine S and R into a single operation A , which requires only 8 multiplications. Then we operate on G with A , which requires 4n multiplications. By “composing” the two operations, we have reduced the total from 8n to 4n + 8 multiplications. Furthermore, we can now build operators with complex actions by combining simple actions.

Example 5.6. 

We can build an operator that stretches objects along a diagonal line by composing scaling and rotation. We must

rotate the diagonal line to the x-axis with R( – θ);

scale with S(s,1); and

rotate back to the original orientation with R(θ).

Figure 1 shows a square being stretched along a 45o line. The composite operator that performs this directional stretching is

(5.24)

Note that the rightmost operator in a product of operators is applied first.

Figure 5.3. 

Rotating and Scaling for Directional Stretching

5.5. Vector Graphics: Homogeneous Coordinates*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

In the previous section we were able to combine rotation and scaling into a single composite operation by matrix multiplication. Unfortunately, translation cannot yet be included in the composite operator since we do it by addition rather than by multiplication.

Example 5.7. 

Suppose we wish to rotate the image G by about the point ( – 10,10). Our rotation matrix R(θ) always rotates about the origin, so we must combine three transformations to accomplish this:

translate the point ( – 10,10) to the origin;

rotate radians about the origin; and

translate the origin back to ( – 10,10).

For step (i), we have and

(5.25) G₁ = G + b₀ 1 T .

For step (ii),

(5.26)

For step (iii), we can use – b₀ from step (i):

(5.27)

In this example we were unable to find a single matrix operator A to do the entire job. The total transformation took the form

(5.28) G new = A G + b 1 T .

This is called an affine transformation because it involves both multiplication by A and addition of a constant matrix. This is in contrast to the more desirable linear transformation, which involves only multiplication by A .

We will now move toward a modified representation of the image and the operators by rewriting the last equation as

(5.29)

where in the example we had and .

Exercise 5.5.1.

Show that, for any matrices A,B,C,D of compatible sizes,

(5.30)

The matrix looks like

(5.31)

and the points are called homogeneous coordinates. We can modify Equation 5 so that the new point matrix is also in homogeneous coordinates:

(5.32)

In the new representation, each point in the image has a third coordinate, which is always a 1. The homogeneous transformation is a 3 × 3 matrix,

(5.33)

which is capable of translation, rotation, and scaling all by matrix multi- plication. Thus, using homogeneous coordinates, we can build composite transformations that include translation.

In homogeneous coordinates, we have

(5.34)

Example 5.8. 

The composite transformation to triple the size of an image and then move it 2 units to the left is

(5.35)

On the other hand, the composite transformation to move an image 2 units to the left and then triple its size is

(5.36)

In the latter case, the distance of the translation is also tripled.

Exercise 5.5.2.

Find a single composite transformation in homogeneous coordinates that rotates an image by angle θ about point as in Example 1.

5.6. Vector Graphics: Three-Dimensional Homogeneous Coordinates*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

We now consider the storage and manipulation of three-dimensional objects. We will continue to use homogeneous coordinates so that translation can be included in composite operators. Homogeneous coordinates in three dimensions will also allow us to do perspective projections so that we can view a three-dimensional object from any point in space.

Image Representation. The three-dimensional form of the point matrix in homogeneous coordinates is

(5.37)

The line matrix H is exactly as before, pointing to pairs of columns in G to connect with lines. Any other grouping matrices for objects, etc., are also unchanged.

Image manipulations are done by a 4 × 4 matrix A. To ensure that the fourth coordinate remains a 1, the operator A must have the structure

(5.38)

The new image has point matrix

(5.39)

Exercise 5.6.1.

If the coordinates of the i th point in G are , what are the coordinates of the i th point in G new = AG when A is as given in Equation 2?

Exercise 5.6.2.

Write down the point matrix G for the unit cube (the cube with sides of length 1, with one corner at the origin and extending in the positive direction along each axis). Draw a sketch of the cube, numbering the vertices according to their order in your point matrix. Now write down the line matrix H to complete the representation of the cube.

Left- and Right-Handed Coordinate Systems. In this book we work exclusively with right-handed coordinate systems. However, it is worth pointing out that there are two ways to arrange the axes in three dimensions. Figure 1(a) shows the usual right-handed coordinates, and the left-handed variation is shown in Figure 1(b). All possible choices of labels x,y , and z for the three coordinate axes can be rotated to fit one of these two figures, but no rotation will go from one to the other.

Figure 5.4. 

(a)

(b)

Three-Dimensional Coordinate Systems; (a) Right-handed, and (b) Left-handed

Be careful to sketch a right-handed coordinate system when you are solving problems in this chapter. Some answers will not be the same for a left-handed system.

Image Transformation. Three-dimensional operations are a little more complicated than their two-dimensional counterparts. For scaling and translation we now have three independent directions, so we generalize the operators of Equation 10 from "Vector Graphics: Homogeneous Coordinates" as

(5.40)

Exercise 5.6.3.

Show that is the inverse of undoes the work of T .

Rotation may be done about any arbitrary line in three dimensions. We will build up to the general case by first presenting the operators that rotate about the three coordinate axes. R x (θ) rotates by angle θ about the x-axis, with positive θ going from the y-axis to the z-axis, as shown in Figure 2. In a similar fashion, positive rotation about the y-axis using R y (θ) goes from z to x , and positive rotation about the z-axis goes from x to y , just as in two dimensions. We have the fundamental rotations

(5.41)

A more general rotation about any line through the origin can be constructed by composition of the three fundamental rotations. Finally, by composing translation with the fundamental rotations, we can build an operator that rotates about any arbitrary line in space.

Figure 5.5. 

Directions of Positive Rotation

Example 5.9. 

To rotate by angle φ about the line L, which lies in the x – y plane in Figure 3, we would

rotate L to the x-axis with R z ( – θ);

rotate by φ about the x-axis with R x (φ); and

rotate back to L with R z (θ).

The composite operation would be

(5.42) A

( θ , φ )

= R z ( θ )

R x ( φ )

R z ( – θ )

.

Figure 5.6. 

Composition of Rotations

Direction Cosines. As discussed in the chapter on Linear Algebra, a vector v may be specified by its coordinates (x,y,z) or by its length and direction. The length of v is ||v||, and the direction can be specified in terms of the three direction cosines ( cosθ x , cosθ y , cosθ z ). The angle θ x is measured between the vector v and the x-axis or, equivalently, between the vector v and the vector e x = [100] T . We have

(5.43)

Likewise, θ y is measured between v and e y = [010] T , and θ z is measured between v and e z = [001] T . Thus

(5.44)

The vector

(5.45)

is a unit vector in the direction of v , so we have

(5.46)

Exercise 5.6.4.

Show that u is a unit vector (i.e. 1 u|| = 1).

The direction cosines are useful for specifying a line in space. Instead of giving two points P ₁ and P ₂ on the line, we can give one point P ₁ plus the direction cosines of any vector that points along the line. One such vector is the vector from P ₁ to P ₂ .

Arc tangents. Consider a vector in two dimensions. We know that

(5.47)

for the angle θ shown in Figure 4. If we know x and y , we can find θ using the arc tangent function

(5.48)

Figure 5.7. 

Tangent and Arc Tangent

In MATLAB,

theta = atan(y/x)

Unfortunately, the arc tangent always gives answers between and corresponding to points v in quadrants I and IV. The problem is that the ratio is the same as the ratio – – – xA so quadrant III cannot be distinguished from quadrant I by the ratio Likewise, quadrants II and IV are indistinguishable.

The solution is the two-argument arc tangent function. In MATLAB,

theta = atan2(y,x)

will give the true angle θ – π and π in any of the four quadrants.

Example 5.10. 

Consider the direction vector , as shown in Figure 5. What is the angle φ y between the projection of u into the x – y plane and the y-axis? This is important because it is that will put u in the y – z plane, and we need to know the angle φ y in order to carry out this rotation. First note that it is not the same as θ y . From the geometry of the figure, we can write

(5.49)

This gives us a formula for φ y in terms of the direction cosines of u . With the two-argument arc tangent, we can write

(5.50) φ y = tan – 1(cosθ x ,cosθ y ).

Figure 5.8. 

Angles in Three Dimensions

Exercise 5.6.5.

Suppose point p ' is in the y – z plane in three dimensions, . Find θ so that R x (θ) will rotate p ' to the positive z-axis. (Hint: Use the two-argument arc tangent. θ will be a function of y ' and z ' .)

Let p be any point in three-dimensional space, p = (x,y,z,1). Find ψ so that R z (ψ) rotates p into the y – z plane. (Hint: Sketch the situation in three-dimensions, then sketch a two-dimensional view looking down at the x – y plane from the positive z-axis. Compare with Example 2.) Your answers to parts (a) and (b) can be composed into an operator Z(p) that rotates a given point p to the positive z-axis, Z(p) = R x (θ)R z (ψ).

Let L be a line in three-dimensional space specified by a point 1 = (x,y,z,1) and the direction cosines ( cosθ x , cosθ y , cosθ z ). Use the following procedure to derive a composite operator R(φ,L) that rotates by angle φ about the line L:

translate 1 to the origin;

let u = ( cosθ x , cosθ y , cosθ z ,1) and use Z(u) to align L with the z-axis;

rotate by φ about the z-axis;

undo step (ii); and

undo step (i).

5.7. Vector Graphics: Projections*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

Computer screens and printers are two-dimensional display devices. We must somehow convert three-dimensional images to two dimensions in order to display them. This task is done by another kind of matrix operator called a projection.

To build a projection, we first choose a projection plane in the three- dimensional space of the object we wish to view. All points of the object are then projected onto the plane. There are many different kinds of projections, corresponding to various geometric rules for mapping points in space onto a plane. We begin with the parallel projection illustrated in Figure 1, wherein the dotted lines between the points and their projections in the plane are all parallel to one another. These dotted lines are called projectors.

Range. The projection plane is called the range of the projection. We will assume for now that the projection plane passes through the origin. If it does not, we may later compose the necessary translations with our projection. Three points, not all in a line, are required to determine a plane. We will take the origin as one of the points and suppose that the other two points are the vectors r ₁ and r ₂ . From these vectors, we form the matrix

(5.51)

which determines the range of the projection.

Figure 5.9. 

Oblique Parallel Projection

Null-Space. To complete the specification of the parallel projection, we must specify the direction of the projectors. The line through the origin in this direction is called the null-space of the projection. Note that any line in the three-dimensional object that is parallel to the null-space will disappear to a point when projected. The null-space may be specified by a vector n pointing in the direction of the projectors.

Orthogonal Projection. If the null-space is perpendicular to the range, we say that the projection is an orthogonal projection. The matrix operator for orthogonal projection is a function of the range. If we assume that R is specified in three-dimensional coordinates (not homogeneous!), we have the following definition for the orthogonal projection onto the range of R :

(5.52)

In homogeneous coordinates, we embed this 3 × 3 projection matrix in the general 4 × 4 transformation:

(5.53)

Example 5.11. 

Suppose we choose the x – y plane as our projection plane. Vectors r ₁ and r ₂ can be any two points in the plane (besides the origin), so let's take

(5.54)

Then the range matrix is

(5.55)

The orthogonal projection looking down on the x – y plane from the z-axis is

(5.56)

In homogeneous coordinates,

(5.57)

Exercise 5.7.1.

Let the vectors and specify the range of an orthogonal projection. Find P(R) and P h (R). Compare with Example 1 and explain any similarities.

Example 5.12. 

In the chapter on linear algebra you learned that the projection of w onto x is given by

(5.58)

This is an orthogonal projection of w onto a line, but it is closely related to the projection into the plane just described. To see the similarity, let's work on the expression for z a little:

(5.59)

So we see that it can be written in the same way as the projection onto the plane. The only difference is that the range is now one-dimensional and specified by x in place of R .

Oblique Projection. More generally, we may have a null-space n that is not perpendicular to the range R. The projection shown in Figure 1 is an oblique projection. Once again we start with nonhomogeneous coordinates for n and R and write the 3 × 3 oblique projection as

(5.60)

where

(5.61)

As with the orthogonal projection, we can return to homogeneous coordinates by

(5.62)

Exercise 5.7.2.

Prove and interpret the following properties of paralle1 projections (both orthogonal and oblique):

P² = P;E² = E.

P R = R;E R = R.

E n = 0;P n = 0. (First show that R T n = 0 when n is orthogonal to R.)

Display from Projections. Even after we have used a projection, the image points are in three-dimensional homogeneous coordinates. How then do we display them on a two-dimensional display? If we had chosen the x – y plane as the range of our projection, we could let the output device represent the x – y plane and ignore the z-coordinate of each point. The z-coordinates of the projected image would all be 0 anyway since the projected points would lie in the x – y plane. The fourth coordinate of 1 may also be ignored for display. But even if the projection plane is not the x – y plane, we may perform a rotation to align the two planes and again let the output device represent the x – y plane.

Perspective Projections. To obtain a perspective projection, we first choose a projection plane as we did for parallel projections. Instead of choosing a null-space parallel to all projectors, we now choose a viewpoint. The projectors all pass through the viewpoint in a perspective projection, as shown in Figure 2. For a viewpoint at z = d on the z-axis and a projection plane coinciding with the x – y plane, the three-dimensional homogeneous operator for perspective projection is

(5.63)

The first thing you should notice about the perspective projection M(d) is that it violates the structure given in Equation 2 from "Vector Graphics: Three-Dimensional Homogeneous Coordinates" by having the 4,3 position equal to rather than to 0. This means that the fourth coordinates will not remain 1 in the new point matrix

(5.64) G new = M

( d )

G .

Figure 5.10. 

A Perspective Projection

This situation is interpreted to mean that the new point ( x , y , z , w ) must be renormalized to before the operation is considered complete. Renormalization adds another computational step to the image transformation process. This is undesirable, but we are stuck with it if we wish to include perspective projections in our repertoire of transformations. Note that renormalization is a point-by-point process since w may be different for each point.

How does renormalization affect composition of operators? We might expect that, each time we perform an operation requiring renormalization, we will be forced to stop and do the renormalization before going on with other operations. In this respect we are fortunate: we may put off the renormalization until all transformations are complete and renormalize once just before displaying the image. Thus we are able to compose perspective projections at will with other transformations.

Exercise 5.7.3.

What is the perspective projection matrix for a viewpoint at infinity, M(∞)? Interpret the result.

Generalization of Projections. “Projection” is a technical term in linear algebra. A square matrix of any size is considered a projection if it obeys the property of Exercise 2(a), P² = P. All of our geometric projection matrices have this property. We have considered only three-dimensional spaces with a two-dimensional range and a one-dimensional null-space. In general, the dimensionality of the space may be split in any way between range and null-space. If we reinterpret homogeneous coordinates as four-dimensional space, we have projections with three-dimensional range and one-dimensional null-space, but the perspective projection is actually not quite a projection in the technical sense even though M²(d) = M(d) because perspective projection also includes renormalization.

5.8. Vector Graphics: Numerical Experiment (Star Field)*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

With Earth at the origin, we may specify the star positions for the Big and Little Dippers in three-dimensional homogeneous coordinates. With light years as our units, we have G T =

1.5441      -1.2064     153.0875    1.0000
-1.0386     8.4588      142.7458    1.0000
-8.7955     26.2870     198.0698    1.0000
-12.8807    19.0964     106.5383    1.0000
-18.8926    17.461      90.6185     1.0000
-45.1364    54.2706     215.1148    1.0000
-9.6222     20.1734     88.0062     1.0000
-33.7097    -8.4048     64.6574     1.0000
-33.7144    -8.7191     52.3806     1.0000
-43.8531    -1.1483     59.7193     1.0000
-36.1086    2.3667      55.7927     1.0000
-34.2309    8.2181      52.1260     1.0000
-30.7876    11.8182     46.9810     1.0000
-61.8581    31.5183     80.7615     1.0000

To make use of this data, we need a function to display it. Enter and save the following generalization of the function from Demo 1 in "Vector Graphics: Two Dimensional Image Representation". Call it vhgraph.m.

function vhgraph (P,L,A,ps,ls);
% vhgraph(P,L,A,PS,LS) graphs images whose points are
% stored in P and whose lines are stored in L. The points
% in P must be in homogeneous coordinates in either 2 or 3
% dimensions, with each column of P representing a point.
% The lines are coded in L, with each column of L containing
% 2 integers pointing to a pair of points in P to be
% connected by a line segment. If A is present, the points
% in P are transformed to A*P beforre graphing. For 3D data
% points, only the first two coordinates are graphed, so A
% should include the desired projection from 3D to 2D. The
% point symbol may be specified in PS and the line type in
% LS, if desired
%
% Richard T. Behrens, October 1989.
%
% The first dection of the program determines the sizes of
% all the input matrices and chacks that they make sense.
[mp,np] = size(P);
if (nargin > 1)
   [ml,nl] = size(L);
else
    ml = 2; nl = 0;
end
if (nargin <=2)
    A = eye(mp);
end
[mA,nA] = size(A);
if ((mp~= nA) | (ml~=2))
    error('Incompatible sizes of input matrices.")
end
if (nargin <= 3);
    ps = '*';
end

P = A*P;    % Performs the transformation A on the points
            % (effect is only local to this function).

% The next section contains a loop that renormalizes the
% homogeneous coordinates of the points if necessary.
renorm = find((P(mA,:)~= 1));
if~isempty(renorm)
    for i = 1:length(renorm)
        P(:,renorm(i)) = P(:,renorm(i))/P(mA,renorm(i));
    end
end

% The next program line sets a fixed scale output window
% from -50 to 50 in both x and y directions on the screen.
% For automatic scaling to include all points of the
% image, we could use instead the line q = min(min(P));
% r = max(max(P));
q = -50; r = 50;
axis([q r q r])
axis('square')
plot(P(1,:),P(2,:),ps)  % Plots the points with symbol ps.
hold on                 % Saves the points while we plot
for i = 1:nl            % lines with line type LS.
plot([P(1,L(1,i))P(1,L(2,i))],..
     [P(2,L(1,i))P(2,L(2,i))],ls)
end
hold off

Enter the point matrix given at rhe beginning of this section (and take its transpose to put it in the usual form). Also enter the line matrix from Demo 1 in "Vector Graphics: Two Dimensional Image Representation". Save these two matrices and try looking at the image

≫ save dip3d
≫ vhgraph(G,H)

No dippers in sight, right? Without specifying a transformation matrix A , we have defaulted to looking down on the x – y plane from z = ∞ (a parallel projection). This is how the constellations would look from a distant galaxy (say, a billion light years north of here) through an enormous telescope. We need a perspective view from the origin (Earth), but first we need a set of functions to give us the fundamental operators with which we can build the desired projection.

Take R y (θ) as an example. The function to build it looks like

    function Ry = vhry (theta) ;
% Rotation matrix about y-axis for 3-D homogeneous
% coordinates.
    Ry = eye(4);
    Ry(1,1) = cos(theta);
    Ry(3,3) = cos(theta);
    Ry(3,1) = -sin(theta);
    Ry(1,3) = sin(theta);

Enter and save hry.m as given. Then write functions for

(5.65)

Useful MATLAB functions for this task include zeros, eye, and diag.

Now build and use a perspective projection with viewpoint Earth and projection plane at z = 1000 behind the dippers:

translate Earth to z = – 1000 so that the projection plane coincides with the x – y plane: T(O,0, – 1000);

use the fundamental perspective projection: M( – 1000); and

translate back: T(O,0,1000).

≫ A = vht(0,0,1000) * vhm(-1000) * vht(0,0,-1000)
≫ vhgraph(G,H,A)

Oops! Now the image is too big; it's mostly off the screen. Scale it down and have another look:

≫ A = vhs(.06,.06,.06) * A
≫ vhgraph(G,H,A)

Now the view should look familiar. Leave A as it is now:

≫ save dip3d

Experiment with scale and rotation about the z-axis. For example, try

≫ vhgraph(G,H,vhrz(pi/2) * A)

The two-dimensional star positions given in Demo 1 in "Vector Graphics: Two Dimensional Image Representation" were obtained from the three-dimensional positions with the composite operator A you are now using. To compare the two, type

≫ Gnew = A * G

≫ for i = 1:14
Gnew(;,i) = Gnew(:,i)/Gnew(4,i);
end

≫ Gnew

and compare the x and y coordinates with those of Demo 1 in "Vector Graphics: Two Dimensional Image Representation".

Astronomers give star positions in equatorial coordinates using right ascension, declination, and distance. The following function converts equatorial coordinates, which are spherical, to Cartesian coordinates with the z-axis pointing north, the x-axis pointing at the vernal (Spring) equinox in the constellation Pisces, and the y-axis pointing toward the Winter solstice in the constellation Opheuchus.

function v = starxyz(rah,ram,decd,decm,dist)
% starxyz is the cartesian coordinates of a star whose
% spherical coordinates ( e.g. from a star catalog) are
%
%          rah     right ascension hours
%          ram     right ascension minutes
%          decd    declination degrees
%          decm    declination minutes (should be negative
%                  if decd is negative)
%          dist    distance (light years)
%
phi = (pi/180) * (decd + decm/60);
theta = (pi/12) * (rah + ram/60);
r = dist;
v = [r * cos(phi) * cos(theta); -r * cos(phi) * sin(theta);
    r * sin(phi)];

Have you ever wondered what the constellations would look like from other places in the galaxy? We will soon see the answer. First we will view the dippers from Alpha Centauri, the nearest star, whose coordinates are

 -1.5680     1.3157      -3.6675.

We will look toward the centroid of the fourteen stars in the dippers, located at

-26.3632      12.8709      100.4714.

To get the desired view, we must

translate the viewpoint to the origin;

rotate the centroid (direction to look) to the z-axis–note that the centroid will have new coordinates after step (1); and

apply the composite A=S(.06,.06,.06)*T(0,0,1000)*M(-1000)*T(0,0,-1000) (as used to view from Earth).

Write a function vhz.m based on Exercise 5 from "Vector Graphics: Three-Dimensional Homogeneous Coordinates" to accomplish step (2). Test it on several random points to make sure it works right. Now write a general perspective projection function called vhview.m. The function vhview should accept as inputs two vectors, the first specifying the viewpoint and the second the point to look toward. Its output should be a composite operator that performs all three of the preceding steps.

Now we want to look toward the centroid of the dippers from Alpha Centauri. To do so, enter the vectors for the two points of interest and construct the view like this:

≫ acentauri = [-1.5680; 1.3157; -3.6675]
≫ centroid = [-26.3632; 12.8709; 100.4714]
≫ A = vhview(acentauri,centroid)
≫ vhgraph(G,H,A)
≫ title('From Alpha Centauri')

The farther we move from Earth, the more distorted the dippers will look in general. It should be easy now to view them from any desired location. Just choose a viewpoint, recalculate the composite operator for that viewpoint using vhview, and use vhgraph. Follow this procedure to view the dippers from each of the stars in the following list. You will need to use starxyz first to convert their coordinates.

Table 5.1. Table 1 is a six-row and four-column table with the first row as a header, the first column describing different stars, and the values in the second through fourth columns describing the corresponding right ascension, declination, and distance for each star.

Star | Right Ascension | Declination | Distance (ly)
Alpha Centauri | 14h 40m | -60° 50' | 4.2
Sirius | 6h 45m | -16°43' | 9.5
Arcturus | 14h 16m | 19°11' | 16.6
Pollux | 7h 45m | 28°02' | 35.9
Betelgeuse | 5h 55m | 7°24' | 313.5

Of course, star viewing is not the only application of vector graphics. Do some experiments with the unit cube (see Exercise 2 from "Vector Graphics: Three-Dimensional Homogeneous Coordinates"). View the cube from location (4,3,2) looking toward the origin using the procedure just outlined for stars. You may need to adjust the scaling to get a meaningful view.

[9] It is possible to extend these techniques to deal with some types of curves, but we will consider only straight lines for the sake of simplicity.

Chapter 6. Filtering

6.1. Filtering: Introduction*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

Notes to Teachers and Students:

Filtering is one of the most important things that electrical and computer engineers do. In this chapter we extend everyday understanding of filters to numerical filters. We then study weighted moving averages and exponential averages. We define the important test signals for electrical and computer engineering and show how filters respond to them. The idea that filters are characterized by their response to simple test signals is fundamental. In the numerical experiment, students explore the frequency response of a simple filter, a concept that forms the basis of circuit theory, electronics, optics and lasers, solid-state devices, communications, and control.

Introduction

A filter is any device that passes material, light, sound, current, velocity, or information according to some rule of selectivity. Material (or mechanical) filters are commonplace in your everyday life:

coffee filters pass flavored water while filtering out coffee grounds;

Goretex fibers pass small, warm perspiration droplets while filtering out large, cool droplets of rain or snow;

fiberglass strands in a furnace filter pass warm air while filtering out particles of dirt and dust;

a centrifuge retains material of low density while spinning out (or filtering out) material of high density; and

an electrostatic precipitator filters out dust and other effluents by attaching charge to them and using an electric field to move the charged particles to a high potential drain.

The first three of these examples selectively pass material according to size; the last two selectively pass material according to its mass density.

Typical filters for light are

UV filters on camera lenses and eyeglasses that pass light in the range of visible wavelengths while blocking light in the invisible (but damaging) ultraviolet range;

polaroid lenses that pass light that is randomly polarized while blocking out glare that is linearly polarized;

green fabrics that reflect green light and absorb other colors;

red taillights that pass light in the long wavelength red range and reflect light in the short wavelength violet range (look at the inside of your taillights to see violet); and

glacial ice that absorbs all but the blue wavelengths so that it appears blue.

Exercise 6.1.1.

List as many examples of natural and man-made sound filters as you can.

Satellite Television. Among current filters, the tuner in a super-heterodyne receiver is, perhaps, the first example that comes to mind. But satellite TV filters are another fascinating example. A typical C-band satellite has twelve transponders (or repeaters), each of which transmits microwave radiation in a personalized 36 MHz band. (The abbreviation MHz stands for megahertz, or 10⁶ Hz, or 10⁶ cycles per second. Other common abbreviations are Hz for 1 Hz, kHz for 10³ Hz, and GHz for 10⁹ Hz.) Each transponder actually transmits two channels of information, one vertically polarized and one horizontally polarized. There is an 8 MHz guard band between each band, and the vertical and horizontal channels are offset by 20 MHz. The transmission scheme for the 24 channels is illustrated in Figure 1. The entire transmission band extends over 540 MHz, from 3.7 × 10⁹ Hz to 4.24 × 10⁹ Hz. The satellite receiver has two different microwave detectors, one for vertical and one for horizontal polarization, and a microwave tuner to tune into the microwave band of interest.

Exercise 6.1.2.

Check that the transmission scheme of Figure 1 consumes 540 MHz of bandwidth.

Exercise 6.1.3.

List as many examples of natural and man-made devices for velocity filtering as you can.

Figure 6.1. 

Satellite TV– Vi , Vertically Polarized Channel i;H j , Horizontally Polarized Channel j

An Aside on Hertz and Seconds. The abbreviation Hz stands for hertz, or cycles/second. It is used to describe the frequency of a sinusoidal signal. For example, house current is 60 Hz, meaning that it has 60 cycles each second. The inverse of Hz is seconds or, more precisely, seconds/cycle, the period of 1 cycle. For example, the period of 1 cycle for house current is 1/60 second. When we are dealing with sound, electricity, and electromagnetic radiation, we need a concise language for dealing with signals and waves whose frequencies range from 0 Hz (called DC or direct current) to 10¹⁸ Hz (visible light). Table 1 summarizes the terms and symbols used to describe the frequency and period of signals that range in frequency from 0 Hz to 10¹² Hz.

Table 6.1. Terms and symbols for Sinusoidal Signals

Frequency | Period |
Hz | Term | Units | Seconds | Term | Units | Example
Hz | hertz | 1 Hz | sec | second | 1 sec | battery current: 0 Hz house current: 60 Hz
kHz | kilohertz | 10³ Hz | msec | millisecond | 10⁻³ sec | midfrequency sound
MHz | megahertz | 10⁶ Hz | µsec | microsecond | 10⁻⁶sec | clock frequencies in microcomputers
GHz | gigahertz | 10⁹ Hz | nsec | nanosecond | 10⁻⁹sec | microwave radiation for satellite communication
THz | terahertz | 10¹² Hz | psec | picosecond | 10⁻¹² sec | infrared radiation

Numerical Filters. Rather amazingly, these ideas extend to the domain of numerical filters, the topic of this chapter. Numerical filters are just schemes for weighting and summing strings of numbers. Stock prices are typically averaged with numerical filters. The curves in Figure 2 illustrate the daily closing average for Kellogg's common stock and two moving averages. The 50-day moving average is obtained by passing the daily closing average through a numerical filter that averages the most current 50 days' worth of closing averages. The 200-day moving average for the stock price is obtained by passing the daily closing prices through a numerical filter that averages the most current 200 days' worth of daily closing averages. The daily closing averages show fine-grained variation but tend to conceal trends. The 50-day and 200-day averages show less fine-grained variation but give a clearer picture of trends. In fact, this is one of the key ideas in numerical filtering: by selecting our method of averaging, we can filter out fine-grained variations and pass long-term trends (or vice versa), or we can filter out periodic variations and pass nonperiodic variations (or vice versa). Figure 2 illustrates that moving averages typically lag increasing sequences of numbers and lead decreasing sequences. Can you explain why?

We will call any algorithm or procedure for transforming one set of numbers into another set of numbers a numerical filter or digital filter. Digital filters, consisting of memories and arithmetic logic units (ALUs), are implemented in VLSI circuits and used for communication, control, and instrumentation. They are also implemented in random–or semicustom–logic circuits and in programmable microcomputer systems. The inputs to a digital filter are typically electronic measurements that are produced by A/D (analog-to-digital) conversion of the output of an electrical or mechanical sensor. The outputs of the filter are “processed,” “filtered,” or “smoothed” versions of the measurements. In your more advanced courses in electrical and computer engineering you will study signal processing and system theory, assembly language programming, microprocessor system development, and computer design. In these courses you will study the design and programming of hardware that may be used for digital filtering.

Figure 6.2. 

Dow-Jones Averages (Adapted from the New York Stock Exchange, Daily Graphs, William O'Neil and Co., Inc., Los Angeles, California)

6.2. Filtering: Simple Averages*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

The simplest numerical filter is the simple averaging filter. This filter is defined by the equation

(6.1)

The filter output x is the average of the N filter inputs u ₁,u ₂,... , uN . These inputs may be real or complex numbers, and x may be real or complex. This simple averaging filter is illustrated in Figure 1.

Figure 6.3. 

A Simple Averaging Filter

Example 6.1. 

If the averaging filter is excited by the constant sequence u ₁ = u ₂ = ⋯ = u N = u , then the output is

(6.2)

The output is, truly, the average of the inputs. Now suppose the filter is excited by the linearly increasing sequence

(6.3)

This sequence is plotted in Figure 2. How do we sum such a sequence in order to produce the average x ? For N even, the average may be written as

(6.4)

Each pair-sum in parentheses equals N + 1, and there are such pair-sums, so the average is

(6.5)

This is certainly a reasonable answer for the average of a linearly increasing sequence. See Figure 2.

Figure 6.4. 

Linearly Increasing Sequence

Exercise 6.2.1.

Write as a sum of pair-sums for N odd. What does x equal?

General Sum Formula. Suppose the input to the simple averaging filter is the polynomial sequence

(6.6)

where k is a non-negative integer such as k = 0,1,2,.... The output of the filter is

(6.7)

We rewrite x as to remind ourselves that we are averaging N numbers, each of which is n k . For example, when N = 8 and k = 2,

(6.8)

Rather than study the average , we will study the sum and divide by N at the very end:

(6.9)

The sum may be rewritten as the sum

(6.10)

This result is very important because it tells us that the sum , viewed as a function of N , obeys a recursion in which is just the sum using one less input, namely, , plus N k . Now, since polynomials are the most general functions that obey such recursions, we know that must be a polynomial of order k + 1 in the variable N :

(6.11)

Let's check to see that this polynomial really can obey the required recursion. First note that is the following polynomial:

(6.12)

The term (N – 1) k + 1 produces . (Remember the binomial expansion?) Therefore the difference between and is

(6.13)

This recursion is general enough to produce the difference Nk provided we can solve for a ₀,a ₁,... , a k + 1 to make c ₀ = c ₁ = ⋯ = c k – 1 = 0 and c k = 1. We know that for N = 0, so we know that a ₀ = 0, meaning that the polynomial for can really be written as

(6.14)

In order to solve for the coefficients of this polynomial, we propose to write out our equation for as follows:

(6.15)

Using the linear algebra we learned earlier, we may write these equations as the matrix equation

(6.16)

The terms on the right-hand side of the equal sign are “initial conditions” that tell us how the sum begins for N = 1,2,...,k + 1. These initial conditions must be computed directly. (For example, ) Then the linear system of (k + 1) equations in (k + 1) unknowns may be solved for a ₁,a ₂,...,a k + 1 . The solution for S k N is then complete, and we may use it to solve for S k N for arbitrary N .

Example 6.2. 

When k = 2, we have the following equation for the coefficients a ₁,a ₂ , and a ₃ in the polynomial :

(6.17)

Exercise 6.2.2.

Solve for a ₁,a ₂,a ₃ in the linear equation of Example 2. Show that obeys the recursion .

Exercise 6.2.3.

(MATLAB) Write a MATLAB program to determine the coefficients a ₁,a ₂,...,a k + 1 for the polynomial . Generate a table of formulas for the averages for k = 1,2,...,5. Evaluate these formulas for N = 2, 4, 8, and 16.

Exponential Sums. When the input to an averaging filter is the sequence

(6.18)

we say that the input is exponential (or geometric). Typical sequences are illustrated in Figure 6.5 for a = 0.9,a = 1, and a = 1.1. Don't let it throw you that we have changed the index to run from 0 to N – 1 rather than from 1 to N . This change is not fundamentally important, but it simplifies our study. The sum of the inputs is

(6.19)

Figure 6.5. 

Exponential Sequences

How do we evaluate this sum? Well, we note that the sum a S N is

(6.20)

Therefore, provided a ≠ 1, the sum S N is

(6.21)

This formula, discovered already in the chapter covering the functions e x and e jθ , works for a ≠ 1. When a = 1, then S N = N :

(6.22)

When |a| < 1, then a N → 0 for N → ∞, and we have the asymptotic formula

(6.23)

Exercise 6.2.4.

Evaluate and for a = 0.9,1, and 1.1 and for N = 1,2,4,8,16, and 32.

Exercise 6.2.5.

Prove that obeys the recursion

(6.24) S N = S N – 1 + a N – 1 .

Prove that S N = N obeys this recursion for a = 1 and that obeys it for a ≠ 1.

Recursive Computation. Every sum of the form

(6.25)

obeys the recursion

(6.26) S N = S N – 1 + u N – 1.

This means that when summing numbers you may “use them and discard them.” That is, you do not need to read them, store them, and sum them.

You may read u ₀ to form S ₁ and discard u ₀ ; add u ₁ to S ₁ and discard u ₁ ; add u ₂ to S ₂ ; and continue.

Figure 6.6. 

The Recursion S n + 1 = S n + u n

This is very important for hardware and software implementations of running sums. You need only store the current sum, not the measurements that produced it. Two illustrations of the recursion S n + 1 = S n + u n are provided in Figure 4. The diagram on the left is self-explanatory. The diagram on the right says that the sum S n is stored in a memory location, to be added to u n to produce S n + 1 , which is then stored back in the memory location to be added to u n + 1 , and so on.

6.3. Filtering: Weighted Averages*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

Weighted, tapered, or windowed averages are straightforward generalizations of simple averages. They take the form

(6.27)

with the constraint that the “weights in the window,” w n , sum to 1:

(6.28)

When then x is the simple average studied in the section on "Simple Averages".

Example 6.3. 

There are many windows that are commonly used in engineering practice. For N odd, the standard triangular window is

(6.29)

This window, illustrated in Figure 1, weights the input u (N + 1) / 2 by and the inputs u ₁ and u N by The most general triangular window takes the form

(6.30)

Figure 6.7. 

Triangular Window

Exercise 6.3.1.

Determine the constraints on α and β to make the general triangular window a valid window (i.e., ). Show that is a valid solution. Propose another solution that you like.

Exercise 6.3.2.

You are taking three 3-credit courses, one 5-credit course, and one 2-credit course. Write down the weighted average for computing your GPA in a system that awards 4.0 points for an A,3.0 points for a B,..., and (horrors!) 0 points for an F .

6.4. Filtering: Moving Averages*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

Moving averages are generalizations of weighted averages. They are designed to “run along an input sequence, computing weighted averages as they go.” A typical moving average over N inputs takes the form

(6.31)

The most current input, u n , is weighted by w ₀ ; the next most current input, u n – 1 , is weighted by w ₁ ; and so on. This weighting is illustrated in Figure 1. The sequence of weights, w ₀ through w N – 1 , is called a “window,” a “weighting sequence,” or a “filter.” In the example illustrated in Figure 6.8, the current value u n is weighted more heavily than the least current value. This is typical (but not essential) because we usually want x n to reflect more of the recent past than the distant past.

Figure 6.8. 

Moving Average

Example 6.4. 

When the weights w ₀,w ₁,...,w N – 1 are all equal to , then the moving average x n is a “simple moving average”:

(6.32)

This is the same as the simple average, but now the simple average moves along the sequence of inputs, averaging the N most current values.

Exercise 6.4.1.

Evaluate the moving average for the inputs

Interpret your findings.

Exercise 6.4.2.

Evaluate the simple moving average when u n is the sequence

(6.33)

Interpret your result.

Example 6.5. 

When the weights w n equal w ₀ a n for n = 0,1,...,N – 1, then the moving average x n takes the form

.

When a < 1, then un is weighted more heavily than u n – (N – 1); when a > 1, u n – (N – 1) is weighted more heavily than un ; when a = 1,u n is weighted the same as u n – (N – 1).

Exercise 6.4.3.

Evaluate w ₀ so that the exponential weighting sequence w n = w ₀ a n (n = 0,1,...,N – 1) is a valid window (i.e., ).

Exercise 6.4.4.

Compute the moving average when the input sequence u n is

(6.34)

What happens when b = a ? Can you explain this?

6.5. Filtering: Exponential Averages and Recursive Filters*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

Suppose we try to extend our method for computing finite moving averages to infinite moving averages of the form

(6.35)

In general, this moving average would require infinite memory for the weighting coefficients w ₀,w ₁,... and for the inputs u n ,u n – 1,.... Furthermore, the hardware for multiplying w k u n – k would have to be infinitely fast to compute the infinite moving average in finite time. All of this is clearly fanciful and implausible (not to mention impossible). But what if the weights take the exponential form

(6.36)

Does any simplification result? There is hope because the weighting sequence obeys the recursion

(6.37)

This recursion may be rewritten as follows, for k ≥ 1:

(6.38)

Let's now manipulate the infinite moving average and use the recursion for the weights to see what happens. You must follow every step:

(6.39)

This result is fundamentally important because it says that the output of the infinite exponential moving average may be computed by scaling the previous output x n – 1 by the constant a, scaling the new input u n by w ₀ , and adding. Only three memory locations must be allocated: one for w ₀ , one for a , and one for x n – 1 . Only two multiplies must be implemented: one for a x n – 1 and one for w ₀ u n . A diagram of the recursion is given in Figure 1. In this recursion, the old value of the exponential moving average, x n – 1 , is scaled by a and added to w ₀ u n to produce the new exponential moving average x n . This new value is stored in memory, where it becomes x n – 1 in the next step of the recursion, and so on.

Figure 6.9. 

Recursive Implementation of an Exponential Moving Average

Exercise 6.5.1.

Try to extend the recursion of the previous paragraphs to the weighted average

(6.40)

What goes wrong?

Exercise 6.5.2.

Compute the output of the exponential moving average x n = a x n – 1 + w ₀ u n when the input is

(6.41)

Plot your result versus n .

Exercise 6.5.3.

Compute w ₀ in the exponential weighting sequence

(6.42)

to make the weighting sequence a valid window. (This is a special case of Exercise 3 from Filtering: Moving Averages.) Assume – 1 < a < 1

6.6. Filtering: Test Sequences*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

When we design a filter, we design it for a purpose. For example, a moving average filter is often designed to pass relatively constant data while averaging out relatively variable data. In an effort to clarify the behavior of a filter, we typically analyze its response to a standard set of test signals. We will call the impulse, the step, and the complex exponential the standard test signals.

Unit Pulse Sequence. The unit pulse sequence is the sequence

(6.43)

This sequence, illustrated in Figure 1, consists of all zeros except for a single one at n = 0. If the unit pulse sequence is passed through a moving average filter (whether finite or not), then the output is called the unit pulse response:

(6.44)

Figure 6.10. 

Unit Pulse Sequence

(Note that δ n – k = 0 unless n = k.) So the unit pulse sequence may be used to read out the weights of a moving average filter. It is common practice to use w k (the k th weight) and h k (the k th impulse response) interchangeably.

Exercise 6.6.1.

Find the unit pulse response for the finite moving average . Caution: You must consider n < 0,0 ≤ n ≤ N – 1, and n ≥ N .

Exercise 6.6.2.

Find the unit pulse response for the recursive filter x n = a x n – 1 + w ₀ u n .

Unit Step Sequence. The unit step sequence is the sequence

(6.45)

This sequence is illustrated in Figure 2. When this sequence is applied to a moving average filter, the result is the unit step response

(6.46)

The unit step response is just the sequence of partial sums of the unit pulse response.

Figure 6.11. 

Unit Step Sequence

Exercise 6.6.3.

Find the unit step response for the finite moving average filter . Specialize your general result to the special case where for k = 0,1,...,N – 1.

Exercise 6.6.4.

Find the unit step response for the recursive filter x n = a x n – 1 + w ₀ u n .

Complex Exponential Sequence. The complex exponential sequence is the sequence

(6.47)

This sequence, illustrated in Figure 3, is a “discrete-time phasor” that “ratchets” counterclockwise (CCW) as k moves to k + 1 and clockwise (CW) as k moves to k – 1. Each time the phasor ratchets, it turns out an angle of θ . Why should such a sequence be a useful test sequence? There are two reasons.

Figure 6.12. 

Discrete-Time Phasor

(i) e jkθ represents (or codes) cosk θ . The real part of the sequence e jkθ is the cosinusoidal sequence cosk θ :

(6.48)

Therefore the discrete-time phasor e jkθ represents (or codes) cosk θ in the same way that the continuous-time phasor e jωt codes cosω t . If the moving average filter

(6.49)

has real coefficients, we can get the response to a cosinusoidal sequence by taking the real part of the following sum:

(6.50)

In this formula, the sum

(6.51)

is called the complex frequency response of the filter and is given the symbol

(6.52)

This complex frequency response is just a complex number, with a magnitude and a phase arg . Therefore the output of the moving average filter is

(6.53)

This remarkable result says that the output is also cosinusoidal, but its amplitude is rather than 1, and its phase is rather than 0. In the examples to follow, we will show that the complex “gain” can be highly selective in θ , meaning that cosines of some angular frequencies are passed with little attenuation while cosines of other frequencies are dramatically attenuated. By choosing the filter coefficients, we can design the frequency selectivity we would like to have.

(ii) e jkθ is a sampled data version of e jωt . The discrete-time phasor e jkθ can be produced physically by sampling the continuous-time phasor e jωt at the periodic sampling instants t k = k T :

(6.54)

The dimensions of θ are radians, the dimensions of ω are radians/second, and the dimensions of T are seconds. We call T the sampling interval and the sampling rate or sampling frequency. If the original angular frequency of the phasor e jωt is increased to , then the discrete-time phasor remains e jkθ :

(6.55) e j[ω + m(2π / T)]t | t = k T = e j(ω k T + k m2π) = e jkθ .

This means that all continuous-time phasors of the form e j[ω + m(2π / T)]t “hide under the same alias” when viewed through the sampling operation. That is, the sampled-data phasor cannot distinguish the frequency ω from the frequency . In your subsequent courses you will study aliasing in more detail and study the Nyquist rule for sampling:

(6.56)

This rule says that you must sample signals at a rate that exceeds the bandwidth of the signal.

Example 6.6. 

Let's pass the cosinusoidal sequence u k = cos kθ through the finite moving average filter

(6.57)

We know from our previous result that the output is

(6.58)

The complex frequency response for this example is

(6.59)

(Do you see your old friend, the finite sum formula, at work?) Let's try to manipulate the result into a more elegant form:

(6.60)

The magnitude of the function is

(6.61)

At θ = 0, corresponding to a “DC phasor,” equals 1; at . The magnitude of the complex frequency response is plotted in Figure 4.

Figure 6.13. 

Frequency Selectivity of a Moving Average Filter

This result shows that the moving average filter is frequency selective, passing low frequencies with gain near 1 and high frequencies with gain near 0.

Exercise 6.6.5.

Compute the phase of the complex frequency response

(6.62)

Exercise 6.6.6.

Choose the filter length N for the filter , so that a 60 Hz cosine, sampled at the rate , is perfectly zeroed out as it comes through the filter.

Exercise 6.6.7.

(MATLAB) Write a MATLAB program to compute and plot the magnitude and the phase arg versus – π < θ < π when

(6.63)

Choose suitable increments for θ .

Exercise 6.6.8.

Compute the complex frequency response for the recursive filter x n = a x n – 1 + w ₀ u n .

6.7. Filtering: Numerical Experiment (Frequency Response of First-Order Filter)*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

Consider the exponential moving average filter

(6.64)

Write out a few terms of the sum to show how the filter works.

Write xn as a recursion and discuss the computer memory required to implement the filter.

Compute the complex frequency response for the filter.

Write a MATLAB program to plot the magnitude and phase of the complex frequency response versus θ for in steps of Do this for two values of a, namely, a = 0.98 and a = – 0.98. Explain your findings.

Write a MATLAB program to pass the following signals through the filter when a = 0.98:

u n = δ n

u n = ξ n

.

Plot the outputs for each case and interpret your findings in terms of the complex frequency response . Repeat step 5 for a = – 0.98. Interpret your findings.

Chapter 7. Binary Codes

7.1. Binary Codes: Introduction*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

Acknowledgment: Richard Hamming's book, Information Theory and Coding, Prentice-Hall, New York (1985) and C. T. Mullis's unpublished notes have influenced our treatment of binary codes. The numerical experiment was developed by Mullis.

We use this chapter to introduce students to the communication paradigm and to show how arbitrary symbols may be represented by binary codes. These symbols and their corresponding binary codes may be computer instructions, integer data, approximations to real data, and so on.

We develop some ad hoc tree codes for representing information and then develop Huffman codes for optimizing the use of bits. Hamming codes add check bits to a binary word so that errors may be detected and corrected. The numerical experiment has the students design a Huffman code for coding Lincoln's Gettysburg Address.

Introduction

It would be stretching our imagination to suggest that Sir Francis had digital audio on his minde (sic) when he wrote the prophetic words

Sir Francis Bacon, 1623

...a man may express and signifie the intentions of his minde, at any distance... by... objects... capable of a twofold difference onely.

Nonetheless, this basic idea forms the basis of everything we do in digital computing, digital communications, and digital audio/video. In 1832, Samuel F. B. Morse used the very same idea to propose that telegram words be coded into binary addresses or binary codes that could be transmitted over telegraph lines and decoded at the receiving end to unravel the telegram. Morse abandoned his scheme, illustrated in Figure 1, as too complicated and, in 1838, proposed his fabled Morse code for coding letters (instead of words) into objects (dots, dashes, spaces) capable of a threefold difference onely (sic).

Figure 7.1. 

Generalized Coder-Decoder

The basic idea of Figure 1 is used today in cryptographic systems, where the “address a i " is an encyphered version of a message w i ; in vector quantizers, where the “address a i " is the address of a close approximation to data w i ; in coded satellite transmissions, where the “address a i " is a data word w i plus parity check bits for detecting and correcting errors; in digital audio systems, where the “address a i " is a stretch of digitized and coded music; and in computer memories, where a i is an address (a coded version of a word of memory) and w i is a word in memory.

In this chapter we study three fundamental questions in the construction of binary addresses or binary codes. First, what are plausible schemes for mapping symbols (such as words, letters, computer instructions, voltages, pressures, etc.) into binary codes? Second, what are plausible schemes for coding likely symbols with short binary words and unlikely symbols with long words in order to minimize the number of binary digits (bits) required to represent a message? Third, what are plausible schemes for “coding” binary words into longer binary words that contain “redundant bits” that may be used to detect and correct errors? These are not new questions. They have occupied the minds of many great thinkers. Sir Francis recognized that arbitrary messages had binary representations. Alan Turing, Alonzo Church, and Kurt Goedel studied binary codes for computations in their study of computable numbers and algorithms. Claude Shannon, R. C. Bose, Irving Reed, Richard Hamming, and many others have studied error control codes. Shan- non, David Huffman, and many others have studied the problem of efficiently coding information.

In this chapter we outline the main ideas in binary coding and illustrate the role that binary coding plays in digital communications. In your subsequent courses in electrical and computer engineering you will study integrated circuits for building coders and decoders and mathematical models for designing good codes.

7.2. Binary Codes: The Communication Paradigm*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

A paradigm is a pattern of ideas that form the foundation for a body of knowledge. A paradigm for (tele-) communication theory is a pattern of basic building blocks that may be applied to the dual problems of (i) reliably transmitting information from source to receiver at high speed or (ii) reliably storing information from source to memory at high density. High-speed communication permits us to accommodate many low-rate sources (such as audio) or one high-rate source (such as video). High-density storage permits us to store a large amount of information in a small space. For example, a typical 1.2 Mbyte floppy disc stores 9.6 × 10⁶ bits of information, whereas a typical CD stores about 2 × 10⁹ bits, enough for one hour's worth of high-quality sound.

Figure 7.2. 

Basic Building Blocks in a (Tele-) Communication System

Figure 1 illustrates the basic building blocks that apply to any problem in the theory of (tele-) communication. The source is an arbitrary source of information. It can be the time-varying voltage at the output of a vibration sensor (such as an integrating accelerometer for measuring motion or a microphone for measuring sound pressure); it can be the charges stored in the CCD array of a solid-state camera; it can be the addresses generated from a sequence of keystrokes at a computer terminal; it can be a sequence of instructions in a computer program. The source coder is a device for turning primitive source outputs into more efficient representations. For example, in a recording studio, the source coder would convert analog voltages into digital approximations using an A / D converter; a fancy source coder would use a fancy A / D converter that finely quantized likely analog values and crudely quantized unlikely values. If the source is a source of discrete symbols like letters and numbers, then a fancy source code would assign short binary sequences to likely symbols (such as e ) and long binary sequences to unlikely symbols (such as z ). The channel coder adds “redundant bits” to the binary output of the source coder so that errors of transmission or storage may be detected and corrected. In the simplest example, a binary string of the form 01001001 would have an extra bit of 1 added to give even parity (an even number of l's) to the string; the string 10110111 would have an extra bit of 0 added to preserve the even parity. If one bit error is introduced in the channel, then the parity is odd and the receiver knows that an error has occurred. The modulator takes outputs of the channel coder, a stream of binary digits, and constructs an analog waveform that represents a block of bits. For example, in a 9600 baud Modem, five bits are used to determine one of 2⁵ = 32 phases that are used to modulate the signal A cos(ω t + φ). Each possible string of five bits has its own personalized phase, φ , and this phase can be determined at the receiver. The signal Acos(ω t + φ) is an analog signal that may be transmitted over a channel (such as a telephone line, a microwave link, or a fiber-optic cable). The channel has a finite bandwidth, meaning that it distorts signals, and it is subject to noise or interference from other electromagnetic radiation. Therefore transmitted information arrives at the demodulator in imperfect form. The demodulator uses filters matched to the modulated signals to demodulate the phase and look up the corresponding bit stream. The channel decoder converts the coded bit stream into the information bit stream, and the source decoder looks up the corresponding symbol. This sequence of steps is illustrated symbolically in Figure 7.3.

Figure 7.3. 

Symbolic Representation of Communication

In your subsequent courses on communication theory you will study each block of Figure 1 in detail. You will find that every source of information has a characteristic complexity, called entropy, that determines the minimum rate at which bits must be generated in order to represent the source. You will also find that every communication channel has a characteristic tolerance for bits, called channel capacity. This capacity depends on signal-to-noise ratio and bandwidth. When the channel capacity exceeds the source entropy, then you can transmit information reliably; if it does not, then you cannot.

7.3. Binary Codes: From Symbols to Binary Codes*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

Perhaps the most fundamental idea in communication theory is that arbitrary symbols may be represented by strings of binary digits. These strings are called binary words, binary addresses, or binary codes. In the simplest of cases, a finite alphabet consisting of the letters or symbols s ₀,s ₁,...,s M – 1 is represented by binary codes. The obvious way to implement the representation is to let the i th binary code be the binary representation for the subscript i :

(7.1)

The number of bits required for the binary code is N where

(7.2)2 N – 1 < M ≤ 2 N .

We say, roughly, that N = log₂ M .

Octal Codes. When the number of symbols is large and the corresponding binary codes contain many bits, then we typically group the bits into groups of three and replace the binary code by its corresponding octal code. For example, a seven-bit binary code maps into a three-digit octal code as follows:

(7.3)

The octal ASCII codes for representing letters, numbers, and special characters are tabulated in Table 1.

Exercise 7.3.1.

Write out the seven-bit ASCII codes for A,q,7,a n d{.

Table 7.1. Octal ASCII Codes (from Donald E. Knuth, The TEXbook, ©1986 by the American Mathematical Society, Providence, Rhode Island p. 367, published by Addison-Wesley Publishing Co.)

| '0 | '1 | '2 | '3 | '4 | '5 | '6 | '7
'00x | ␀ | ␁ | ␂ | ␃ | ␄ | ␅ | ␆ | ␇
'01x | ␈ | ␉ | ␊ | ␋ | ␌ | ␍ | ␎ | ␏
'02x | ␐ | ␑ | ␒ | ␓ | ␔ | ␕ | ␖ | ␗
'03x | ␘ | ␙ | ␚ | ␛ | ␜ | ␝ | ␞ | ␟
'04x | ␠ | ! | " | # | $ | % | & | '
'05x | ( | ) | * | + | , | - | . | /
'06x | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7
'07x | 8 | 9 | : | ; | < | = | > | ?
'10x | @ | A | B | C | D | E | F | G
'11x | H | I | J | K | L | M | N | O
'12x | P | Q | R | S | T | U | V | W
'13x | X | Y | Z | [ | \ | ] | ^ | _
'14x | ` | a | b | c | d | e | f | g
'15x | h | i | j | k | l | m | n | o
'16x | p | q | r | s | t | u | v | w
'17x | x | y | z | { | | | } | ~ | ␡

Exercise 7.3.2.

Add a 1 or a 0 to the most significant (left-most) position of the seven-bit ASCII code to produce an eight-bit code that has even parity (even number of 1's). Give the resulting eight-bit ASCII codes and the corresponding three-digit octal codes for %, u,f,8, and + .

Quantizers and A/D Converters. What if the source alphabet is infinite? Our only hope is to approximate it with a finite collection of finite binary words. For example, suppose the output of the source is an analog voltage that lies between – V ₀ and + V ₀ . We might break this peak-to-peak range up into little voltage cells of size and approximate the voltage in each cell by its midpoint. This scheme is illustrated in Figure 1. In the figure, the cell C i is defined to be the set of voltages that fall between and :

(7.4)

The mapping from continuous values of V to a finite set of approximations is

(7.5)

That is, V is replaced by the quantized approximation whenever V lies in cell C i . We may represent the quantized values with binary codes by simply representing the subscript of the cell by a binary word. In a subsequent course on digital electronics and microprocessors you will study A / D (analog-to-digital) converters for quantizing variables.

Figure 7.4. 

A Quantizer

Example 7.1. 

If M = 8, corresponding to a three-bit quantizer, we may associate quantizer cells and quantized levels with binary codes as follows:

(7.6)

This particular code is called a sign-magnitude code, wherein the most significant bit is a sign bit and the remaining bits are magnitude bits (e.g., 110 ∼ – 2 and 010 ∼ 2). One of the defects of the sign-magnitude code is that it wastes one code by using 000 for 0 and 100 for-O. An alternative code that has many other advantages is the 2's complement code. The 2 ' s complement codes for positive numbers are the same as the sign-magnitude codes, but the codes for negative numbers are generated by complementing all bits for the corresponding positive number and adding 1:

(7.7)

Exercise 7.3.3.

Generate the four-bit sign-magnitude and four-bit 2's complement binary codes for the numbers – 8, – 7,..., – 1,0,1,2,...,7.

Exercise 7.3.4.

Prove that, in the 2 ' s complement representation, the binary codes for – nand + n sum to zero. For example,

(7.8)

In your courses on computer arithmetic you will learn how to do arithmetic in various binary-coded systems. The following problem illustrates how easy arithmetic is in 2's complement.

Exercise 7.3.5.

Generate a table of sums for all 2 ' s complement numbers between – 4 and +3. Show that the sums are correct. Use 0 + 0 = 0,0 + 1 = 1,1 + 0 = 1, and 1 + 1 = 0 with a carry into the next bit. For example, 001 + 001 = 010.

Binary Trees and Variable-Length Codes. The codes we have constructed so far are constant-length codes for finite alphabets that contain exactly M = 2 N symbols. In the case where M = 8 and N = 3, then the eight possible three-bit codes may be represented as leaves on the branching tree illustrated in Figure 2(a). The tree grows a left branch for a 0 and a right branch for a 1, until it terminates after three branchings. The three-bit codes we have studied so far reside at the terminating leaves of the binary tree. But what if our source alphabet contains just five symbols or letters? We can represent these five symbols as the three-bit symbols 000 through 100 on the binary tree. This generates a constant-length code with three unused, or illegal, symbols 101 through 111. These are marked with an " x " in Figure 2(a). These unused leaves and the branches leading to them may be pruned to produce the binary tree of Figure 2(b).

If we admit variable-length codes, then we have several other options for using a binary tree to construct binary codes. Two of these codes and their corresponding binary trees are illustrated in Figure 3. If we disabuse ourselves of the notion that each code word must contain three or fewer bits, then we may construct binary trees like those of Figure 4 and generate their corresponding binary codes. In Figure 4(a), we grow a right branch after each left branch and label each leaf with a code word. In Figure 4(b), we prune off the last right branch and associatea code word with the leaf on the last left branch.

Figure 7.5. 

(a)

(b)

Binary Trees and Constant-Length Codes; (a) Binary Tree, and (b) Pruned Binary Tree

All of the codes we have generated so far are organized in Table 2. For each code, the average number of bits/symbol is tabulated. This average ranges from 2.4 to 3.0. If all symbols are equally likely to appear, then the best variable-length code would be code 2.

Figure 7.6. 

(a)

(b)

Binary Trees and Variable-Length Codes; (a) Binary Tree for Variable-length Code, and (b) Another Binary Tree for Variable-length Code

All of the codes we have constructed have a common characteristic: each code word is a terminating leaf on a binary tree, meaning that no code word lies along a limb of branches to another code word. We say that no code word is a prefix to another code word. This property makes each of the codes instantaneously decodable, meaning that each bit in a string of bits may be processed instantaneously (or independently) without dependence on subsequent bits.

Exercise 7.3.6.

Decode the following sequence of bits using code 2:

(7.9) 0111001111000000101100111 .

Figure 7.7. 

(a)

(b)

Left-Handed Binary Trees for Variable-Length Codes; (a) Left-handed Binary Tree, and (b) Pruned Binary Tree

Table 7.2. Variable Length Codes

Code # | S ₀ | S ₁ | S ₂ | S ₃ | S ₄ | Average Bits/Symbol
1 | 000 | 001 | 010 | 011 | 100 | 15 / 5 = 3.0
2 | 000 | 001 | 01 | 10 | 11 | 12 / 5 = 2.4
3 | 000 | 001 | 010 | 011 | 1 | 13 / 5 = 2.6
4 | 1 | 01 | 001 | 0001 | 00001 | 15 / 5 = 3.0
5 | 1 | 01 | 001 | 0001 | 0000 | 14 / 5 = 2.8

Exercise 7.3.7.

Illustrate the following codes on a binary tree. Which of them are instantaneously decodable? Which can be pruned and remain instantaneously decodable?

(7.10)

Code #2 generated in Table 2 seems like a better code than code #5 because its average number of bits/symbol (2.4) is smaller. But what if symbol S ₀ is a very likely symbol and symbol S ₄ is a very unlikely one? Then it may well turn out that the average number of bits used by code #5 is less than the average number used by code #2. So what is the best code? The answer depends on the relative frequency of use for each symbol. We explore this question in the next section.

7.4. Binary Codes: Huffman Codes for Source Coding*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

In 1838, Samuel Morse was struggling with the problem of designing an efficient code for transmitting information over telegraph lines. He reasoned that an efficient code would use short code words for common letters and long code words for uncommon letters. (Can you see the profit motive at work?) In order to turn this reasoned principle into workable practice, Morse rummaged around in the composition trays for typeface in a printshop. He discovered that typesetters use many more e ' s than s's. He then formed a table that showed the relative frequency with which each letter was used. His ingenious, variable-length Morse code assigned short codes to likely letters (like “dot” for e ' ) and long codes to unlikely letters (like “dash dash dot dot” for z ' ). We now know that Morse came within about 15% of the theoretical minimum for the average code word length for English language text.

A Variation on Morse's Experiment. In order to set the stage for our study of efficient source codes, let's run a variation on Morse's experiment to see if we can independently arrive at a way of designing codes. Instead of giving ourselves a composition tray, let's start with a communication source that generates five symbols or letters S ₀,S ₁,S ₂,S ₃,S ₄ . We run the source for 100 transmissions and observe the following numbers of transmissions for each symbol:

(7.11)

We will assume that these “source statistics” are typical, meaning that 1000 transmissions would yield 500 S ' ₀s and so on.

The most primitive binary code we could build for our source would use three bits for each symbol:

(7.12)

This code is inefficient in two ways. First, it leaves three illegal code words that correspond to no source symbol. Second, it uses the same code word length for an unlikely symbol (like S ₄ ) that it uses for a likely symbol (like S ₀ ). The first defect we can correct by concatenating consecutive symbols into symbol blocks, or composite symbols. If we form a composite symbol consisting of M source symbols, then a typical composite symbol is S ₁ S ₀ S ₁ S ₄ S ₂ S ₃ S ₁ S ₂ S ₀ . The number of such composite symbols that can be generated is 5 M . The binary code for these 5 M composite symbols must contain N binary digits where

(7.13)

The number of bits per source symbol is

(7.14)

This scheme improves on the best variable length code of Table 2 from "Binary Codes: From Symbols to Binary Codes" by 0.08 bits/symbol.

Exercise 7.4.1.

Suppose your source of information generates the 26 lowercase roman letters used in English language text. These letters are to be concatenated into blocks of length M. Complete the following table of N (number of bits) versus M (number of letters in a block) and show that approaches log₂26.

Table 7.3.

| M
| 1 | 2 | 3 | 4 | 5 | 6
N | 5 | 10 |  |  |  |
N / M | 5 | 5 |  |  |  |

Now let's reason, as Morse did, that an efficient code would use short codes for likely symbols and long codes for unlikely symbols. Let's pick code #5 from Table 2 from "Binary Codes: From Symbols to Binary Codes" for this purpose:

(7.15)

This is a variable-length code. If we use this code on the 100 symbols that generated our source statistic, the average number of bits/symbol is

(7.16)

Exercise 7.4.2.

Use the source statistics of Equation 1 to determine the average number of bits/symbol for each code in Table 2 from "Binary Codes: From Symbols to Binary Codes".

Entropy. So far, each ad hoc scheme we have tried has produced an improvement in the average number of bits/symbol. How far can this go? The answer is given by Shannon's source coding theorem, which says that the minimum number of bits/symbol is

(7.17)

where pi is the probability that symbol Si is generated and is a fundamental property of the source called entropy. For our five-symbol example, the table of pi and – logp i is given in Table 2. The entropy is 1.861, and the bound on bits/symbol is

(7.18)

Code #5 comes within 0.039 of this lower bound. As we will see in the next paragraphs, this is as close as we can come without coding composite symbols.

Table 7.4. Source statistics for Five-Symbol Source

Symbol | Probability | ~ Log Probability
S ₀ | 0.5 | 1
S ₁ | 0.2 | 2.32
S ₂ | 02 | 2.32
S ₃ | 0.05 | 4.32
S ₄ | 0.05 | 4.32

Exercise 7.4.3.

Select an arbitrary page of English text. Build a table of source statistics containing p i (relative frequencies) and-log p i for a through z . (Ignore distinction between upper and lower case and ignore punctuation and other special symbols.) Compute the entropy .

Huffman Codes. In the late 1950s, David Huffman discovered an algorithm for designing variable-length codes that minimize the average number of bits/symbol. Huffman's algorithm uses a principle of optimality that says, “the optimal code for M letters has imbedded in it the optimal code for the M – 1 letters that result from aggregating the two least likely symbols.” When this principle is iterated, then we have an algorithm for generating the binary tree for a Huffman code:

label all symbols as “children”;

“twin” the two least probable children and give the twin the sum of the probabilities:

Figure 7.8. 

(regard the twin as a child; and

repeat steps (ii) and (iii) until all children are accounted for.

This tree is now labeled with 1's and 0's to obtain the Huffman code. The labeling procedure is to label each right branch with a 1 and each left branch with a 0. The procedure for laying out symbols and constructing Huffman trees and codes is illustrated in the following examples.

Example 7.2. 

Consider the source statistics

(7.19)

for which the Huffman algorithm produces the following binary tree and its corresponding code:

Figure 7.9. 

Example 7.3. 

The Huffman code for the source statistics

(7.20)

is illustrated next:

Figure 7.10. 

Exercise 7.4.4.

Generate binary trees and Huffman codes for the following source statistics:

(7.21)

Coding a FAX Machine. Symbols can arise in unusual ways and be defined quite arbitrarily. To illustrate this point, we consider the design of a hypothetical FAX machine. For our design we will assume that a laser scanner reads a page of black and white text or pictures, producing a high voltage for a black spot and a low voltage for a white spot. We will also assume that the laser scanner resolves the page at 1024 lines, with 1024 spots/line. This means that each page is represented by a two-dimensional array, or matrix, of pixels (picture elements), each pixel being 1 or 0. If we simply transmitted these l's and O ' s, then we would need 1024 × 1024 = 1,059,576 bits. If these were transmitted over a 9600 baud phone line, then it would take almost 2 minutes to transmit the FAX. This is a long time.

Let's think about a typical scan line for a printed page. It will contain long runs of 0's, corresponding to long runs of white, interrupted by short bursts of l's, corresponding to short runs of black where the scanner encounters a line or a part of a letter. So why not try to define a symbol to be “a run of k 0's" and code these runs? The resulting code is called a “run length code.” Let's define eight symbols, corresponding to run lengths from 0 to 7 (a run length of 0 is a 1):

(7.22)

If we simply used a simple three-bit binary code for these eight symbols, then for each scan line we would generate anywhere from 3 × 1024 bits (for a scan line consisting of all 1's) to 3 × 1024 / 7 ≅ 400 bits (for a scan line consisting of all 0's). But what if we ran an experiment to determine the relative frequency of the run lengths S ₀ through S ₇ and used a Huffman code to “run length encode” the run lengths? The following problem explores this possibility and produces an efficient FAX code.

Exercise 7.4.5.

An experiment conducted on FAXed documents produces the following statistics for run lengths of white ranging from 0 to 7:

(7.23)

These statistics indicate that only 1% of a typical page is black. Construct the Huffman code for this source. Use your Huffman code to code and decode these scan lines:

Figure 7.11. 

7.5. Binary Codes: Hamming Codes for Channel Coding*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

The idea behind Hamming codes is to intersperse, or append, extra binary digits to a binary code so that errors in transmission of the code over a channel may be detected and corrected. For example, suppose we transmit the code 01101001, and it is received as 01001001. In this transmission, the third most significant bit is received erroneously. Let's define the following “modulo-2 addition” of binary numbers:

(7.24)

Multiplication in modulo-2 arithmetic is simply 0 · 0 = 0 · 1 = 1 · 0 = 0 and 1 · 1 = 1. Then we can say that the error sequence 00100000 is “added” to the transmission 01101001 to produce the erroneous reception:

(7.25)

Hamming error correcting codes will permit us to receive the erroneous transmission and to detect and correct the error. This is obviously of great value in transmitting and storing information. (Imagine how upset you would be to have the binary code for your checking account confused with that of Mrs. Joan Kroc.)

Choosing the Number of Check Bits. Let's suppose we have N bits of information that we wish to transmit and that we wish to intersperse “check bits” that will enable us to detect and correct any single bit error in the transmission. If we use N information bits and n check bits, then we will transmit a code word containing N + n bits. The n check bits can code 2 n events, and we want these events to indicate whether or not any errors occurred and, if so, where they occurred. Therefore we require

where (N + n) is the number of single error events that can occur and +1 is the number of no-error events. For example, when N = 4, we require n = 3 so that 2³ ≥ (4 + 3) + 1.

How many check bits do you require to code seven bits of information for single error correction?

Code Construction. Let's suppose we have constructed an (N,n) Hamming code consisting of N information bits and n check bits (or parity bits). We denote the information bits by x ₁,x ₂,...,x N and the check bits by c ₁,c ₂,...,c n . These bits may be interspersed. When N = 4 and n = 3, then a typical array of bits within a code word would be one of the following:

(7.26)

The first ordering is “natural” (as we will see), and the second is “systematic” (a term that is used to describe any code whose head is information and whose tail is check). If a single error occurs in an (N,n) code, then the received code word will be the modulo-2 sum of the code word and the error word that contains a 1 in its i th position:

(7.27)

We would like to operate on this received code word in such a way that the location of the error bit can be determined. If there were no code word, then an obvious solution would be to premultiply the error word by the parity check matrix

(7.28)

The i th column of A T is just the binary code for i . When A T premultiplies an error word, the error bit picks out the column that codes the error position:

Figure 7.12. 

If the error word contains no error bits, then the product is 0, indicating no errors.

This seems like a good idea, but what about the effect of the code word? In Exercise 1, you are asked to show that the effect of the parity check matrix A T applied to the modulo-2 sum of a code word x and an error word e is

(7.29)A T (x⊕e) = A T x⊕A T e.

In this equation all sums and products obey the rules of modulo-2 arithmetic.

Exercise 7.5.1.

Let y = x⊕e denote the modulo-2 sum of a code word x and an error word e;A T is a parity check matrix. Show that

(7.30)

We have designed the parity check matrix A T so that the syndrome A T e produces a binary code for the error location. (The location of the error is th ' e syndrome for the error word.) The product A T x will interfere with this syndrome unless A T x = 0 . Therefore we will require that the code word x satisfy the constraint

(7.31)A T x = 0.

This constraint actually defines the Hamming code. Let's illustrate this point by applying the constraint to a code word in its “natural format” x T = .

Natural Codes. When the information bits and the check bits are coded in their natural order , then we may determine the check bits by writing A T x as follows:

(7.32)

We use the rules of modulo-2 arithmetic to write these constraints as

(7.33)

Therefore the check bits c ₁,c ₂ , and c ₃ are simply the following modulo-2 sums

(7.34)

This finding may be organized into the matrix equation

(7.35)

This equation shows how the code word x is built from the information bits . We call the matrix that defines the construction a coder matrix and write it as H :

(7.36)

(7.37)

This summarizes the construction of a Hamming code x .

Exercise 7.5.2.

Check to see that the product of the parity check matrix A T and the coder matrix H is A T H = 0. Interpret this result.

Exercise 7.5.3.

Fill in the following table to show what the Hamming (4,3) code is:

(7.38)

Exercise 7.5.4.

Design a Hamming (11,n) code for coding eleven information bits against single errors. Show your equations for c ₁,c ₂,...,c n and write out the coder matrix H for x = HΘ.

Decoding. To decode a Hamming code, we form the syndrome A T y for the received (and possibly erroneous) code word y = x⊕e. Because A T x = 0, the syndrome is

(7.39)s = A T e.

Convert this binary number into its corresponding integer location and change the bit of y in that location. If the location is zero, do nothing. Now strip off the information bits. This is the decoding algorithm.

Exercise 7.5.5.

Use the table of Hamming (4,3) codes from Exercise 3 to construct a table of received codes that contain either no bit errors or exactly one bit error. Apply the decoding algorithm to construct and show that all received code words with one or fewer errors are correctly decoded.

Digital Hardware. The tables you have constructed in Exercise 3 and 5 for coding and decoding Hamming (4,3) codes may be stored in digital logic chips. Their functionality is illustrated in Figure 1. The coder chip accepts as its address and generates a coded word. The decoder chip accepts as its address and generates a decoded word. In your courses on digital logic you will study circuits for implementing coders and decoders.

Figure 7.13. 

Digital Logic for Hamming Code

Exercise 7.5.6.

Discuss the possibility of detecting a received (4,3) code word that is neither a valid code word nor a code word with a single error. How would you use such a detector?

Exercise 7.5.7.

What fraction of received seven-bit words can be correctly decoded as Hamming (4,3) codes?

Systematic Codes. Systematic Hamming codes are codes whose information bits lead and whose check bits trail. The format for a (4,3) code is then . The construction of a (4,3) code word from the information bits may be written as

(7.40)

The coder matrix takes the form

(7.41)

The problem is to find the matrix C that defines the construction of check bits. The constraint A T x = 0 produces the constraint A T H = 0 so that A T HΘ = 0. The constraints A T H = 0 may be written out as

These constraints produce all the equations we need (twelve equations in twelve unknowns) to determine the c ij .

Exercise 7.5.8.

Solve Equation 19 for the c ij . Show that the coder matrix for a systematic Hamming (4,3) code is

(7.42)

Exercise 7.5.9.

Show that the coder matrix of Exercise 7 is a permutation of the coder matrix in Equation 14. (That is, the rows are reordered.)

Exercise 7.5.10.

(MATLAB) Write a MATLAB program that builds Hamming (4,3) codes from information bits and decodes Hamming (4,3) codes to obtain information bits . Synthesize all seven-bit binary codes and show that your decoder correctly decodes correct codes and one-bit error codes.

7.6. Binary Codes: Numerical Experiment (Huffman Codes)*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

The paragraph below contains Lincoln's Gettysburg Address. Ignore special symbols like periods and ignore the distinction between lowercase and uppercase letters. Compute the relative frequency of occurrence for each of the 26 letters of the alphabet. Write a MATLAB program to generate a binary tree for the Huffman code of the Gettysburg Address. When you have generated the code, compute the average code word length

(7.43)

where is the relative frequency for symbol i and l i is the code word length for symbol i . Compare L with the entropy

(7.44)

and compare L to log₂26. Interpret your findings.

Gettysburg Address

Fourscore and seven years ago, our fathers brought forth upon this
continent a new nation, conceived in liberty and dedicated to the proposition
that all men are created equal. Now we are engaged in a great civil war,
testing whether that nation or any nation so conceived and so dedicated can
long endure. We are met on a great battlefield of that war. We have come
to dedicate a portion of that field as a final resting-place for those who have
given their lives that that nation might live. It is altogether fitting and proper
that we should do this. But in a larger sense, we cannot dedicate, we cannot
consecrate, we cannot hallow this ground. The brave men, living and dead,
who struggled here have consecrated it far above our power to add or detract.
The world will little note nor long remember what we say here, but it can
never forget what they did here. It is for us the living rather to be dedicated
here to the unfinished work which they who fought here have thus far so nobly
advanced. It is rather for us to be here dedicated to the great task remaining
before us–that from these honored dead we take increased devotion to that
cause for which they gave the last full measure of devotion–that we here
highly resolve that these dead should not have died in vain, that this nation
under God shall have a new birth of freedom, and that government of the
people, by the people, for the people shall not perish from the Earth.
              -Abraham Lincoln, November 19, 1863

Chapter 8. An Introduction to MATLAB

8.1. An Introduction to MATLAB: Introduction*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

Acknowledgment: This appendix, written with assistance from Cédric J. Demeure and Peter Massey, was inspired by the MATLAB user's manual from The MATHWORKS, Inc. The MA TLAB Primer, available through the MATLAB User's Group, is a useful learning aid for teachers and students. To join the MATLAB User's Group, send your request via E-mail to matlab-users request@mcs.anl.gov.

Introduction

MATLAB stands for “Matrix Laboratory.” It is a computing environment specifically designed for matrix computations. The program is ideally suited to circuit analysis, signal processing, filter design, control system analysis, and much more. Beyond that, its versatility with complex numbers and graphics makes it an attractive choice for many other programming tasks. MATLAB can be thought of as a programming language like PASCAL, FORTRAN, C, or BASIC. Like most versions of BASIC, MATLAB can be used in an interactive mode wherein statements are executed immediately as they are typed. Alternatively, a program can be written in advance and saved to a disc file using an editor and then executed in MATLAB. You will find both modes of operation useful.

8.2. An Introduction to MATLAB: Running MATLAB (Macintosh)*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

In order to run MATLAB on a Macintosh SE or PLUS computer, you need the program called EDU-MATLAB. The program requires at least 1 Mbyte of memory, System 3.0 or above, Finder version 3.0 or above, and an 800K drive. A hard disc drive is highly recommended. In order to run MATLAB on a Macintosh II, IIx, IIcx, or SE / 30, you need the program called MacII-MATLAB, and the same system requirements apply.

To start MATLAB, you may need to open the folder containing the MATLAB program. Then just “double-click” the program icon or the program name (for example, EDU-MATLAB). Figure A.1 shows a typical organization of the folder containing Mac II-MATLAB. It contains the main program, the settings file, the demonstrations folder, and any toolbox folders. The double-click on Mac II-MATLAB produces the Command window as shown in Figure A.2. You will also see a Graph window partially hidden behind it. (The fact that the window is not in ffont means that it is opened but not currently active.) If you do not know what “clicking,” “dragging,” “pop-up menu,” and “trash” mean, you should stop reading now and familiarize yourself with the Macintosh.

Figure 8.1. 

The MATLAB Folder (©Apple Computer, Inc., used with permission.)

In the command window, you should see the prompt ≫ . The program interpreter is waiting for you to enter instructions. At this point it is a good idea to run the demonstration programs that are available in the “About MATLAB” menu under the Apple menu. Just click on the “demos” button and select a demo. During pauses, strike any key to continue. Whenever you have a MATLAB file in any folder, then you may double-click the file to launch the program. This allows you to have your own folder containing your own MATLAB files, separated from the MATLAB folder.

Figure 8.2. 

The Command Window (©Apple Computer, Inc., used with permission.)

MATLAB has four types of windows:

Command for computing, programming, and designing input/output displays;

Graph for displaying plots and graphs;

Edit for creating and modifying your own files; and

Help for getting on-line help and for running demos.

All windows follow the traditional behavior of Macintosh windows. You can resize them (actually the help window has a fixed size) or move them. For more details on menus and windows, see the Macintosh and MATLAB manuals.

8.3. An Introduction to MATLAB: Running MATLAB (PC)*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

In order to run MATLAB (Version 3.5) on an IBM or compatible personal computer, you must have a floating point math coprocessor (80x87)installed and at least 512 kbytes of memory. The program is called PCMAT-LAB.EXE, but it is usually invoked via the batch file MATLAB.BAT in the MATLAB subdirectory. If you are using a menu system and MATLAB is one of your choices, just choose it. Otherwise, go to the MATLAB suiUirectory and type MATLAB.

You may be able to usea more powerful implementation of MATLAB if you have an 80286 or 80386 machine. AT-MATLAB runs on an 80286 with at least 1 Mbyte of extended memory. AT-MATLAB is distributed with PC-MATLAB. 386-MATLAB, a special version for 80386 or 80486 machines with virtual memory support and no limits on variable size, is sold separately.

When you run MATLAB, you should see the prompt ≫ . The program interpreter is waiting for you to enter instructions. Some MATLAB instructions, such as plot, are graphics-type instructions which plot results and data. Execution of one of these graphics instructions puts the PC screen into the graphics mode, which displays the resulting plot. No instructions can be executed in the graphics mode other than a screen-dump function. Striking any other key will return the PC to the command mode, but the graphics are temporarily stored (like variables) and can be recalled by the shg (show graphics) instruction. If you wish, you may run some of the demonstration programs now by entering demo and following the on-screen instructions.

8.4. An Introduction to MATLAB: Interactive Mode*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

In command mode, MATLAB displays a prompt ( ≫ ) and waits for your input. You may type any legal mathematical expression for immediate evaluation. Try the following three examples (press “enter” or “return” at the end of each line):

≫ 2+2
≫ 5^2
≫ 2*sin(pi/4)

The variable pi = 3.14 is built into MATLAB, as are the sin function and hundreds of other functions. When you entered each of the preceding lines, MATLAB stored the results in a variable called ans for answer. The value of ans was then displayed. The last line should have produced the square root of 2. We can manipulate ans to find out

≫ ans^2

The new answer is very close to 2, as expected. Let's see what the roundoff error is:

≫ ans-2

8.5. An Introduction to MATLAB: Variables*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

Any result you wish to keep for a while may be assigned to a variable other than ans:

≫ x = pi/7
≫ cos(x)
≫ y = sin(x)^2+cos(x)^2;
≫ y

A semicolon (;) at the end of the line suppresses printing of the result, as when we calculated y in the next-to-last line just shown. This feature is especially useful when writing MATLAB programs where intermediate results are not of interest and when working with large matrices.

MATLAB supports the dynamic creation of variables. You can create your own variables by just assigning a value to a variable. For example, type x = 3.5+4.2. Then the real variable x contains the value 7.7. Variable names must start with an alphabetical character and be less than nineteen characters long. If you type x = -3*4.0, the content 7.7 is replaced by the value -12. Some commands allow you to keep track of all the variables that you have already created in your session. Type who or whos to get the list and names of the variables currently in memory (whos gives more information than who). To clear all the variables, type in clear. To clear a single variable (or several) from the list, follow the command clear by the name of the variable you want to delete or by a list of variable names separated by spaces. Try it now.

MATLAB is case sensitive. In other words, x and X are two different variables. You can control the case sensitivity of MATLAB by entering the command casesen, which toggles the sensitivity. The command casesen on enforces case sensitivity, and casesen off cancels it.

If one line is not enough to enter your command, then finish the first line with two dots (. . ) and continue on the next line. You can enter more than one command per line by separating them with commas if you want the result displayed or with semicolons if you do not want the result displayed. For example, type

≫ theta = pi/7; x = cos(theta); y = sin(theta);
≫ x,y

to first compute theta,cos(theta), and sin(theta) and then to print x and y .

8.6. An Introduction to MATLAB: Complex Variables*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

The number is predefined in MATLAB and stored in the two variable locations denoted by i and j. This double definition comes from the preference of mathematicians for using i and the preference of engineers for using j (with i denoting electrical current). i and j are variables, and their contents may be changed. If you type j = 5, then this is the value for j and j no longer contains . Type in j = sqrt(-1) to restore the original value. Note the way a complex variable is displayed. If you type i, you should get the answer

i =
    0+1.0000i.

The same value will be displayed for j. Try it. Using j, you can now enter complex variables. For example, enter z1 = 1+2*j and z2 = 2+1.5*j. As j is a variable, you have to use the multiplication sign *. Otherwise, you will get an error message. MATLAB does not differentiate (except in storage) between a real and a complex variable. Therefore variables may be added, subtracted, multiplied, or even divided. For example, type in x = 2, z = 4.5*j, and z/x. The real and imaginary parts of z are both divided by x. MATLAB just treats the real variable x as a complex variable with a zero imaginary part. A complex variable that happens to have a zero imaginary part is treated like a real variable. Subtract 2*j from z1 and display the result.

MATLAB contains several built-in functions to manipulate complex numbers. For example, real (z) extracts the real part of the complex number z. Type

≫ z = 2+1.5*j, real(z)

to get the result

z =
   2.000+1.500i

ans =
   2

Similarly, imag(z) extracts the imaginary part of the complex number z. The functions abs(z) and angle(z) compute the absolute value (magnitude) of the complex number z and its angle (in radians). For example, type

≫ z = 2+2*j;
≫ r = abs(z)
≫ theta = angle(z)
≫ z = r*exp(j*theta)

The last command shows how to get back the original complex number from its magnitude and angle. This is clarified in Chapter 1: Complex Numbers.

Another useful function, conj (z), returns the complex conjugate of the complex number z. If z = x+j*y where x and y are real, then conj (z) is equal to x-j*y. Verify this for several complex numbers by using the function conj (z).

8.7. An Introduction to MATLAB: Vectors and Matrices*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

As its name indicates, MATLAB is especially designed to handle matrices. The simplest way to enter a matrix is to use an explicit list. In the list, the elements are separated by blanks or commas, and the semicolon (;) is used to indicate the end of a row. The list is surrounded by square brackets . For example, the statement

≫ A = [1 2 3;4 5 6;7 8 9]

results in the output

A =
   1 2 3
   4 5 6
   7 8 9

The variable A is a matrix of size 3 × 3. Matrix elements can be any MATLAB expression. For example, the command

≫ x = [-1.3 sqrt(3) (1+2+3)*4/5]

results in the matrix

x =
   -1.3000  1.7321  4.8000

We call a matrix with just one row or one column a vector, and a 1 × 1 matrix is a scalar. Individual matrix elements can be referenced with indices that are placed inside parentheses. Type x(5) = abs(x(1)) to produce the new vector

x =
   -1.3000  1.7321  4.8000  0.000  1.3000

Note that the size of x has been automatically adjusted to accommodate the new element and that elements not referenced are set equal to 0 (here x(4)). New rows or columns can be added very easily. Try typing r = [10 11 12],A = [A;r]. Dimensions in the command must coincide. Try r = [13 14],A = [A;r].

The command size(A) gives the number of rows and the number of columns of A. The output from size(A) is itself a matrix of size 1 × 2. These numbers can be stored if necessary by the command [m n] = size(A). In our previous example, A = [A;r] is a 4 × 3 matrix, so the variable m will contain the number 4 and n will contain the number 3. A vector is a matrix for which either m or n is equal to 1. If m is equal to 1, the matrix is a row vector; if n is equal to 1, the matrix is a column vector. Matrices and vectors may contain complex numbers. For example, the statement

≫ A = [1 2;3 4]+j*[5 6;7 8]

and the statement

≫ A = [1+5*j 2+6*j;3+7*j 4+8*j]

are equivalent, and they both produce the matrix

A =
   1.0000+5.0000i    2.0000+6.0000i
   3.0000+7.0000i    4.0000+8.0000i

Note that blanks must be avoided in the second expression for A. Try typing

≫A = [1 + 5*j 2 + 6*j 2 + 6*j;3 +7*j 4 + 8*j]

What is the size of A now?

MATLAB has several built-in functions to manipulate matrices. The special character, ', for prime denotes the transpose of a matrix. The statement A = [ 1 2 3;4 5 6;7 8 9]' produces the matrix

A =
   1 4 7
   2 5 8
   3 6 9

The rows of A' are the column of A, and vice versa. If A is a complex matrix, then A is its complex conjugate transpose or hermitian transpose. For an “unconjugate” transpose, use the two-character operator dot-prime (. '). Matrix and vector variables can be added, subtracted, and multiplied as regular variables if the sizes match. Only matrices of the same size can be added or subtracted. There is, however, an easy way to add or subtract a common scalar from each element of a matrix. For example, x = [1 2 3 4],x = x-1 produces the output

x =
   1 2 3 4

x =
   0 1 2 3

As discussed in the chapter on linear algebra, multiplication of two matrices is only valid if the inner sizes of the matrices are equal. In other words, A*B is valid if the second size of A (number of columns) is the same as the first size of B (number of rows). Let a i,j represent the element of A in the i th row and the j th column. Then the matrix A*B consists of elements

(8.1)

where n is the number of columns of A and the number of rows of B. Try typing A = [1 2 3;4 5 6];B = [7;8;9]; A*B. You should get the result

ans =
    50
    112

The inner product between two column vectors x and y is the scalar defined as the product x'*y or equivalently as y'*x For example, x = [1;2],y = [3;4],x'*y, leads to the result

ans =
    11

Similarly, for row vectors the inner product is defined as x*y'. The Euclidean norm of a vector is defined as the square root of the inner product between a vector and itself. Try to compute the norm of the vector [1 2 3 4]. You should get 5.4772. The outer product of two column (row) vectors is the matrix x*y' (x'*y).

Any scalar can multiply or be multiplied by a matrix. The multiplication is then performed element by element. Try A = [1 2 3;4 5 6;7 8 9];A*2. You should get

ans =
    2  4  6
    8 10 12
   14 26 28

Verify that 2*A gives the same result.

The inverse of a matrix is computed by using the function inv(A) and is only valid if A is square. If the matrix is singular, meaning that it has no inverse, a message will appear. Try typing inv(A). You should get

Warning: Matrix is close to singular or badly scaled.
   Results may be inaccurate. RCOND=2.937385e-18

ans =
     1.0e+16*
     0.3152  -0.6304   0.3152
    -0.6304   1.2609  -0.6304
     0.3152  -0.6304   0.3152

The inverse of a matrix may be used to solve a linear system of equations. For example, to solve the system

you could type A = [1 2 3;1 -2 4;0 -2 1]; b = [2;7;3]; inv(A)*b and get

ans =
    1
   -1
    1

(8.2)

Check to see that this is the correct answer by typing A*[1;-1;1]. What do you see?

MATLAB offers another way to solve linear systems, based on Gauss elimination, that is faster than computing the inverse. The syntax is A\b and is valid whenever A has the same number of rows as b. Try it.

The “Dot” Operator. Sometimes you may want to perform an operation element by element. In MATLAB, these element-by-element operations are called array operations. Of course, matrix addition and subtraction are already element-by-element operations. The operation A. *B denotes the multiplication, element by element, of the matrices A and B. Make two 3 × 3 matrices, A and B, and try

≫ A*B
≫ A.*B

Suppose we want to find the square of each number in A. The proper way to specify this calculation is

≫ A_squared=A.^2

where the period (dot) indicates an “array operation” to be performed on each element of A. Without the dot, A is multiplied by A according to the rules of matrix multiplication described in Chapter 4, giving a totally different result:

≫ A^2

Subtleties. Because MATLAB can do so many different mathematical functions with just a few keystrokes, there are times when a very slight change in what you type will lead to a different result. Using the matrix A entered earlier, type the following two lines:

≫ 2.^A
≫ 2 .^A                        %with a space after the 2.

In the first case, the dot is “absorbed” by the 2 as a decimal point and the ^ is taken as a matrix exponential. But, when the dot is separated from the 2 by a space, it becomes part of the operator and specifies that 2 should be raised to the power of each element in A. The point is, you should be very careful to type what you mean in an unambiguous way until you are familiar enough with MATLAB to know how the subtle situations wiIl be interpreted. An unambiguous way of typing the preceding lines is

≫ (2.)^A                     %for matrix exponential
≫ (2).^A                     %for array exponential.

8.8. An Introduction to MATLAB: The Colon*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

You can use the colon several ways in MATLAB (see help :). Its basic meaning is a vector of sequential values. For example, type

≫ x = 3:9

to get

x =
   3 4 5 6 7 8 9

For increments other than 1, use statements like

≫ x = 1:0.5:4
≫ x = 6:-1:0

Most MATLAB functions will accept vector inputs and produce vector out- puts. The statement

≫ y = sqrt(1:10)

builds a vector of integers from 1 to 10 and takes the square root of each of those numbers. Try it.

Now for another subtlety–what is the effect of each of the following statements and why?

≫ 1+1:5
≫ 1+(1:5)

Appending to a Matrix or Vector. A matrix or vector can be enlarged in size by appending new values to the old values. Let x=[1 3 5]:

(8.3)

8.9. An Introduction to MATLAB: Graphics*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

In "Complex Numbers", you learn that complex variables can be represented as points in the plane. MATLAB makes it easy for you to plot complex variables in a graph. Type z1 = 1+.5*j;plot(z1,'o'). The graph window should be activated and the point z1 displayed by a 'o'. You must specify the symbol for display, and the authorized symbols for point display are '.', 'o', '+', and '*'. When you are displaying a curve (to come later), no type is necessary. MATLAB automatically adjusts the scale on a graph to accommodate the value of the point being plotted. In this case, the range is [0,2] for the real part and [0,1] for the imaginary part.

Let's now plot a second complex number by typing z2 = 2+1.5*j; plot(za,'o'). Note that the second plot command erases the first plot and changes the scaling to [0,4] and [0,3]. Sometimes you may want to have the points plotted on the same graph. To achieve this, you have to use the command hold on after the first plot. Try the following:

≫ plot(z2,'o')
≫ hold on
≫ plot(z1,'o')
≫ hold off

The advantage in using the hold command is that there is no limit to the number of plot commands you can type before the hold is turned off, and these plots may involve the same variable plotted over a range of values. You can also use different point displays. A disadvantage of the hold command is that the scaling is enforced by the first plot and is not adjusted for subsequent plots. This is why we plot the point z2 first. Try reversing the order of the plots and see what happens. This means that points outside the scaling will not be displayed. The command hold off permits erasing the current graph for the next plot command.

You can freeze the scaling of the graph by using the command axis. MATLAB gives you the message

Axis scales frozen
ans =
   0 4 0 3

This freezes the current axis scaling for subsequent plots. Similarly, if you type axis a second time, MATLAB resumes the automatic scaling feature and prints the message

Axis scales auto-ranged
ans =
   0 4 0 3

The axis scaling can also be manually enforced by using the command

≫ axis([xmin xmax ymin ymax])

where (xmin, ymin) is the lower left corner and (xmax, ymax) is the upper right corner of the graph. This scaling remains in effect until the next axis command is entered (with or without arguments).

Another way to plot several complex numbers on the same graph is to display them as a curve. For this purpose, you have to store the numbers in a vector. For example, type z1 = z1,z(2) = z2,plot(z). Note that the two points are at the two extremes of the line plotted on the graph. If you specify a symbol, then no line is drawn, just the extreme points. Try plot(z,'o') .

If you examine your current graph carefully, you will notice that the unit lengths on the x and y axes are not quite the same. In fact, MATLAB adjusts the length of an axis to conform to the overall size of the graph window. What this means is that a 45o line will actually be displayed at an angle depending on the overall aspect ratio of the graph window. To ensure that the aspect ratio is equal to 1, you may enter the command axis('square'). MATLAB will then enforce an aspect ratio equal to 1, regardless of the aspect ratio for the outside graph window. This ensures that circles appear as circles and not as ellipses. MATLAB will make the square graph as large as possible to fit within the graph window. To go back to the default ratio, just type in axis('normal').

To add labels to your graph, the functions xlabel('text'), ylabel('text'), and title('text') are useful and self-explanatory. The argument text contains a string of characters. Add the label Real on the horizontal axis and the label Imaginary on the vertical axis of your graph. The command grid draws a grid on your graph. The grid does not remain in effect for the next plots. Try it.

The plot Instruction. The plot instruction in MATLAB is very versatile. It can be used to plot several different types of data. Its syntax is plot(x,y,' symbol and/or color') or plot(y,'symbol and/or color'). The instruction will plot a vector of data versus another vector of data. The first vector is referenced to the horizontal axis and the second to the vertical axis. If only one vector is used, then it is plotted with reference to the vertical axis while the horizontal axis is automatically forced to be the index of the vector for the corresponding data point. The notation inside the apostrophes is optionally used to designate whether each element of the vector is to be plotted as a single point with a certain symbol or as a curve with a straight line drawn between each data value. The colors can also be specified. Possible symbols are (*,o,+,.), and colors are (r,g,b,w) (red, green, blue, and white). Complex valued vectors are plotted by making the horizontal axis the real part of the vector and the vertical axis the imaginary part. Warning: A complex valued vector will automatically be plotted correctly on the complex plane (instead of real versus imaginary) only if every element of the vector is complex valued. Try plot (x,y,'*') , plot (x,'*'), plot (y,'*'), plot (y,x,'*') and plot (z,'*'), plot (Real (z), Imag (z),'*') to clarify your understanding of plot. Use x = [1357],y = [2468],z = [1 + j,2 + 2j,3 + 3j].

We may summarize as follows:

Table 8.1.

plot(x,'*r') | (red star—points with the values of x on vertical and indicies on horizontal)
plot(y) | (line—connected curve of the value of y on vertical and the value of x on horizontal)
plot(x,y,'og') | (line—connected curve of the value of y on vertical and the value of x on horizontal)
plot(x,y,'og') | (circle—points of the value of y on vertical and the value of x on horizontal)
plot(real(z),imag(z)) | (line—connectedplot of z on the complex plane)
plot(real(z),imag(z),'+b') | (blue plus—points of z on the complex plane).

Figure 8.3. 

Subplots. It is possible to split the graphics screen up into several separate smaller graphs rather than just one large graph. As many as four subplots can be created. The MATLAB instruction subplot (mnp) signifies which of the smaller graphs is to be accessed with the next plot statement. The mnp argument consists of three digits. The m and n are the numbers of rows (m) and columns (n) into which the screen should be divided. The p designates which of the matrix elements is to be used. For example,

Help and Demos. MATLAB has on-line help and a collection of demonstrations. For a list of available functions, type

≫ help

For help on a specific function, sin for example, type

≫ help sin

To learn how to use colon (;, a very important and versatile character) in MATLAB, type

≫ help :

The demos will also help you become more familiar with MATLAB and its capabilities. To run them, type

≫ demo

8.10. An Introduction to MATLAB: Editing Files and Creating Functions (Macintosh)*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

If you quit MATLAB now, all the commands you have typed will be lost. This is where the Edit window is useful. If you choose new from the File menu, a new window appears with the title Edit 1: Untitled. In this window, you should type in all the commands you would like MATLAB to execute at once. When you are finished typing, you may save the file by choosing save or save as in the file menu and by entering a name for the file. If the edit window is active (that is, if it appears in front), then choose save and go from the file menu to save the file and execute it. If the command window is active, then you can execute the file by entering its name.

Editing Files. To test your understanding of file editing, enter the following commands in a file named myfile:

clear, clg                      %Clear variables and graphics
j=sqrt(-1);                     %To be sure
z1=1+.5*j,z2=2+1.5*j            %Enter variables
z3=z1+z2,z4=z1*z2               %Compute sum and product
axis([0 4 0 4]),                %First plot
  axis('square'),plot(z1,'o')
hold on                         %Allow overplot
plot(z2,'o'),plot(z3,'+'),      %Other plots
    plot(z4,'*')
hold off

You do not have to type the % sign and the text that follows it. These are simply comments in a file. They are ignored by the MATLAB interpreter. You should, however, make a habit of adding comments (preceded by %) to your file if you want to be able to understand programs that have been written long ago.

Do not forget to save your file. Such a file is called a script file. It contains MATLAB commands that could have been entered one by one in the command window. You have three ways to execute a script file:

with the edit window active, choose save and go from the file menu;

with the command window active, enter the file name; or

with the command window active, choose run script... from the M-file menu. In this case, a menu pops up to ask you which file you want to execute.

Try each of these three methods in order to get used to their differences. Figure 1 shows the plot that you should get.

Creating Functions. MATLAB puts many commands at your disposal, and you just have to enter their names (with or without arguments) to execute them. Some commands are built in to MATLAB, and others are contained in files to which you have access (not to modify them, but to see how things are done). You can try to display the contents of a command file in the command window by entering the command type filename. Enter type plot. You should get the message

??? Built-in function.

Figure 8.4. 

A Typical Graph (©Apple Computer, Inc., used with permission.)

This means that the command plot is a build-in function and thus cannot be typed by the user (it is compiled with the program). Similarly, try to type the function file containing the command sinh by entering the command type sinh. You should get

function y=sinh(x)
%SINH SINH(X) is the hyperbolic sine of the elements of X.
y=(exp(x)-exp(-x))/2;

This is a typical example of a function file. It differs from a script file by the fact that the first line contains the word function. What it does is make the command y = sinh(x) equivalent to the command y=(exp(x)-exp(-x))/2 . The variable x is the input argument (there could be more than one), and y is the output argument (there also could be more than one). The second line contains comments about the function and its arguments. They can spread to several lines. Display them by typing the command help sinh. The third line contains the actual function commands (there could also be several lines).

As an exercise, enter the function perp(x):

function y=perp(x)
%PERP PERP(X) is a complex value perpendicular to X.
j=sqrt(-1);
y=j*real(x)-imag(x);

Evaluate perp on various complex numbers. Replace the last line by y = x*j;. Do you get the same result? Why?

Note that local variables are just that–local. Only the input and out- put arguments are kept in memory after the function is called and executed. For more on functions and multiple arguments, see the MATLAB manual. In the previous example, the variable j is local. If, before using the function perp, you use the same variable j , its contents will not be affected by the command perp(z). Verify this.

Normally, while a file (script or function) is executed, the commands are not displayed on the command window. Just the results are displayed. The command echo allows you to view all the instructions. This is useful for debugging and demonstrations. It is turned off by typing echo off.

8.11. An Introduction to MATLAB: Editing Files and Creating Functions (PC)*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

You should write a MATLAB program whenever you anticipate exe- cuting some sequence of statements several times or again in a later session. On an IBM PC, you may use any text editor to write a program, as long as the file can be saved in ASCII format without the control codes used by most word processors. Applicable text editors include Edix, Wordstar, XTree Pro, and Turbo Pascal's editor.

Editing Files. If you have enough memory, you can run your editor without leaving MATLAB by using the exclamation point (!), like this:

≫ !EDIX

The exclamation point may be used to execute any DOS command or program from MATLAB. When the command or program finishes, your MATLAB variables are just as you left them. Use your editor to write program lines just as you would type them in MATLAB's command mode. Then save the file with extension .m in the directory where you will run MATLAB. Such MATLAB programs are called m-files. You may run your m-file by typing the file name (without the .m extension) at the MATLAB command prompt ( ≫ ).

Script Files. There are two kinds of m-files, called script files and functions. Running a script file is exactly like typing the commands it contains at the ≫ prompt. Your m-file will automatically be a script file unless you specify otherwise, as described later. Practice by entering, saving, and running plotsin.m as listed next:

t = -6:.2:6;
y = sin(t);
plot(t,y)
title('SINE')
pause
grid
xlabel('t')
ylabel('sin(t)')

When the pause is executed, you will need to press a key to go on. If you type whos after running plotsin, you will see that the variables t and y remain in memory. Comments are important to a script file. They are marked with the symbol %. Anything following this symbol on a line is assumed to be a comment and is ignored by the MATLAB program interpreter.

Functions. Functions differ from script files in that they have designated input and output variables. Any other variables used within a function are local variables, which do not remain after the function terminates and which have no effect on variables outside the function. Many of the functions supplied with MATLAB are actually m-files. A good example is triu.m:

≫ type triu

The word function at the beginning of the file makes it a function rather than a script file. The function name in this line must match the file name. The input variables of triu are x and k, meaning that the first input argument will be referred to as x and the second as k within the function. Likewise, the function line designates y as the output. There is nothing special about the variable names x,k, and y when the function is used. It is only that whatever inputs and output you use will be referred to as x,k, and y inside the function. The variables m,n,j, and i are created temporarily when triu runs and disappear when it terminates. They are local variables and have no effect on variables with the same names outside the function. In contrast, a script file has no local variables and does no substitution of input and output variable names.

As an exercise, enter and save the function perp.m:

      function y=perp(x)
      % PERP(x) is a complex number perpendicular to x.
      j = sqrt(-1);
      y = j*real(x)-imag(x);

Evaluate perp on various complex numbers. Replace the last line by y = x*j;. Do you get the same result? Why?

Printing Files and Graphics. To display an m-file on the screen, use the instruction ≫ type filename. To make a copy at a printer, use the DOS command ≫ !print filename.m. Graphics hardcopy is available through the commands meta and gpp. See the MATLAB manual for more information.

8.12. An Introduction to MATLAB: Loops and Control*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

MATLAB has control statements like those in most computer languages. We will only study the for loop here. See the MATLAB manual for details on if and while statements.

What for loops do is allow a statement or a group of statements to be repeated. For example,

for i = 1:n,x(i) = 0,end

assigns the value 0 to the first n elements of the array x. If n is less than 1, the instruction is still valid but nothing will be done. If n is not defined, then the following message will appear:

??? Undefined function or variable.
Symbol in question==>n

If n contains a real value, the integer part of n is used. If n is a complex number, the integer part of the real part is taken. (This should, however, be avoided.) If x is not declared earlier or is of smaller dimension than n, then extra memory is allocated.

The command end must follow the command for. If it is not present, MATLAB will wait for remaining statements until you type t he command end, and nothing will be executed in the meantime.

More than one statement may be included in the loop. The statement 1:n is the way to specify all the integer values between 1 and n. A step different than 1 may be specified. For example, the statement for i=1:2:5,x(i)=1,end is equivalent to x(1)=1,x(5)=1. Negative steps are also allowed, as in i=n:-1:1.

We may use a for loop to draw a circle of radius 1. Type

≫ j=sqrt(-1);
≫ n=360;
≫ for i=1:n,circle(1)=exp(2*j*i*pi/n);end;
≫plot(circle)

Note how easy it is to plot a curve. But also note how slowly the for loop is executed. MATLAB is not very good at executing things one by one. It prefers, by far, a vector-oriented statement. Using the range specification as in the for loop, it is possible to speed up the process by replacing the explicit for loop by an implicit for loop using the colon, like this:

≫ circle = exp((2*j*pi/n)*[1:n]);

Note how much faster this command is executed. In general, for loops should be avoided as much as possible. For example, the first for loop you typed could have been replaced by the command x=zeros(1,n);, which is much more efficient. The function zeros fills the variable with 0's to the specified size. Similarly, the function ones fills the variable with 1's. The size can also be determined by the size of the argument. If A is a matrix of size m,n, then the command B=ones(a) fills the matrix B with 1's and forces the matrix B to have exactly the same size as the matrix A.

Avoiding for Loops. Since for loops are very inefficient in MATLAB (they are computed sequentially, adding several more computations for every loop), it is preferable to use the matrix capabilities of MATLAB to replace for loops and speed up processing time.

Replace the for loop

≫ for i = 1:10,
≫    x(i) = i;
≫ end;

with

≫ x = 1:10

to get

x = [1 2 3 4 5 6 7 8 9 10]

Replace the for loop

≫ z = something;
≫ for i = 1:10,
≫   x(i) = z*i;
≫ end;

with

≫ z = something
≫ x = z*(1:10);

to get

x = [z 2*z 3*z ... 10*z]

Replace the for loop

≫ z = something;
≫ x(1) = z;
≫ for i = 2:10.
≫    x(i) = z*x(i-1);
≫ end;

with

≫ z = something;
≫ x = z.^(1:10);

to get

x = [z z**2 z**3 ... z**10]

Replace the for loop

≫ for i = 0:2:100,
≫    x(i) = 1.0*exp(j*2*pi*i/100);
≫ end;

with

≫ x = 1.0*exp(j*2*pi*(0:2:100)/100);

to get

x = [exp(0) exp(j*4*pi/100) ... exp(j*200*pi/100)]

Chapter 9. The Edix Editor*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

On a PC, creating and editing m-files is done outside of MATLAB. The Edix editor is one of the visual types of editors. It can be accessed either from DOS by using > edix filename.m or from MATLAB by using ≫ !edix filename.m. The latter will return you back to MATLAB after you quit the editor. MATLAB script files must end with a .m extension. They are also called m-files. Once in the Edix editor, a help list of the editor's commands can be displayed on the screen by typing “Alt-h.” (Note: The “Alt” key must be held down while the "h" key is depressed.) Pressing the “space” key will get you out of the help system.

A new file is created by calling the Edix editor with a file name that has not been previously used. If the file name already exists on disc, then that file is retrieved and displayed on the screen and is ready to be edited. Since Edix is a visual editor, changes are made by moving the cursor to wherever something needs to be typed in or deleted. Once changes have been made to a file, then that file needs to be saved (written to disc) by typing “Alt-w.” When all editing is finished, you may exit from the Edix editor by typing “Alt-x.” Following are some useful Edix editor commands:

Table 9.1.

Alt-h(F7) | help-list of all the Edix commands;
Alt-w | saves (writes) the file to disc;
Alt-x | exits the Edix editor;
Alt-d | deletes the current line;
Alt-m (F8) | the first time used, marks the beginning of a block of lines to be edited; the second time, marks the end of a block of lines to be edited and highlights this block in white; the third time, "un-marks" the highlighted block;
Alt-c | makes a copy of the highlighted block at the current cursor location (can be repeated);
Alt-m | (moves the highlighted block to the current cursor location (can be repeated);
(F5) | put the cursor at the beginning of the line; and
(F6) | puts the cursor at the end of the line.

Chapter 10. Useful Mathematical Identities*

Note

This module is part of the collection, A First Course in Electrical and Computer Engineering. The LaTeX source files for this collection were created using an optical character recognition technology, and because of this process there may be more errors than usual. Please contact us if you discover any errors.

10.1.  e jθ , cos θ, and sin θ

10.2. Trigonometric Identities

(10.1) sin² θ + cos² θ = 1

(10.2)sin ( θ + φ ) = sin θ cos φ + cos θ sin φ

(10.3) cos ( θ + φ ) = cos θ cos φ –

sin θ sin φ

(10.4) sin ( θ – φ ) = sin θ cos φ –

cos θ sin φ

(10.5) cos ( θ – φ ) = cos θ cos φ + sinθsin φ

10.3. Euler's Equations

(10.6) e jθ = cosθ + jsin θ

(10.7)

(10.8)

10.4. De Moivre's Identity

(10.9)(cosθ + jsinθ) n = cosn θ + jsinnθ

10.5. Binomial Expansion

(10.10)

(10.11)

10.6. Geometric Sums

(10.12)

(10.13)

10.7. Taylor's Series

(10.14)

(10.15)

Appendix A. Attributions

Collection: A First Course in Electrical and Computer Engineering

Edited by: Louis Scharf

URL: http://cnx.org/content/col10685/1.2/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Dedication of "A First Course in Electrical and Computer Engineering"

By: Louis Scharf

URL: http://cnx.org/content/m21796/1.3/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Preface to "A First Course in Electrical and Computer Engineering"

By: Louis Scharf

URL: http://cnx.org/content/m21476/1.7/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Complex Numbers: Introduction

By: Louis Scharf

URL: http://cnx.org/content/m21412/1.6/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Complex Numbers: Geometry of Complex Numbers

By: Louis Scharf

URL: http://cnx.org/content/m21411/1.6/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Complex Numbers: Algebra of Complex Numbers

By: Louis Scharf

URL: http://cnx.org/content/m21408/1.6/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Complex Numbers: Roots of Quadratic Equations

By: Louis Scharf

URL: http://cnx.org/content/m21415/1.6/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Complex Numbers: Representing Complex Numbers in a Vector Space

By: Louis Scharf

URL: http://cnx.org/content/m21414/1.6/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Complex Numbers: An Electric Field Computation

By: Louis Scharf

URL: http://cnx.org/content/m21410/1.6/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Complex Numbers: Numerical Experiment (Quadratic Roots)

By: Louis Scharf

URL: http://cnx.org/content/m21413/1.6/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: The Functions e^x and e^jθ: Introduction

By: Louis Scharf

URL: http://cnx.org/content/m21479/1.5/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: The Functions e^x and e^jθ: The Function e^x

By: Louis Scharf

URL: http://cnx.org/content/m21485/1.7/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: The Functions e^x and e^jθ: The Function e^jθ and the Unit Circle

By: Louis Scharf

URL: http://cnx.org/content/m21486/1.7/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: The Functions e^x and e^jθ: The Euler and De Moivre Identities

By: Louis Scharf

URL: http://cnx.org/content/m21487/1.6/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: The Functions e^x and e^jθ: Roots of Unity and Related Topics

By: Louis Scharf

URL: http://cnx.org/content/m21482/1.7/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: The Functions e^x and e^jθ: Second-Order Differential and Difference Equations

By: Louis Scharf

URL: http://cnx.org/content/m21483/1.7/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: The Functions e^x and e^jθ: Numerical Experiment (Approximating e^jθ)

By: Louis Scharf

URL: http://cnx.org/content/m21481/1.7/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Phasors: Introduction

By: Louis Scharf

URL: http://cnx.org/content/m21469/1.6/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Phasors: Phasor Representation of Signals

By: Louis Scharf

URL: http://cnx.org/content/m21474/1.7/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Phasors: Beating between Tones

By: Louis Scharf

URL: http://cnx.org/content/m21468/1.6/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Phasors: Multiphase Power

By: Louis Scharf

URL: http://cnx.org/content/m21472/1.6/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Phasors: Lissajous Figures

By: Louis Scharf

URL: http://cnx.org/content/m21471/1.6/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Phasors: Sinusoidal Steady State and the Series RLC Circuit

By: Louis Scharf

URL: http://cnx.org/content/m21475/1.6/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Phasors: Light Scattering by a Slit

By: Louis Scharf

URL: http://cnx.org/content/m21470/1.7/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Phasors: Numerical Experiment (Interference Patterns)

By: Louis Scharf

URL: http://cnx.org/content/m21473/1.6/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Linear Algebra: Introduction

By: Louis Scharf

URL: http://cnx.org/content/m21454/1.7/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Linear Algebra: Vectors

By: Louis Scharf

URL: http://cnx.org/content/m21467/1.6/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Linear Algebra: Inner Product and Euclidean Norm

By: Louis Scharf

URL: http://cnx.org/content/m21452/1.6/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Linear Algebra: Direction Cosines

By: Louis Scharf

URL: http://cnx.org/content/m21449/1.8/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Linear Algebra: Projections

By: Louis Scharf

URL: http://cnx.org/content/m21465/1.6/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Linear Algebra: Other Norms

By: Louis Scharf

URL: http://cnx.org/content/m21463/1.7/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Linear Algebra: Matrices

By: Louis Scharf

URL: http://cnx.org/content/m21457/1.6/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Linear Algebra: Solving Linear Systems of Equations

By: Louis Scharf

URL: http://cnx.org/content/m21466/1.6/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Linear Algebra: Circuit Analysis

By: Louis Scharf

URL: http://cnx.org/content/m21447/1.8/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Linear Algebra: Numerical Experiment (Circuit Design)

By: Louis Scharf

URL: http://cnx.org/content/m21462/1.6/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Vector Graphics: Introduction

By: Louis Scharf

URL: http://cnx.org/content/m21491/1.6/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Vector Graphics: Two-Dimensional Image Representation

By: Louis Scharf

URL: http://cnx.org/content/m21495/1.6/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Vector Graphics: Two-Dimensional Image Transformations

By: Louis Scharf

URL: http://cnx.org/content/m21409/1.6/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Vector Graphics: Composition of Transformations

By: Louis Scharf

URL: http://cnx.org/content/m21489/1.6/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Vector Graphics: Homogeneous Coordinates

By: Louis Scharf

URL: http://cnx.org/content/m21490/1.6/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Vector Graphics: Three-Dimensional Homogeneous Coordinates

By: Louis Scharf

URL: http://cnx.org/content/m21494/1.6/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Vector Graphics: Projections

By: Louis Scharf

URL: http://cnx.org/content/m21493/1.6/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Vector Graphics: Numerical Experiment (Star Field)

By: Louis Scharf

URL: http://cnx.org/content/m21492/1.6/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Filtering: Introduction

By: Louis Scharf

URL: http://cnx.org/content/m21417/1.6/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Filtering: Simple Averages

By: Louis Scharf

URL: http://cnx.org/content/m21421/1.6/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Filtering: Weighted Averages

By: Louis Scharf

URL: http://cnx.org/content/m21423/1.5/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Filtering: Moving Averages

By: Louis Scharf

URL: http://cnx.org/content/m21418/1.5/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Filtering: Exponential Averages and Recursive Filters

By: Louis Scharf

URL: http://cnx.org/content/m21416/1.5/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Filtering: Test Sequences

By: Louis Scharf

URL: http://cnx.org/content/m21422/1.5/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Filtering: Numerical Experiment (Frequency Response of First-Order Filter)

By: Louis Scharf

URL: http://cnx.org/content/m21419/1.4/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Binary Codes: Introduction

By: Louis Scharf

URL: http://cnx.org/content/m21405/1.5/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Binary Codes: The Communication Paradigm

By: Louis Scharf

URL: http://cnx.org/content/m21407/1.5/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Binary Codes: From Symbols to Binary Codes

By: Louis Scharf

URL: http://cnx.org/content/m21399/1.7/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Binary Codes: Huffman Codes for Source Coding

By: Louis Scharf

URL: http://cnx.org/content/m21404/1.8/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Binary Codes: Hamming Codes for Channel Coding

By: Louis Scharf

URL: http://cnx.org/content/m21403/1.5/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Binary Codes: Numerical Experiment (Huffman Codes)

By: Louis Scharf

URL: http://cnx.org/content/m21406/1.5/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: An Introduction to MATLAB: Introduction

By: Louis Scharf

URL: http://cnx.org/content/m21391/1.4/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: An Introduction to MATLAB: Running MATLAB (Macintosh)

By: Louis Scharf

URL: http://cnx.org/content/m21394/1.4/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: An Introduction to MATLAB: Running MATLAB (PC)

By: Louis Scharf

URL: http://cnx.org/content/m21393/1.4/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: An Introduction to MATLAB: Interactive Mode

By: Louis Scharf

URL: http://cnx.org/content/m21390/1.4/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: An Introduction to MATLAB: Variables

By: Louis Scharf

URL: http://cnx.org/content/m21397/1.4/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: An Introduction to MATLAB: Complex Variables

By: Louis Scharf

URL: http://cnx.org/content/m21384/1.4/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: An Introduction to MATLAB: Vectors and Matrices

By: Louis Scharf

URL: http://cnx.org/content/m21398/1.5/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: An Introduction to MATLAB: The Colon

By: Louis Scharf

URL: http://cnx.org/content/m21395/1.4/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: An Introduction to MATLAB: Graphics

By: Louis Scharf

URL: http://cnx.org/content/m21388/1.5/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: An Introduction to MATLAB: Editing Files and Creating Functions (Macintosh)

By: Louis Scharf

URL: http://cnx.org/content/m21385/1.6/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: An Introduction to MATLAB: Editing Files and Creating Functions (PC)

By: Louis Scharf

URL: http://cnx.org/content/m21387/1.4/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: An Introduction to MATLAB: Loops and Control

By: Louis Scharf

URL: http://cnx.org/content/m21392/1.5/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: The Edix Editor

By: Louis Scharf

URL: http://cnx.org/content/m21477/1.4/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

Module: Useful Mathematical Identities

By: Louis Scharf

URL: http://cnx.org/content/m21488/1.4/

Copyright: Louis Scharf

License: http://creativecommons.org/licenses/by/3.0/

About Connexions

Since 1999, Connexions has been pioneering a global system where anyone can create course materials and make them fully accessible and easily reusable free of charge. We are a Web-based authoring, teaching and learning environment open to anyone interested in education, including students, teachers, professors and lifelong learners. We connect ideas and facilitate educational communities. Connexions's modular, interactive courses are in use worldwide by universities, community colleges, K-12 schools, distance learners, and lifelong learners. Connexions materials are in many languages, including English, Spanish, Chinese, Japanese, Italian, Vietnamese, French, Portuguese, and Thai.