## Sampling and the Mysterious Scaling Factor

This post treats the signals in continuous time which is different than the approach I adopted in my book. The book deals exclusively in discrete time.

Some time ago, I came across an interesting problem. In the explanation of sampling process, a representation of impulse sampling shown in Figure below is illustrated in almost every textbook on DSP and communications. The question is: how is it possible that during sampling, the frequency axis gets scaled by — a very large number? For an ADC operating at 10 MHz for example, the amplitude of the desired spectrum and spectral replicas is ! I thought that there must be something wrong somewhere.

Figure 1: Sampling in time domain creates spectral replicas in frequency domain, each of which gets scaled by

I asked a few DSP experts this question. They did not know the answer as well. Slowly I started to understand the reason why it is true, and in fact, makes perfect sense. The answer is quite straightforward but I did not realize it immediately. Since this is a blog post, I can take the liberty of explaining the route I took for this simple revelation.

## A Unit Impulse

My first impression was that whenever impulses are involved, everything that branches from there becomes fictitious as well. A continuous-time unit impulse is defined as shown in Figure 2.

Area under a rectangle = , or

Figure 2: Definition of a unit impulse

Clearly, the sampling scale factor is independent of how we define a unit impulse and its infinite height. The next step might be to look into a sequence of unit impulses: an impulse train.

## An Impulse Train

An impulse train is a sequence of impulses with period defined as

. The standard method to derive its Fourier Transform is through expressing the above periodic signal as a Fourier series and then finding Fourier series coefficients as follows.

First, remember that a signal with period can be expressed as

where and it can be seen as the fundamental harmonic in the sequence of frequencies with ranging from to . Here, are the Fourier series coefficients given as

When is an impulse train, one period between and only contains a single impulse . Thus, can be found to be

Plugging this in definition above and using the fact that and , we get

Figure 3: Fourier Transform of an impulse train in time domain is another impulse train in frequency domain

The resulting time and frequency domain sequences are drawn in Figure 3 above which is a standard figure in DSP literature. However, it really comes as a surprise that an impulse train has a Fourier Transform as another impulse train, although the Fourier Transform of a unit impulse is a constant.
What happened to the concept that a signal narrow in time domain has a wide spectral representation? Going through this method, it is not immediately clear why.

So I thought of deriving it through some other technique. Note that I prefer to use the frequency variable instead of .

From here, there are many techniques to prove the final expression, and the mathematics becomes quite cumbersome. So I just take the route of visualizing it and see where it leads.

## Where the Impulses Come From

First, the Fourier Transform of a time domain impulse train can be seen as the sum of frequency domain complex sinusoids with “frequencies” equal to , or periods equal to . Furthermore, the above signal is periodic with period of first harmonic .

Therefore, for the sake of better visualization, we limit ourselves to the range when drawing the figures in frequency domain. Whatever happens here gets repeated elsewhere. Figure 4 illustrates 3 frequency domain complex sinusoids for and , with frequencies and , respectively. Note how within a period of , the first completes one cycle, while the other two complete two and three cycles, respectively. Here, we deliberately refrained from drawing the result for which is a straight line at on axis.

Figure 4: 3 frequency domain complex sinusoids  for and , with frequencies and , respectively, in the range

For a closer look, see the signal decomposed into and axes. Clearly, the axis consists of and the axis consists of again with and . This is shown in Figure 5.

Figure 5: and decomposition of Figure 4. The axis consists of and the axis consists of again with and , in the range

Having understood the basic concept, we now divide the range into segments each with width .

This implies that for integer ,

The result of summing the sinusoids shown in Figure 5 for a large is drawn in Figure 6.

Figure 6: Summing the and parts from Figure 5 but for a very large in the range . The part is seen converging to 0 while the part approaches an impulse, half of which is visible at and half at

The rectangle at has a height of since it is a sum of for all values of . It can now be concluded that

• arm — the expression — approaches at in each period . As , and it becomes an impulse (strictly speaking, the limit does not exist as a function and is rather known as a “distribution”).
• On the other hand, the arm — the expression — vanishes.

The scaling factor here can be found again by taking the area under the first rectangle. Thus,

Area under the rectangle ,

the same as found in Figure 3 (except the factor of ). Since it is a periodic signal, it consists of impulses repeating at integer multiples of — just what we saw in Figure 3 (except the factor as we are dealing in now). The complete sequence is now drawn in Figure 7.

Figure 7: Displaying the branch from Figure 6 in the range for large

It was then I realized my problem. All the graphs and curves I have drawn, and DSP textbooks draw for that matter, use the wrong scaling on the axes! Although the time and frequency domain impulse sequences look pretty much similar to each other, they are not. Imagine a very large number, say . Now if I draw the time and frequency domain representations of an impulse train within the range to , the signals would look like as illustrated in Figure 8 for a sample rate of MHz. As it is evident, all the confusion in the y-axis was due to incorrect scaling at the x-axis!

Figure 8: For a MHz sample rate, there are (10 million) impulses in time domain for just 1 in frequency domain

Here, it is easier to recognize that for an ADC operating at 10 MHz, there are (10 million) impulses on the time axis, for just one such impulse on the frequency axis. Let us express the equivalence of energy in time and frequency domains (known as Parseval’s relation) in a different way.

Clearly, the scaling of the frequency axis by seems perfectly logical. A signal narrow in time domain is wide in frequency domain, finally! I must say that this is the same figure as Figure 3 but until I went this route, I did not figure (pun intended) this out.

This reminded me of drawing the structure of an atom along with the solar system to indicate some similarity, without mentioning their respective scales, as in Figure 9.

Figure 9: An atom and the solar system

This also reinforced my concept of using the frequency variable instead of . While has its advantages at places, the variable connects us with what is out there: the reality. The commonly used ISM band is GHz, not G-radians/second.

## Going into Discrete Domain

Let us turn our thoughts in discrete domain now. There are two points to consider in this context.

1. The main difference between a continuous-time impulse and a discrete-time impulse is that a discrete-time impulse actually has an amplitude of unity; there is no need to define the concept related to the area under the curve.
2. There is a factor of that gets injected into expressions due to the way we define Fourier Transforms. This is the equivalent of the factor for continuous frequency domain in Discrete-Time Fourier Transform (DTFT).

A discrete impulse train is a sequence of unit impulses repeating with a period within our observation window (and owing to DFT input periodicity, outside the observation window as well). Figure 10 illustrates it in time domain for and .

Figure 10: Discrete impulse train in time domain for and

To compute its -point DFT ,

A few things to note in this derivation are as follows.

• I write DFT emphasizing on the expression because it corresponds to the continuous-time frequency , explicitly exposing as the time variable similar to its counterpart (think of , or ).
• From going to second equation from the first, I use the fact that except where .
• The third equation can be derived, again, in many ways such as the one described in the continuous-time case described before.

The final DFT output is shown in Figure 11 for and .

Figure 11: Discrete impulse train in frequency domain for and

Note how impulse train in time domain is an impulse train in frequency domain. So for and a discrete-time impulse train with a period of , the DFT is also an impulse train with a period of . Moreover, the unit impulses in frequency domain have an amplitude of as well. Observe that a shorter period in time domain results in a longer period in frequency domain. Also, the amplitude of those frequency domain impulses is times larger as well: compare with in Figure 3.

This is the same result as in continuous frequency domain with impulse train having a period of as well as a scaling factor of . However, it is much clearer to recognize in discrete domain due to the finite DFT size . That is why I talked about limiting the continuous frequency range within for a large and wrote Parseval’s relation as above.

## Sampling in Discrete Domain: Downsampling

With the above findings, we want to see what happens when a discrete-time signal is sampled with a discrete-time impulse train. In DSP terminology, this process is known as downsampling. Although it is common to discuss downsampling as a separate topic from sampling a continuous-time signal, both processes are intimately related and offer valuable insights into one another.

A signal can be downsampled by a factor of by retaining every th sample and discarding the remaining samples. Such a process can be imagined as multiplying with an impulse train of period and then throwing off the intermediate zero samples (in practice, no such multiplication occurs as zero valued samples are to be discarded). As a result, there is a convolution between and and spectral copies of arise: one at frequency bin and others at , all having a spectral amplitude of . As increases, the spectral replicas come closer and their amplitudes decrease proportionally. This is illustrated in Figure 12 for downsampling a signal by .

Figure 12: Downsampling a signal by in time and frequency domains

This is the same concept as sampling a continuous-time signal. Just as we decrease the amplitudes and spread of the resulting spectral copies by reducing the sampling rate, we decrease the amplitudes and spread of the resulting spectral copies by throwing off more samples, i.e., reducing the sampling rate. In the former, this happens in proportion to ; in the latter, in proportion to .

Put in this way, is choosing out of every samples. And is choosing out of every samples if there ever were. In this context, is the probability density function (pdf) of a uniform random variable in the range .

## An Alternative View

Carrying on, now we can look into an alternative view on sampling. First, consider the scaling property of the Fourier Transform.

Second, instead of impulse sampling, we can just see it as a scaling of the time axis. Taking the constant as ,

Here, is the counterpart of from impulse sampling with the difference being a real number instead of an integer . Also, instead of nano and micro seconds, has no units and we just deal it in the limiting case where the interval is divided into infinitesimally small segments and is just a positive number. In this way, this case becomes similar to downsampling where is always greater than .

The spectrum x-axis also gets scaled by leading to the fundamental relationship between continuous-time frequency and discrete-time frequency .

Within this framework, it is only after discarding the intermediate “samples” that the spectral replicas emerge. This is a result of defining the frequency axis in relation to periodic signals. There might be an alternative definition of frequency that would make processing the samples easier.

If you found this article useful, you might want to subscribe to my email list below to receive new articles.

## A Unit Impulse in Continuous-Time

This post treats the signals in continuous time which is different than the approach I adopted in my book. The book deals exclusively in discrete time.

A unit impulse is defined as

The result is an impulse with zero width and infinite height, but a consequence of defining it in this way is that the area under the curve is unity.

This is shown in Figure below.

Stated in another way,

A consequence of this property is that theoretically a particular value of a signal can be extracted in the following manner.

(1)

The Fourier Transform of a unit impulse can be derived through the help of Eq (1).

which is a constant for all frequencies from to . This is shown in Figure below.

Here, there is a chance of not emphasizing the underlying concept of a unit impulse — a rectangle with width approaching zero. An alternative and beautiful approach in this context is transforming a rectangular signal and taking the limit . Since the Fourier Transform of a rectangle is a sinc function (),

where we used the approximation for small . With becoming small and approaching zero, the mainlobe of the sinc becomes wider and approaches . Imagine an eagle spreading its wings. This illustrated in Figure below and perfectly demonstrates the concept that a signal narrow in time domain has a wide frequency domain representation and vice versa.

## The Fundamental Problem of Synchronization

We have seen in the effect of phase rotation that the matched filter outputs do not map back perfectly onto the expected constellation, even in the absence of noise and no other distortion. Unless this rotation is small enough, it causes the symbol-spaced optimal samples to cross the decision boundary and fall in the wrong decision zone. And even for small rotations, relatively less amount of noise can cause decision errors in this case, i.e., noise margin is reduced. In fact, for higher-order modulation, the rotation becomes even worse because the signals are closely spaced with each other for the same Tx power. Similarly, sampling the incoming Rx signal anywhere other than the zero-ISI instants causes inter-symbol interference (ISI) among neighbouring symbols. A synchronization unit is vital to ensure the proper functionality of a wireless communication system. We explain the fundamental problem of synchronization through the help of the phase rotation impairment.

The task of a phase synchronization unit in a Rx is to estimate this phase offset and de-rotate (since the rotation of a complex number is defined with respect to anticlockwise fashion, de-rotation implies rotating the input in a clockwise direction) the matched filter outputs by this estimate either in a feedforward or a feedback manner. In such a way, the carrier at the Rx can be said to become synchronized with the carrier used at the Tx. We need to analyze the Rx phase as follows.

Breakdown of the Rx phase

The modulated signal arriving at the Rx consists of two different phase components:

1. Unknown phase shifts arising due to modulating data occurring at symbol rate . For example, in BPSK modulation, phase changes by or at the boundary of each symbol interval . For QPSK, there are four different phase shift possibilities, i.e., , , , or . A similar argument holds for Quadrature Amplitude Modulated (QAM) signals as well.
2. Unknown phase difference between Tx and Rx local oscillator, .

Imagine a QPSK signal input directly into a black box designed to estimate and compensate for the phase and frequency of a sinusoid (a PLL for example). Depending on its design parameters, the mechanism implemented in the black box will try to lock onto the incoming phase within a specific time duration. However, that phase is jumping around by , or at every symbol boundary due to modulating data, never allowing our mechanism to converge. This is the fundamental problem every synchronization subsystem needs to address and is illustrated in Figure below.

We are faced with two options in this situation. Either transmit the synchronization signal in parallel to the data signal that costs increased power and/or bandwidth, or some procedure needs to be invoked to remove the modulation induced phase shifts in the data signal as a result of which

[for a phase offset] the output should become a constant complex number whose phase is our unknown parameter, and
[for a frequency offset] the output should become a simple complex sinusoid whose frequency is our unknown parameter.

Then, either a feedforward (one-shot) estimator or a feedback mechanism (a PLL) can detect this unknown phase. We can either utilize this observation directly to come up with some intuitive phase estimators, or employ our master algorithm — the correlation — to this problem and see where it leads. Interestingly, it turns out that the derivation of phase estimators through the maximum correlation process actually leads to the same intuitively satisfying results. Maximum correlation is a result of a procedure termed as maximum likelihood estimation. Almost all ad hoc synchronization algorithms can be derived through different approximations of the maximum likelihood estimation. For Gaussian noise under some constraints, maximum likelihood and maximum correlation are one and the same thing.

Example

Suppose that the training symbols and are known (a data-aided system), then their angle can be subtracted from the angle of the matched filter output at each symbol time as

(1)

The intuition this approach is simple: the phase difference between received and expected symbols is the remaining phase offset.

We discussed earlier that Pulse Amplitude Modulation (PAM) transmits information through amplitude scaling of the pulse according to the symbol value. To understand QAM, we have to differentiate between baseband and passband signals.

A baseband signal has a spectral magnitude that is nonzero only for frequencies around origin () and negligible elsewhere. An example spectral plot for a PAM waveform is shown below for 500 2-PAM symbols shaped by a Square-Root Raised Cosine pulse with excess bandwidth and samples/symbol . The PAM signal has its spectral contents around zero and hence it is a baseband signal.

For wireless transmission, this information must be transmitted through space as an electromagnetic wave. Also observe that once this spectrum is occupied, no other entity in the surrounding region can share the wireless channel for the purpose of communications. For these reasons (and a few others as well), a wireless system is allocated a certain bandwidth around a higher carrier frequency and the generated baseband signal is shifted to that specific portion of the spectrum. Such signals are called passband signals. That is why wireless signals in everyday use such FM radio, WiFi, cellular like 4G, 5G, and Bluetooth are all passband signals which execute their transmissions within their respective frequency bands sharing the same physical medium.

The easiest method to shift the spectrum to a designated carrier frequency is by multiplying, or mixing, it with a sinusoidal waveform. This is because time domain multiplication is frequency domain convolution between spectra of the sinusoid and the desired signal. Now in the article on DFT examples, we saw that the spectrum of a pure sinusoid at any frequency is composed of two impulses at that frequency (one positive and one negative and half amplitude). Also, convolution of a signal with a unit impulse results in the same signal shifted at the location of that impulse. Hence, a PAM waveform can be multiplied with a carrier sinusoid at frequency to “park” the actual spectrum in its allocated slot. This is drawn in Figure below.

After this spectral upconversion, both positive and negative portions of the baseband spectrum appear at and . The bandwidth — positive portion of the spectrum — hence becomes double. However, a relief comes from the fact that both and can be used to carry independent waveforms due to their orthogonality (having phases apart) to each other over a complete period, i.e.,

That is the birth of Quadrature Amplitude Modulation (QAM), in which two independent PAM waveforms are communicated through mixing one with a cosine and the other with a sine. Just like constellation, the term quadrature also comes from astronomy to describe position of two objects apart.

For mathematical expression for QAM, first consider two independent PAM waveforms with symbol streams and , respectively.

(1)

Next, we multiply the resulting complex signal with a complex sinusoid at carrier frequency and collect its real part, i.e., multiply arm with and arm with . The reason of including these two factors, and the negative sign, will be discussed later during QAM detection. Consequently, a general QAM waveform can be written as

(2)

(3)

where we have used the fundamental relation between continuous and discrete frequencies , and corresponds to the carrier frequency . Observe that is a real signal with no component. After digital to analog conversion (DAC), the continuous-time signal can be expressed as

(4)

In Eq (4), symbols determine the signal while control the signal. Such representation of the QAM waveform as a sum of amplitude scaled and pulse shaped sinusoids is known as the rectangular form. Using trigonometric identity , Eq (4) can also be written as

Called the polar form, here we have a single sinusoid whose amplitude and phase are determined by some combination of symbols and .

## Constellation Diagram

Some examples of QAM constellations are discussed below.

• For an even power of , square QAM is a constellation whose points are spaced on a grid in the form of a square. It is formed by a product of two -PAM constellations, one on axis and the other on axis. For example, a -QAM constellation is formed by two PAM constellations as drawn in Figure below, while some other square QAM constellations are also shown in the next Figure.

Notice how constellation points in higher-order QAM are closer to each other compared to lower-order QAM. A relatively lower noise power is then enough to cause a decision error by moving the received symbol over the decision boundary. This is the cost of increasing data rate by packing more bits in the same symbol. We will have more to say about it when we discuss Bit Error Rates (BER) for each constellation in another article.

For , the average symbol energy in square QAM is derived using Pythagoras theorem as

For general , a similar derivation as in the case of PAM yields

(5)

• A special case of -QAM is -PSK, which stands for Phase Shift Keying. As the name PSK implies, the amplitude remains constant in this configuration while the information is conveyed by different phases. Examples of -PSK, -PSK and -PSK are drawn in Figure below.

Notice that

• -PSK constellation (also known as BPSK) looks similar to -PAM. However, the difference is that there is no carrier upconversion in PAM systems.
• -PSK constellation (also known as QPSK) is exactly the same as square -QAM.
• For , square QAM packs the constellation points more efficiently than and hence the modulation of choice in many wireless standards. Points come relatively closer in PSK and the closer the points, the larger the probability of receiving a symbol in error due to a jump across the decision boundary. On the other hand, QAM heavily depends on overall system linearity due to information conveyed in the signal amplitude. A constant envelope of the modulated signal is much more suitable for transmission over nonlinear channels where PSK is the preferred choice. In such situations, the performance of PSK is quite insensitive to nonlinear distortion and heavy filtering.
• Since any constellation that uses both amplitude and phase modulation fall under the general category of QAM, there can be many other rearrangements of QAM constellation points. However, we focus on square QAM and PSK in this text which are widely used in practice. Non-regular constellations can be formed which yield a small performance advantage, though it is usually not much significant. Moreover, the more points are in the constellation, the lesser is the difference.

## QAM Modulator

To build a conceptual QAM modulator, we follow similar steps as in a PAM modulator. The block diagram for a -QAM modulator is drawn in Figure below.

• Every seconds, a new bit arrives at the input forming a serial bit stream.
• A serial-to-parallel (S/P) converter collects such bits every seconds that are used as an address to access two Look-Up Tables (LUT). One LUT stores symbol values while the other symbol values specified by the constellation.
• To produce a QAM waveform, the symbol sequences and are converted to discrete-time impulse trains in separate arms (one and the other ) through upsampling by , where is samples/symbol defined in as ratio of symbol time to sample time , or equivalently sample rate to symbol rate .
• As explained in sample rate conversion, upsampling inserts zeros between each symbol after which the interpolated intermediate samples can be raised from dead with the help of a pulse shaping filter in each arm that — in addition to shaping the spectrum — suppresses all the spectral replicas arising from upsampling except the primary one. These are the two and PAM waveforms forming the signal .
• Next, PAM waveform is upconverted by mixing with the carrier while the PAM waveform with , which are then summed to form the QAM signal . The carriers are generated through an oscillator at the Tx.
• This discrete-time signal is converted to a continuous-time signal by a DAC.

The mathematical derivation for the QAM modulator was shown in Eq (2) and Eq (4).

In the meantime, if you found this article useful, you might want to subscribe to my email list below to receive new articles.

## QAM Detector

The received signal is the same as the transmitted signal but with the addition of additive white Gaussian noise (AWGN) . The symbols are detected through the following steps illustrated in Figure below.

• Out of the infinite spectrum, the desired signal is selected with the help of a Bandpass Filter (BPF).
• Through an analog to digital converter (ADC), this signal is sampled at a rate of samples/s to produce a sequence of -spaced samples .
• Next, a complex signal is produced when by downconverted by mixing with two carriers, and which are generated through an oscillator at the Rx.
• The resulting complex waveform in and arms is processed through two matched filters thus generating and . As discussed earlier, the output of the matched filters are continuous correlations of the symbol-scaled pulse shape with an unscaled and time-reversed pulse shape.
• These and outputs are downsampled by at optimal sampling instants

to produce -spaced numbers and back from the signal.

• The minimum distance decision rule is employed in -plane to find the symbol estimates and to decide on the final constellation point.

Let us discuss the mathematical details of this process for a noiseless received signal as in Eq (4),

(6)

where is the carrier frequency and and are defined in Eq (1). After bandlimiting the incoming signal through a bandpass filter, it is sampled by the ADC operating at samples/second to produce

where the relation is used. To produce a complex baseband signal from the received signal, the samples of this waveform are input to a mixer which multiplies them with discrete-time quadrature sinusoids and yielding

Using the identities , and ,

Now we can observe why the two factors, a with both sinusoids and a negative sign with , were inserted.

• A at the modulator and later another in the detector result in a gain of , which cancels the halving of sinusoid amplitudes in above trigonometric multiplications.
• When a complex signal is multiplied with a complex sinusoid, a negative sign appears for phase addition of and in the section of cumulative waveform (see the multiplication rule for clarity). This section of the cumulative waveform is the real part of the complex product which is transmitted through the channel. Another negative sign with at the Rx delivers a positive at the output.

The matched filter output is written as

where comes into play from the definition of auto-correlation function. The double frequency terms in the above equation are filtered out by the matched filter , which also acts a lowpass filter due to its spectrum limitation in the range . To generate symbol decisions, -spaced samples of the matched filter output are required at . Downsampling the matched filter output generates

and

A square-root Nyquist pulse has an auto-correlation that satisfies no-ISI criterion,

Thus,

In conclusion, the downsampled matched filter outputs map back to the Tx symbols in the absence of noise. If the world was simple, that would have been an end to it! But the world is complicated, and there are layers of issues that happen between the Tx information generation and Rx decision making. Anything we do after this will be to combat a subset of signal distortions and towards recovering the original information.

Observe that the system shown in Figure above is a multirate system. In the QAM detector, for example, the ADC and the matched filters operate at the sample rate . After the outputs of the matched filters are downsampled by , the symbol decisions are made at the symbol rate . Furthermore, there are some hidden assumptions in the QAM detector:

[Resampling] The ADC in general does not produce an integer number of samples per symbol, i.e., is not an integer. As we will see later, a resampling system is required in the Rx chain that changes the sample rate from the ADC rate to a rate that is an integer multiple of the symbol rate.
[Symbol Timing Synchronization] The peak samples at the end of symbol durations in both and arms are not known in advance at the Rx and in fact do not necessarily coincide with generated samples as well. This is because ADC just samples the incoming continuous waveform without any information about the symbol boundaries. This is a symbol timing synchronization problem which we will discuss later.
[Carrier Frequency Synchronization] The carrier frequency of the oscillator at the Tx and that at the Rx are not exactly the same, equal to . Instead, if the oscillator frequency at the Tx is denoted as , then at the Rx, we have , where is the difference between the two and can be either positive or negative. Any movement by the Tx, the Rx or within the environment between them also causes a shift in frequency (known as Doppler shift) that needs to be compensated. We will discuss carrier frequency synchronization in another article.
[Carrier Phase Synchronization] Furthermore, the carrier phase at the Rx oscillators is not known beforehand and needs to be estimated which will be explained in another article.

## QAM Eye Diagram and Scatter Plot

As discussed above, a QAM signal consists of two PAM signals riding on orthogonal carriers. At baseband, these two PAM signals appear as and components of a complex signal at the Tx and Rx. Therefore, there are two eye diagrams for a QAM modulated signal, one for and the other for and both of them are exactly the same as PAM.

The case of scatter plot is a little different. After downsampling the matched filter output to symbol rate, the samples thus obtained are mapped back to the constellation, previously illustrated in QAM detector block diagram and now drawn in Figure below for -QAM modulation. This cloud of samples around the constellation points is now -dimensional (for PAM, there was no component) and can also be understood as a plot of versus for each mapped value.

Looking at the scatter plot, one can readily deduce a lot of features for the particular transmission system. At this stage, however, it is enough to observe that the diameter of these clouds is a rough measure of the noise power corrupting the signal. For the ideal case of no noise, this diameter is zero and all the optimally timed samples coincide with their respective constellation points.

## Finite Impulse Response (FIR) Filters

We learned in the concept of frequency that most signals of practical interest can be considered as a sum of complex sinusoids oscillating at different frequencies. The amplitudes and phases of these sinusoids shape the frequency contents of that signal and are drawn through magnitude response and phase response, respectively. In DSP, a regular goal is to modify these frequency contents of an input signal to obtain a desired representation at the output. This operation is called filtering and it is the most fundamental function in the whole field of DSP.

From this Eq, one can observe that it is possible to design a system, or filter, such that some desired frequency contents of the signal can be passed through by specifying for those values of , while other frequency contents can be suppressed by specifying . The terms lowpass, highpass and bandpass then make intuitive sense that low, high and band refer to desired spectral contents that need to be passed while blocking the rest.

## Filter Frequency Response

Figure below shows the magnitude response (as a function of continuous frequency) of an ideal lowpass filter. A lowpass filter passes frequencies near while blocks the remaining frequencies.

As explained in the discussion about sampling, in a continuous frequency world, the middle filter is all that exists. However, in a sampled world, the frequency response of the filter — just like a sampled signal — repeats at intervals of (sampling frequency) and there are infinite spectral replicas on both sides shown with dashed lines. The primary zone of the sampled frequency response is from to shown within dotted red lines with its spectral contents in solid line. From this Eq on sampling, we know that filter response near is the same as that near and the filter response around is the same as around .

On the other hand, a highpass filter passes both positive and negative frequencies near while blocking everything else. Observe the symmetry around in Figure below. Similarly, bandpass filters only pass a specific portion of the spectrum.

## Filter Impulse Response

Since a filter is a system that modifies the spectral contents of a signal, it naturally has an impulse response just like a system. When this impulse response is finite, the filter is called Finite Impulse Response (FIR). Impulse response of an FIR filter, i.e., the sequence of its coefficients, can be found by taking the iDFT of the frequency response .

## FIR Filter Design

Understanding of FIR filter design starts from the following argument for continuous frequency domain. Since ideal filters have unity passband and zero stopband and a direct transition between these two, this finite frequency domain support produces an impulse response in time domain which is infinite and hence non-realizable, as explained in the discussion about frequency. When this is truncated on the two sides, it is equivalent to multiplication of the ideal infinite by a rectangular window as

As mentioned previously, windowing is a process of pruning an infinite sequence to a finite sequence. We know that multiplication in time domain is convolution in frequency domain, and that the spectrum of a rectangular window is a sinc signal. Thus, convolution between the spectrum of the ideal filter and this sinc signal generates the actual frequency response of the filter.

This sinc signal has decaying oscillations that extend from to . Convolution with these decaying oscillations is the reason why a realizable FIR filter always exhibits ripple in the passband and in the stopband as well as a transition bandwidth between the two. Naturally, this transition bandwidth is equal to the width of the sinc mainlobe, which in turn is a function of the length of the rectangular window in time. Truncation with a rectangular window causes the FIR filters to have an impulse response of finite duration.

Sidelobes appearing in the stopband of the filter, transition bandwidth and ripple in the passband characterize the performance of a filter. These parameters are usually specified in terms of decibels (or dBs). The filter magnitude in dB is given by

It should be remembered that the multiplicative factor above is replaced by for power conversions (because power is a function of voltage or current squared). The advantage of converting magnitudes to dBs for plotting purpose is that very small and very large quantities can be displayed clearly on a single plot.

A filter is designed in accordance with the parameters in Table below and shown graphically in the next Figure.

FIR filter design basically requires finding the values of filter taps (or coefficients) that translate into a desired frequency response. Many software routines are available to accomplish this task. A standard method for FIR filter design is the Parks-McClellan algorithm. The Parks-McClellan algorithm is an iterative algorithm for finding the optimal FIR filter such that the maximum error between the desired frequency response and the actual frequency response is minimized. Filters designed this way exhibit an equiripple behavior in their frequency responses and are sometimes called equiripple filters (equiripple implies equal ripple within the passband and within the stopband, but not necessarily equal in both bands).

Example

We design a lowpass FIR filter through Parks-McClellan algorithm in Matlab that meets the following specifications.

First, noting from above Figure that the maximum amplitude in the passband of frequency domain is , we can convert dB units into linear terms as

For the stopband,

A Matlab function firpm() returns the coefficients of a length linear phase (which implies real and symmetric coefficients, see the concept of phase) FIR filter. For finding an approximate , the Matlab command firpmord() — which stands for FIR Parks-McClellan order — can be used.

where the vector specifies the desired amplitude pointing towards passband and stopband, respectively. For our example,

A word of caution: it can raise confusion in the sense that firpmord() uses the band widths for input amplitudes while firpm() uses the band edges. For example, firpmord() uses the desired amplitude vector to denote passband and stopband. On the other hand, firpm() uses the amplitude vector to denote band edges.

For a trial value of ,

where the vector specifies the desired amplitude at band edges of passband and stopband, respectively. The normalization by is of no significance as it is just a requirement of this function. Furthermore, the filter is designed with coefficients by Matlab and that is why we used as its argument.

If you try this little code, you will see that the result thus obtained had stopband levels of only around dB and a passband ripple of dB. The sidelobes do not meet the dB attenuation while the passband ripple exceeds the requirement of dB. Therefore, can be increased until dB attenuation is achieved for . To avoid excessive filter length increase which increases the processor workload, different weighting factors can be employed for passband and stopband ripples as well. For example, a weighting factor of for passband to stopband ( in vector form) yields,

In this case, the sidelobe level reached dB, a dB improvement over the same when there were no band weights. Next, the filter order is gradually increased until it meets the desired specifications at , a difference of taps from and no weighting factors. Such trial and error approach is often necessary to design a filter with given specifications.

The impulse response and frequency response of the final filter are drawn in Figure below. Note the stopband attenuation of dB and passband ripple within dB.

## Filtering as Convolution

We also learned above that the higher the number of taps, the more closely the filter follows an expected response (e.g., lowpass, highpass, bandpass, etc.) at a cost of increased computational complexity. Therefore, the defining feature of an FIR filter is the number of coefficients that determines the response length, the number of multipliers and the delay in samples required to compute each output. An FIR filter is built of multipliers and adders. A delay element, which is just a clocked register, is used between coefficients. A single delay element is represented with a symbol. Figure below illustrates a 5-tap FIR filter.

Taking into account the structure in Figure above, this -tap filter output can be mathematically written as

For a general FIR filter of length ,

(1)

If this equation sounds familiar to you, you guessed it right. Compare it with convolution Eq, and this is nothing but a convolution between the input signal and the filter coefficients. Just like convolution, at each clock cycle, samples from input data and filter taps are cross-multiplied and the outputs of all multipliers are added to form a single output for that clock cycle. On the next clock cycle, the input data samples are shifted right by relative to filter taps, and the process continues.

## Filter Delay

Due to convolution, the length of the output signal is given as in the article on convolution, which is repeated below.

which is longer than the input by samples. We can say that every FIR filter causes the output signal to be delayed by a number of samples given by

(2)

As a result, samples of the output at the start — when the filter response is moving into the input sequence — can be discarded. Similarly, the last samples — when the filter is moving out of the input sequence — can be discarded as well.

## Filter Phase Response

As discussed in the concept of phase, the phase response of filters is also important because the shape of the signal in time domain is determined by phase alignment of its constituent complex sinusoids. A filter with unity magnitude across the spectrum does not affect the amplitude of those sinusoids but if its phase response is not linear, it can completely change the shape of the filtered signal due to different sinusoids adding up with different phases at the output. A linear phase on the other hand just delays the output without distorting it. In the post on convolution, we discussed that the output response of an LTI system to a complex sinusoid at its input is nothing but the same complex sinusoid at the same frequency but with different amplitude and phase. If this phase of the filter is a linear function of frequency , then rearranging Eq on time and phase shifts,

for all complex sinusoids in its spectrum. Fortunately, maintaining a linear phase response in FIR filters is straightforward by making the coefficients symmetric, see this Figure as an example. This is one of the main reasons for widespread use of FIR filters in DSP applications.