For more information:
e-mail: enquiries@singmai.com
Tel: +66 (0) 36 275465
Previous issues:
SingMai Electronics wishes all our customers, (and potential customers), a happy Christmas and a successful and healthy New Year. Christmas is not a holiday in Thailand so we will be open for business as usual throughout the holiday period.
You may have read about the recent flooding in Thailand, the worst for 50 years. SingMai did what it could by providing food for some of the flood victims and will be continuing this initiative on a regular basis to the poorer members of our local community.
We have received prototypes of our first standalone product, the SD video pattern generator and encoder, SM01. You can expect the product to be launched in early 2011. Please contact us for advance information on this product.
SingMai now has a page on Linked-In. If you wish you can follow us by clicking on the following link.
We are completing final testing of two more video IP cores.
The first of these is PT15, an SD RGB to YCbCr colour space converter (CSC) which can be used independently or as a front end to our video encoders to allow them to accept RGB component inputs. The input to the CSC is independent 8/10 bit RGB together with the associated 13.5MHz clock. The output is 4:2:2 (13.5/6.75/6.75MHz) Y,Cb,Cr. The CSC includes the appropriate decimating filters for the chroma.
PT11 is a 3x3 median filter which is useful for the reduction of impulse noise in images. The input is a BT656 formatted video source on which the 3x3 median filter is performed in real time on the luma only. The output is also BT656. Again the median filter may be windowed to only operate over a region of the image.
Further details on these IP cores will be provided in the next newsletter but in the meantime, for advance information, please send us an e-mail.
This article first appeared in EDN in 2008.
FPGAs are used extensively for customizing electronics products. However in cost sensitive consumer electronic’s products, although customization is a highly desirable feature for discriminating against competitor products, the cost of the FPGA usually means it cannot be afforded in the low to mid range products.
In this article I will look at one area of consumer electronics, the video and audio acquisition block, used in hard-disc and DVD recorders, televisions and set top boxes for example, and see how an FPGA may be made affordable whilst also allow customization of the product through some unique features.
In addition I will consider some of the custom features that may be added and the technical implications for integrating them. Custom features I will consider will include video and audio demodulation, Teletext and closed caption decoding and innovative 3D comb filters.
Figure 1 illustrates the typical blocks that constitute the front end acquisition circuits. The sound intermediate frequency, (SIF, e.g. NICAM, A2, BTSC, EIA-J), and video, (e.g. NTSC, PAL, SECAM), decoding is usually achieved through the use of one or two ICs that are offered by a number of manufacturers. In addition external memory is required if a 3D comb filter is implemented and a baseband audio stereo ADC is usually required. These blocks are repeated if dual tuners need to be incorporated.
Video and audio decoding IP cores are available which allow the replacement of these I.C.s with an FPGA. Area efficient implementations allow the functions to be incorporated into one of the more cost effective FPGA products, for example the Altera Cyclone or Xilinx Spartan range. One way in which the cost of the FPGA can be kept down is to allow different images for different standards with the main control processor downloading the appropriate image depending on the detected standard. For example image sets might comprise NTSC/BTSC or PAL/NICAM. Using the main control processor to initialize the FPGA also offsets the cost of an EEPROM to hold the FPGA image.
Of course the FPGA does not allow analogue functionality so an external IC is required to perform the analogue to digital conversion. Such ICs are available from a number of manufacturers, e.g. the AD9981 from Analog Devices. This IC also performs sync separation and clock generation functions and can be made to interface to CVBS, (composite), YC, YPbPr and RGB inputs. Having an external ADC usually achieves a better signal to noise ratio over the integrated solutions but the cost of implementation will be about 50% higher than a ‘standard’ I.C. solution.
However we can look at integrating some additional functionality into the FPGA which can mitigate against this cost increase.
For example, it is common for recorders to allow the recording of one channel whilst watching another. This requires two digital and two analogue tuners and decoders and in the non-FPGA implementation, to have two ICs. However it is possible to have two decoders in one small FPGA so the additional cost is limited to one additional ADC. To reduce area it may be possible limit the decoder options for the secondary channel, for example to not have SECAM decoding. The total cost of the FPGA solution is now similar or even lower than the IC solution.
We can now consider how to include additional functionality and discriminating features in the FPGA.
The conventional metal can analogue tuner contains two main functions, RF tuner and demodulator. We can reduce the cost of the tuner by performing the demodulation digitally within the FPGA. The video ADC can be used to digitize the video IF. This saving on the cost of the tuner will be by a factor of two in the case of the dual tuner products. Additionally this solution also now opens up the possibility of using a silicon tuner as their performance is now adequate for most consumer products. The silicon tuner, with its all region capability, brings possibilities for cost effective all-region consumer products with the corresponding manufacturing savings that can bring, (usually metal can tuners are region specific for cost reasons).
Another possibility that having an FPGA in these systems brings is to add some local intelligence through the integration of a small microcontroller. This microcontroller can then be used for all the front end functionality, off-loading that requirement from the main controller. For example a simple high level instruction to tune channels could be issued. Advantages of this approach can be faster initial channel acquisition times and the ability to power down more of the main controller under standby conditions, a key criterion in today’s green society.
Additional functionality for higher end consumer products can be added by integrating the serial digital interface (SDI) receiver. FPGAs now offer LVDS receivers and clock recovery which permit an SDI receiver to be implemented with just an external cable equalizer IC, considerably cheaper than using a complete SDI receiver IC.
The IC video decoder solutions usually also strip off the information in the vertical blanking area of the signal, but leave the processing of that signal to the main processor. Again having some specialized hardware and a local controller permits, for example, closed caption decoding to be performed. Decoding closed caption or Teletext subtitles is usual for televisions of course, but less so for PVRs and DVD recorders. However superimposing the subtitle on the output BT656 video, something the FPGA can do, allows the recording of subtitles for the deaf and hard of hearing, something currently not possible. The decoded subtitles could also be embedded in the MPEG metadata so the subtitle can be read in the normal way for a more elegant solution.
Another function that may be integrated is an improved comb filter. Most consumer products use a 2D comb which leaves decoding artifacts which are undesirable to display and use valuable bandwidth if compressed. A good 3D comb can improve the performance noticeably but requires an external memory device. The 3D comb IC solutions that are available use a symmetrical frame comb which, for PAL, requires 4 frames x 625 lines x 1440 pixels x 8 or 10 bits = 36Mbits which is why an external memory device, usually an SDRAM, is required. This requirement can be halved by using an asymmetrical comb which has the advantage of not requiring a compensating audio delay. It can be halved again, for PAL, by using techniques such as PAL modifiers in the comb architecture and halved again by using a field comb. A single tap 262 line, (for NTSC), or 312 line, (for PAL), gives excellent results although does not permit ‘perfect’ decoding of complex still frames. However for ‘real’ images the wide aperture of the PAL frame comb in particular often means that the 3D comb will fail on moving images and drops down to line comb mode. A field comb offers a good compromise between memory requirements and performance. A field comb for PAL then requires 312 lines x 1440 pixels x 8 bits minimum which is 3.6Mbits – unfortunately still too large to use the integrated memory of small FPGAs.
However it is possible to implement a ‘pseudo’ 3D comb within the memory requirements of even the smallest FPGAs. Normally three comb modes are available to the decoder, the 3D comb, a 2D line comb and a ‘simple’ mode which is either a low pass filter or a notch. The comb mode is chosen based on failure signals that indicate failure conditions such as motion, preventing the 3D comb from operating, or diagonals, preventing the 2D comb from operating. There is an assumed priority than 3D is always the preferred mode and simple the least desirable. This is not always true; for example on flat areas of colour simple can often be the best mode, (by best I mean the mode with highest signal to noise ratio or least visible artifacts). The reason for this can be that the wide aperture of the 3D comb can mean that clock jitter, and just 1ns of clock jitter across a 80ms tap distance, can leave residual subcarrier. It is possible therefore to consider a comb architecture where the 2D and simple modes are used to decode an image wherever possible. Only where neither of these modes can operate do we choose the 3D comb. What we create is a 1-bit positional flag which is stored, along with the frame/field delayed data for that flag position. On the subsequent frame/field we can then choose this 3D comb aperture for these positions in addition to the 2D and simple modes. On a number of tested images, including the ubiquitous Snell and Wilcox moving zone plate, it is surprising how little time the 3D is required. It is still necessary to determine if the 3D comb is not failing - it will have failed if there is substantial motion in the image, especially if a frame aperture is chosen – but a considerably reduced memory requirement is the benefit. It is necessary to have a 1 bit plane for the 3D comb aperture chosen, for example for a 312 line field comb, 312x720 = 224640 bits are required. Then at the flagged locations where both the 2D and simple modes fail we need memory to store the delayed information. Surprisingly this can be limited to just 32kbytes and still produce substantial improvements in the decoded image. In other words, for a large range of images, there are just 32,000 pixels where the 3D comb is essential. This combined amount of memory is available in small FPGAs allowing the pseudo 3D comb to operater on at least one channel of the decoder.
Another improvement in decoded quality is possible by using a modified comb architecture. Video Cassette Recorders use a technique known as color-under for recording. The luminance information is FM modulated onto a carrier of approximately 3.5MHz, whereas the chrominance information, (and also the luminance information contained within it), is re-modulated onto a lower frequency carrier, (approximately 600 kHz), to avoid problems recording at higher frequencies. Because of this re-modulation of the chrominance component onto another carrier the phase relationship with the original sub-carrier is lost. Without this phase relationship it is no longer possible to comb the higher frequency component to separate the luminance and chrominance, and the higher frequency information is therefore discarded leaving a low resolution image.
The majority of comb filters use a complementary baseband comb. The composite video is demodulated using a phase locked subcarrier waveform and then low pass filtered to produce the U and V chrominance outputs. The U and V signals are then combed to remove the luminance components and then remodulated and added to produced a ‘clean’ chrominance only signal, now centred on the subcarrier frequency and in phase with it. When this is subtracted from the composite video the clean luminance is produced.
Another comb architecture can be envisaged where the re-modulation and the subtraction from the composite it done before the comb filter. This produces ‘simple’ U and V components and a clean notched luminance. If we then comb the U and V, (to produce our combed chrominance outputs), and subtract these from their simple versions we are left with the high frequency luminance signal which can then be re-modulated and added into the notched luminance, effectively filling in the information missing from the notch. This architecture is illustrated in Figure 2.
The advantage of this architecture is the high frequency luminance can be added into the notched luminance using a different phase of the sub-carrier. When the HF luminance is re-modulated with the original phase relationship original waveform is reconstructed with a higher bandwidth.
The phase of the second re-modulator is adjusted based on information derived from a high pass filter. The correct phase for the addition of the luminance signals is found by detecting the improved sharpness of the luminance signal for a certain sub-carrier phase. This can be done with a high pass filter, a square law function, (to rectify the high pass filter output and increase the weighting given to slope of the signals), and an accumulator to measure the amount of edge detail in the image. The adjustment of the correct phase is done as an iterative process, (but the phase does not change quickly so this is satisfactory). The additional phase offset is added into the sub-carrier phase derived from the input signal. This method is capable of substantially improving the perceived sharpness of any VCR source and reducing the discrepancy between the performance of the VCR and the DVD; ideal for transcriptions.
It is not uncommon for some video sources, for example VCRs, to have the luminance and chrominance mistimed. The non-coincidence of the vertical edges leads to a lack of clarity of the video image and the vertical edge is smeared. A number of video decoders offer a YC delay control which allows the user to vary the comparative delay. However the problem with this manual method of control is the user needs to know the delay to be able to compensate for it, effectively rendering the control redundant. Such adjustment is very difficult to do ‘by eye’, especially for an unskilled user and requires specific video test patterns to perform the adjustment. The delay can also vary in time, especially for mechanical mechanisms such as VCRs. However the FPGA can incorporate methods to automatically re-time the luminance and chrominance which are not found in off-the-shelf IC decoders.
Audio is accepted in one of two formats, either a sound intermediate frequency (SIF) from the tuner (e.g. NICAM or BTSC), or base-band stereo from the composite or component video inputs. I.C.s exist to demodulate and decode the SIF signal and similarly IP cores are available for the FPGA. As mentioned earlier the FPGA can take advantage of having different configurations for different standards to reduce the area impact and can also incorporate two such decoders for dual tuner products.
In I.C. solutions the base-band audio ADC may be integrated with the video decoder, but this approach usually gives poor signal to noise ratio, or may be an external IC. However it is possible to consider the use of one of the video ADCs for the audio. By substantially over-sampling the audio and then decimating the result we can theoretically achieve the additional bits we need without an external device, leading to further cost savings.
The proposed FPGA video/audio front end is shown in Figure 3. In this article I hope to have shown that FPGAs do not need to be restricted to high end applications. Low cost devices have sufficient resources to be used in even the most cost sensitive products and can also create a number of unique discriminatory features not available in off the shelf I.C. solutions.