Donald G. Nonlinear Distortion in Wireless Systems. Khaled M. Jose M.
About this book
Gernot Hueber. Kwonhue Choi. Satellite Communications Payload and System. Teresa M. Satellite and Terrestrial Radio Positioning Techniques. Davide Dardari. High Performance Integrated Circuit Design. Advanced Circuits for Emerging Technologies. Krzysztof Iniewski. Microelectronic Systems. Albert Heuberger. Guillaume de la Roche. Wolfgang Engel. Power Line Communications.
Hendrik C. Hyunchul Shin. The Cable and Telecommunications Professionals' Reference. Goff Hill. Vishram S. Jacopo Iannacci. Ramon Vilanova. Akram Alomainy. Implementing Software Defined Radio. Eugene Grayver. Li-Minn Ang. Electrical Overstress EOS. Ivan Kaminow. Real-Time Digital Signal Processing.
Sen M. Communication in Transportation Systems. Otto Strobel. Microwave and Millimeter Wave Circuits and Systems. Apostolos Georgiadis.
Fa-Long Luo. Eric Verhulst. Rino Micheloni. SystemVerilog for Verification. Chris Spear. Antonis Kalis. Sanjay Churiwala. Soft Computing Techniques in Engineering Applications. Srikanta Patnaik. Pallab Chatterjee. Millimeter Wave Communication Systems. Kao-Cheng Huang. Error Control for Network-on-Chip Links. Bo Fu. Manfred Dietrich. Abbas Sheibanyrad. Adaptable Embedded Systems.
- Focal-Plane Sensor-Processor Chips | Ákos Zarándy | Springer.
- Agency Micro-sites!
- Anatomy of the focal-plane sensor-processor arrays.
- How To Cure Nail Fungus - Top Nail Fungus Remedies.
- Search In:!
- VTLS Chameleon iPortal System Error Occurred..
- Shaman King, Vol. 2: Kung-Fu Master!
Antonio Carlos Schneider Beck. Communications Engineering Desk Reference. Bear in mind that the image in Figure 6c is the outcome after photointegration, pixel copy, interconnection setting, charge redistribution and A-to-D conversion. This last stage is key to remove the aforementioned artifacts. During A-to-D conversion, we copy the corresponding original pixel value still stored in the capacitors holding V ij for the regions where full resolution is affordable in terms of privacy. Otherwise, averaging is allowed. On-chip obfuscation without artifacts can thus be achieved, as shown in Figure 7.
In this case, we set interconnection patterns for progressively coarser simultaneous pixelation of two different image regions containing faces. Finally, as an example of on-the-fly focal-plane reconfiguration conducted by a vision algorithm, we set up a closed loop between our test system and the Viola-Jones frontal face detector provided by OpenCV, as depicted in Figure 8.
The sensor captures images that are sent to a PC from the test board. The Viola-Jones detector is run on these images on the PC. If faces are detected, the coordinates of the corresponding bounding rectangles are sent back to the test board for the vision sensor to reconfigure the image capture in real time. Pixelation of the face regions will take place from that moment on at the focal plane. The degree of pixelation of these regions is adjustable through a button of the test board.
A sequence extracted during the execution of this loop can be download from [ 43 ]. Notice that all of the reconfiguration and control of the array must be carried out externally for this prototype. Real-time focal-plane reconfiguration conducted by the Viola-Jones frontal face detector. If faces are detected, the coordinates of the corresponding bounding rectangles are sent back to the test board for the vision sensor to reconfigure the image capture on-the-fly.
One of the fundamental problems that visual sensors have to cope with is the inspection of scenes featuring a wide range of illuminations. In such cases, assuming the usual photocurrent integration mode, the application of the same exposure time across the pixel array leads to either over-exposed or under-exposed regions. The consequent low contrast and missing details can make the processing of these regions unfeasible. The processing primitive described next is specially suitable for vision algorithms requiring tracking of regions of interest ROI [ 44 , 45 ] in such circumstances.
The two photodiodes and two sensing capacitances included at the elementary cell in Figure 3 are required to implement this low-level operation. If the voltage V S ij starts from a value above the input threshold voltage of the inverter, V th inv , the reset of the pixel capacitance holding V ij is concurrently initiated. Otherwise, as is the case in the illustrating timing diagram of Figure 9 , this reset will be delayed slightly until V S ij reaches that threshold.
A key aspect of the circuit is that the inverter must be designed in such a way that V th inv is located just at the middle point of the signal range, that is:. At that time instant, photointegration begins concurrently in both the pixel capacitance and the averaging capacitance. Keep in mind that these signals come from the peripheral circuitry previously mentioned. As a result, the voltage excursion For the sake of simplicity, we define the voltage excursion as the difference between the initial voltage, V rst , and the final voltage.
This permits getting rid of the minus sign. It must be observed that, since the averaging capacitances are interconnected, the following equation holds:. The photointegration at the pixel capacitances will finish when V AV k reaches V th inv , that is, from Equation 3 :. If the effect of the dark currents can be neglected, this current is directly proportional to the average incident illumination on the block.
By substituting Equation 5 in Equation 2 , the following voltage excursion for each pixel is obtained:. This property, when applied to the whole pixel array, means a DR enhancement of only 6 dB [ 46 ]. However, when confined to any particular rectangular-shaped block, it endows our hardware with the capability of dealing with regions of interest ROIs across frames featuring a total intra-scene dynamic range of up to dB. The dynamics of two arbitrary pixels belonging to the same block along with the corresponding local averaging voltage is depicted in Figure 9.
V m,n represents a pixel whose illumination is below the average block illumination, whereas V p,q represents a pixel receiving illumination above that average. Notice that the integration period for all of the pixels of the block, including V m,n and V p,q arbitrarily selected for representation, ends at the same time instant without requiring any external control.
That instant is set when the sensing of the average incident illumination, denoted by V AV k , reaches the middle point of the signal range.
Table of Contents
This example can be extended to the whole focal-plane array. Thus, once the different blocks for the next image to be captured are established, a maximum photo-integration period for all of them starts to run. During this period, each block adjusts automatically and independently its integration time according to its particular mean illumination.
Timing diagram illustrating the operation of block-wise HDR. The dynamics of two pixels belonging to the same block along with the corresponding local averaging voltage is depicted.
An example of this operation is shown in Figure Global control of the integration time is applied to the left image. All pixels undergo the same integration time, which is set to ms according to the mean illumination of the scene. Details about the lamp are missed due to the extreme deviation with respect to this mean illumination. However, such details can be retrieved by confining the control of the integration period to the region of interest, as can be seen in the right image.
On-chip block-wise intra-frame integration time control. In the right image, the integration time of the region around the center of the lamp adjusts locally and asynchronously to its mean illumination. The so-called integral image is a common intermediate image representation used by well-known vision algorithms, e. It is defined as:. That is, each pixel composing II x , y is given by the sum of all the pixels above and to the left of the corresponding pixel at the input image.
Focal Plane array processors with adaptive visual range and millimeter wave sensors
We exploit the focal-plane reconfigurability sketched in Figure 5 to compute the integral image. For example, the computation of its first row simply requires to disable all row connections between pixels and then progressively enable column connections, and thereby averaging, for each pixel of the row. This control pattern repeats for the rest of the rows provided that the interconnection between them is progressively enabled as the computation of the integral image progresses. In order to deal with the extremely wide signal range required to represent an integral image, charge redistribution plays a key role as the underlying physical operation supporting the computation at the focal plane.
It permits to keep the signal swing within the range of individual pixels, no matter how many pixels of the original image are involved in the computation of the current integral image's pixel. The average value obtained for each case simply requires externally keeping track of the position of the pixel being calculated. Thus, we only need to multiply that average value by the number of row and column associated with the corresponding pixel. In other words, the array is capable of computing an averaged version of the integral image mathematically described as:.
- Reward Yourself?
- Uncovering Student Thinking About Mathematics in the Common Core, Grades K–2: 20 Formative Assessment Probes.
- Twilight of the Vuvuzelas?
- A Cowboys First Time (Gay Erotica).
The array can also compute an averaged version of the square integral image. To this end, we make use of the squarer experimentally tested and reported in [ 48 ]. Thus, by precharging the capacitor holding V SQ ij to V DD and exploiting its discharge for a short period of time through the transistor M SQ working in the saturation region, the value of the pixel square can be computed. Then, charge redistribution takes over, just as for the integral image. Indeed, apart from the required previous computation of the pixel square, the procedure to obtain the averaged versions of the integral image and the square integral image is exactly the same, applied respectively to the voltages V S ij and V SQ ij at the pixel level.
A timing diagram showing two consecutive computations of integral image pixels is depicted in Figure In order to read out and convert these pixels, we must simply connect V S 1,1 and V SQ 1,1 to the respective analog-to-digital converters. These voltages will always contain the targeted calculation for each pixel, according to the definition of integral image and the proposed hardware implementation based on charge redistribution. Timing diagram showing the signals involved in the computation of two consecutive integral image pixels. An example of on-chip integral image computation is shown in Figure In this case, we can visualize the averaged integral image delivered by the chip and the integral image that can be directly derived from it.
On-chip focal-plane integral image computation and comparison with the corresponding off-chip ideal computation. The combination of focal-plane reconfigurability, charge redistribution and distributed memory enables subsequent reduced kernel filtering by adjusting which pixels merge their values and in which order. Very useful image filtering kernels, like the binomial filter mask for image Gaussian smoothing or the Sobel operators for edge detection, fall into the category of reducible kernels [ 49 ].
Operating on the pre-processed image with one of them represents a smaller number of operations per pixel than realizing all of the multiply-accumulate operations needed to apply the corresponding original kernels. Memory accesses are reduced in the same fraction. An example of Gaussian filtering is depicted in Figure On-chip focal-plane Gaussian filtering and comparison with the corresponding off-chip ideal computation. This section describes how the proposed on-chip reconfiguration scheme can be adapted to speed up object recognition and even to concurrently combine this task with privacy protection.
Note that our sensor was not designed with this application in mind. The connection between the different concepts involved came later when exploring image features suitable for our focal-plane sensing-processing approach. Therefore, key parameters of the prototype have not been tuned for reaching real-time operation in this case. Nevertheless, we consider that the hardware-software co-design framework sketched next is a completely new idea in the field of embedded vision systems. It will constitute a solid base supporting the process of setting specifications for a future sensor chip.
There are two major approaches for generic object recognition in computer vision: window-based object representations and part-based object representations [ 50 ]. Window-based representations perform better for recognition of rigid objects with a certain regularity in their spatial layout. They implement a holistic description of appearance based on a dense sampling of features obtained at multiple scales.
To determine the presence or absence of a certain object in a new image, this approach makes use of a sliding window testing all possible sub-windows of the image with a previously trained classifier. On the other hand, part-based representations usually perform better for non-rigid deformable objects, since these describe the appearance of an object through local parts following a certain geometric interconnection pattern.
They therefore incorporate more detailed spatial relations into the recognition procedure, requiring complex search procedures for the detection of image keypoints. Conversely, they require fewer training examples. The so-called granular space for feature extraction belongs to the sliding-window category. It was first defined in [ 17 ], being applied for multi-view face detection.
Focal Plane Sensor Processor Chips by BrandenBeaman - Issuu
It simply consists in a multi-resolution space composed of the so-called granules that observe each image instance at different locations and scales. The granules are represented by the average value of their constituting pixels. By combining different granules across the image, a new sparse feature is obtained. A large number of these sparse features are learned during training for a particular object. They are later analyzed during real operation for new images where such an object could be present. A scheme of the procedure is depicted in Figure The white and black granules are respectively weighted positively and negatively when it comes to attaining a representative value of the corresponding sparse feature.
This value is then compared with that of another feature or with a threshold, according to the classification technique applied.
- San Pablo en sus cartas (Spanish Edition)!
- Puntos de referencia (Vida y Pensamiento de Mexico) (Spanish Edition).
- Focal Plane array processors with adaptive visual range and millimeter wave sensors | magoxuluti.tk.
Object recognition pipeline based on the granular space. The original work in [ 17 ] has been extended during the last few years mostly by introducing variations in the training and feature search strategy [ 51 — 54 ]. The recognition of other objects, like pedestrians, has also been addressed.
Still, the raw material feeding the whole processing chain keeps being the granular space. It is on building this space where focal-plane sensing-processing can contribute to boosting the global performance. The granular space is ultimately based on block-wise averaging across the image plane. Coincidentally, the operation of averaging, physically supported by charge redistribution, is one of the most efficient computations to be realized in parallel at the focal plane. As we described in Section 3.
This scheme permits to apply division grids enabling the concurrent computation of rectangular-shaped granules of any size across the whole image, as depicted in Figure As mentioned earlier, the chip was not designed to provide several scales of the granular space per frame in real time as would be required by an algorithm of object recognition based on sparse feature extraction in such a space. Finally, notice that the computation of granules at the focal plane, while meaningful from the perspective of object recognition, implies a significant distortion of the image content.
Such a distortion can be exploited for privacy protection as long as the subsequent classifier can afford to work with granules of a large enough minimum size. Examples of focal-plane division grids concurrently providing granules of different sizes across the whole image. The resulting distortion can be exploited for privacy protection as long as the subsequent classifier can afford to work with granules of a large enough minimum size.
A great deal of tradeoffs affecting both hardware and software will have to be considered for the design and implementation of a future network visual node based on the focal-plane generation of the granular space. Key parameters for the performance of such a node, like the resolution, the frame rate or the detection rate, must be determined by carefully adjusting design variables, like the pixel size, sensitivity or conversion time hardware level , together with others, like the size of the granules or the training technique software level.
From our point of view, software simulations must first set the initial tentative specifications for the hardware. To this end, we are starting to build a tunable classifier based on features extracted from the granular space. The design loop will be closed by hardware simulations derived from the initial specifications provided by the classifier. These simulations will then feedback software simulations and vice versa in a constant process of parameter adjustment and performance evaluation. Eventually, we will obtain accurate requirements for a smart imager tailored for a privacy-aware vision system providing reliable object recognition at low power consumption.
The deployment of ubiquitous networked visual sensors seamlessly integrated in the Internet of Things is still a challenge to be accomplished. Privacy and power consumption are two major barriers to overcome. Once visual information is disconnected from private data and locally processed by low-power trusted computing devices, a wide range of innovative applications and services will be able to be addressed; applications and services now impossible to be implemented because current technology collides with legal constraints and public rejection.
The pixel-parallel processing on the sensor plane enables very high speed, and reduces high-cost time, energy long-distance data transfers. Complex algorithms can be performed completely within the KOVA1 chip. The pixel cells in the array are physically interconnected within their local neighborhoods, allowing direct information exhange in sensor-level image analysis operations. Local, pixel-level memories allow multiple full images or intermediate processing results to be stored on the sensor plane. Efficient integration to higher level image content analysis systems.