File Name: vehicle license plate recognition based on extremal regions and restricted boltzmann machines.zip
- A Robust Attentional Framework for License Plate Recognition in the Wild
- Vehicle License Plate Recognition Based on Text-line Construction and Multilevel RBF Neural Network
- Multi-Oriented and Scale-Invariant License Plate Detection Based on Convolutional Neural Networks
This disclosure relates to optical recognition methods and systems and more particularly, to an intelligent camera for character recognition. Most image processing systems for industrial applications and video surveillance are still based on a personal computer PC , a frame grabber and a separate CCD camera.
A Robust Attentional Framework for License Plate Recognition in the Wild
License plate detection LPD is the first and key step in license plate recognition. State-of-the-art object-detection algorithms based on deep learning provide a promising form of LPD.
However, there still exist two main challenges. First, existing methods often enclose objects with horizontal rectangles. However, horizontal rectangles are not always suitable since license plates in images are multi-oriented, reflected by rotation and perspective distortion. Second, the scale of license plates often varies, leading to the difficulty of multi-scale detection.
To address the aforementioned problems, we propose a novel method of multi-oriented and scale-invariant license plate detection MOSI-LPD based on convolutional neural networks. To obtain bounding parallelograms, we first parameterize the edge points of license plates by relative positions. Next, we design mapping functions between oriented regions and horizontal proposals. Then, we enforce the symmetry constraints in the loss function and train the model with a multi-task loss.
Finally, we map region proposals to three edge points of a nearby license plate, and infer the fourth point to form bounding parallelograms. To achieve scale invariance, we first design anchor boxes based on inherent shapes of license plates.
Next, we search different layers to generate region proposals with multiple scales. Finally, we up-sample the last layer and combine proposal features extracted from different layers to recognize true license plates. Experimental results have demonstrated that the proposed method outperforms existing approaches in terms of detecting license plates with different orientations and multiple scales. License plate recognition is a key technology for intelligent transportation systems.
It has been widely applied in traffic surveillance and road management. Typically, license plate recognition consists of three procedures: license plate detection LPD , character segmentation, and text recognition. Among them, LPD is the first and key step, since it directly affects follow-up tasks and determines the overall accuracy [ 1 ].
As the need for automatic detection of motor vehicles grows, to design an effective and efficient LPD method is becoming increasingly important. To detect license plates in images, representative features should be extracted to distinguish the target license plates from the background.
Traditional LPD methods carefully handcraft features based on inherent attributes of license plates e. These methods can achieve satisfactory performance under certain conditions. However, the handcrafting process is labor-intensive, and the extracted features only reveal local and low-level characteristics [ 21 ]. In the practical use, multiple kinds of features are often combined to improve the accuracy.
For example, Yuan et al. They first down-scaled the input images and applied a line density filter to extract candidate regions. Then they trained a cascaded classifier with color saliency features and used the classifier to detect true license plates. This model is accurate and achieves state-of-the-art performance among traditional LPD methods. Therefore, it is chosen as representative of the traditional LPD methods to be compared with our approach in the experiments.
In recent years, deep learning methods based on convolutional neural networks CNN have achieved remarkable performance in general object-detection tasks. Driven by the great success, several methods [ 23 , 24 , 25 , 26 ] have been proposed to adapt the object-detection algorithms for the LPD task. In CNN, features are automatically learned from data and reveal high-level characteristics of the inputs.
Therefore, LPD methods based on deep learning are less labor-intensive and more accurate than traditional ones. To improve the efficiency, Rafique et al. Faster R-CNN [ 27 ] is the classic algorithm among region-based deep learning methods [ 27 , 28 , 29 , 30 ]. It consists of a region proposal network RPN to generate high-quality proposals, and a detection network to recognize and locate true objects.
These proposal-free methods directly estimate object locations without generating region proposals. Therefore, they are faster but relatively less accurate and robust than the region-based ones. Our work adopts the region-based mechanism and uses Faster R-CNN [ 27 ] as the backbone architecture. However, directly training the deep models for the LPD task may not achieve good enough performance. For the practical LPD applications, there still exist two main challenges.
The first challenge is that the detected regions are not accurate enough. Unlike general object-detection, LPD is often the prerequisite for character recognition. Since the license plates should be rectified ahead of recognition, the localization needs to be highly accurate.
However, general object-detection algorithms enclose objects by horizontal rectangles. As shown in Figure 1 a 1 , horizontal rectangles cannot tightly enclose the multi-oriented license plates. Two main challenges for existing methods. Existing methods based on horizontal or rotated rectangles cannot tightly enclose the multi-oriented license plates.
The detected regions and actual license plates are presented by red horizontal rectangles and green polygons, respectively. It is difficult to detect license plates with multiple scales, especially the tiny ones.
Works for text detection [ 33 , 34 , 35 , 36 , 37 ] can provide some insights for the multi-oriented detection issue. In [ 33 , 34 ], fully convolutional network FCN [ 38 ] was used to predict salient maps, and geometric approaches were applied to estimate the orientations.
These methods partly solve the problem, but the prerequisite segmentation is time-consuming. To improve the efficiency, end-to-end systems based on detection networks have been proposed in [ 35 , 36 ].
These methods adapted the object-detection networks to directly regress rotated rectangles from inclined proposals or boxes. They are relatively more efficient than the segmentation-based methods, but the added inclined hypotheses still produce heavy computation cost. More importantly, the rotated rectangles are still not accurate enough for practical applications. As shown in Figure 1 a 2 , rotated rectangles have right angles, while the skewed license plates have free angles.
The mismatch results in inaccurate enclosure around the corners. Recently, Liao et al. Since quadrilaterals have free orientations and angles, they can fit arbitrary regions. However, the lack of geometric constraints brings difficulty to model training and harms the recognition performance. Besides, its proposal-free architecture is less accurate and robust than the region-based ones.
Therefore, ref. The second challenge is that the detection of license plates with multiple scales has not been well solved. A traditional way to improve the scale invariance is to use the scattering operator as in the works of [ 39 , 40 , 41 ]. The scattering descriptor contains high-frequency information of the wavelet coefficients, and is robust to the scale variance of inputs.
In deep learning methods, Faster R-CNN [ 27 ] deals with the scale issue simply by referring to anchor boxes with multiple scales and aspect ratios. This is effective in some way, but there is still a severe inconsistency between the objects with various scales, and filter receptive fields with very limited scale ranges [ 42 ]. As shown in Figure 1 b, the detection performance is particularly poor for small targets.
To improve the performance, He et al. This method works well, but it requires inputs with multiple scales, which brings high computational cost. To improve the efficiency, works of [ 32 , 42 , 44 , 45 ] built feature pyramids instead of image pyramids by exploiting different convolutional layers.
In [ 32 , 42 ], region proposals or default boxes were generated on multiple layers. In [ 44 , 45 , 46 ], features were extracted from different layers.
These methods have achieved good performance and are inspiring to our work. However, they only take advantage of the multiple layers for one task, while the layers can be further used. The proposed MOSI-LPD tightly encloses license plates with bounding parallelograms, and is highly invariant to the scale discrepancy of license plates.
The main contributions of our work can be summarized as follows. We propose novel strategies to tightly enclose the multi-oriented license plates with bounding parallelograms. Both the network architecture and the loss function are elaborately designed to directly regress bounding parallelograms from horizontal proposals.
Our method significantly improves the localization precision and guarantees a high detection accuracy simultaneously. We design effective strategies to detect license plates with multiple scales. Multiple convolutional layers are exploited both for proposal generation and feature extraction. The priori knowledge regarding inherent shapes of license plates is considered for anchor box design. Our method is highly invariant to the scale discrepancy of license plates, and effectively detects tiny license plates that are only several pixels.
We construct a large license plate dataset. The dataset contains more than images, and all the license plates are labeled by the exact edge points. The two sub-networks share the fundamental CNN structure. The backbone framework is Faster R-CNN [ 27 ], the classic region-based deep learning network for object detection.
To achieve multi-oriented and scale-invariant detection, several vital modifications are proposed. The anchor boxes are set based on the priori knowledge regarding inherent shapes of license plates. We estimate three edge points of the license plates by regressing relative positions from horizontal proposals. The fourth edge point is inferred based on the symmetry property to form final bounding parallelograms that tightly enclose the multi-oriented license plates.
Right after the first four convolutional layers, there is also a Max-pooling layer. In each sliding window, the dimension convolutional features are extracted, and we simultaneously refer to 9 anchor boxes. The scales and aspect ratios of the anchor boxes are set based on the priori knowledge regarding the inherent shapes of license plates. The features are then fed into two fully connected layers: the first layer classifies the regions as license plate proposals or the background, and the second layer regresses functions that adjust proposal positions.
Vehicle License Plate Recognition Based on Text-line Construction and Multilevel RBF Neural Network
Recognizing car license plates in natural scene images is an important yet still challenging task in realistic applications. Many existing approaches perform well for license plates collected under constrained conditions, eg, shooting in frontal and horizontal view-angles and under good lighting conditions. However, their performance drops significantly in an unconstrained environment that features rotation, distortion, occlusion, blurring, shading or extreme dark or bright conditions. In this work, we propose a robust framework for license plate recognition in the wild. It is composed of a tailored CycleGAN model for license plate image generation and an elaborate designed image-to-sequence network for plate recognition. On one hand, the CycleGAN based plate generation engine alleviates the exhausting human annotation work.
The core technology of the system is built using a sequence of deep Convolutional Neural Networks CNNs interlaced with accurate and efficient algorithms. The CNNs are trained and fine-tuned so that they are robust under different conditions e. For quantitative analysis, we show that our system outperforms the leading license plate detection and recognition technology i. ALPR on several benchmarks. In the past, there has been considerable focus on license plate detection [ 1 , 2 , 3 ] and recognition techniques [ 4 , 5 , 6 , 7 , 8 , 9 , 10 ]. From traffic and toll violations to accident monitoring, the ability to automatically detect and recognize plates is one of the key tools used by law enforcement agencies around the world. Contrary to popular belief, license plate detection and recognition is still a challenging problem due to the variability in conditions and license plate types.
Index Terms—License plate detection, license plate recognition, hybrid discriminative restricted Boltzmann machines, extremal regions. I. INTRODUCTION.
Multi-Oriented and Scale-Invariant License Plate Detection Based on Convolutional Neural Networks
License plate detection LPD is the first and key step in license plate recognition. State-of-the-art object-detection algorithms based on deep learning provide a promising form of LPD. However, there still exist two main challenges.
Skip to search form Skip to main content You are currently offline. Some features of the site may not work correctly. DOI:
С учетом обстоятельств, я полагаю, - сказала Сьюзан, - вам все же нужно позвонить директору. Стратмор покачал головой, и капля пота с его лба упала на стол. - Я не хочу никоим образом нарушать покой директора и говорить с ним о кризисе, в то время как он не в состоянии предпринять хоть что-нибудь.
Наконец парень посмотрел на. - Scusi? - Он оказался итальянцем. - Аегорortо. Per favore.