Over the last few years, FPGAs have moved from glue logic to computing platforms. They effectively provide a reconfigurable hardware platform for implementing logic and algorithms. Being fine-grained hardware, FPGAs are able to exploit the parallelism inherent within a hardware design while at the same time maintaining the reconfigurability and programmability of software. This has led to FPGAs being used as a platform for accelerating computationally intensive tasks. This is particularly seen in the field of image processing, where the FPGA-based acceleration of imaging algorithms has become mainstream.
Field programmable gate arrays (FPGAs) are increasingly being used for the implementation of image processing applications. This is especially the case for real-time embedded applications, where latency and power are important considerations. An FPGA embedded in a smart camera is able to perform much of the image processing directly as the image is streamed from the sensor, with the camera providing a processed output data stream, rather than a sequence of images. The parallelism of hardware is able to exploit the spatial (data level) and temporal (task level) parallelism implicit within many image processing tasks. Unfortunately, simply porting a software algorithm onto an FPGA often gives disappointing results, because many image processing algorithms have been optimized for a serial processor. It is usually necessary to transform the algorithm to efficiently exploit the parallelism and resources available on an FPGA.
Automatic Number plate recognition (ANPR) may be found useful in many applications ranging from applications such as monitoring parking lots to identifying speeding vehicles in expressways. The procedure of implementing such systems may be found tedious due to problems such as: maintaining a constant distance with the rear view of the vehicle, varying lighting conditions, or the number plate being obstructed by some other object or dirt. Although the above-mentioned factors make the task of isolating and recognizing a number plate tedious, the speed of processing is a great concern because character recognition in real-time is vital in such applications. This research is focused mainly on the prime requirement for an improved number plate recognition system - the speed of processing. Hence, this system is based on a Field Programmable Gate Array (FPGA). Also, the implemented ANPR system has the ability to recognize characters of the number plate even if it is slightly obscured. The proposed Methodology of this project involves extracting the number plate from a rearview image of a vehicle and processing it in a suitable manner to perform character recognition using a template matching technique. This system is implemented based on several assumptions such as; the number plate being exposed to constant lighting conditions and that the frame of the number plate is captured for processing at a constant distance from the camera. These assumptions were made so that more focus could be given to faster processing, character recognition, and other concerns when using image processing techniques in hardware descriptive languages.
Before processing an image we need to load the image to the FPGA but Verilog cannot read image directly. To read a .bmp image in Verilog, the image is required to be converted from bitmap format to the hexadecimal format. An image can be easily converted to hexadecimal/binary data using Matlab, C, Python, etc. Below is a Matlab example code to convert a bitmap image to a .hex file. The input image size is 768x512 and the .hex file includes R, G, B data of the bitmap image.
For Verilog code, you can use $readmemh(for hexadecimal data) or $readmemb(for binary data) command to load a converted binary/hexadecimal text file directly. Example Verilog code for loading a text file or an image into FPGA using $readmemh.
b=imread('kodim24.bmp'); % 24-bit BMP image RGB888
k=1;
for i=512:-1:1 % image is written from the last row to the first row
for j=1:768
a(k)=b(i,j,1);
a(k+1)=b(i,j,2);
a(k+2)=b(i,j,3);
k=k+3;
end
end
fid = fopen('kodim24.hex', 'wt');
fprintf(fid, '%x\n', a);
disp('Text file write done');disp(' ');
fclose(fid);
% fpga4student.com FPGA projects, Verilog projects, VHDL projects
To read the image hexadecimal data file, Verilog uses this command: $readmemh or $readmemb if the image data is in a binary text file. After reading the image .hex file, the RGB image data are saved into memory and read out for processing.
Below is the Verilog code to the image reading:
/******************************************************************************/
/****************** Module for reading image **************/
/******************************************************************************/
`include "parameter.v" // Include definition file
module image_read
#(
parameter WIDTH = 768, // Image width
HEIGHT = 512, // Image height
INFILE = "./img/kodim01.hex", // image file
START_UP_DELAY = 100, // Delay during start up time
HSYNC_DELAY = 160,// Delay between HSYNC pulses
VALUE= 100, // value for Brightness operation
THRESHOLD= 90, // Threshold value for Threshold operation
SIGN=1 // Sign value using for brightness operation
// SIGN = 0: Brightness subtraction
// SIGN = 1: Brightness addition
)
(
input HCLK, // clock
input HRESETn, // Reset (active low)
output VSYNC, // Vertical synchronous pulse
// This signal is often a way to indicate that one entire image is transmitted.
// Just create and is not used, will be used once a video or many images are transmitted.
output reg HSYNC, // Horizontal synchronous pulse
// An HSYNC indicates that one line of the image is transmitted.
// Used to be a horizontal synchronous signals for writing bmp file.
output reg [7:0] DATA_R0, // 8 bit Red data (even)
output reg [7:0] DATA_G0, // 8 bit Green data (even)
output reg [7:0] DATA_B0, // 8 bit Blue data (even)
output reg [7:0] DATA_R1, // 8 bit Red data (odd)
output reg [7:0] DATA_G1, // 8 bit Green data (odd)
output reg [7:0] DATA_B1, // 8 bit Blue data (odd)
// Process and transmit 2 pixels in parallel to make the process faster, you can modify to transmit 1 pixels or more if needed
output ctrl_done // Done flag
);
//-------------------------------------------------
// Internal Signals
//-------------------------------------------------
parameter sizeOfWidth = 8; // data width
parameter sizeOfLengthReal = 1179648; // image data : 1179648 bytes: 512 * 768 *3
// local parameters for FSM
localparam ST_IDLE = 2'b00,// idle state
ST_VSYNC = 2'b01,// state for creating vsync
ST_HSYNC = 2'b10,// state for creating hsync
ST_DATA = 2'b11;// state for data processing
reg [1:0] cstate, // current state
nstate; // next state
reg start; // start signal: trigger Finite state machine beginning to operate
reg HRESETn_d; // delayed reset signal: use to create start signal
reg ctrl_vsync_run; // control signal for vsync counter
reg [8:0] ctrl_vsync_cnt; // counter for vsync
reg ctrl_hsync_run; // control signal for hsync counter
reg [8:0] ctrl_hsync_cnt; // counter for hsync
reg ctrl_data_run; // control signal for data processing
reg [7 : 0] total_memory [0 : sizeOfLengthReal-1];// memory to store 8-bit data image
// temporary memory to save image data : size will be WIDTH*HEIGHT*3
integer temp_BMP [0 : WIDTH*HEIGHT*3 - 1];
integer org_R [0 : WIDTH*HEIGHT - 1]; // temporary storage for R component
integer org_G [0 : WIDTH*HEIGHT - 1]; // temporary storage for G component
integer org_B [0 : WIDTH*HEIGHT - 1]; // temporary storage for B component
// counting variables
integer i, j;
// temporary signals for calculation: details in the paper.
integer tempR0,tempR1,tempG0,tempG1,tempB0,tempB1; // temporary variables in contrast and brightness operation
integer value,value1,value2,value4;// temporary variables in invert and threshold operation
reg [ 9:0] row; // row index of the image
reg [10:0] col; // column index of the image
reg [18:0] data_count; // data counting for entire pixels of the image
//-------------------------------------------------//
// -------- Reading data from input file ----------//
//-------------------------------------------------//
initial begin
$readmemh(INFILE,total_memory,0,sizeOfLengthReal-1); // read file from INFILE
end
// use 3 intermediate signals RGB to save image data
always@(start) begin
if(start == 1'b1) begin
for(i=0; i<WIDTH*HEIGHT*3 ; i=i+1) begin
temp_BMP[i] = total_memory[i+0][7:0];
end
for(i=0; i<HEIGHT; i=i+1) begin
for(j=0; j<WIDTH; j=j+1) begin
// Matlab code writes image from the last row to the first row
// Verilog code does the same in reading to correctly save image pixels into 3 separate RGB mem
org_R[WIDTH*i+j] = temp_BMP[WIDTH*3*(HEIGHT-i-1)+3*j+0]; // save Red component
org_G[WIDTH*i+j] = temp_BMP[WIDTH*3*(HEIGHT-i-1)+3*j+1];// save Green component
org_B[WIDTH*i+j] = temp_BMP[WIDTH*3*(HEIGHT-i-1)+3*j+2];// save Blue component
end
end
end
end
//----------------------------------------------------//
// ---Begin to read image file once reset was high ---//
// ---by creating a starting pulse (start)------------//
//----------------------------------------------------//
always@(posedge HCLK, negedge HRESETn)
begin
if(!HRESETn) begin
start <= 0;
HRESETn_d <= 0;
end
else begin // ______
HRESETn_d <= HRESETn; // | |
if(HRESETn == 1'b1 && HRESETn_d == 1'b0) // __0___| 1 |___0____ : starting pulse
start <= 1'b1;
else
start <= 1'b0;
end
end
//-----------------------------------------------------------------------------------------------//
// Finite state machine for reading RGB888 data from memory and creating hsync and vsync pulses --//
//-----------------------------------------------------------------------------------------------//
always@(posedge HCLK, negedge HRESETn)
begin
if(~HRESETn) begin
cstate <= ST_IDLE;
end
else begin
cstate <= nstate; // update next state
end
end
//-----------------------------------------//
//--------- State Transition --------------//
//-----------------------------------------//
// IDLE . VSYNC . HSYNC . DATA
always @(*) begin
case(cstate)
ST_IDLE: begin
if(start)
nstate = ST_VSYNC;
else
nstate = ST_IDLE;
end
ST_VSYNC: begin
if(ctrl_vsync_cnt == START_UP_DELAY)
nstate = ST_HSYNC;
else
nstate = ST_VSYNC;
end
ST_HSYNC: begin
if(ctrl_hsync_cnt == HSYNC_DELAY)
nstate = ST_DATA;
else
nstate = ST_HSYNC;
end
ST_DATA: begin
if(ctrl_done)
nstate = ST_IDLE;
else begin
if(col == WIDTH - 2)
nstate = ST_HSYNC;
else
nstate = ST_DATA;
end
end
endcase
end
// ------------------------------------------------------------------- //
// --- counting for time period of vsync, hsync, data processing ---- //
// ------------------------------------------------------------------- //
always @(*) begin
ctrl_vsync_run = 0;
ctrl_hsync_run = 0;
ctrl_data_run = 0;
case(cstate)
ST_VSYNC: begin ctrl_vsync_run = 1; end // trigger counting for vsync
ST_HSYNC: begin ctrl_hsync_run = 1; end // trigger counting for hsync
ST_DATA: begin ctrl_data_run = 1; end // trigger counting for data processing
endcase
end
// counters for vsync, hsync
always@(posedge HCLK, negedge HRESETn)
begin
if(~HRESETn) begin
ctrl_vsync_cnt <= 0;
ctrl_hsync_cnt <= 0;
end
else begin
if(ctrl_vsync_run)
ctrl_vsync_cnt <= ctrl_vsync_cnt + 1; // counting for vsync
else
ctrl_vsync_cnt <= 0;
if(ctrl_hsync_run)
ctrl_hsync_cnt <= ctrl_hsync_cnt + 1; // counting for hsync
else
ctrl_hsync_cnt <= 0;
end
end
// counting column and row index for reading memory
always@(posedge HCLK, negedge HRESETn)
begin
if(~HRESETn) begin
row <= 0;
col <= 0;
end
else begin
if(ctrl_data_run) begin
if(col == WIDTH - 2) begin
row <= row + 1;
end
if(col == WIDTH - 2)
col <= 0;
else
col <= col + 2; // reading 2 pixels in parallel
end
end
end
//-------------------------------------------------//
//----------------Data counting---------- ---------//
//-------------------------------------------------//
always@(posedge HCLK, negedge HRESETn)
begin
if(~HRESETn) begin
data_count <= 0;
end
else begin
if(ctrl_data_run)
data_count <= data_count + 1;
end
end
assign VSYNC = ctrl_vsync_run;
assign ctrl_done = (data_count == 196607)? 1'b1: 1'b0; // done flag
