Real-time object recognition in complex background - algorithm

Question

I'm trying to make a real-time advertisement billboard detection in road using android smartphone. The goal is to crop the area of the advertisement billboard object (regions of interest) and save it to database.
For example:
enter image description here
enter image description here
For preprocessing, I used grayscaling and Canny Edge Detection (Otsu thresholding is used to set the upper and lower threshold). Then, I used contour-based method to detect whether the object is rectangular by checking the point. I use Java OpenCV in android studio for implementation. When I run the program, it only detect rectangular object in plain background and if the rectangular having a high contrast from the background. Currently, it can only detect rectangle with 90 degree and it failed to detect object with rounded rectangle shape. Furthermore, my program failed completely to detect rectangular object in a more complex background, like road scene where the object I'm trying to detect is having similar color to the background/low contrast or there are many occlusions like tree, traffic light, and cables which caused the detection to fail.
This is the code I use for edge detection
Mat destination = new Mat(oriMat.rows(), oriMat.cols(), oriMat.type());
Imgproc.cvtColor(oriMat, destination, Imgproc.COLOR_RGBA2GRAY);
Imgproc.GaussianBlur(destination, destination, new Size(3,3), 0, 0, Imgproc.BORDER_DEFAULT);
double otsuThresholdValue = Imgproc.threshold(destination, destination, 0, 255, Imgproc.THRESH_BINARY | Imgproc.THRESH_OTSU);
double lowerThreshold = otsuThresholdValue*0.5;
double upperThreshold = otsuThresholdValue;
Mat canny = new Mat();
Imgproc.Canny(destination, canny, lowerThreshold, upperThreshold);
Mat abs = new Mat();
Core.convertScaleAbs(canny, abs);
Mat result = new Mat();
Core.addWeighted(abs, 0.5, abs, 0.5, 0, result);
Here is the code I use for contour-based detection
ArrayList<MatOfPoint> contours = new ArrayList<>();
// find contours and store them all as a list
Imgproc.findContours(matData.monoChrome.clone(), contours, new Mat(), Imgproc.RETR_EXTERNAL, Imgproc.CHAIN_APPROX_SIMPLE);
final int width = matData.monoChrome.rows();
final int height = matData.monoChrome.cols();
int matArea = width * height;
for (int i = 0; i < contours.size(); i++) {
double contoursArea = Imgproc.contourArea(contours.get(i));
MatOfPoint2f approx = new MatOfPoint2f();
MatOfPoint2f contour = new MatOfPoint2f(contours.get(i).toArray());
double epsilon = Imgproc.arcLength(contour, true) * 0.1;
// Imgproc.minAreaRect(contour);
// approximate contour with accuracy proportional to the contour perimeter
Imgproc.approxPolyDP(contour, approx, epsilon, true);
if (Math.abs(contoursArea) < matArea * 0.01 ||
!Imgproc.isContourConvex(new MatOfPoint(approx.toArray()))) {
continue;
}
Imgproc.drawContours(matData.resizeMat, contours, i, new Scalar(0, 255, 0));
List<Point> points = approx.toList();
int pointCount = points.size();
LinkedList<Double> cos = new LinkedList<>();
for (int j = 2; j < pointCount + 1; j++) {
cos.addLast(angle(points.get(j % pointCount), points.get(j - 2), points.get(j - 1)));
}
Collections.sort(cos, (lhs, rhs) -> lhs.intValue() - rhs.intValue());
double mincos = cos.getFirst();
double maxcos = cos.getLast();
if (points.size() == 4 && mincos >= -0.3 && maxcos <= 0.5) {
for (int j = 0; j < points.size(); j++) {
Core.circle(matData.resizeMat, points.get(j), 6, new Scalar(255, 0, 0), 6);
}
matData.points = points;
break;
}
}
Is there any method I can use to recognize advertisement billboard in road?
I would appreciate any answers and ideas. Thank you!