Brake lights classifier

[danielv012]

No description provided.

[saiperam]

09/09/2024 Update:

• Status: In Progress - So far I was able to research robust datasets online that have vehicle brake lights with annotations ON/OFF. I also was able to successfully run a script that spawns vehicles in CARLA, randomly configures vehicles to have brake lights ON/OFF and publishes their status, and capture screenshots in preparing a dataset in CARLA. I also was able to modify images with different weather conditions like sunny, and rain. I also successfully completed HSV and LAB Colorspace filtering in preparing for preprocessing datasets.
• Next Steps: I plan on working on splitting the data with an ideal 70% train 30% test. I also plan on soon implementing an object detection model and training a random forest classifier or a region based CNN for effectively classifying brake lights ON/OFF. I also plan on applying both colorspace filtering on current real-time datasets of various real car brake lights.
• Projected Completion: Finish dataset collection and processing by September 15.
• Update: Successfully able to highlight the benefit of leveraging LAB Colorspace over HSV colorspace.

[saiperam]

09/16/2024 Update: Brake Light Classifier

• Status: In Progress - I successfully developed scripts for applying LAB and HSV Colorspace Filtering on the captured dataset of images. This week, I successfully wrote scripts that automatically annotate the captured dataset of car brake lights based on the statuses reported by CARLA, and also allow me to draw bounding boxes on car brake lights across various weather conditions and statuses.
• Next Steps: - I'm currently working on preprocessing, organizing, and cleaning up the dataset as I noticed there were some subfolders in the dataset directory that had no images possibly due to some technical error. This week, I fixed the bug in my script and images are correctly captured into the right directories. I plan on soon training an object detection model like YOLOv3 to recognize car brake lights given any image of a car rear, and to also recognize whether these lights are on or off based on their specific red LAB colorspace threshold values (the more brighter the red contrast in LAB colorspace filtered images brake lights are, the more likely it is to be on.)
• Projected Completion: Finish organizing and cleaning up the dataset, applying bounding boxes on car brake lights, and annotations of statuses to each image by September 25.
• Update: I'm successfully able to annotate and draw bounding boxes of brake lights of car rear image captures in CARLA. These annotations and bounding boxes are successfully stored as displayed by the images shown below.

[saiperam]

09/23/2024 Update:
Status: In Progress - The past week I researched the current pre-existing YOLOv7 Object Detection Model that performs brake light classification: https://medium.com/@armaan.sandhu.2002/training-yolov7-to-detect-vehicle-braking-e8e7e9db1b3b#3a49 and I'm researching how to integrate this model in the Navigator Repository. I have already cloned the GitHub repository onto my folder and have already set up the respective directories to store the dataset used to train this model accordingly (folders for train/test/validate as well as for labels). I also pulled all required dependencies for this model.
Next Steps: I plan on integrating the model further by looking at source documentation and understanding how the model functions as a whole to detect brake lights in advance. As well, for my CARLA dataset of images, I plan on applying data augmentation techniques like re-orienting or flipping images of the dataset I currently have to make it more robust.
Projected Completion: I plan on completing the integration of the current model ideally by the end of this month. Then, going forward, I plan on adjusting the bounding box identification techniques: having one box for each brake light instead of one big box for the vehicle rear.
Update: I was successfully able to pull the YOLOv7 model onto my docker container and also was able to update it with the corresponding dependencies.

[saiperam]

09/30/2024 Update:
Status: In Progress - This week I dove deeper into the YOLOv7 Brake Light Classification Model with regards to cloning the github repository into the navigator repo within my Docker container. I also was able to successfully download the provided dataset of images with 8 classifications:

1. Left Blinker - No Brakes: Going left and moving
2. Left Blinker - Brakes: Going left and stopping
3. Right Blinker - No Brakes: Going right and moving
4. Right blinker - brakes: Going right and stopping
5. Hazards (both blinkers) - Brakes
6. Hazards - No Brakes
7. No Blinkers - Brakes: Just stopping
8. No Blinkers - No Brakes: Just moving
   Next Steps: I plan on working on splitting this data first and identifying annotation and bounding boxes as I have in the past for the CARLA data of brake lights. I then plan on preprocessing and cleaning up the data to ensure no anomalies are present. Moving forward, I'll transition into integrating the existing model and training it.
   Projected Completion: Finish cleaning up and starting to train the model by mid October.
   Update: Successfully downloaded and copied the dataset of rear signals of every class into my docker container and am currently working on splitting it for using a portion in training the model.

[saiperam]

10/07/2024 Update:
Status: In Progress - This week, I was able to pull and load the pre-trained weights of the model: yolov7.pt. I was also able to download the COCO dataset which is standardized for YOLO models. I was able to run a script - detect.py - that successfully detected random objects in input sample images: horse, tie, apple, shoes...
Next Steps: I plan on fine-tuning the model to recognize brake lights of a car, and look into automating the process of drawing bounding boxes based on LAB Colorspace filtering threshold values recognized by the model.
Projected Completion: Finish fine-tuning the model by the end of October and complete dataset cleansing by end of October.
Update: Successfully downloaded the pretrained weights and ran a YOLOv7 object detection model on detecting arbitrary objects in a given image. I'm currently working on fine-tuning this model to recognize brake lights and soon training it with the dataset above downloaded.

[saiperam]

10/21/2024 Update:
Status: In Progress - This week, I researched the method of effectively merging the background images with the normal and braking tail light images.
Next Steps: I plan on automating the process of preprocessing the dataset, including augmenting the images and labeling them with the correct tail light status (normal/braking). This will prepare the data for fine-tuning the YOLOv7 model specifically for brake light detection.
Update: Successfully downloaded the pretrained weights and ran a YOLOv7 object detection model on detecting arbitrary objects in a given image. I'm currently working on further fine-tuning this model to recognize brake lights and will soon start training it with the newly processed dataset.

[saiperam]

10/28/2024 Update:
Status: In Progress - This week, I progressed by generating merged images, combining the background images with both normal and braking tail light images. This step is preparing the dataset needed to effectively train and test the YOLOv7 model on detecting brake lights.
Next Steps: I plan to finalize the dataset by labeling these merged images accurately and automating the dataset preparation process. This will allow me to proceed with fine-tuning the YOLOv7 model to identify red regions specifically in the context of car brake lights. Once the model shows reliable performance in predicting brake lights, I will integrate it into the ROS2 node for real-time brake light detection.
Update: Successfully generated merged images with background and both normal and braking tail light images. This dataset will enhance the model's ability to distinguish brake light status accurately. I also started fine-tuning the YOLOv7 model to focus on predicting red regions specific to car brake lights, aiming to increase its accuracy in identifying braking actions.

[saiperam]

11/04/2024 Update:
Status: In Progress - This week, I successfully trained the YOLOv7 model for 40 epochs and achieved reliable detection of vehicle rears in images. The model can now detect multiple vehicles when more than one car is present in an image, as well as trucks and people.

Next Steps: To meet the project’s goal of detecting brake light statuses (i.e., "brakes on" vs. "brakes off"), I plan to further fine-tune the model to focus specifically on this classification. Currently, it identifies the rears of vehicles but does not specifically categorize brake light status. I plan to review and debug the current YOLOv7 model configuration to see if it can be fine-tuned for brake light status classification directly. If needed, I’ll then explore adding a colorspace filtering technique to isolate red regions within the bounding boxes, which would allow the model to focus specifically on brake lights and enhance braking detection.

Update: I was successfully able to train and run the YOLOv7 model described in https://medium.com/@armaan.sandhu.2002/training-yolov7-to-detect-vehicle-braking-e8e7e9db1b3b#3a49. During setup, I encountered a shared memory allocation issue which was limiting the GPU's ability to process data with CUDA. I resolved this by launching a new Docker container instance with increased shared memory allocation to fully support the NVIDIA CUDA platform, allowing smoother model training and execution.

The following are a few outputs of the YOLOv7 drawing bounding boxes, identifying car rears and people.

[saiperam]

10/28/2024 Update:
Status: In Progress - This week, I was able to run the YOLOv7 model to make detections on cars in an image.
Next Steps: I have a clear vision of fine-tuning the model to detect just the 2 classes of Braking and Normal.
Update: Successfully determined the weights needed to perform detection on a given image and identified the corrupted weights I had before. I now have a clear idea and plan on working with generating a customized YOLOv7 based dataset to predict brakes only based off the general weights.