Changing the Yolov8-OBB head to output Polygonal Bounding Box with Four Corners instead of Oriented Bounding Box

[Nixson-Okila]

Search before asking

• I have searched the YOLOv8 issues and discussions and found no similar questions.

Question

I would like to modify Yolov8-OBB to give an output in a polygonal form shown below.

Could you guide me how and which files to configure

Additional

No response

[github-actions]

👋 Hello @Nixson-Okila, thank you for your interest in Ultralytics YOLOv8 🚀! We recommend a visit to the Docs for new users where you can find many Python and CLI usage examples and where many of the most common questions may already be answered.

If this is a 🐛 Bug Report, please provide a minimum reproducible example to help us debug it.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset image examples and training logs, and verify you are following our Tips for Best Training Results.

Join the vibrant Ultralytics Discord 🎧 community for real-time conversations and collaborations. This platform offers a perfect space to inquire, showcase your work, and connect with fellow Ultralytics users.

Install

Pip install the ultralytics package including all requirements in a Python>=3.8 environment with PyTorch>=1.8.

pip install ultralytics

Environments

YOLOv8 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all Ultralytics CI tests are currently passing. CI tests verify correct operation of all YOLOv8 Modes and Tasks on macOS, Windows, and Ubuntu every 24 hours and on every commit.

[glenn-jocher]

Hello! Modifying the YOLOv8-OBB model to output polygonal bounding boxes (PBB) with four corners instead of the standard oriented bounding boxes (OBB) involves a few changes to the model's architecture and post-processing steps.

Here’s a brief guide on how to approach this:

1. Model Output Modification: You'll need to adjust the model's head to output eight values (x1, y1, x2, y2, x3, y3, x4, y4) representing the coordinates of the four corners of the bounding box. This can be done in the model definition file (typically a .yaml file).

2. Post-Processing Changes: Modify the post-processing code to handle these eight output values correctly. This involves adjusting the code that interprets the model outputs to create bounding boxes from these coordinates.

3. Loss Function Adjustment: Ensure that the loss function used during training can handle the difference in bounding box representation. You might need to customize it to calculate the loss based on the distances between the predicted corners and the actual corners.

Here's a simple example of what changes in the model definition might look like (assuming you're familiar with the structure of the .yaml files used to configure YOLOv8 models):

# Assuming 'head' is the section of the model where outputs are defined
head:
  - [Conv, [256, 3, 1]]
  - [Conv, [8, 1, 1]]  # Outputting 8 values (x1, y1, x2, y2, x3, y3, x4, y4)

Remember, these changes require a good understanding of both the model architecture and the training process. Testing and validation are crucial to ensure that the model performs as expected with the new bounding box format.

If you need more detailed guidance, feel free to ask! 😊

[Nixson-Okila]

Thank you so much for the guidance. Could you help me with the code modification of the postprocess() function in the predict.py file. I will greatly appreciate

…

On Mon, May 20, 2024 at 1:44 AM Glenn Jocher ***@***.***> wrote: Hello! Modifying the YOLOv8-OBB model to output polygonal bounding boxes (PBB) with four corners instead of the standard oriented bounding boxes (OBB) involves a few changes to the model's architecture and post-processing steps. Here’s a brief guide on how to approach this: 1. *Model Output Modification*: You'll need to adjust the model's head to output eight values (x1, y1, x2, y2, x3, y3, x4, y4) representing the coordinates of the four corners of the bounding box. This can be done in the model definition file (typically a .yaml file). 2. *Post-Processing Changes*: Modify the post-processing code to handle these eight output values correctly. This involves adjusting the code that interprets the model outputs to create bounding boxes from these coordinates. 3. *Loss Function Adjustment*: Ensure that the loss function used during training can handle the difference in bounding box representation. You might need to customize it to calculate the loss based on the distances between the predicted corners and the actual corners. Here's a simple example of what changes in the model definition might look like (assuming you're familiar with the structure of the .yaml files used to configure YOLOv8 models): # Assuming 'head' is the section of the model where outputs are definedhead: - [Conv, [256, 3, 1]] - [Conv, [8, 1, 1]] # Outputting 8 values (x1, y1, x2, y2, x3, y3, x4, y4) Remember, these changes require a good understanding of both the model architecture and the training process. Testing and validation are crucial to ensure that the model performs as expected with the new bounding box format. If you need more detailed guidance, feel free to ask! 😊 — Reply to this email directly, view it on GitHub <#12783 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A3UOUB2UYUHP7ESDL45FYQ3ZDETNRAVCNFSM6AAAAABH4TB5P6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMJZGM4DGNBYHA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[glenn-jocher]

@Nixson-Okila hello!

Happy to help with the postprocess() function modification. Here's a concise example to handle the eight output values for the polygonal bounding boxes:

def postprocess(self, preds, img, orig_imgs):
    # Assuming preds are the raw model outputs
    results = []
    for pred, orig_img in zip(preds, orig_imgs):
        # Convert model outputs to polygon coordinates
        polygons = pred[:, :8].reshape(-1, 4, 2)  # Reshape to (num_boxes, 4 points, 2 coords)
        polygons = ops.scale_boxes(img.shape[2:], polygons, orig_img.shape[:2])
        
        # Create Results object
        results.append(Results(orig_img, polygons=polygons))
    return results

This snippet assumes that your model outputs the coordinates in a flat format and that you have a utility function scale_boxes to adjust the coordinates to the original image size. Make sure to adapt it to fit the exact output format and utility functions available in your setup.

Let me know if you need further assistance! 😊

[Nixson-Okila]

Awesome, thanks for the info. I will check it out

…

On Fri, May 24, 2024 at 10:47 AM Glenn Jocher ***@***.***> wrote: @Nixson-Okila <https://github.com/Nixson-Okila> hello! Happy to help with the postprocess() function modification. Here's a concise example to handle the eight output values for the polygonal bounding boxes: def postprocess(self, preds, img, orig_imgs): # Assuming preds are the raw model outputs results = [] for pred, orig_img in zip(preds, orig_imgs): # Convert model outputs to polygon coordinates polygons = pred[:, :8].reshape(-1, 4, 2) # Reshape to (num_boxes, 4 points, 2 coords) polygons = ops.scale_boxes(img.shape[2:], polygons, orig_img.shape[:2]) # Create Results object results.append(Results(orig_img, polygons=polygons)) return results This snippet assumes that your model outputs the coordinates in a flat format and that you have a utility function scale_boxes to adjust the coordinates to the original image size. Make sure to adapt it to fit the exact output format and utility functions available in your setup. Let me know if you need further assistance! 😊 — Reply to this email directly, view it on GitHub <#12783 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A3UOUB3ZY5LABNPBUHCOBCLZD3WANAVCNFSM6AAAAABH4TB5P6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMRYHAZDONBWGA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[glenn-jocher]

Great to hear that, @Nixson-Okila! If you run into any snags or have more questions as you implement the changes, don't hesitate to reach out. Happy coding! 😊

[Nixson-Okila]

Thanks a lot for your kind advice. I have still hit a snag with the modification. I have failed to modify the scale_box() for rescaling the bounding box into the format xyxyxyxy. As an inquiry, could it not be easier to modify into polygonal bounding detection the rectangular bounding box detection than oriented bounding box detection. Thanks

…

On Fri, May 24, 2024 at 10:42 PM Glenn Jocher ***@***.***> wrote: Great to hear that, @Nixson-Okila <https://github.com/Nixson-Okila>! If you run into any snags or have more questions as you implement the changes, don't hesitate to reach out. Happy coding! 😊 — Reply to this email directly, view it on GitHub <#12783 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A3UOUB22EKP4RIP3PKP7EIDZD6J3JAVCNFSM6AAAAABH4TB5P6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMZQGI2DSMRXHA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[glenn-jocher]

Hi @Nixson-Okila,

Thanks for your feedback! 😊

For the scale_boxes() function, you can modify it to handle the xyxyxyxy format by scaling each coordinate pair individually. Here’s a quick example:

def scale_boxes(img1_shape, boxes, img0_shape):
    gain = min(img1_shape[0] / img0_shape[0], img1_shape[1] / img0_shape[1])
    pad = (img1_shape[1] - img0_shape[1] * gain) / 2, (img1_shape[0] - img0_shape[0] * gain) / 2  # wh padding
    boxes[:, [0, 2, 4, 6]] -= pad[0]  # x padding
    boxes[:, [1, 3, 5, 7]] -= pad[1]  # y padding
    boxes[:, :8] /= gain
    return boxes

Regarding your inquiry, modifying the rectangular bounding box detection to polygonal bounding detection can indeed be simpler in some cases. This is because rectangular bounding boxes are axis-aligned and easier to manipulate. However, the choice depends on your specific use case and the nature of the objects you are detecting.

Feel free to reach out if you have more questions or need further assistance!

[Nixson-Okila]

Thanks for the guidance. I will get back when stuck

…

On Mon, May 27, 2024 at 8:38 PM Glenn Jocher ***@***.***> wrote: Hi @Nixson-Okila <https://github.com/Nixson-Okila>, Thanks for your feedback! 😊 For the scale_boxes() function, you can modify it to handle the xyxyxyxy format by scaling each coordinate pair individually. Here’s a quick example: def scale_boxes(img1_shape, boxes, img0_shape): gain = min(img1_shape[0] / img0_shape[0], img1_shape[1] / img0_shape[1]) pad = (img1_shape[1] - img0_shape[1] * gain) / 2, (img1_shape[0] - img0_shape[0] * gain) / 2 # wh padding boxes[:, [0, 2, 4, 6]] -= pad[0] # x padding boxes[:, [1, 3, 5, 7]] -= pad[1] # y padding boxes[:, :8] /= gain return boxes Regarding your inquiry, modifying the rectangular bounding box detection to polygonal bounding detection can indeed be simpler in some cases. This is because rectangular bounding boxes are axis-aligned and easier to manipulate. However, the choice depends on your specific use case and the nature of the objects you are detecting. Feel free to reach out if you have more questions or need further assistance! — Reply to this email directly, view it on GitHub <#12783 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A3UOUB6I3QDO5YKYVGNTZPTZENVRVAVCNFSM6AAAAABH4TB5P6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMZTHA3TCNRRGU> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[glenn-jocher]

You're welcome! Feel free to reach out if you encounter any issues or need further assistance. We're here to help! 😊

Best of luck with your modifications!

[Nixson-Okila]

Hello, When I run the command : !python path/to/train.py --data path/to/my_dataset.yaml --cfg path/to/default.yaml --weights ' ' --batch-size 16 --epochs 10 using the Yolov8 model which I have git cloned locally, training does not take place and no error is given either. I have started with the defaults parameters without going into what I have modified. Could you help how to fix this. Thanks

…

On Tue, May 28, 2024 at 3:07 PM Glenn Jocher ***@***.***> wrote: You're welcome! Feel free to reach out if you encounter any issues or need further assistance. We're here to help! 😊 Best of luck with your modifications! — Reply to this email directly, view it on GitHub <#12783 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A3UOUB5MTZMP4JBBME547VDZERXQ3AVCNFSM6AAAAABH4TB5P6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMZVGA2TMMRSGE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[glenn-jocher]

Hello,

Thank you for reaching out. It seems unusual that the training does not start without any error messages. Here are a few steps you can take to diagnose and potentially resolve the issue:

1. Check Python Environment: Ensure that your Python environment is set up correctly and all dependencies are installed. Sometimes, missing libraries can cause silent failures.

2. Verify Paths: Double-check the paths you've provided in the command to ensure they are correct and accessible. This includes path/to/train.py, path/to/my_dataset.yaml, and path/to/default.yaml.

3. Console Output: Run the command directly in a terminal (outside of any notebooks if you're using one) to see if there are any output messages that might not be showing up in your current environment.

4. Logs: Check if there are any logs generated in the directory where you are running the command. They might contain clues about what's going wrong.

5. Minimal Configuration: Try running the training with a minimal configuration using a well-known dataset like COCO128 to rule out any issues with your custom dataset or configuration.

If these steps do not resolve the issue, please provide any additional information about changes you made to the configuration or other relevant details, and we'll be glad to assist further!

[Nixson-Okila]

Thank you, I will do that.

…

On Sun, Jun 2, 2024 at 11:18 PM Glenn Jocher ***@***.***> wrote: Hello, Thank you for reaching out. It seems unusual that the training does not start without any error messages. Here are a few steps you can take to diagnose and potentially resolve the issue: 1. *Check Python Environment*: Ensure that your Python environment is set up correctly and all dependencies are installed. Sometimes, missing libraries can cause silent failures. 2. *Verify Paths*: Double-check the paths you've provided in the command to ensure they are correct and accessible. This includes path/to/train.py, path/to/my_dataset.yaml, and path/to/default.yaml. 3. *Console Output*: Run the command directly in a terminal (outside of any notebooks if you're using one) to see if there are any output messages that might not be showing up in your current environment. 4. *Logs*: Check if there are any logs generated in the directory where you are running the command. They might contain clues about what's going wrong. 5. *Minimal Configuration*: Try running the training with a minimal configuration using a well-known dataset like COCO128 to rule out any issues with your custom dataset or configuration. If these steps do not resolve the issue, please provide any additional information about changes you made to the configuration or other relevant details, and we'll be glad to assist further! — Reply to this email directly, view it on GitHub <#12783 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A3UOUB2FRY6IFIX3IGCXF6TZFN42ZAVCNFSM6AAAAABH4TB5P6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNBUGAYDQNZSHA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[glenn-jocher]

@Nixson-Okila you're welcome! If you encounter any further issues or have questions as you proceed, don't hesitate to reach out. We're here to help! 😊