GSoC 2013 Ankit Agrawal Implementation of STAR and Binary Feature Detectors and Descriptors
Name : Ankit Agrawal
Email : aaaagrawal@iitb.ac.in
github : ankit-maverick
IRC nick : AnkitAgrawal
Skype : aaaagrawal
Blog : http://skimager.blogspot.com
University : Indian Institute of Technology(IIT), Bombay
Major : Electrical Engineering with Masters in Communication and Signal Processing
Current Year : Third year
Expected Graduation date : June 2015
Degree : Dual Degree(B.Tech + Masters in 5 yrs)
I am Ankit Agrawal, a third year student enrolled in a 5 year Dual Degree Program(B.Tech and Masters) in Electrical Engineering at IIT Bombay. My Masters specialization is in Communication and Signal Processing and I intend to pursue my Master's project in Computer Vision during the next two years at my university. I want to work on the idea of 'Implementation of STAR and Binary Feature Detectors and Descriptors'. The relevant coursework that I have done in the past related to this project are Linear Algebra, Probability and Stochastic Processes, Digital Signal Processing, Image Processing, Machine Learning and Computer Vision. Python is the programming language I am most confident with. I also know C++, Java, Matlab and R and have used them in various projects.
I have been using many open source softwares for my daily programming purposes and have always felt the desire to contribute back to the open source community. The last few weeks have been very satisfying to me because of the learning curve I have ascended by writing good quality test-driven code in a collaborative environment while contributing to some open source projects in Python. Becoming a part of development team of such open source projects requires time to gain trust in the code you write and the decisions you make. Google Summer of Code provides a perfect platform for someone like me to become a part of well established open source project.
Feature Detectors and Descriptors are at the center of many tasks in Image Processing and Computer Vision. Good Feature Detectors extract out the keypoints that are robust to image transformations like scaling, different viewpoints, change in intensity etc. Good Feature Descriptors extract a compact description of a keypoint which can be used to accurately match or correspond keypoints in multiple images and hence are widely used for tasks like Image Registration, Object Recognition and Tracking, Stereo Vision etc. This project focuses on implementing STAR(CenSurE) and Binary feature detectors and descriptors like BRIEF, ORB(Oriented BRIEF) and FREAK to extend the features module in scikit-image.
- Image Processing course project: Mini-project 1 and Mini-project 2 based on the paper 'Undersampled Radial MRI with Multiple Coils. Iterative Image Reconstruction Using a Total Variation Constraint' in MATLAB
- Computer Vision Project(WIP) : Parallel Tracking and Mapping in 3D using Kinect based on the paper 'Parallel Tracking and Mapping for Small AR Workspaces' using OpenCV and Point Cloud Library
- Implementation of Search Engine features : Predicting Similar Queries, Query Auto-completion, Spell checker using 20M query data from AOL
- Machine Learning competitions on Kaggle
- Basic Pixelwise Image Transforms(WIP) https://github.com/scikit-image/scikit-image/pull/505
- Resampling of nD arrays(WIP) https://github.com/scikit-image/scikit-image/pull/511
- Deprecating is_rgb and is_gray(WIP) https://github.com/scikit-image/scikit-image/pull/532
By the end of the summer, I intend to have completed the implementations of the following feature detectors and descriptors :
- BRIEF feature point descriptor and descriptor Matcher
- STAR(CenSurE) feature detector
- ORB(Oriented BRIEF) feature descriptor
- FREAK feature descriptor
I don't have any other commitment during summer hence I will work the normal 40 hr/week throughout the summer. I will commit at least four times a week to ensure that my code is reviewed regularly by the mentors and I can work on the received feedback immediately. I will reply to mails within 12 hrs of their reception. I will report my weekly progress on this blog.
- Reading the papers in the reference section more thoroughly.
- Discussing implementation details with mentors and the community.
- Start working on implementing BRIEF descriptor to get a head-start.
- Implementing the BRIEF Descriptor and Matcher[1] in the first week.
- Image Smoothing
- Sampling Binary tests
- Hamming Distance based Matching
- Improving the implementation, adding tests and documentation in the second week.
- Adding the STAR feature detector[2] in the first two weeks.
- Difference of Boxes
- Line Suppression
- Non Maximal Suppression
- Third week for improving the implementation, adding tests and documentation.
- Buffer period to finish off any previous incomplete work
- Code Optimization
- Adding Examples
- Implementing ORB(Oriented BRIEF) descriptor[3] in the first week.
- Steered BRIEF
- Greedy search to obtain rBRIEF
- Second week for improving the implementation, adding tests and documentation.
- Implementing the FREAK descriptor[4] in the first three weeks.
- Coarse to Fine Ordering of Difference of Gaussians
- Saccadic Search
- Rotation Estimation of Keypoint
- Fourth week for improving the implementation, adding tests and documentation.
- Buffer period to finish off any previous incomplete work
- Code Optimization
- Adding Examples
- Implementing FAST feature detector[9], BRISK feature detector and descriptor[10]
- Implementing Machine Learning based Computer Vision algorithms like face detection in scikit-image
- Implementing Descriptor Matchers[7] like FLANN, Brute Force Matcher that can be used for Image Registration, Object Recognition etc.
(1) BRIEF: Binary Robust Independent Elementary Features
(2) CenSurE: Center Surround Extremas for Realtime Feature Detection and Matching
(3) ORB: an efficient alternative to SIFT or SURF
(4) FREAK: Fast Retina Keypoint
(5) Comparative Evaluation of Binary Features
(6) Computer Vision by Richard Szeliski, Page no. 205, Chapter 4, Feature detection and matching
(7), (8) OpenCV features2D module
(9) FAST Corner Detection
(10) BRISK: Binary Robust Invariant Scalable Keypoints