A Rule based Urdu Q/A System that inputs a document and a question file and outputs the sentence or line that contains the answer to that question
Run the prototype.py file with parameters (document_file, question_file) placed in the input folder. The output is in the answer.txt in the output file. Added a result.xlxs file that tracks the models accuracy on articles from the SQUAD Dataset (Using Google Translate for both document and question). Link to the articles below. for every passage, there are 5 questions. Results saved in the form of yes if the answer matched the websites answer and no if not. Accuracy computed.
Link to Articles:
Harvard_University: https://rajpurkar.github.io/SQuAD-explorer/explore/v2.0/dev/Harvard_University.html?model=nlnet%20(single%20model)%20(Microsoft%20Research%20Asia)&version=v2.0
Version 0.1:
- developed first prototype using maximum keyword matching
- stripping stopwords from sentence tokens to retrieve keywords
Version 0.2:
- Added rules for Who and How Many Question type
- improved the sentence score system
- using a postfix stemmer to retrieve the root of the word
- using Person Dictionary with names and titles to perform NER
- Important: use the keywords (kitnay/kitni) near the object you wish to retrieve the quantity for: Example: Dhamakay mai kitnay afrad halak hoay?
Version 0.3:
- Added rules for When and Where question type
- using date related words for time expressions
- using locations and city names, along with locational pronouns to identify locations in a sentence
- tested against some SQUAD dataset articles
Version 0.4:
- Improved Question type detection with more variations in keywords (kis nai)(kis saal)(kon sa mulk)
- Added improved rules for What and Why Question types