This project has been finalized and submitted, and thus will no longer be worked on for the forseeable future.
People Analytics for the AU Library
The main aim of our project is to create a system or a package in which the library can use to replace the staff manually counting people that enter the library. Our project will focus on using pre-trained people detection models to create a system to detect, track, count people entering the library, and then using the counting data, display people analytics through our website. This repository consists of the backend program which utilizes pre-trained people detection models and a pre-made Kalman Filter algorithm.
See requirements.txt for software requirements. Software setups may vary.
Not all files are included in this repository
Some files are created by other authors
Note:
Inside run.py the video stream should be changed to a local file or stream IP address depending.
Website can be found here: link
More details can be found here: Github
Prototype Vers.
│ run.py - Main Program. Contains folder paths, video input and model inputs.
│
├───model - Model used ssd_mobilenet_v2_coco
│ │ checkpoint
│ │ frozen_inference_graph.pb
│ │ model.ckpt.data-00000-of-00001
│ │ model.ckpt.index
│ │ model.ckpt.meta
│ │ pipeline.config
│ │
│ └───saved_model
│ │ saved_model.pb
│ │
│ └───variables
├───output - Output folder for csv file and optional video output.
│ 16-46.avi
│ peoplecount.csv
│
├───utils
│ │ boundary.py - Contains classes to create boundary lines and an algorithm to check line intersection.
│ │ detector.py - Contains the class to initialize tensorflow models and perform detection.
│ │ filevideostream.py - Contains the classes for receiving a local video.
│ │ gcloud.py - Contains a function used to upload the csv file to GCloud.
│ │ person.py - Contains the tracker algorithm and the enter/exit conditions.
│ │ stream.py - Contains the classes for receiving various types of video streams.
│ │ tracker.py - Contains the class for the tracker for each person.
│ │ util.py - Contains formulas and algorithms for retrieving midpoints, distances, and gaussian.
│ │
When the program is run, the boundary lines are created and a connection to the IP camera is made with OpenCV. Once it is established, the video is passed frame by frame using imutils to the detector where it uses the pre-trained “ssd_mobilenet_v2_coco” object detection model and determines if a person is present.
If a person is detected, then bounding boxes are drawn around the person and their coordinates are passed to the people tracker class. Here, it keeps track of their coordinates and then it uses the Kalman Filter to predict their movements. Then a check is made to see if they meet the conditions of enter or exit. If their midpoints, or centroids pass the boundaries in a specific sequence, top to bottom or bottom to top, it counts them accordingly.
Counts are then stored as a csv file locally and then uploaded via Google Cloud API to a Google Cloud storage bucket. The file then acts as a sort of database that can be accessed with SQL queries. The website was created using Bootstrap and is also hosted on the same Google Cloud server as the file. From there the website sends SQL queries--via Google Cloud API--requesting data from the uploaded csv file, and then displays the data in charts and graphs.