WIN1201-20 Fleming College Interactive Greeting System


 

Brandon D’cunha: Team Leader

Worked on: Navigation, Webpages, Backend, Detection, and automation. Development, integration and testing of the system.

Skills: C/C++, Java, Python,My SQL, Web Development, Flask framework, Networking, Photoshop and Premiere pro.

Cassandra Dsouza: Communication Leader

Worked on: Face Recognition, Search Engine, Backend and Detection. Development, Integration and testing.

Skills: Android Application Development, Matlab, R, C, Java, Speech Processing, Image Processing, Python, Web development, Flask framework, and MySQL

Amanpreet Kaur: Project Manager

Worked on: UX/UI and Security

Skills: C, C++, Java, PHP, HTML, MySQL, Android Development, and Photoshop.

Rahul Kumar: Documentation

Worked on: Voice to Text Development and Integration.

Skills: MATLAB, C++ and Python.

To the FIRST PLACE PRIZE winners of Fleming College Innovation and Technology Showcase 2021.

A hearty congratulations to this team for their consistent delivery of excellent work in competition with 43 teams in 3 Technology Programs, Wireless Information Networking (WIN), Computer Investigations (CSI), Computer Technologies (CTY) and Mechatronics (MEC).

The Judges were highly impressed. Keep up the great work and your careers will be very rewarding.

Best of luck, Rahul, Amanpreet, Cassandra and Brandon.

Regards
Alwyn Appiah

Professor, IT Applied Projects
School of Business and IT

www.flemingcollege.ca


Team Name:

#suDoers

Team Members Names:

Brandon D'cunha, Cassandra Dsouza, Amanpreet Kaur, Rahul Kumar

Program of Study:

Wireless Information Networking

Mentor Name:

Cathy Smits

Project Name:

Fleming College Interactive Greeting System

Sponsor Name:

George Seto



Sponsor Organisation:

Fleming College


Brief Description Of Project:

This project includes two systems, each at different entrances for anyone visiting Fleming College. The system will detect the user, initiate the display to greet them, assist with their queries, and guide them to their destination. Facial recognition is implemented for the user and is optional while registration can be completed using credentials. Once you have chosen either voice or text input you can enter the question in the search engine. The system will then display a list of possible locations for them to choose from. Once a destination is chosen, the direction to the location is displayed using a map of the college. The user can continue navigating after exiting the system using their device.

Project Objective:

  • To detect the user when they arrive at the system
  • Recognize and register users using facial recognition
  • To display locations in the college and provide directions
  • For systems to synchronize with each other
  • To search a question and display possible results
  • To detect speech and search the users queries

Project Innovation:

  • Our systems runs on Raspberry Pi, we needed the processing power for facial recognition and to perform other various automated processes simultaneously.
  • Implemented a Passive Infrared Sensor (PIR), user will be detected after coming within 2m range of the system.
  • Used OpenCV and facial recognition module to register and recognize the user, this was implemented using Python and flask.
  • Instead of saving the images of the user we saved encoding of the facial features.
  • Implemented search to easily display relevant results by extracting keywords from the user’s questions.
  • Used a LAMP server provided by the college as a database to store user information and System Backups.
  • Implemented Voice to Text recognition so that the user doesn’t have to type the question on the screen.
  • Our system performs indoor navigation which requires additional components. These components were not provided which is why we displayed the map with QR codes. Now the user can interact with the maps on their devices and navigate to their destinations.
  • College Floor Plan was converted into a Campus Map using photoshop and different layers of navigation were created.
  • Due to restriction of navigation indoors without additional components we used QR codes to display the current location of the user, if they lose their way, they can continue navigation from their current location to their destination. Click here from a mobile device or scan the QR code next to the video below
  • The systems need to be in sync because the user needs to be able to access both, this was achieved by using Python and Shell Scripts.
  • Due to the system being implemented at the entrance the processes such as Syncing, Frequent Backups and User Detection had to be automated which was done using various Python Scripts and Shell Scripts running as Cron Jobs.

Challenges & Solutions:

  •  Majority of our project was working on the physical devices and various components which could not be done due to the pandemic. Alternatively we set up the device with remote access and connected the components so that each Member had access to the device and we could work on it simultaneously.
  • We started developing the project in HTML/CSS and used PHP. The rest of the modules were coded in Python, integrating Python and PHP is not feasible which is why we decided to implement all our applications using flask by converting PHP to HTML/flask.
  • Flask application could not be connected to the server because the server did not accept connections from an external IP address. This was fixed by granting permissions to the Raspberry Pi’s IP addresses and setting privileges.
  • We were provided with the college floor plan and a blueprint which needed to be converted into maps to be displayed this was achieved using Photoshop.
  • We attempted to implement interactive navigation using HTML/CSS but mobile browsers could not fully support it hence we abandoned the idea. Click Here from a Desktop Browser to check it out.
  • Due to Covid-19 restrictions it was not possible to access the college and we could not add photos of the route as an alternate guide.
  • It was a challenge to work together on various modules in the project such as Face recognition, Navigation, Search, Speech detection and Process Automation. We overcame this by setting up remote connections to devices and scheduling work together on the specific module at the same time.

Project Results or Progress:

The Results Achieved so far are:

  • Detecting the user within 2m of the system
  • Registering the user using Facial Registering
  • Logging in to the system using Facial Recognition with 90% accuracy
  • Displaying a list of various locations provided by the sponsor
  • Displaying the navigation route to the destination on the users device after leaving the system
  • Creating a Search Bar to display the possible results for the questions asked by the users
  • Communication between two systems so that the systems are in sync with each other every 5 minutes
  • Performing regular Backups of the systems at midnight

Lessons Learned:

The Team Learned the following :

  • Remotely access hardware and implement solutions and programs on it.
  • Working of navigation system and to display and guide a person indoors to reach the destination without the need of additional sensors and components.
  • Python programming and implementing OpenCV to detect, register and recognize a face.
  • Deploying a LAMP server and also providing connections to the applications.
  • Integration of webpages and Python was difficult and understanding how Python works with Flask and its integration.
  • Working together on a project remotely and on a particular issue at the same time.

Things that the team could have done differently:

  • Start working on smaller components and features of a module rather than working towards the bigger picture/ functionality.
  • Mistakes we made were to implement a particular functionality and trying to make other features work according to it, rather than researching and developing a feature to work seamlessly.

Short Video: