HackGT 2017 Repository

Yu Fu, Zijin Luo, Jinghua Zhang, Zhu Zhuang

A lot of schools and colleges are in high demand of note takers for students with certain disabilities and impairment. Therefore, we decided to design an application to transcribe photos that students took from lectures or audio recordings to notes in Word Documents automatically to help students to study.

Our application can recognize typed and hand-written words, detect and convert recorded audio to text, and integrate all data into organized text. This Android App will be a great helper for those who cannot go to classes and for instructor to send notes.

We used Android development (with Java programming) for front-end of the application, which is a mobile app that presents the physical structure of our work. We also used python to develop the backend server that hosts the application of machine learning results.

We encountered some problems involving accuracy of hand-written text recognition, consecutive voice detection, and stable and instant backend server establishment.

We learned about the new concept of convolution neural networks and deep learning. We are very proud that we successfully developed the application using those new ideas and the app also has a high accuracy in detecting hand-written words and vocal sounds.

We would like to enhance and add more features to the application, such as adding the PDF function which can transform the transcribed text into formal PDF file for students to use.

Android-studio, Java, Google Cloud Platform, Convolutional Nereul Network,
backend-server, Firebase, Deep-learning, Python