Math Homework Machine
Project Overview
To achieve every student’s dream, I created a homework machine to read worksheets and reveal their answers. As a ten-week assignment, problems were limited to simple multiplication problems, called “times tables” in many US schools, where third grade students multiply whole numbers inclusively between 0 and 12.
The core project components were a LEGO prototype for projecting the machine’s plausibility, Tesseract for transcribing image files to text, OpenCV for visualizing Tesseract’s bounding boxes/overlaying answers on the photocopy, and a microcontroller for operating the scanner and conversing with the Python code via serial.
The input and output iteration explains the order of the program.
The core project components were a LEGO prototype for projecting the machine’s plausibility, Tesseract for transcribing image files to text, OpenCV for visualizing Tesseract’s bounding boxes/overlaying answers on the photocopy, and a microcontroller for operating the scanner and conversing with the Python code via serial.
The input and output iteration explains the order of the program.
LEGO® Prototype
As a serious LEGO® fan, I built an XY plotter capable of writing any numbers through slight code modifications. Hitting a push button caused a motor to pull in a worksheet, moving the vertical axis. Another motor moved a conveyor belt, sliding an attached pen to traverse the horizontal axis. A third motor controlled a cam, consequently lifting and lowering the pen between characters.
Tesseract
The library’s image_to_boxes function returned bounding box locations for characters found in the worksheet’s digital photocopy. Contextual methods for finding text were bypassed by selecting a specific page segmentation method. Consequently, I made my own sorting algorithm to find neighboring characters. From here, identifying an expression by spotting the multiplication sign, “x”, preceded and proceeded by two, nearby numbers was performed.
OpenCV
Besides reading, writing and binarising images, OpenCV contributed to overlaying (a) bounding boxes, (b) spotted multiplication signs and (c) worksheet answers. Bounding boxes and spotted multiplication signs visually help debugging. Meanwhile, OpenCV prints answers with it’s `putText` function for students to see the solutions.
Before implementing OpenCV vs after implementing openCV
Microcontroller
An Arduino UNO microcontroller coordinated a push-button to power a NEMA17 stepper motor for worksheet scanning, and the microcontroller conversed with Python serially. When one-ninth of the page was shifted for the next scan, the microcontroller sent a serial message to inform the webcam to take a photo. After all nine scans, the Arduino program ended and computer vision work in Python ensued.
The microcontroller controls a driver, in turn controlling the motor
Future Possibilities
Besides implementing motors to slide and lower the pen, solvable math problems could be easily expanded to addition, division, subtraction and fundamental calculus problems. Word problems would be more ambitious yet plausible, which could lend to standardized math tests and demonstrate mathematical comprehension of a text.
For a more thorough, technical analysis of the multiplication homework machine, visit the machine’s GitHub repository