Computer Vision Automation Tool

This project started as a practical experiment: could I use computer vision to automate a repetitive in-game task and measure the efficiency gain?

The Problem

Certain games involve highly repetitive item collection loops — clicking the same patterns hundreds of times. I wanted to see if I could build a lightweight tool that handles this automatically using screen analysis.

How It Works

The tool captures a region of the screen, runs template matching via OpenCV to locate target objects, and then triggers mouse inputs through the Win32 API to interact with them. The detection-to-action loop runs in real-time with minimal latency.

Detection Pipeline

1. Screen capture using Windows GDI for low-overhead frame grabbing
2. Template matching with OpenCV's matchTemplate to find objects at known scales
3. Thresholding and filtering to reject false positives
4. Coordinate mapping from captured region to screen space

Input Simulation

Once a target is identified, the tool uses Win32 SendInput to generate mouse events at the correct screen coordinates. Timing is randomized slightly to mimic human behavior.

Results

The tool achieved a 5x efficiency increase in item collection compared to manual input. More importantly, it was a great exercise in integrating image processing with system-level APIs in C++.

Tech Stack

• C++ for performance and low-level system access
• OpenCV for image processing and template matching
• Win32 API for screen capture and input simulation

What I Learned

This project reinforced the value of combining domains — game knowledge with computer vision and systems programming. It also gave me hands-on experience with real-time processing constraints and the practical challenges of working with screen-space coordinates.