Vision system is an important part of autonomous robots. Adding a vision system onboard to a robot requires a control system with a lot of processing power at high speed. Traditional systems with microprocessors or DSPs find it extremely difficult to meet real time requirements of high speed vision systems for small robots. This project demonstrates the use of SoC approach in designing vision based robots. The video data is passed directly from CMOS image sensor to the FPGA where preprocessing of the data is done. Color recognition and edge detection is implemented in hardware. This relieves CPUs or DSPs from tedious repetitive tasks and allows them to focus on more complicated tasks. Software written to MicroBlaze, a 32-bit soft-core RISC processor, within the same FPGA then performs object recognition and provides commands to stepper controller. Stepper controller, a hardware module implemented within the FPGA, generates required signals to drive different motors present in Microbot TeachMover for a desired motion. The system at present recognizes objects of red, white and black. The complete system stands as a starting base platform for more complex vision tasks and planning of control actions for autonomous robots. Team Members: Bishesh Khanal, Deepak Parajuli, Dinesh Twanabasu, and Suman Raj Bista.