In the design of a visually controlled robotic vehicle, a microcontroller based design is used to send serial data from a computer to a two wheeled robotic vehicle via an Arduino board. This is done to control the vehicle using the color detection technique (blob analysis) via the webcam of a computer on receiving an input from the user. The webcam of the computer detects a particular color of an object placed in front of it. The direction of movement of the robot is decided by the object movements made by the user. This direction is then serially sent via a USB to TTL Converter module from the computer port to the robotic vehicle through the Arduino microcontroller and the motor driver circuit. Hence, the robot can be made to move in all four directions by moving the object of a particular color in front of the webcam in a certain manner. The robotic vehicle also detects obstacle in its path with the help of an ultrasonic sensor put on the back of the vehicle. An ultrasonic sensor transmits ultrasonic waves from its sensor head and again receives the ultrasonic waves reflected from an object in its line of sight. It detects the position of this object by measuring the length of time from the transmission to reception of the sonic wave. Henceforth, when the vehicle is made to move backward, the sensor put on the vehicle detects the object and is triggered to stop when the distance between the object and the vehicle becomes less than 15 cm thereby facilitating safe parking of the vehicle. The visually controlled movements of the robotic vehicle as well as its obstacle detection capabilities make the robotic vehicle worthy of being replaced as a wheelchair for paralytic patients. © 2006-2014 Asian Research Publishing Network (ARPN).