What is Computer Vision?

This article explores computer vision, its core tasks and its growing applications across industries. From enhancing email security and powering self-driving cars to improving medical imaging and cybersecurity: Discover how computer vision is transforming technology and securing systems in our digital world.

Table of Contents

• What is computer vision?
• How does computer vision work?
• computer vision examples
• computer vision in cybersecurity

What is computer vision?

Computer Vision is a field of computer science that uses Artificial Intelligence (AI) to understand visual data (imagery). If AI is used to “think” by analyzing and interpreting data inputs, Computer Vision is used to “see” by analyzing and interpreting visual inputs.

Typical Computer Vision Tasks

Visual data used in Computer Vision includes anything with pixels—images, videos, icons, etc.— with the goal of being able to interpret imagery the way humans do. While the technology is rapidly expanding, below are three main Computer Vision tasks:

• Object classification:
  
  The ability to find or classify an object in visual data. At the most basic level, this would include the computer being able to identify the contents of an image—such as trees, the ocean, or a dog.
• Object localization:
  
  Object localization is the ability to locate a single object within visual data. For example, if you have a picture of a dog, the computer could identify and locate the dog within the image.
• Object detection:
  
  Object detection is a more advanced form of Computer Vision technology. Object detection capabilities allow computers to accurately identify and locate where multiple objects are within visual data. For example, imagine you have an image of both a dog and a cat together. With object detection, the computer could differentiate between the two objects and also accurately identify and locate both.

How does computer vision work?

Computer Vision works off of pattern recognition. It uses algorithms to scan an immense amount of visual data and make sense of it.

Training for Computer Vision involves exposing computers to massive amounts of visual data so they can learn to distinguish key features and patterns. For example—to teach a computer to identify boats, data scientists must feed it images of boats, as well as related objects. Over time, the computer learns to distinguish what is and isn’t a boat, as well as the key features of a boat.

Once the computer accurately identifies what an image is, it can then segment, detect, and classify it for various applications. Through advancements in AI, Computer Vision technology is as good as — and even oftentimes superior than — their human counterparts at image recognition.

Computer vision examples

Computer vision has a wide array of use cases that is rapidly expanding as the technology becomes more accessible. Some of the most common examples of Computer Vision technology today include:

• Email security
  
  Computer Vision enhances email security by analyzing images and identifying features typically used in phishing attacks like brand logos, QR codes, and suspicious copy. Image manipulation—a tactic hackers use where they slightly alter a company’s logo in order to bypass blacklists—is an increasing threat to organizations. Computer Vision provides an added layer of security to defend against these highly sophisticated email phishing attacks by recognizing image manipulation, even in remote images.
• Facial recognition technology
  
  Computer Vision powers software that analyzes a person’s facial features to verify their identity. While the use cases for facial recognition software are rapidly expanding, some popular ones include—security checkpoints such as airports and country borders, criminal suspect identification, and smartphone login.
• Self-driving cars
  
  Computer Vision is at the crux of self-driving technology. Self-driving cars like Tesla use object classification, identification, and localization to safely maneuver the vehicle with minimal need for human intervention.
• Traffic light and pedestrian detection
  
  Traffic-light detection is used to optimize the flow of traffic. The technology relies on digital sensors to recognize key traffic moments—cars waiting for the light to change, when emergency vehicles are nearby, or even when a car runs a red-light. Similarly, pedestrian detection is used to identify pedestrians at crosswalks and various points in the road.
• Medical imaging (X-Rays, CT Scans, etc.)
  
  Radiologists and other medical specialists rely on Computer Vision to help them more accurately interpret medical imaging. The AI model helps reveal abnormalities not easily detected by the human eye, vastly improving accuracy and even prognosis.

Computer vision in cybersecurity

In 2019, the cybersecurity community turned towards Computer Vision technology to help with security threat detection. Using Machine Learning, researchers were able to create a program that could detect malware based on the images associated with it. Overtime, this AI technology evolved to include the detection of phishing websites, emails, and more.

At a high-level, Computer Vision works to identify cybersecurity threats by using binary visualization to identify detectable patterns. Binary visualization allows the software to detect patterns that are malicious or safe. Malicious files tend to include either ASCII characters or more vibrant imagery.

Traditionally, protecting against malicious websites involved blacklisting suspicious domains. Computer Vision enhances cybersecurity by detecting malware from images sent via phishing emails. This adds a layer of protection before the recipient clicks on the domain that leaves them susceptible to attacks. What it’s called may vary, but the ultimate goal is the same—to deceive users into trusting a phishing email or website. Even savvy internet users can be duped by this spoof, as cousin domains are often indistinguishable from a legitimate domain. For example, hackers may substitute Latin-script characters for those used in the Cyrillic alphabet.

How computer vision enhances email security

Using Computer Vision technology, email security filters continually scans images found within emails to identify malicious content. Cybercriminals use three main techniques:

• Image manipulation:
  
  This tactic uses slightly altered images to evade email security solutions using fingerprint technology.
• Text-based images:
  
  This advanced phishing technique uses images that contain text that needs to be extracted with OCR (Optical Character Recognition) to bypass security analysis.
• Remotely hosted images:
  
  Remote images, images hosted elsewhere online, make it easier to sidestep email security since they’re often hosted on reputable sites like GitHub, making detecting malware more difficult.